arxiv: v1 [physics.data-an] 27 Oct 2014

Transcription

1 CERN-OPEN October 29, 2014 arxiv: v1 [physics.data-an] 27 Oct 2014 Interpolation between multi-dimensional histograms using a new non-linear moment morphing method M. Baak a, S. Gadatsch b, R. Harrington c, W. Verkerke b a CERN, Geneva, Switzerland b Nikhef, Amsterdam, The Netherlands c University of Edinburgh, Edinburgh, Scotland Abstract A prescription is presented for the interpolation between multi-dimensional distribution templates based on one or multiple model parameters. The technique uses a linear combination of templates, each created using fixed values of the model s parameters and transformed according to a specific procedure, to model a non-linear dependency on model parameters and the dependency between them. By construction the technique scales well with the number of input templates used, which is a useful feature in modern day particle physics, where a large number of templates is often required to model the impact of systematic uncertainties.

2 1 Introduction 1 Introduction In particle physics experiments, data analyses generally use shapes of kinematical distributions of reconstructed particles to interpret the observed data. These shapes are simulated using Standard Model or other theoretical predictions, and are determined separately for signal and background processes. Simulations of known fundamental physics processes are run through a detailed detector simulation, and are subsequently reconstructed with the same algorithms as the observed data. These simulated samples may depend on one or multiple model parameters, for example the simulated Higgs particle mass, and a set of such samples may be required to scan over the various parameter values. Since Monte Carlo simulation can be time-consuming, there is often a need to interpolate between the limited number of available Monte Carlo simulation templates. In particular, the statistical tests widely used in particle physics, e.g. for the construction of confidence intervals on model parameters or the discovery of new phenomena, rely strongly on continuous and smooth parametric models that describe the signal and background processes in the data 1. These parametric models describe parameters of interest, such as a shifting mass parameter or the rate of a signal process, and so-called nuisance parameters that parametrize the impact of systematic uncertainties. As such, the models are often constructed in terms of those parameters by interpolating between simulated Monte Carlo templates, thereby ensuring continuity in those parameters. Several algorithms exist that can be used to interpolate between Monte Carlo sample distributions [1, 2]. Interpolation techniques have been used on multiple occasions in particle physics, for example to predict kinematic distributions for intermediate values of a model parameter, e.g. the simulated Higgs boson, W boson or top quark mass, or to describe the impact of systematic uncertainties, which are often modeled as shape or rate variations about a nominal template of a kinematic distribution. This work describes a new morphing technique, moment morphing, which has the advantage over existing methods in that it is fast, numerically stable, allows for both binned histogram and continuous templates, has proper vertical as well as horizontal morphing (explained in Section 2), and is not restricted in the number of input templates, the number of model parameters or the number of input observables. In particular, the latter feature allows the moment morphing technique to model the impact of a non-factorizable response between different model parameters, where varying one model parameter at a time is insufficient to capture the full response function. The paper is organized as follows: Section 2 describes in detail how the moment morphing function, used to interpolate between histograms, is constructed using one or more morphing parameters. Section 3 describes how the moment morphing technique can be used to properly take into account systematic uncertainties in a high energy physics analysis, giving an example of a typical application. A comparison in terms of accuracy of moment morphing with alternative morphing algorithms is provided in Section 4. Section 5 describes the implementation of the moment morphing algorithm in publicly available C++ code, including benchmarking of its performance. 1 Typically, a statistical test involves the maximization of a likelihood function, which has been built from both the parametric model and the observed data. The maximization procedure relies on the derivatives of the likelihood with respect to the model s parameters. 2

3 2 Construction of the morphing p.d.f. 2 Construction of the morphing p.d.f. This section details the construction of the moment morphing probability density function (p.d.f.). The method proposed here is based on the linear combination of input templates. The dependency on the morphing parameter(s) can be non-linear, and is captured in multiplicative coefficients and a transformation of the template observables. Interpolation using a single morphing parameter is described in Section 2.1. Section 2.2 describes interpolation using multiple morphing parameters and shows that dependencies between morphing parameters can be readily modeled. Other choices of basis functions for the construction of the morphing p.d.f. are considered in Section Interpolation with a single morphing parameter Consider an arbitrary p.d.f. f(x m), where f depends on the single morphing parameter m and describes the observables x. The true dependency on m is not known or difficult to obtain. Instead, the p.d.f. f has been sampled at n different values of m, with each f(x m i ) representing a known input template shape for a single value of the morphing parameter, labeled m i. In the following the goal is to construct a parametric approximation of f(x m) for arbitrary m, which is continuous and smooth in the model parameter, as required for example by the statistical tests used in particle physics alluded to in Section 1. There are two steps to this. First, given the sampling points, f(x m) can be expanded in a Taylor series up to order n 1 around reference value m 0, f(x m) n 1 j=0 d (j) f(x m 0 ) (m m 0 ) j dm (j) j! n 1 = f j(x m 0 )(m m 0 ) j, (1) where the second equality defines f (x m). For the n given values of m follows the vector equation: j=0 n 1 n 1 f(x m i ) (m i m 0 ) j f j(x m 0 ) = M ij f j(x m 0 ), (2) j=0 where M ij = (m i m 0 ) j defines a n n transformation matrix. Inverting Eqn. 2 gives j=0 n 1 f j(x m ( 0 ) = M 1 ) ji f(x m i) (3) i=0 which allows to determine the n values f j (x m 0). Substituting this in Eqn. 1, f(x m) reads f(x m) n 1 (m m 0 ) j ( M 1) ji f(x m i), (4) i,j=0 which can be used to predict the template shape at any new value of the morphing parameter given by m, n 1 f pred (x m ) = c i (m )f(x m i ), (5) i=0 3

4 2.1 Interpolation with a single morphing parameter which is a linear combination of the input templates f(x m i ), each multiplied by a coefficient c i (m ), n 1 c i (m ) = (m m 0 ) j ( M 1) ji, (6) j=0 which themselves are non-linear and depend only on the distance to the reference points. This approach of weighting the input templates is also known as vertical morphing. Note that the coefficients c i are independent of the derivatives of f with respect to morphing parameters or to the observable set x, making their computation easy. The coefficient for a point included in the set of input templates is one, i.e. c i (m j ) = δ ij, (7) and by construction the sum of all coefficients c i equals one: c i (m) = 1. (8) This turns out to be a useful normalization, as will be seen below. i To illustrate, one can consider a morphed p.d.f. using only input templates at two values of the morphing parameter, m min and m max. The coefficients c i (m) become linear in m and reduce to the simple fractions: c imin = 1 m frac (9) c imax = m frac, () where m frac = (m m min )/(m max m min ), c imin and c imax sum up to one, and all other coefficients are zero. Second, it may be that the sampled input p.d.f.s f i describe distributions in x that vary strongly as a function of m in shape and location. This is equivalent to the first and second moments (i.e. the means and variances) of the input distributions having a dependence on the morphing parameter m. Since the input p.d.f.s in Eqn. 5 are summed linearly, it is imperative to translate all input distributions f i (x) in the sum before combining in the morphed p.d.f. such that their locations match up. The process of translating the input observables (but not scaling; see below) is also called horizontal morphing. In addition it is necessary to take in account the change in the width of the input distributions as a function of the morphing parameter. To achieve this, the mean µ ij and width σ ij of each input distribution i and observable x j are shifted to the common values of µ j (m) and σ j (m). These are obtained by multiplying the underlying means and widths with the coefficients c i (m) of Eqn. 6 µ j(m) = i σ j(m) = i c i (m) µ ij (11) c i (m) σ ij (12) 4

5 2.1 Interpolation with a single morphing parameter In order to shift the input p.d.f.s a linear transformation of each observable is applied. For each p.d.f. i and observable j define x ij = a ij x j + b ij, (13) with slope and offset a ij = σ ij σ j to replace the original observables x j in the input p.d.f.s (14) b ij = µ ij µ ja ij. (15) f(x m i ) f(x m i ). (16) Since only a linear transformation is applied to each observable, i.e. only the first two moments of the input p.d.f.s are modified, the normalization of the scaled input template is analytically related to the normalization of the original template as + f(x m i )dx = 1 j a j(m) + f i (x m i )dx, (17) with the slope a j = σ j /σ j, where j refers to the observable x j. The construction of the complete morphed p.d.f. as the sum of the transformed input p.d.f.s then gives p(x m ) = i c i (m )f(x, m i ) j a j (m ). (18) As the coefficients c i add up to 1 by construction, the morphed p.d.f. of Eqn. 18 is unit-normalized as well for normalized input templates. This leads to an important computational advantage and novelty: for models where the input templates are constant, such as histogram-based templates, no normalization integrals need to be recalculated during the minimization of the likelihood function, which is often a bottle-neck when using morphed p.d.f.s. As a result, the number of input templates is generally increasable without significant performance loss. Note that the self-normalization of Eqn. 17 remains valid when applying a rotation to the set of (multiple) observables, which would introduce covariance moments to the modified input p.d.f.s. Though technically possible, such rotations are avoided here as they obscure the physical interpretation of the observable set. A consequence of this on the accuracy of the morphed p.d.f. to model changing correlations between observables is discussed in Section 4.2. The processes of vertical and horizontal morphing (i.e. summing and translating) and of scaling the input morphed p.d.f.s are illustrated in Fig. 1a, which morphs between two normal distributions. The technique proposed also accurately models the evolution of rapidly changing distributions as illustrated in Fig. 1b. In the sample, the application of moment morphing is used to describe the non-linear transition of a Cauchy distribution via a Crystal Ball line shape into a normal distribution. The parameters of the used p.d.f.s are chosen such that the positions of their means as well as their shapes vary substantially as a function of the morphing parameter α, in particular in the tails of the distributions which change dynamically along the morphing path. 5

6 2.2 Interpolation with multiple morphing parameters morphing parameter Projection of p.d.f.s st 1 input template (mapped to m=0) nd 2 input template (mapped to m=1) Vertical morphing interpolation (at m=0.5) MomentMorph interpolation (at m=0.5) (accounting only for shifted mean) observable Events / ( 0.4 x 1 ) morphing parameter observable (a) (b) Figure 1: Two examples of moment morphing. (a) Construction of the morphed p.d.f. The interpolation is done between two normal distributions, shown as solid lines, corresponding to values 0 and 1 of the morphing parameter. After vertical morphing, the mean of the templates is shifted to the common value and their widths are adjusted accordingly. The dashed p.d.f. shows the morphed p.d.f., which is a linear combination of the modified inputs. (b) Non-linear morphing of a Cauchy distribution (m = 0) via a Crystal Ball line shape (m = 5) into a normal distribution (m = ). Fig. 2 shows an application of the technique described in this article to a complex physics and detector simulation. The reconstructed invariant mass distribution for a Standard Model Higgs boson with mass 125 GeV decaying to four leptons is described by non-linearly interpolating between a series of templates corresponding to simulation response estimates for four assumed Higgs boson masses, at 123, 124, 126 and 127 GeV. As reference templates kernel estimation p.d.f.s modeling events simulated with MadGraph [3] and an ATLAS-type PGS [4] simulation are used. Despite rapidly evolving features, the template predicted by the morphing technique reproduces the true template at 125 GeV. The resulting morphed p.d.f. is parametrized in terms of the true Higgs boson mass, as opposed to the reconstructed invariant mass, and a fit of the morphed p.d.f. to a dataset with an assumed true Higgs boson of 125 GeV directly and accurately measures that true Higgs boson mass. 2.2 Interpolation with multiple morphing parameters The non-linear implementation of multiple morphing parameters is analogous to the single parameter case of Section 2.1. We illustrate here the expansion to two parameters, including correlated effects in the template distribution caused by changing of two or more morphing parameters simultaneously, before generalizing the approach to an arbitrary number of parameters. The general Taylor series expansion of f(x m 1, m 2 ), depending on parameters m 1 and m 2, around 6

7 2.2 Interpolation with multiple morphing parameters Events/0.06 GeV 1400 Simulated data Reference template 1200 Prediction (m =125 GeV) 4l Fitted template (m = l 00 True template GeV) [GeV] m 4l Figure 2: Example of p.d.f. interpolation. The reconstructed invariant mass distribution of for a 125 GeV Standard Model Higgs boson decaying to four leptons (solid, red) has been predicted by non-linearly interpolating between the reference distributions for surrounding mass hypotheses (solid, green), as described in the text. The prediction is compared to the true 125 GeV template (dashed, yellow) derived from the simulated data for this hypothesis and a fit of the morphed p.d.f. to this dataset (solid, blue). The predicted, true, and fitted curves match almost perfectly. (m, m 20 ) reads: f(x m 1, m 2 ) = f(x m, m 20 ) + 1 [ ( m 1 ) f(x m, m 20 ) + ( m 2 ) ] f(x m, m 20 ) 1! m 1 m [( m 1 ) ! m 2 f(x m, m 20 ) + 2( m 1 )( m 2 ) f(x m, m 20 ) 1 m 1 m 2 ] + ( m 2 ) 2 2 m 2 f(x m, m 20 ) +..., 2 (19) which, for a 2 2 square grid with reference points surrounding (m 1, m 2 ), is approximated by: f(x m 1, m 2 ) = f(x m, m 20 ) + ( m 1 ) f(x m, m 20 ) + ( m 2 ) f(x m, m 20 ) m 1 m ( m 1 )( m 2 ) f(x m, m 20 ). m 1 m 2 (20) Here the middle two terms are the linear expansions along m 1 and m 2, and the last term represents the expansion along both m 1 and m 2 simultaneously. The addition of the fourth corner point allows one to model functions that cannot be factorized into functions depending only on m 1 or m 2. Repeating the Taylor series expansion for a grid of k l reference points, and again writing the truncated series as a vectorial equation, leads to Eqn. 2 with additional mixed terms in the 7

8 2.2 Interpolation with multiple morphing parameters transformation matrix M and the derivatives f j. The transformation matrix M now reads ( m 2 ) ( m 2 ) k ( m 1 ) ( m 1 )( m 2 ) k ( m 1 ) l ( m 2 ) k M =......, 1 ( m 2n0 ) ( m 2n0 ) k ( m 1n0 ) ( m 1n0 )( m 2n0 ) k ( m 1n0 ) l ( m 2n0 ) k (21) where m in0 = m in m i0 is a short notation for the distance between reference point n and reference point 0 in the ith dimension of the parameter space. The distance vector m is in multiple dimensions defined as m = ( 1 ( m 2l0 ) ( m 2l0 ) k ( m 1l0 ) ( m 1l0 )( m 2l0 ) k ( m 2l0 ) l ( m 2l0 ) k) T. (22) Following Eqn. 5, the coefficients c i for a new point (m 1, m 2 ) = (m 1 q, m 2q ) are now given by c i (m 1, m 2) = (k l) 1 j=0 ( M 1 ) ji ( m) j. (23) The construction of the morphed p.d.f. p(x m 1, m 2 ) using these coefficients is as in Eqn. 18. The extension to an arbitrary number of morphing parameters, n, is a matter of using an n- dimensional grid of input p.d.f.s, with k l... grid points, and consistently expanding the transformation matrix M with the additional higher order terms. Returning to the 2 2 square grid of input p.d.f.s with reference values surrounding (m 1, m 2 ), the coefficients in (m 1, m 2 ) reduce to c 00 (m 1, m 2 ) = (1 m 1,frac ) (1 m 2,frac ) (24) c (m 1, m 2 ) = m 1,frac (1 m 2,frac ) (25) c 01 (m 1, m 2 ) = (1 m 1,frac ) m 2,frac (26) c 11 (m 1, m 2 ) = m 1,frac m 2,frac, (27) with m 1,frac = (m 1 m 0 )/(m 1 m 0 ) and m 1,frac = (m 2 m 200 )/(m 201 m 200 ). Note that the coefficients are all positive when staying within the boundaries of the grid, and add up to 1. Towards the corner point (m 111, m 211 ), the nominal coefficient c 00 and linear-expansion coefficients c and c 01 are turned off, and the quadratic term m 1,frac m 2,frac in c 11 is turned on in full, describing the change caused by changing m 1 and m 2 simultaneously. Shown in Fig. 3 is an illustration of a multivariate normal distribution modeling the dependency between two observables. The covariance matrix as well as the mean change as a function of two parameters α 1 and α 2. In the given example the distributions are known for 2 2 grid points fulfilling α i = {0, 1}, i = 1, 2. The information is used to linearly interpolate the p.d.f. to any desired point in the two-dimensional parameter space. The yellow contours in the center of the grid, i.e. at (α 1, α 2 ) = (0.5, 0.5), represent the true template, which is compared with the prediction by the moment morphed p.d.f. (red) and the prediction by vertical morphing only (green). Similar to the example presented in Fig. 1a, vertical morphing only does not yield a central template with shifted mean. The difference between the moment morphed template and 8

9 2.3 Other choices of basis functions morphing parameter 1 observable Input templates True template Moment morphing Vertical morphing 68% CL 95% CL morphing parameter observable 1 Figure 3: Example of multi-dimensional moment morphing. Linear interpolation between multivariate normal distributions in a multi-dimensional parameter space. For illustration purposes only, the plot has a 1-to-1 mapping between the corners of the morphing parameter and observable space. The example is constructed such that mean and covariance of the multivariate normal distributions follow an analytic description, in particular the variation of Pearson s correlation coefficient can be described bi-linearly. The corner contours (blue) of the grid represent the input templates to the morphing algorithm. Dashed and solid contours indicate one and two standard deviations from the mean respectively. The true template in the center of the grid, i.e. at (0.5, 0.5), is shown in yellow. The prediction of the morphed p.d.f. for the central point is shown in red. The green contours represent the prediction from vertical interpolation. the true template is induced by the change of the correlation between the two observables with varying morphing parameter values. In case of constant correlation between the observables of the input templates, the prediction of the moment morphed p.d.f. is exact. More discussion on this follows in Section Other choices of basis functions Finally, having derived the morphed p.d.f. using a Taylor series expansion, the dependency on a morphing parameter m can be easily re-expressed in any orthonormal basis {Ψ 0,..., Ψ n 1 } = {Ψ i } i=0,...,n 1. The choice of the basis functions can simplify the studied problem and thus lead to a better approximation, even to an exact description. For example, if the dependency on model parameters is expected to be periodic, it can be expressed in Fourier space as a linear combination of the eigenvectors Ψ i : n 1 p(x m) = d i (x)ψ i (m), (28) where the coefficients d i read d i (x) = i=0 p(x m)ψ i (m)dm. (29) 9

10 3 A p.d.f. for modeling systematic uncertainties Thus, the dependency of the morphed p.d.f. on the observables enters through the coefficients d i (x) only, while the basis functions are independent of x. Note that the number of sampling points limits the number of utilizable basis functions Ψ i (m). Analogously to Eqn. 5, p can then be predicted at any new point m, p pred (x m ) = n 1 i,j=0 with the modified n n transformation matrix M ij = Ψ j (m i ). Ψ j (m ) ( M 1) ji p(x m i), (30) 3 A p.d.f. for modeling systematic uncertainties In a typical analysis performed in a particle physics experiment, the impact of systematic uncertainties is typically quantified by varying one-by-one the model parameters relating to detector modeling and physics models (e.g. energy calibrations, factorization scales) and to record the template distribution with these modified settings. The resulting pairs of alternate templates, corresponding to up and down variations of the uncertain model parameters, must then be incorporated in the likelihood model of the physics analysis, in the form of a model nuisance parameter that causes the template distribution to deform as prescribed by the pair of alternate templates. For the most widely-used statistical test at the LHC, the profile likelihood ratio, it is required that the modeling of the systematic uncertainty in terms of a nuisance parameters is done in a continuous way. In particular, the maximization of the likelihood function, as needed (twice) in the profile likelihood ratio, requires continuous and smooth parametric models to describe the signal and background processes present in the data 2. This section builds a parametrized p.d.f. describing a set of systematic variations about the nominal prediction for a signal or background process, using the morphing technique of Section 2. Each systematic uncertainty i is described with a nuisance parameter, η i, that continuously morphs between the variation and nominal templates such that η i = ±1 corresponds to the ±1σ (1σ = 1 standard deviation) variations, and η i = 0 corresponds to the nominal template 3. Additional variation templates may be added for different values of η i. 2 Specifically, the likelihood for a physics measurement, L(µ, η), where µ is the physics parameter of interest and η are the nuisance parameters that parametrize the impact of systematic uncertainties on the signal and background predictions, must be defined for all values of µ and η. The profile likelihood ratio is given by λ(µ) = L(µ, ˆη) L(ˆµ, ˆη), (31) where ˆµ and ˆη represent the unconditional maximum likelihood estimates of µ and η, and ˆη represents the conditional maximum likelihood estimate for the chosen value of µ. Therefore λ is a function of µ. (Note that the data is omitted in the short hand notation of L.) For a more detailed discussion on the profile likelihood ratio test statistic, see Ref. [5]. 3 A Gaussian constraint is applied separately for each systematic uncertainty to account for uncertainty in the external measurement. This constraint preferably centers each systematic variation around the nominal prediction, with a reduced likelihood for potential shifts; however, a combined fit to the observed data may of course prefer shifts in the nuisance parameters.

11 4 Accuracy of moment morphing and comparison to alternative morphing algorithms The response of the likelihood function to changes in the nuisance parameters is here assumed to be factorized, i.e. it is assumed that the effect of a simultaneous changes in two or more nuisance parameters can be described as a superposition of the effects of changing each nuisance parameter individually. (Unfactorizable uncertainties are not discussed here; their treatment is handled following the recipe of Section 2.2.) Where n unfactorizable uncertainties would require a full n-dimensional grid of input templates, this assumption reduces the number of required inputs to a set of n one-dimensional variations, with only the nominal template in common a star shape in the nuisance parameter space. The construction of the morphed p.d.f. as the sum of the input templates becomes p pred (x η) = (1 c ij (η i )) p(x, 0) + c ij (η i ) p ij (x η i = j), (32) i=1 j=±1,±m i=1 j=±1,±m where the double-sum runs over the implemented systematic uncertainties and their available ±1... ± m σ variations. The morphed p.d.f. is self-normalized, as by construction all coefficients add up to one. In the publicly available implementation, detailed in Section 5.1, the coefficients c ij (η i ) are linear, and depend only the two closest input points surrounding η i, as in Eqn. 9. Also, the linear transformation of the observables x x, responsible for scaling and horizontal morphing, can be turned on or off. (By default it is off.) Henceforth this p.d.f. is called the star-morphed p.d.f. This type of star-morphed p.d.f. has been used in the analysis of Higgs decay to 4 leptons (H ZZ 4l) by the ATLAS experiment [6] to describe the various background components contributing the ZZ mass spectrum, each including the variations of all relevant systematic uncertainties. Fig. 4 shows an example application of the star-morphed p.d.f., used to describe the dominant background process for a typical H 4l analysis, labeled q q ZZ. As in Section 2.1, this background prediction has been obtained from events simulated with MadGraph [3] and PGS [4]. For illustration, three non-physical systematic uncertainties have been added to the star-morphed p.d.f. One is a shape variation in the low mass region, the second is a shape variation in the high mass region, and the third is an overall normalization uncertainty that affects the entire mass region. The effects of these systematic uncertainties on the template distribution are shown in Fig Accuracy of moment morphing and comparison to alternative morphing algorithms The accuracy of a moment morphing function f pred (x α) is assessed here using a series of benchmark models for which the true distribution f true (x α) is defined. A priori one can demonstrate that moment morphing provides an exact solution for any set of true distributions in which the first and second moment (i.e. the mean and width) of f true (x α) change linearly with the morphing parameter α and all other moments are constant (i.e. the shape of the distribution does not change other than through its first and second moment). The simplest 11

12 4 Accuracy of moment morphing and comparison to alternative morphing algorithms Events/0.5 GeV Fitted template Total uncertainty Flat uncertainty component Low uncertainty component High uncertainty component [GeV] m 4l Figure 4: Example application of the star-morphed p.d.f. to describe the effect of different uncertainties. Three example uncertainties affecting different regions of the shown spectrum have been added to the starmorphed p.d.f. The black curve shows the fitted template for the q q ZZ process along with the total uncertainty derived from all included sources indicated by the yellow band. The green, red and blue curves show the partial uncertainties corresponding to a flat, low mass and high mass effect, respectively. example of such a distribution is a Gaussian distribution with a linearly changing mean and width. However, the class of true distributions that are exactly reproduced by moment morphing is not restricted to this example: any distribution with linearly changing first and second moments and fixed higher-order moments is exactly described. In particular, when considering use cases in particle physics, the physics parameters of interest (particle masses, resonance widths, etc.) are often related to the first and second moments; hence a correct description of these is most critical. Deviations from the exact solution can occur in two ways: a) the dependence of the first and second moment of f true (x α) become non-linear in α, or b) higher moments of the shape of the distribution depend on α, either linearly or non-linearly. The magnitude of both types of deviations are under explicit control of the user, as the moment morphing algorithm allows templates to be specified at any number of freely choosable values of α. Deviations of the first type are generally unproblematic as it is always possible to construct a configuration of templates in which: a) the dependence of the first and second moments between any pair of adjacent templates is sufficiently close to linear (when choosing the piece-wise linear interpolation option), or b) where the dependence of the moments over the entire domain of α is well approximated by a n th -order polynomial for n templates (when choosing the non-linear interpolation option). For this reason, no attempt is made here to quantify the accuracy of moment morphing due to (local) non-linearity of the first and second moment of f true (x α). Deviations of the second type, introduced by changing higher-order moments, may be more limiting in cases where higher-order moments of the true distribution change rapidly as a function of α. It should be noted that changes in the truth distribution related to higher-order moments are empirically accounted for in the morphing algorithm, by gradually changing the weight of momentadjusted input templates as a function of α. However, the morphed distribution does not guarantee 12

13 4.1 Performance on benchmark models Name Model description Dependence on morphing parameters N µ Gaussian(x µ, σ = 1) µ = 2 α N σ Gaussian(x µ = 0, σ) σ = α N µσ Gaussian(x µ, σ) µ = 2 α σ = α Γ k GammaDist(x k, θ = 1) k = α Γ θ GammaDist(x k = 3, θ) θ = α Γ kθ GammaDist(x k, θ) k = α θ = α C 1 Chebychev(x a 1, a 2 = 0) a 1 = α C 2 Chebychev(x a 1 = 0.5, a 2 ) a 2 = 0.4 α C 12 Chebychev(x a 1, a 2 ) a 2 = α a 2 = 0.4 α Table 1: Benchmark models used to quantify accuracy of moment morphing and alternative morphing algorithms. a linear change in these higher-order moments as a function of α, as is done for the first and second moments. The impact of this empirical modelling of changes in higher-order moments is quantified in the following sub-sections. 4.1 Performance on benchmark models The accuracy of moment morphing is evaluated here on nine analytical benchmark models that are similar to various particle physics use cases, and the accuracy is compared to two other morphing algorithms that have historically been used in particle physics: vertical morphing and integral morphing. In the vertical morphing approach 4 templates are interpolated with a simple weighting strategy: V (x α) = (1 α)/2 T L (x) + (α 1)/2 T R (x) where T L/R are the left and right template models corresponding to α = ±1. The vertical morphing approach is widely used in LHC physics analyses, notably in the modeling of distributions in the discovery analysis of the Higgs boson. In contrast, in the integral morphing approach [1] interpolation occurs between the cumulative distribution functions of the templates T L/R. The integral morphing approach is more suited to models with rapidly shifting means, like moment morphing and unlike vertical morphing, but is computationally intensive due to (numeric) integration and root-finding steps, and is restricted to the description of one-dimensional distributions. The integral morphing approach has been used, among others, in physics analyses published by the D0 and CDF collaborations. The nine analytical benchmark models tested here are detailed in Table 1. For the first three benchmark models N µ, N σ and N µσ, based on the normal distribution, moment morphing provides an exact solution, and these models are included in the benchmark to facilitate accuracy performance 4 Note that the moment morphing algorithm reduces to vertical morphing strategy if the adjustment step of the first and second moments of the input templates is omitted. 13

14 4.1 Performance on benchmark models Moment morphing Vertical morphing Integral morphing Name KS(α = 0) KS max KS(α = 0) KS max KS(α = 0) KS max N µ N σ N µσ Γ k Γ θ Γ kθ C C C Table 2: Accuracy of the moment morphing, vertical morphing and integral morphing algorithms on the nine benchmark models define in Table 1, expressed in the KS distance at α = 0 and the largest KS distance that occurs in the range 1 < α < 1. comparisons with the other morphing approaches. The second set of benchmark models is based on the Gamma distribution: Γ(x k, θ) = xk 1 e x/θ θ k, Γ(k) and are included as an example of a distribution where also higher-order moments of the true distribution changes as a function of the morphing parameter. The last set of three benchmark models is based on the 2 nd -order Chebychev polynomials: C(x a 1, a 2 ) = 1 + a 1 x + a 2 (2x 2 1), as a distribution that is representative of typical background distributions. Given a pair of input templates positioned at α = ±1, the accuracy of the morphing methods is quantified using the Kolmogorov-Smirnov (KS) distance. This is the largest distance between the cumulative distribution functions of the truth model and the cumulative distribution function of the morphing interpolation model for any value of x, as a function of the morphing parameter α. The KS distance values range between zero (perfect agreement between the two distributions) and one (complete spatial separation of the distributions in x). Fig. 5 illustrates the accuracy of the three morphing approaches on the N µσ, Γ kθ and C 1 models, along with the KS statistic as a function of α for all morphing approaches. Accuracy metrics for all nine benchmark models are given in Table 2, which summarizes for each benchmark model and each morphing strategy the KS distance at α = 0 (the morphing mid-point), and the worst KS distance that occurs in the morphing range α [ 1, 1]. As expected, the moment morphing approach exactly replicates the N µσ benchmark model, unlike the vertical morphing approach. The integral morphing algorithm can also exactly replicate the N µσ model, but is limited to an accuracy of about due to numerical precision limitations, related to integration and root-finding in the algorithm s implementation. For the Γ kθ benchmark model, no morphing algorithm can exactly reproduce the truth model due to changing higherorder moments, but the moment and integral morphing algorithms perform about equally, and 14

15 4.1 Performance on benchmark models Projection of p.d.f.s Projection of p.d.f.s Projection of p.d.f.s Input templates True templates Moment morphing Integral morphing Vertical morphing Input templates True templates observable Moment morphing Integral morphing Vertical morphing Input templates True templates observable Moment morphing Integral morphing Vertical morphing observable KS distance KS distance KS distance morphing parameter morphing parameter morphing parameter Figure 5: Accuracy of the moment morphing, vertical morphing and integral morphing algorithms on three of the benchmark models defined in Table 1. Top row: the N µσ model, center row: the Γ k model, bottom row: the C 1 model. The plots in the left column overlay the truth model distributions for various values of the morphing parameter α (α = ±1 shown in dark blue, which also serve as templates for the morphing algorithms, other values of α shown in yellow dashed) on the predicted distribution by moment morphing (red), vertical morphing (green) and integral morphing (light blue). To quantify the level of agreement between truth and morphing predictions, the right column shows the KS distance between the truth model and predicted distributions as a function of the morphing parameter α by the three types of morphing models, using the same colour codings as the left column plots. Some of the morphing predictions overlap (almost) perfectly, making them difficult to distinguish visually. They are discussed further in the text. 15

16 4.2 Accuracy for multi-dimensional distributions and multi-parameter morphing significantly better than the vertical morphing algorithm. For the C 1 benchmark model both the moment morphing and vertical morphing perform identically, as the first and second moment of the true distribution do not depend on α, hence both approaches reduce to the same algorithm, which happens to describe the C 1 model perfectly 5. In summary, in these tests the moment morphing approach is as good as and usually better than the two alternative morphing methods. 4.2 Accuracy for multi-dimensional distributions and multi-parameter morphing For morphing models with multiple morphing parameters, and/or with input distributions with multiple observables, it is challenging to capture the accuracy with a limited set of benchmark models as KS tests have no simple equivalent for multidimensional distributions, and the number of benchmark permutations to test becomes large with multiple morphing parameters. Nevertheless, one can illustrate the challenges of morphing and the performance of moment morphing with a single simple yet challenging 2-parameter benchmark model for 2-dimensional distributions: a multivariate normal distribution: f true (x α, β) = Gaussian(x µ, V), for which all five degrees of freedom, µ x, µ y, σ x V xx, σ y V yy and ρ V xy / V xx V yy depend linearly on two morphing parameters α, β. In the case where the correlation coefficient ρ is independent of α, β, moment morphing provides the exact solution for this multi-variate Gaussian model, similar to the one-dimensional case, even if the dependence of µ x, µ y, σ x, σ y is strong, as long as it is linear. Conversely, vertical morphing will perform poorly, especially if µ x or µ y depend strongly on α, β, while integral morphing is not available for multi-dimensional distributions. The more challenge scenario where also ρ depends on α, β is visualized in Fig. 3. Moment morphing will not provide an exact solution for this class of models as the covariance moments are not explicitly corrected for, nevertheless the covariance of the interpolated shape at (α, β) = (0.5, 0.5) reasonably matches the covariance of the true model, although the shape is no longer perfectly Gaussian. 5 Implementation This section discusses the publicly available morph classes, and is followed by details of the chosen extrapolation approach of a morph parameter beyond the provided input range. 5 This perfect performance is specific to the C 1 benchmark that changes the first Chebychev coefficient only. Truth models that change high-order coefficients are imperfectly described my all morphing algorithms, as detailed in Table 2. 16

17 5.1 Available morph classes 5.1 Available morph classes Moment morphing has been implemented in C ++ for the RooFit toolkit [7]. As of ROOT release , the morphing features described in this document are available in the RooFit models library. Common to all classes is the ability to handle one or multiple observables, as well as the implementation of a cache that stores pre-calculated expensive components such as numerically computed moments, e.g. the means and widths of each input template, required for the translation of the corresponding observables. The following moment morph classes are available in RooFit. The RooMomentMorph p.d.f. can be used to interpolate between an arbitrary number of reference distributions using a single morphing parameter. The algorithms settings described in this paper, e.g. linear or truly non-linear, can be used. Furthermore, a sine-linear variant transforms m frac to sin(π/2 m frac ) before calculating the coefficients, thus ensuring a continuous and differentiable transition when crossing between two adjacent sets of enclosing grid points. In addition, non-linear coefficients for adjusting the moments of the p.d.f.s can be mixed with linear coefficients when constructing the morph p.d.f., and an option is available to select positive non-linear coefficients only. RooStarMomentMorph is the natural extension for combining multiple one-dimensional RooMomentMorph p.d.f.s with one common sampling point. The class supports linear and sinelinear interpolation. The transformation of the template observables can be turned on or off. RooMomentMorphND allows the parametrization of a n-dimensional parameter space, interpolating linearly or sine-linearly between reference points sitting on a hyper-cube or arbitrary size. Roo1DMomentMorphFunction and Roo2DMomentMorphFunction are similar to the top moment morph p.d.f., but can be used to interpolate between functions, not p.d.f.s. Available for one or two morph parameters. Example code of how to use the moment morph p.d.f. is given below. Listing 1: Sample code to build the model shown in Fig. 1a. // This example b u i l d s two normal d i s t r i b u t i o n s and uses moment morphing // to i n t e r p o l a t e between the templates using RooFit. using namespace RooFit ; // Create a p e r s i s t a b l e c o n t a i n e r f o r RooFit p r o j e c t s, a l l o w i n g to use // a s i m p l i f i e d s c r i p t i n g language to b u i l d the p. d. f. s needed in t h i s // example. RooWorkspace w( w, 1 ) ; // Build two normal d i s t r i b u t i o n s, corresponding to d i f f e r e n t v a l u e s in 17

18 5.2 Extrapolation beyond input boundaries // the morph parameter space. They share the same observable, but have // o t h e r w i s e d i f f e r e n t moments, i. e. mean and width. w. f a c t o r y ( RooGaussian : : gaussian1 ( obs [ 0, ], 5 0, 1 5 ) ) ; w. f a c t o r y ( RooGaussian : : gaussian2 ( obs, 3 0 0, 4 0 ) ) ; // Build a RooMomentMorph p. d. f. which i n t e r p o l a t e s between the normal // d i s t r i b u t i o n s c r e a t e d b e f o r e. The i n t e r p o l a t i o n i s parametrized by // the parameter alpha and the r e f e r e n c e templates map to alpha=0 and // alpha=1 r e s p e c t i v e l y. w. f a c t o r y ( RooMomentMorph : : morphpdf ( alpha [ 0, 1 ], obs, { gaussian1, gaussian2 }, { 0, 1 } ) ) ; // Set the morphing parameter alpha e x p l i c i t l y to w : : alpha >setval ( 0. 5 ) ; // Create a frame to draw the p. d. f. s from b e f o r e and show the input // templates as s o l i d blue curves and the moment morph pdf at alpha =0.5 // in dashed red. RooPlot frame = w : : obs >frame ( ) ; w : : gaussian1 >ploton ( frame, LineColor ( kblue ), L i n e S t y l e ( k S o l i d ) ) ; w : : gaussian2 >ploton ( frame, LineColor ( kblue ), L i n e S t y l e ( k S o l i d ) ) ; w : : morphpdf >ploton ( frame, LineColor ( kred ), L i n e S t y l e ( kdashed ) ) ; frame >Draw ( ) ; 5.2 Extrapolation beyond input boundaries By construction, the validity of the morph p.d.f. of Eqn. 18 is highest when interpolating its morph parameter(s) within the provided range(s) of input values. Beyond these the predictive power is a priori unknown, but of course can be interesting to investigate. Some ad hoc choices need to be made for extrapolation cases, which were not covered by the description of the algorithm so far. As apparent from Eqn. 6, when extrapolating m beyond the input boundary values m min and m max, the coefficients c i (m) increase in size and may become highly negative. In this situation the morph p.d.f. can become smaller than zero, and as such ill-defined. To prevent this, the following extrapolation approach is implemented. Whenever one of the morph parameters extends beyond the input range, i.e. m < m min or m > m max, the coefficient multiplied with the nearest input p.d.f. is forced to one, and all other coefficients are set to zero. The same is done for the transformed width of Eqn. 11, which is to remain greater than zero. Beyond the input boundaries, the transformed mean of Eqn. 11 does remain well-defined, and uses the coefficients of either Eqn. 6 or Eqn. 9, depending on the linearity setting used. 18

19 5.3 Computational performance of moment morphing Evaluation time [ms] 0.12 Non-linear interpolation Linear interpolation Number of reference templates Figure 6: Benchmark results for RooMomentMorph. The figure shows the average CPU time for the evaluation of the morph p.d.f. as a function of the reference templates used for the interpolation for linear (red) and non-linear (blue) algorithms settings. This was measured on an Intel(R) Xeon(R) CPU 2. GHz with 2 GB of memory per core. As reference p.d.f.s normal distributions with varying mean are used. 5.3 Computational performance of moment morphing Template morphing is one of the computationally limiting factors in current HEP analyses, which makes understanding the performance of the algorithm at hand crucial. Fig. 6 compares the performance of the linear and non-linear algorithms settings in terms of average CPU time needed for the evaluation of the morph p.d.f. as a function of the used reference templates. The benchmark makes use of the caching described in section 5.1. It excludes the computation time needed for the calculation of the moment integrals over the input templates, which is a one-time calculation, the result of which can be cached in the RooFit workspace file along with the model if desired by the user. Details of the setup are described in the caption of the figure. For the interpretation of performance numbers it should be noted that the evaluation time for non-linear morphing models depends quadratically on the number of reference templates n, as for every template the associated coefficient is a product of n distances. In the linear case, the evaluation time is driven by the efficiency of the algorithm finding the two closest reference points surrounding the point at which the p.d.f. is evaluated. For (20) reference templates, the ratio of non-linear over linear evaluation time is 1.4 (2.2). 6 Conclusion We have described a new algorithm to interpolate between Monte Carlo sample distributions, called moment morphing. The proposed technique is based on a linear combination of the input templates, has a non-linear dependence on the parameters that control the morphing interpolation, and can be constructed in any orthogonal basis. 19

20 REFERENCES REFERENCES Compared with existing methods, it allows for both binned histogram and continuous templates, has both horizontal and vertical morphing, and is not restricted in the number of input templates, observables, or model parameters. In particular, the latter feature allows the moment morphing technique to model the impact of a non-factorizable response between different model parameters. The moment-morphed p.d.f. is self-normalized, and therefore, in terms of speed and stability, scales well with the number of input templates used. This is a useful feature in modern day particle physics where large numbers of templates are often required to model the variations by systematic uncertainties. In a suite of comparison tests the moment morphing method proofs to be as accurate as, or better than, two commonly used alternative techniques. Various implementations of the moment morphing technique are publicly available through the RooFit toolkit. Acknowledgments This work was supported by CERN, Switzerland; FOM and NWO, the Netherlands; and STFC, United Kingdom. References [1] A. L. Read, Linear interpolation of histograms, Nucl.Instrum.Meth. A425 (1999) [2] K. Cranmer, G. Lewis, L. Moneta, A. Shibata, W. Verkerke, HistFactory: A tool for creating statistical models for use with RooFit and RooStats, CERN-OPEN [3] J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, et al., The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations, arxiv: [4] J. Conway, et al., conway/research/software/pgs/pgs4- general.htm. [5] G. Cowan, K. Cranmer, E. Gross, O. Vitells, Asymptotic formulae for likelihood-based tests of new physics, Eur.Phys.J. C71 (2011) arxiv: [6] G. Aad, et al., Measurements of Higgs boson production and couplings in the four-lepton channel in pp collisions at center-of-mass energies of 7 and 8 TeV with the ATLAS detector, arxiv: [7] W. Verkerke, D. P. Kirkby, The RooFit toolkit for data modeling, econf C (2003) MOLT007. arxiv:physics/