Derivative-based global sensitivity measures for interactions

Size: px

Start display at page:

Download "Derivative-based global sensitivity measures for interactions"

Elvin Adams
6 years ago
Views:

1 Derivative-based global sensitivity measures for interactions Olivier Roustant École Nationale Supérieure des Mines de Saint-Étienne Joint work with J. Fruth, B. Iooss and S. Kuhnt SAMO 13 Olivier Roustant (EMSE) DGSM for interactions SAMO 13 1 / 16

2 Outline 1 Background and motivation Variance-based and derivative-based sensitivity measures 1st-order analysis. Screening : Total indices & DGSM. 2nd-order analysis. Interaction screening. Crossed DGSM 2 The main result : A link between superset importance and crossed DGSM 3 Applications A 6-dimensional example When the gradient is supplied Olivier Roustant (EMSE) DGSM for interactions SAMO 13 2 / 16

3 Background and motivation Variance-based and derivative-based sensitivity measures Let X = (X 1,..., X d ) be a vector of independent input variables with distribution µ 1 µ d, and g : R d R such that g(x) L 2 (µ). Sobol-Hoeffding decomposition [Sobol, 1993, Efron and Stein, 1981] d g(x) = g 0 + g i (X i ) + g (X i, X j ) + + g 1,...,d (X 1,..., X d ) = i=1 I {1,...,d} g I (X I ) 1 i<j d (1) The g I s are centered and orthogonal. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 3 / 16

4 Background and motivation Variance-based and derivative-based sensitivity measures Global sensitivity measures Variance-based measures Partial variances : D I = var(g I (X I )), and Sobol indices S I = D I /D D := var(g(x)) = I D I, 1 = I S I Olivier Roustant (EMSE) DGSM for interactions SAMO 13 4 / 16

5 Background and motivation Variance-based and derivative-based sensitivity measures Global sensitivity measures Variance-based measures Partial variances : D I = var(g I (X I )), and Sobol indices S I = D I /D D := var(g(x)) = I D I, 1 = I S I Total index : D T i = J {i} D J, S T i = DT i D. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 4 / 16

6 Background and motivation Variance-based and derivative-based sensitivity measures Global sensitivity measures Variance-based measures Partial variances : D I = var(g I (X I )), and Sobol indices S I = D I /D D := var(g(x)) = I D I, 1 = I S I Total index : Di T = J {i} D J, Si T = DT i D. Superset importance [Liu and Owen, 2006] : D super I := J I D J, S super I = Dsuper I D D super := J {} D J "total interaction index" [Fruth et al., 2013] Olivier Roustant (EMSE) DGSM for interactions SAMO 13 4 / 16

7 Background and motivation Variance-based and derivative-based sensitivity measures Global sensitivity measures Variance-based measures Partial variances : D I = var(g I (X I )), and Sobol indices S I = D I /D D := var(g(x)) = I D I, 1 = I S I Total index : Di T = J {i} D J, Si T = DT i D. Superset importance [Liu and Owen, 2006] : D super I := J I D J, S super I = Dsuper I D D super := J {} D J "total interaction index" [Fruth et al., 2013] Derivative-based measures In sensitivity analysis [Sobol and Gresham, 1995] : ν i = ( g(x) x i ) 2 dµ(x), called DGSM Olivier Roustant (EMSE) DGSM for interactions SAMO 13 4 / 16

8 Background and motivation Variance-based and derivative-based sensitivity measures Global sensitivity measures Variance-based measures Partial variances : D I = var(g I (X I )), and Sobol indices S I = D I /D D := var(g(x)) = I D I, 1 = I S I Total index : Di T = J {i} D J, Si T = DT i D. Superset importance [Liu and Owen, 2006] : D super I := J I D J, S super I = Dsuper I D D super := J {} D J "total interaction index" [Fruth et al., 2013] Derivative-based measures In sensitivity analysis [Sobol and Gresham, 1995] : ν i = ( g(x) x i ) 2 dµ(x), called DGSM In statistical learning [Friedman and Popescu, 2008] : ν = ( 2 g(x) x i x j ) 2 dµ(x), νi = ( I g(x) x I ) 2 dµ(x). "crossed DGSM". Olivier Roustant (EMSE) DGSM for interactions SAMO 13 4 / 16

9 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. First-order analysis considers single variables. One aim is screening : detection of non-influential input variables. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 5 / 16

10 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. First-order analysis considers single variables. One aim is screening : detection of non-influential input variables. Screening with total indices or DGSMs If either D T i = 0 or ν i = 0, than X i is non influential. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 5 / 16

11 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. First-order analysis considers single variables. One aim is screening : detection of non-influential input variables. Screening with total indices or DGSMs If either Di T = 0 or ν i = 0, than X i is non influential. There is a Poincaré-type inequality between total indices and DGSMs D i D T i C(µ i )ν i Proved by [Sobol and Kucherenko, 2009] for the uniform and normal distributions, [Lamboni et al., 2013] for the general case. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 5 / 16

12 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. Poincaré inequality (1-dimensional case) A distribution µ satisfies a Poincaré inequality if for all h in L 2 (µ) such that h(x)dµ(x) = 0, and h (x) L 2 (µ) : h(x) 2 dµ(x) C(µ) h (x) 2 dµ(x) The best constant is denoted C opt (µ). Olivier Roustant (EMSE) DGSM for interactions SAMO 13 6 / 16

13 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. Poincaré inequality (1-dimensional case) A distribution µ satisfies a Poincaré inequality if for all h in L 2 (µ) such that h(x)dµ(x) = 0, and h (x) L 2 (µ) : h(x) 2 dµ(x) C(µ) h (x) 2 dµ(x) The best constant is denoted C opt (µ). Examples ([Sobol and Kucherenko, 2009], [Ané et al., 2000], [Lamboni et al., 2013], [Bobkov and Houdré, 1997, Bobkov, 1999]) Distribution C opt (µ) A case of equality Uniform U[a, b] (b a) 2 /π 2 g(x) = cos Normal N (µ, σ 2 ) σ 2 g(x) = x µ ( π(x a) b a ) Olivier Roustant (EMSE) DGSM for interactions SAMO 13 6 / 16

14 Background and motivation 1st-order analysis. Screening : Total indices & DGSM. Poincaré inequality (1-dimensional case) A distribution µ satisfies a Poincaré inequality if for all h in L 2 (µ) such that h(x)dµ(x) = 0, and h (x) L 2 (µ) : h(x) 2 dµ(x) C(µ) h (x) 2 dµ(x) The best constant is denoted C opt (µ). Examples ([Sobol and Kucherenko, 2009], [Ané et al., 2000], [Lamboni et al., 2013], [Bobkov and Houdré, 1997, Bobkov, 1999]) Distribution C opt (µ) A case of equality Uniform U[a, b] (b a) 2 /π 2 g(x) = cos Normal N (µ, σ 2 ) σ 2 g(x) = x µ Properties of µ ( π(x a) b a A Poincaré constant C(µ) [ ] 2 Continuous 4 sup x R ) min(f(x),1 F(x)) f (x) log-concave 1/f (m) 2 ( log-concave, truncated on [a, b] (F(b) F(a)) 2 /f q ( F (a)+f(b) 2 Olivier Roustant (EMSE) DGSM for interactions SAMO 13 6 / 16 )) 2

15 Background and motivation 2nd-order analysis. Interaction screening. Crossed DGSM Second-order analysis considers pairs of variables. It allows interaction screening : To detect {X i, X j } that do not interact together (D super = 0). Olivier Roustant (EMSE) DGSM for interactions SAMO 13 7 / 16

16 Background and motivation 2nd-order analysis. Interaction screening. Crossed DGSM Second-order analysis considers pairs of variables. It allows interaction screening : To detect {X i, X j } that do not interact together (D super = 0). 2nd-order analysis and additive structures If either ν = 0 or D super = 0, then g can be written as a sum of two functions, one that does not depend on x i, the other that does not depend on x j [Hooker, 2004, Friedman and Popescu, 2008] : g(x) = g i (x i ) + g j (x j ) Olivier Roustant (EMSE) DGSM for interactions SAMO 13 7 / 16

17 Background and motivation 2nd-order analysis. Interaction screening. Crossed DGSM Second-order analysis considers pairs of variables. It allows interaction screening : To detect {X i, X j } that do not interact together (D super = 0). 2nd-order analysis and additive structures If either ν = 0 or D super = 0, then g can be written as a sum of two functions, one that does not depend on x i, the other that does not depend on x j [Hooker, 2004, Friedman and Popescu, 2008] : g(x) = g i (x i ) + g j (x j ) [Hooker, 2004] uses it in machine learning [Muehlenstaedt et al., 2012] use it in computer experiments (see after). Olivier Roustant (EMSE) DGSM for interactions SAMO 13 7 / 16

18 Background and motivation 2nd-order analysis. Interaction screening. Crossed DGSM An application of 2nd-order analysis Here, the estimated interaction structure is used to define a suitable covariance kernel for the Ishigami function f (x 1, x 2, x 3 ) = sin(x 1 ) + 7 sin(x 2 ) x 4 3 sin(x 1 ) FIGURE: Left : Kriging with standard kernel ; Middle : Kriging with a block-additive structure estimated from the FANOVA graph (right). See [Muehlenstaedt et al., 2012] for more details. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 8 / 16

19 The main result : A link between superset importance and crossed DGSM Theorem - A link between superset importance and crossed DGSM Assume that all µ i (i = 1,..., d) satisfy a Poincaré inequality. Then for all pairs {i, j} (1 i, j n), D D super C(µ i )C(µ j )ν. and C opt (µ i )C opt (µ j ) is the best constant. Generalizes to more than pairs. Olivier Roustant (EMSE) DGSM for interactions SAMO 13 9 / 16

20 The main result : A link between superset importance and crossed DGSM Theorem - A link between superset importance and crossed DGSM Assume that all µ i (i = 1,..., d) satisfy a Poincaré inequality. Then for all pairs {i, j} (1 i, j n), D D super C(µ i )C(µ j )ν. and C opt (µ i )C opt (µ j ) is the best constant. Generalizes to more than pairs. Sketch of Proof. Denote g super (x) := J {} g J(x J ). Then : Olivier Roustant (EMSE) DGSM for interactions SAMO 13 9 / 16

21 The main result : A link between superset importance and crossed DGSM Theorem - A link between superset importance and crossed DGSM Assume that all µ i (i = 1,..., d) satisfy a Poincaré inequality. Then for all pairs {i, j} (1 i, j n), D D super C(µ i )C(µ j )ν. and C opt (µ i )C opt (µ j ) is the best constant. Generalizes to more than pairs. Sketch of Proof. Denote g super (x) := J {} g J(x J ). Then : 1 D super = var(g super (x)) = ( 2 g super (x)) dµ(x) Olivier Roustant (EMSE) DGSM for interactions SAMO 13 9 / 16

22 The main result : A link between superset importance and crossed DGSM Theorem - A link between superset importance and crossed DGSM Assume that all µ i (i = 1,..., d) satisfy a Poincaré inequality. Then for all pairs {i, j} (1 i, j n), D D super C(µ i )C(µ j )ν. and C opt (µ i )C opt (µ j ) is the best constant. Generalizes to more than pairs. Sketch of Proof. Denote g super (x) := J {} g J(x J ). Then : 1 D super = var(g super (x)) = ( 2 g super (x)) dµ(x) 2 ν = ( ) 2 2 g(x) ( ) 2 x i x j dµ(x) = 2 g super (x) dµ(x) x i x j Olivier Roustant (EMSE) DGSM for interactions SAMO 13 9 / 16

23 The main result : A link between superset importance and crossed DGSM Theorem - A link between superset importance and crossed DGSM Assume that all µ i (i = 1,..., d) satisfy a Poincaré inequality. Then for all pairs {i, j} (1 i, j n), D D super C(µ i )C(µ j )ν. and C opt (µ i )C opt (µ j ) is the best constant. Generalizes to more than pairs. Sketch of Proof. Denote g super (x) := J {} g J(x J ). Then : 1 D super = var(g super (x)) = ( 2 g super (x)) dµ(x) 2 ν = ( ) 2 2 g(x) ( ) 2 x i x j dµ(x) = 2 g super (x) dµ(x) x i x j Finally combine (by integrating) the two Poincaré inequalities : ( 2 g super (x)) ( g super ) 2 (x) dµi (x i ) C(µ i ) dµ i (x i ) x i ( g super x i (x) ) 2 ( dµ j (x j ) C(µ j ) x j g super x i ) 2 (x) dµ j (x j ) Olivier Roustant (EMSE) DGSM for interactions SAMO 13 9 / 16

24 Applications Applications In applications, we would base the results on the upper bounds : ST i U i := C(µ i ) ν i D U := C(µ i )C(µ j ) ν D S super Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

25 Applications Applications In applications, we would base the results on the upper bounds : ST i U i := C(µ i ) ν i D U := C(µ i )C(µ j ) ν D S super The estimation of Di T and D super can be done by MC (or QMC) from [Jansen, 1999] [Liu and Owen, 2006] : D T i = 1 2 [f (x) f (zi, x i )] 2 dµ(x)dµ i (z i ) D super = 1 [f (x) f (xi, z j, x ) 4 f (z i, x j, x ) + f (z i, z j, x ) ] 2 dµ(x)dµi (z i )dµ j (z j ) Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

26 Applications A 6-dimensional example A 6-dimensional example We consider the 6-dimensional function in L 2 ([Muehlenstaedt et al., 2012]) : a(x 1,..., X 6 ) = cos([1, X 1, X 5, X 3 ]φ) + sin([1, X 4, X 2, X 6 ]γ) with φ = [ 0.8, 1.1, 1.1, 1] T, γ = [ 0.5, 0.9, 1, 1.1] T, and where X 1,..., X 6 are assumed i.i.d uniform on [ 1, 1]. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

27 Applications A 6-dimensional example A 6-dimensional example We consider the 6-dimensional function in L 2 ([Muehlenstaedt et al., 2012]) : a(x 1,..., X 6 ) = cos([1, X 1, X 5, X 3 ]φ) + sin([1, X 4, X 2, X 6 ]γ) with φ = [ 0.8, 1.1, 1.1, 1] T, γ = [ 0.5, 0.9, 1, 1.1] T, and where X 1,..., X 6 are assumed i.i.d uniform on [ 1, 1]. First-order analysis Input S i Si T Ŝi T sd U i Û i sd X (0.012) (0.007) X (0.009) (0.005) X (0.01) (0.006) X (0.008) (0.004) X (0.011) (0.007) X (0.011) (0.007) Screening does not discard any inputs here. Ranking is different (cf. [Sobol and Kucherenko, 2009]) Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

28 Applications A 6-dimensional example Second-order analysis Inputs pair S S super Ŝ super sd U Û sd X 1 : X (0) 0 0 (0) X 1 : X (0.005) (0.003) X 1 : X (0) 0 0 (0) X 1 : X (0.006) (0.004) X 1 : X (0) 0 0 (0) X 2 : X (0) 0 0 (0) X 2 : X (0.004) (0.002) X 2 : X (0) 0 0 (0) X 2 : X (0.005) (0.003) X 3 : X (0) 0 0 (0) X 3 : X (0.005) (0.003) X 3 : X (0) 0 0 (0) X 4 : X (0) 0 0 (0) X 4 : X (0.004) (0.002) X 5 : X (0) 0 0 (0) Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

29 Applications A 6-dimensional example Visualization of the second-order analysis FIGURE: FANOVA graphs of Ŝsuper (left) and Û (right). Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

30 Applications When the gradient is supplied An advantageous situation : When the gradient is supplied Suppose that one run gives both f (x) and f (x) = ( f (x) x i )1 i d. Then crossed DGSM estimation requires d/2 fewer evaluations : Function only Gradient supplied Total indices (d + 1)N no change DGSMs (d + 1)N N superset importance crossed DGSMs (d d(d 1) 2 )N no change (d d(d 1) 2 )N (d + 1)N TABLE: Computational cost for Monte Carlo estimation. N is the sample size. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

31 Applications When the gradient is supplied Convergence study when the gradient is supplied Example for the 6-dimensional function a x1x3 x1x5 x2x Number of model evaluations Number of model evaluations Number of model evaluations x2x6 x3x5 x4x Number of model evaluations Number of model evaluations Number of model evaluations FIGURE: Blue : U (upper bounds for crossed DGSM) ; Red : S super. Dotted : True value ; Solid : MC estimates. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

32 Conclusion Conclusion This work is about 2nd-order analysis, which considers pairs of inputs. There is a Poincaré-type inequality between superset importance (total interaction index) & crossed DGSM : D ( D super 2 ) 2 g(x) C(µ i )C(µ j ) dµ(x) x i x j Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

33 Conclusion Conclusion This work is about 2nd-order analysis, which considers pairs of inputs. There is a Poincaré-type inequality between superset importance (total interaction index) & crossed DGSM : D ( D super 2 ) 2 g(x) C(µ i )C(µ j ) dµ(x) x i x j Crossed DGSM can be used for interaction screening detection of additive structures. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

34 Conclusion Conclusion This work is about 2nd-order analysis, which considers pairs of inputs. There is a Poincaré-type inequality between superset importance (total interaction index) & crossed DGSM : D ( D super 2 ) 2 g(x) C(µ i )C(µ j ) dµ(x) x i x j Crossed DGSM can be used for interaction screening detection of additive structures. Crossed DGSM are especially useful when the gradient is supplied. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

35 Conclusion Conclusion This work is about 2nd-order analysis, which considers pairs of inputs. There is a Poincaré-type inequality between superset importance (total interaction index) & crossed DGSM : D ( D super 2 ) 2 g(x) C(µ i )C(µ j ) dµ(x) x i x j Crossed DGSM can be used for interaction screening detection of additive structures. Crossed DGSM are especially useful when the gradient is supplied. Limitations : As for DGSM, crossed DGSM must NOT be used to rank interactions. They may be give poor results for functions with sharp variations, as well as when some of the C(µ i ) s are large. Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

36 References Ané, C., Blachère, S., Chafaï, D., Fougères, P., Gentil, I., Malrieu, F., Roberto, C., and Scheffer, G. (2000). Sur les inégalités de Sobolev logarithmiques, volume 10 of Panoramas et Synthèses. Société Mathématique de France, Paris. Bobkov, S. G. (1999). Isoperimetric and analytic inequalities for log-concave probability measures. The Annals of Probability, 27(4) : Bobkov, S. G. and Houdré, C. (1997). Isoperimetric constants for product probability measures. The Annals of Probability, 25(1) : Efron, B. and Stein, C. (1981). The jackknife estimate of variance. The Annals of Statistics, 9(3) : Friedman, J. and Popescu, B. (2008). Predictive Learning via Rule Ensembles. The Annals of Applied Statistics, 2(3) : Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

37 References Fruth, J., Roustant, O., and Kuhnt, S. (2013). Total interaction index : A variance-based sensitivity index for interaction screening. Submitted. Hooker, G. (2004). Discovering additive structure in black box functions. In Proceedings of KDD 2004, pages ACM DL. Jansen, M. (1999). Analysis of variance designs for model output. Computer Physics Communication, 117 : Lamboni, M., Iooss, B., Popelin, A.-L., and Gamboa, F. (2013). Derivative-based global sensitivity measures : General links with Sobol indices and numerical tests. Mathematics and Computers in Simulation, 87 : Liu, R. and Owen, A. (2006). Estimating mean dimensionality of analysis of variance decompositions. Journal of the American Statistical Association, 101(474) : Muehlenstaedt, T., Roustant, O., Carraro, L., and Kuhnt, S. (2012). Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

38 References Data-driven Kriging models based on FANOVA-decomposition. Statistics & Computing, 22 : Sobol, I. (1993). Sensitivity estimates for non linear mathematical models. Mathematical Modelling and Computational Experiments, 1 : Sobol, I. and Gresham, A. (1995). On an alternative global sensitivity estimators. In Proceedings of SAMO 1995, pages 40 42, Belgirate. Sobol, I. and Kucherenko, S. (2009). Derivative-based global sensitivity measures and the link with global sensitivity indices. Mathematics and Computers in Simulation, 79 : Olivier Roustant (EMSE) DGSM for interactions SAMO / 16

Poincaré inequalities revisited for dimension reduction

Poincaré inequalities revisited for dimension reduction Olivier Roustant*, Franck Barthe** and Bertrand Iooss**, *** * Mines Saint-Étienne ** Institut de Mathématiques de Toulouse *** EDF, Chatou March