Global Sensitivity Analysis - PDF Free Download

Global Sensitivity Analysis Elmar Plischke Institut für Endlagerforschung TU Clausthal Risk Institute Easter School, Liverpool, April 217 IELF, TU Clausthal GSA 1

Contents Variance-based Sensitivity Analysis Sobol Method FAST: Motivation FAST: Implementation Variants of FAST General Framework for Sensitivity Measures Moment-Independent Importance Measures Transformation Invariance Given Data Estimation IELF, TU Clausthal GSA 2

Global Sensivity Analysis Global Sensitivity Analysis: Identify sources of variation in the output S y=g(x) Input factors System model Output statistics Uncertainty in inputs: random variables X i, i = 1,..., k Deterministic simulator producing scalar output y (as RV: Y) (Q)MC sampling for propagating uncertainties IELF, TU Clausthal GSA 3

Functional ANOVA decomposition Any multivariate integrable mapping g can be decomposed as follows: g(x) = g + n i=1 g i (x i ) + j>i g i,j (x i, x j ) + + g 1,2,...,n (x 1, x 2,..., x n ) if the input probability distribution is independent, F X (x) = i F i(x i ); g = g(x)df X (x) g i (x i ) = g(x) df k (x k ) g k i g i,j (x i, x j ) = g(x) df k (x k ) g i (x i ) g j (x j ) g k i,j IELF, TU Clausthal GSA 4

Functional ANOVA decomposition A strong annihilating condition holds: If l {i 1,..., i k } then g i1,...,i k (x i1,..., x ik )df l (x l ) =. IELF, TU Clausthal GSA 5

Functional ANOVA decomposition A strong annihilating condition holds: If l {i 1,..., i k } then g i1,...,i k (x i1,..., x ik )df l (x l ) =. This implies orthogonality of the g α family. The output variance can be decomposed into V[Y] = α V α where V α = (g α (x α )) 2 df α (x α ) ( with multiindex α). IELF, TU Clausthal GSA 5

Functional ANOVA decomposition S i = V {i} V[Y] S T i = S sub α = S sup α = D eff i α Vα V[Y] β α V β V[Y] β α V β V[Y] Sobol main/first order effect, correlation ratio total effect subset importance superset importance mean = k i=1 α: α =i α V α = k i=1 S T i mean effective dimensionality [Liu and Owen, 26] IELF, TU Clausthal GSA 6

Variance-Based Sensitivity Analysis First order effects: Factor prioritization, model structure (sum of all first order effects) Total effects: Factor fixing Sobol method or extended Fourier Amplitude Sensitivity Test (efast) need special sampling schemes (e.g. Sobol ): IELF, TU Clausthal GSA 7

Sobol Method 1. Generate a (quasi-)random sample with N points in 2k dimensions. 2. Split the matrix so that the first k columns are denoted matrix A, and the remaining k columns are denoted B. 3. For a given input parameter i, construct a matrix A i B which consists of all the columns of A, except the ith column, which is taken from B. 4. Now an estimation of V i, the numerator of S i, is given by ˆV i = 1 N g(b)t (g(a i B ) g(a)) 5. An estimation of S T i is given by Jansen s formula 1 2NˆV[Y] (g(ai B ) g(a))t (g(a i B ) g(a)) which is shown to be numerically more stable than 1 1 NˆV[Y] g(a)t (g(a i B ) g(b)) IELF, TU Clausthal GSA 8

y y y y y y y y Sobol Method Sobol indices are constructed from multiple OAT designs! Total costs: N(1 + k) for Jansen, N(2 + k) for Sobol /Ishigami/Homma/Saltelli approach Sobol method uses correlation coefficients between the output y A or y B and y i. Ishigami: main effect Ishigami: main effect Ishigami: main effect Ishigami: main effect 15 15 15 15 1 1 1 1 5 5 5 5-5 -5-5 -5-1 -1-5 5 1152 y_{1} -1-1-5 5 1152 y_{2} -1-1 -5 5 1 15 y_{3} -1-1 -5 5 1 15 y_{4} Ishigami: total effect Ishigami: total effect Ishigami: total effect Ishigami: total effect 15 15 15 15 1 1 1 1 5 5 5 5-5 -5-5 -5-1 -1-5 5 1152 y_{1} -1-1-5 5 1152 y_{2} -1-1 -5 5 1 15 y_{3} -1-1 -5 5 1 15 y_{4} For main effect: Large correlation corresponds to large sensitivity, for total effect: Small correlation corresponds to large sensitivity IELF, TU Clausthal GSA 9

Random vs. Quasi-Random Sampling Sampling for the Sobol method can be derived from 1. a random (pseudo-random, if algorithmically generated) source; or 2. a quasi-random source, like Sobol LPτ or Halton low-discrepancy series. IELF, TU Clausthal GSA 1

1.8.6.4.2.2.4.6.8 1 1.8.6.4.2.2.4.6.8 1 Random vs. Quasi-Random Sampling Sampling for the Sobol method can be derived from 1. a random (pseudo-random, if algorithmically generated) source; or 2. a quasi-random source, like Sobol LPτ or Halton low-discrepancy series. ( ) ( ) The MC error is of order o 1 log N while the QMC error is of order o k N N. IELF, TU Clausthal GSA 1

Fourier Amplitude Sensitivity Test FAST introduced in [Cukier et al., 1973, Schaibly and Shuler, 1973, Cukier et al., 1975, Cukier et al., 1978] Artificial timeframe: Each input gets assigned its unique frequency The output is scanned for resonances Attributing the resonances to the the frequencies gives the contribution to output variance IELF, TU Clausthal GSA 11

Motivation Ingredients 1. Ergodicity 2. Search Curves 3. Superposition Principle 4. Parseval s Theorem IELF, TU Clausthal GSA 12

Ergodic Theorem Ergodic Theorem [Arnold and Avez, 1968] Let Φ t be a dynamical system on (T, M, µ). If Φ t is ergodic then space average equals time averages: for f L 1 (M, µ) and almost all x M it holds that M 1 T f(x)dµ(x) = lim f(φ t x )dt. T T Φ t mixing & measure-preserving Trajectory visits every point IELF, TU Clausthal GSA 13

Example for Ergodic Systems: Toroidal Shifts ẋ 1 = x 1 mod 1 ẋ 2 = ωx 2 mod 1, with ω Q: Space-filling trajectory along a torus 1 t > ( 1 /sqrt(2) t, 1 /sqrt(3) t) mod 1.9.8.7 x 2.6.5 3 4.4.3.2 1.5.5 2 1 1.1 1 2 4 2 3.2.4.6.8 1 x 2 IELF, TU Clausthal 1 4 GSA 14 4

Unfortunately... Space-filling only for infinite time Wrap-around introduces discontinuities: Periodicity needed f(x, ) = f(x, 1), f(, y) = f(1, y)? IELF, TU Clausthal GSA 15

Unfortunately... Space-filling only for infinite time Wrap-around introduces discontinuities: Periodicity needed f(x, ) = f(x, 1), f(, y) = f(1, y)? The way out: Search Curves Reflexion instead of wrap-around Closed curves IELF, TU Clausthal GSA 15

Search curve 1 t > ( 5/11 t, 5/9 t) mod 1 1 With Reflexion 1 Power Spectrum.9.9.9 x 1 x 2.8.8.8.7.7.7.6.6.6 x 2.5 x 2.5.5.4.4.4.3.3.3.2.2.2.1.1.1.2.4.6.8 1 x 1.2.4.6.8 1 x 1 5 1 15 2 Frequency Different input factors/dimensions can be identified by different frequencies! IELF, TU Clausthal GSA 16

Superposition Principle Product of harmonic functions = sum of harmonic functions 2 cos α cos β = cos(α + β) + cos(α β) 2 sin α cos β = sin(α + β) + sin(α β) 2 sin α sin β = cos(α β) cos(α + β) Powers: Multiples of the frequencies (higher harmonics) Interactions: Resonances in sums and diffs of the frequencies 1.5.5 1 2 4 6 Multiplication 1.5.5 2 1.5 1.5.5 1 1.5 1.5.5 1 2 4 6 Addition 1.5.5 1 2 4 6 2 2 4 6 1 2 4 6 IELF, TU Clausthal GSA 17

Power Spectrum The power spectrum gives the portion of a signal s power falling within given frequency bins MathWorld. Additive decomposition of the signal s energy. Variance is the signal s energy! IELF, TU Clausthal GSA 18

Parseval s Theorem Variance is invariant under orthonormal transformations: V[Y] = V[FY] where F is the Fourier transformation. Identifying the contributions from input parameters and interactions to the output Functional ANOVA decomposition Power spectrum gives first- and higher order effects IELF, TU Clausthal GSA 19

Putting Things Together Choose maximal harmonic M as interference factor Assign frequencies ω i. Sample size: Shannon sampling theorem requires n > 2M k i=1 ω i (Nyquist frequency) Sample (u j,i ) from multi-dimensional search curve Apply a transformation using inverse cdfs, x j,i = F 1 i (u j,i ) Evaluate model y j = f(x j,1,..., x j,k ) Apply a Fast Fourier Transform (FFT) to (y j ) yielding complex Fourier coefficients c m Collect the resonances from the power spectrum for first order effects: M m=1 S i = 2 cmω i m 2. cm 2 IELF, TU Clausthal GSA 2

Input factors x 1 x 2 Model f Output y k x k Spectral analysis Output Variance V v k v 1 v 2 Resonances Fourier amplitude Sensitivity Index = v i i V Frequencies IELF, TU Clausthal GSA 21

Detail: Maximum Harmonic Normally, max. harmonic is M = 4 to 6. If the simulation model is continuous, the Fourier coefficients decay quadratically. More harmonics are needed if the function is discontinuous. 1 f1 2 Power Spectrum f1 1 f2 25 Power Spectrum f2.8 15.8 2.6.4 1.6.4 15 1.2 5 Very Flat.2 5 More Noise.5 1 1 2 3 4.5 1 1 2 3 4 IELF, TU Clausthal GSA 22

Detail: Frequency assignment The choice ω i = ω i 1 with ω = 2M + 1 allows to identify all effects uniquely upto harmonic M. More elaborate algorithms are available optimizing the use of the frequencies. IELF, TU Clausthal GSA 23

Detail: Sample Design Fill the unit hypercube along a search curve: u j,i = 1 π arccos(cos(2πω i(r i + j n ))), i = 1,..., k j = 1,..., n Here r i is an additional random shift IELF, TU Clausthal GSA 24

Detail: Fourier Transformation Written explictly, c m = n j=1 y j ζ n (j 1)m, m =,..., n 1 with complex unit root ζ n = e 2πi n The powers of the unit root can be cleverly reused, resulting in fast implementations. 1 1 1 F 3 = 1 1 2 ( 1 + 1 3i) 2 ( 1 1 1 1 1 3i) 1 1 2 ( 1 1 3i) 2 (1 +, F 4 = 1 1i 1 1i 1 1 1 1 3i) 1 1i 1 1i Not all coefficent are needed, the total variance can be computed the classical way (as sum of squares). IELF, TU Clausthal GSA 25

FAST with sample size 8192: Ishigami function Ishigami FAST 4 Inputs 2 2 4 1 2 3 4 5 Index Ishigami FAST 6 7 8 1 2 3 4 5 Index Power Spectrum of Output 6 7 8 Output 2 Variance 2.4.2 IELF, TU Clausthal 5 1 15 2 25 3 Frequency 35 4 45 5 GSA 26

Lower Plot: Explanations of Resonances Frequencies used: ω 1 = 1, ω 2 = 9, ω 3 = 81 (max. harmonic M = 4) Blue lines Main effects. First line: linear part Green lines Two-term interaction effects. Symmetry! Red lines Three-term interaction effects: Nothing visible. Active : x 1, x 3 1, x4 2, x 1x 2 3 x3 1 x2 3, x 1x 4 3 x3 1 x4 3 IELF, TU Clausthal GSA 27

FAST: A minimal MATLAB implementation % k, model(), trafo() provided M=4; freq=(2*m+1).^(:(k 1)); n=2*(2*m+1)^k; % Full %M=4; freq =[11,21,31];n=2*M*sum(freq); % Manual u=acos(cos(2*pi*linspace(1/2/n,1 1/2/n,n)'*freq))/pi; x=trafo(u);y=model(x); % Model evaluation spect=(abs(fft(y))).^2/n; V=sum(spect(2:n)); % Spectrum stem(2*spect(2:(floor(n/2)))/v); % Visualization Si=2*sum(spect(1+(1:M)'*freq))/V % Main effects IELF, TU Clausthal GSA 28

What about totals? With the above-mentioned frequency scheme, ω M k 1 l= ωl = 1 2 (ωk 1) can be uniquely decomposed: ω = k i=1 α i (ω)ω i 1, α i (ω) { M,..., 1,, 1,..., M}. If α i (ω) then ω contributes to the total effect of input factor i. Ŝ Ti = 2 α i (ω) c ω 2 α m c m 2, Ŝ Ti = 1 2 i (ω)= c ω 2 m c m 2 Higher order effects: Combining the zero patterns of the α i (ω) IELF, TU Clausthal GSA 29

Extended FAST (efast) [Saltelli et al., 1999]: Frequency selection scheme for first and total effects A factor i of interest is assigned to a relative large frequency ω i 1 and all others are assigned to low frequencies (say, ω j i = 1). Total effects: all frequencies below ω T = ω i M j i ω j do not contribute to the variance from factor i up to the Mth order. k (small) sample blocks are needed. m=ω Ŝ Ti = 1 2 T 1 c mω i 2 m c m 2 IELF, TU Clausthal GSA 3

Random Balance Design (RBD) [Tarantola et al., 26]: For first order effects Create a uniform sample u [, 1] n by sampling from u : s 1 2s 1 = 1 π arccos(cos(2πs)) Find permutations π j such that u j = π j (u) are uncorrelated. Transform the marginal distributions x j = F 1 j (u j ). Evaluate the model output y = f(x 1,..., x k ). Apply inverse permutations to the output, y j = π j (y). Transform the permuted output y j via DFT which yields cm j = n l=1 exp ( 2πi(l 1) m ) j n y l, m =, ±1,..., ± n 2. ( M Estimate the sensitivity Ŝj = 2 cmω j 2) ( m m=1 cm j 2) 1. IELF, TU Clausthal GSA 31

RBD: A minimal MATLAB implementation % M, n, k, model(), trafo() provided s=(2*(1:n)' (n+1))/n; u=acos( cos(pi*s))/pi; [,perm]=sort(rand(n,k)); % Random Permutation x=trafo(u(perm));y=model(x); ys=zeros(n,k);for i=1:k; ys(perm(:,i),i)=y; end spect=(abs(fft(ys))).^2/n; V=sum(spect(2:n,1)); Si=2*sum(spect(1+(1:M),:))/V IELF, TU Clausthal GSA 32

Effective Algorithm for Sensitivity Indices (EASI) [Plischke, 21]: Sort and shuffle the positions in the sample: First order effects Sort-of inverse RBD: Construct the permutations from the observations 1 Triangular shape via sorting.9.8 x i x (i) x [i].7.6.5.4.3.2.1 1 2 3 4 5 Index IELF, TU Clausthal GSA 33

EASI: A minimal MATLAB implementation % x,y,m provided [n,k]=size(x); [,index]=sort(x); odd=mod(n,2); shuffle=[1:2:(n 1+odd), (n odd): 2:2]; ys=y(index(shuffle,:)); % Rearrange output spect=(abs(fft(ys))).^2/n; V=sum(spect(2:n,1)); Si=2*sum(spect(1+(1:M),:))/V % Collect Resonances IELF, TU Clausthal GSA 34

Example EASI 2 Ishigami Fourier Trafo 2 Ishigami Fourier Trafo 15 15 1 1 y 5 y 5 5 5 1 4 2 2 4 x 1 1 4 2 2 4 x 2 2 Ishigami Fourier Trafo 2 Ishigami Fourier Trafo 15 15 1 1 y 5 y 5 5 5 1 4 2 2 4 x 3 1 4 2 2 4 x 4 Two regression curves for even and odd indices IELF, TU Clausthal GSA 35

Second order effects (and higher if you dare) Given data triplets {(x 1 i, x2 i, y i)} (as realizations of some RVs), compute the joint influence of X 1 and X 2 on Y Method in a nutshell Sort the (x 1, x 2 ) data along a search curve ( nearest neighbor) with a distinct frequency behaviour. Reorder output accordingly Look out for resonances. IELF, TU Clausthal GSA 36

Search Curve: Plow Track Code the (x 1, x 2 ) position by the length of the search curve Curve has detectable frequency behaviour per dimension IELF, TU Clausthal GSA 37

Search Curve: Ping-Pong Alternative curve with large freedom of choosing the frequencies IELF, TU Clausthal GSA 38

Indexed Inputs Ishigami Plow track 4 x 2 1 x 3 2 4 1 2 3 4 5 6 7 8 9 1 Hyperindex Ishigami Plow track 2 Output 1 Fraction of Variance 1 1 2 3 4 5 6 7 8 9 1 Hyperindex Power Spectrum of Output.2.1 1 2 3 4 5 6 7 8 9 Frequency Parameter 1 3 1,3 η 2.2864.163.3154 IELF, TU Clausthal GSA 39

5 Ishigami Ping Pong Curve Inputs (sorted) x 1 x 3 Variance Contribution 5 1 2 3 4 5 6 7 8 9 1 Index Ishigami Ping Pong Curve Output (reordered) 2 1 1 1 2 3 4 5 6 7 8 9 1 Index Ishigami Ping Pong Curve Power Spectrum of Output.2.1 2 4 6 8 1 12 14 Frequency Parameter 1 3 1,3 η 2.2424.161.2327 IELF, TU Clausthal GSA 4

What if Variance is not a suitable Measure of Uncertainty? Use of suitable output transformations But: interpretation of the results on the original scale is difficult IELF, TU Clausthal GSA 41

What if Variance is not a suitable Measure of Uncertainty? Use of suitable output transformations But: interpretation of the results on the original scale is difficult Instead of variance-based, use moment-independent indicators which consider the whole distribution instead of single moments. IELF, TU Clausthal GSA 41

General Frameworks for Sensitivity and Importance Measures Comparing the joint distribution with the product of the marginals If d(, ) is a 2-dimensional functional distance/divergence: d((x i, Y), X i Y) Copula-Based Approaches, Discrepancy, Tests of statistical independence IELF, TU Clausthal GSA 42

General Frameworks for Sensitivity and Importance Measures Average of comparing the input distribution with the conditional input distibutions If d(, ) is a 1-dimensional functional distance/divergence: Reliability / Regionalized Sensitivity E[d(X, X Y i )] IELF, TU Clausthal GSA 42

General Frameworks for Sensitivity and Importance Measures Average of comparing the input distribution with the conditional input distibutions If d(, ) is a 1-dimensional functional distance/divergence: E[d(X, X Y i )] Reliability / Regionalized Sensitivity Mostly of interest for extreme-valued output IELF, TU Clausthal GSA 42

General Frameworks for Sensitivity and Importance Measures Average of comparing the output distribution with the conditional output distibutions If d(, ) is a 1-dimensional functional distance/divergence: Importance measures E[d(Y, Y X i )] IELF, TU Clausthal GSA 42

General Frameworks for Sensitivity and Importance Measures Average of comparing the output distribution with the conditional output distibutions If d(, ) is a 1-dimensional functional distance/divergence: E[d(Y, Y X i )] Importance measures Rest of talk focusses on these moment-independent importance measures. IELF, TU Clausthal GSA 42

Visual impressions: Non-functional dependence Letter P.9.8.7.6.5.4.3.2.1.1.1.2.3.4.5.6.7.8 IELF, TU Clausthal GSA 43

Visual impressions: Non-functional dependence 1 Linear Regression.9.8.7.6.5.4.3.2 R 2 =.121.1.1.1.2.3.4.5.6.7.8.9 IELF, TU Clausthal GSA 43

Visual impressions: Non-functional dependence 1 Nonlinear Regression.9.8.7.6.5.4.3.2 η 2 =.154.1.1.1.2.3.4.5.6.7.8 IELF, TU Clausthal GSA 43

Visual impressions: Non-functional dependence Product of Marginals.9.8.7.6.5.4.3.2 D * =.96.1.1.1.2.3.4.5.6.7.8 IELF, TU Clausthal GSA 43

Visual impressions: Non-functional dependence Conditioning on y.4.3.2.1 1.8.2.4.4.6.6.2 IELF, TU Clausthal GSA 43.8

Visual impressions: Non-functional dependence.45 Conditioning on y.4.35.3.25.2 delta(y,x)=.327.15.1.5.1.2.3.4.5.6.7.8 IELF, TU Clausthal GSA 43

Visual impressions: Non-functional dependence Conditioning on x.4.3.2.1.8.2.4 IELF, TU Clausthal GSA 43.6.8.2.4.6

Visual impressions: Non-functional dependence.45 Conditioning on x.4.35.3 delta(x,y)=.338.25.2.15.1.5.1.2.3.4.5.6.7.8.9 IELF, TU Clausthal GSA 43

Moment-Independent Importance Measures ζ i = E[d(Y, Y X i )] d(, ): Shift or separation function (functional metric) Bayesian Interpretation: Degree of belief before and after getting to know that X i = x i, averaged over all possible X i IELF, TU Clausthal GSA 44

Examples for Shift/Separation Measures ζ EI (µ Y, µ Y X=x ) = max{µ Y X=x, } max{µ Y, } EVPI, null alternative ζ SI (µ Y, µ Y X=x ) = σ 2 Y (µ Y µ Y X=x ) 2 Main Effect ζ KS (F Y, F Y X=x ) = sup FY F Y X=x Kolmogorov-Smirnov ζ Ku (F Y, F Y X=x ) = sup ( F Y F Y X=x ) inf ( FY F Y X=x ) Kuiper ζ CvM (F Y, F Y X=x ) = 1 2 (FY X=x (y) F Y (y) ) 2 dy Cramér, L 2 (cdf) fy X=x ζ Bo (f Y, f Y X=x ) = 2 1 (y) f Y (y) dy Borgonovo, L 1 (pdf) ζ KL (f Y, f Y X=x ) = f Y X=x (y) log f Y X=x(y) dy f Y (y) Kullback-Leibler ζ He (f Y, f Y X=x ) = 1 f Y (y) f Y X=x (y)dy Hellinger IELF, TU Clausthal GSA 45

Which separation to use? Looking for sensitivity importance measures which are Simple to interpret Easy to estimate Invariant under monotonic transformations of inputs and outputs Detecting strong functional links: Y = g(x) = E[ζ(Y, Y X)] = 1 Offer a test for independence: E[ζ(Y, Y X)] = Y and X are independent No one size fits all sensitivity method IELF, TU Clausthal GSA 46

Moment-Independent Importance Measures II For moment-independent importance, separation measures are between 1. Cumulative Distribution Functions 2. Probabilistic Density Functions 3. Characteristic Functions IELF, TU Clausthal GSA 47

CDF-based Measures Kolmogorov-Smirnov and Kuiper separation 1 Cumulative Distributions.8.6.4.2 4 3 2 1 1 2 3 4 KS: largest distance Kuiper: max. positive distance minus min. negative distance IELF, TU Clausthal GSA 48

PDF-based Measures Borgonovo separation: (signed) area under the curves Kullback-Leibler: Entropy.4 Densities.35.3.25.2.15.1.5 4 3 2 1 1 2 3 4 IELF, TU Clausthal GSA 49

CF-based Measure CF: φ X (s) = E[e isx ] = e isx f X (x)dx Inverse Fourier transform of pdf: Complex-valued, no finite support Distance Covariance [Székely and Rizzo, 213]: φ dcov 2 X,Y (s, t) φ X (s)φ Y (t) 2 (X, Y) = C R 2 s 2 t 2 dsdt Parseval s Theorem: Sampling-based estimators are available. IELF, TU Clausthal GSA 5

Properties of MIM For Variance-Based Sensitivity Measures: log transformation of the output switches from additive (ANOVA) decomposition to multiplicative decompositions, other transformations are also available (Box Cox, probit,logit). Wanted: A Sensitivity Measure that is invariant with respect to transformations (Sensitivity then becomes topological property). [Borgonovo et al., 214] The sensitivity measure ξ is transformation invariant if the separation is given by ζ(p, Q) = sup A A h ( P(A) Q(A) ) (generalized Birnbaum Orlicz) ζ(f Y, f Z ) = ) H f Y (y)dy (Csiszár divergence) ( fz (y) f Y (y) IELF, TU Clausthal GSA 51

Given Data Methodology X Y k-dimensional random vector random variable (quantity of interest for time series) IELF, TU Clausthal GSA 52

Given Data Methodology X Y k-dimensional random vector random variable (quantity of interest for time series) Physical observations Uncertainty propagation through model Y = g(x) Simple random sampling of X Latin Hypercube sampling of X Quasi Monte Carlo sampling (Sobol LPτ,... ) of X But not fast multidimensional/sparse grid quadrature designs IELF, TU Clausthal GSA 52

Examples for 2D Uniform [, 1] Input Samples x 2 x 2 Simple Random Sample 1.8.6.4.2.5 1 x 1 Uniform Design 1.8.6.4.2.5 1 x 1 x 2 x 2 Latin Hypercube Sample 1.8.6.4.2.5 1 x 1 Full Factorial Design 1.8.6.4.2.5 1 x 1 x 2 x 2 Quasi Monte Carlo Sample 1.8.6.4.2.5 1 x 1 Sparse Grid Design 1.8.6.4.2.5 1 x 1 Red: Bad setup. But fine for a meta-modeling layer Worse space-filling properties in higher dimensions IELF, TU Clausthal GSA 53

Going beyond Linear Regression? [Pearson, 1912]: Nothing can be learnt of association by assuming linearity in a case with a regression line (plane, etc.) like A, much in a case like B. A sensitivity measure has always to be interpreted with respect to the used method Report the goodness-of-fit for the method R 2 for linear regression R 2 for rank linear regression n i S i for variance-based first order effects Sum of variance-based first order and higher order effects Successively use more advanced techniques IELF, TU Clausthal GSA 54

F Back to the Roots Correlation Ratios [Pearson, 195]: piecewise constant regression model for local means E[Y X i = x i ] Histogram binning: Estimate local cdfs/pdfs for use with separation measures 1.5 78 76 74 72 7 68 Y 66 135 13 64 125 62 12 115 6 X 1 IELF, TU Clausthal GSA 55

Thank You! Questions, Comments mailto:elmar.plischke@tu-clausthal.de Preprints, Scripts, Stuff http://www.immr.tu-clausthal.de/~epl/ IELF, TU Clausthal GSA 56

References I Arnold, V. I. and Avez, A. (1968). Ergodic Problems of Classical Mechanics. Benjamin, New York. Borgonovo, E., Tarantola, S., Plischke, E., and Morris, M. D. (214). Transformations and invariance in the sensitivity analysis of computer experiments. Journal of the Royal Statistical Society, Series B, 76:925 947. Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., and Schaibly, J. H. (1973). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I. Theory. J. Chem. Phys., 59:3873 3878. Cukier, R. I., Levine, H. B., and Shuler, K. E. (1978). Nonlinear sensitivity analysis of multiparameter model systems. J. Comput. Phys., 26(1):1 42. Cukier, R. I., Schaibly, J. H., and Shuler, K. E. (1975). Study of the sensitivity of coupled reaction systems to uncertainties in rate cofficients. III. Analysis of the approximations. J. Chem. Phys., 63:114 1149. Liu, R. and Owen, A. B. (26). Estimating mean dimensionality of analysis of variance decompositions. Journal of the American Statistical Association, 11(474):712 721. IELF, TU Clausthal GSA 57

References II Pearson, K. (195). On the General Theory of Skew Correlation and Non-linear Regression, volume XIV of Mathematical Contributions to the Theory of Evolution, Drapers Company Research Memoirs. Dulau & Co., London. Pearson, K. (1912). On the general theory of the influence of selection on correlation and variation. Biometrika, 8(3 4):437 443. Plischke, E. (21). An effective algorithm for computing global sensitivity indices (EASI). Reliability Engineering&System Safety, 95(4):354 36. Saltelli, A., Tarantola, S., and Chan, K. (1999). A quantitative, model independent method for global sensitivity analysis of model output. Technometrics, 41:39 56. Schaibly, J. H. and Shuler, K. E. (1973). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. II. Applications. J. Chem. Phys., 59:3879 3888. IELF, TU Clausthal GSA 58

References III Székely, G. J. and Rizzo, M. L. (213). Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143:1249 1272. Tarantola, S., Gatelli, D., and Mara, T. A. (26). Random balance designs for the estimation of first order global sensitivity indices. Reliability Engineering&System Safety, 91:717 727. IELF, TU Clausthal GSA 59