Metamodeling with Gaussian processes

Transcription

1 Metamodelng wth Gaussan processes Bertrand Iooss EDF R&D /06/2014

2 Uncertantes n energy producton systems Many uncertantes for the energy producton and the safety due to: hazards (demand, weather, ), ncomplete system knowledge (ageng, physcs, ), nternal agressons (falures, ) external agressons (earthquake, ) In order to better understand, prove the safety and optmze ts ndustral processes, EDF R&D develops some physcal numercal smulaton codes Problems of uncertanty management n computer experments Summer school SAMO B. Iooss - 26/06/14 2

3 Example: Smulaton of thermal-hydraulc accdent Scenaro : Loss of prmary coolant accdent due to a large break n cold leg [ De Crecy et al ] Pressurzed water nuclear reactor K ~ nput random varables : geometry, materal propertes, envronmental condtons, Computer code Y =f () Tme cost ~ h - n ~ Interest output varable Y : Peak of claddng temperature Goals: numercal model exploraton, senstvty analyss, uncertanty analyss Source: CEA Summer school SAMO B. Iooss - 26/06/14 3

4 Man problems n global senstvty analyss P0a) f (.) s complex (nteractons, non monotonc, dscontnuous, ) Sobol (varance), entropy, dstrbuton-based dstances, P0b) f (.) s costly (nb of smulatons : n << 1000) screenng, metamodels P1) The dmenson of Y s large or Y have a functonal (temporal/spatal) nature functonal decompostons P2) The dmenson of s large (K >> 10) or have a functonal nature screenng, groups of nputs, usng the dervatves [ see Sergeï s lecture ] P3) The quantty of nterest s lnked to rare events (falure probablty, hgh quantle) [ see Lemaître et al., 2014 and Paul Lemaître PhD thess (2014) ] P4) The nputs are dependent [ see Therry s lecture ] P5) f (.) s stochastc [ see Marrel et al., 2012 ] Summer school SAMO B. Iooss - 26/06/14 4

5 Global senstvty analyss: Identfyng nfluent nputs The model : Y = f ( 1,, K ) Calculatons of all types of ndces (Sobol, dstrbuton-based, ) + man effects E(Y ) Complexty/regularty of model f Non monotonc Screenng Morrs Metamodel Varance decomposton Sobol ndces Monotonc + nteractons Monotonc wthout nteracton Lnear 1 st degree Super screenng 0 Desgn of experment Monte-Carlo samplng Rank regresson Lnear regresson K 2K 10K 1000K Summer school SAMO B. Iooss - 26/06/14 5 Number of model evaluatons

6 Metamodelng steps Desgn of experments: Ponts to perform smulatons Smulaton : Performng smulatons Metamodelng : Approxmaton of the computer code 1 Computer code 2 Dfferent type of metamodels: [ Smpson et al ] [ Storle & Helton 2008 ] Lnear regresson - Polynomals - Splnes Addtve models/gam Regresson trees - Neural networks Support Vector Machnes Chaos polynomals Krgng/Gaussan process Summer school SAMO B. Iooss - 26/06/14 6

7 Usng krgng metamodels to estmate Sobol ndces Man effects: S = Var[E( Y Var( Y ) )] ; Total effects: S T = 1 Var[E( Y Var( Y ) )] Many methods to estmate Sobol ndces: Samplng (Monte Carlo, quas-mc, spectral approaches), smoothng methods, metamodels, To reduce the cost (number of model evaluatons), the krgng metamodel s effcent and allows to propagate the metamodel error on S and S T estmates Summer school SAMO B. Iooss - 26/06/14 7

8 Gaussan process (krgng) metamodel Summer school SAMO B. Iooss - 26/06/14 8

9 Krgng metamodel Krgng [ Matheron 63 ] for computer codes reles on the dea to nterpolate the code outputs n dmenson K [ Sacks et al. 89 ] as a spatal cartography Krgng (or Gaussan process) s nterestng because: t nterpolates the outputs, t gves predctor assocated wth confdence bands Example n 1D : Theoretcal functon (K =1) : Y = f ( ) = sn( ) Smulaton of n = 7 computaton ponts Summer school SAMO B. Iooss - 26/06/14 [ Chevaler, ]

10 Spatal statstcs: krgng nterpolaton Lnear combnaton of N data: Y * (u) = N = 1 λ Y( u ) Krgng can take nto account the data confguraton, the dstance between data and target, the spatal correlatons and potental external nformaton u 7 u 1 u 5 u 6 u u 2 u 4 Y * (u) u 3 u 8 Probablstc framework Estmaton wthout bas: E [ Y * (u) Y (u) ] = 0 the mean of the errors s zero Estmaton Y * (u) s optmal: Var [ Y * (u) Y (u) ] s mnmal the dsperson of the errors s reduced Summer school SAMO B. Iooss - 26/06/14 10

11 Stochastc model for Y(x) The random feld Y (x), wth Y R and x R p, s characterzed by ts mean and ts covarance Y (x) s statonary of second order: 1. E[Y (x)] = m does not depend on x 2. Covarance: Cov[ ( x), Y( x+ h)] = Ε[ Y( x+ h) Y( x) ] E[ Y( x+ h) ] E[ Y( x) ] Y = C( h) does not depend on x C(h) varance Covarance functon h Summer school SAMO B. Iooss - 26/06/14 11

12 In practce varance C(h) N( h) = 1 [ ] 2 Y( x + h) Y( x ) C( h) = m Var[Y(x)] = C(0) Y(x) Y(x+h) Y(x) Y(x+2h) Nugget effect (measure error or mcrostructure) Covarance functon Range = maxmal dstance of correlaton = correlaton length h Summer school SAMO B. Iooss - 26/06/14 12

13 Examples of stochastc processes (Gaussan) 1D [ from: Marcotte ] 2D 3D [ from: Bag, 2003 ] Summer school SAMO B. Iooss - 26/06/14 13

14 Smple krgng (known mean) Y * (u) N = = 1 λ ( u ) Mn { E [ Y * (u) Y (u) ] 2 } λ multple lnear regresson by least squares Best Lnear Unbased Predctor (BLUP) [ Y( u ) m ] + m (m = known constant) Krgng weghts λ (u) for Y (u ) are obtaned by: N j = 1 λ j ( u ) C( u u j ) = C ( u u ) = 1K N System of N lnear equatons wth N unknowns whch have an unque soluton (for non sngular covarance matrx) Krgng varance (estmaton error): does not depend on the Y values 2 σ K ( u) = C(0) λ ( u) C( u => Vsualsaton of regons wth mprecse estmatons => Put new observaton ponts n these regons u) Summer school SAMO B. Iooss - 26/06/14 14 N = 1

15 Example : cartography of ar polluton Varogram 73 measures of benzene concentraton (Rouen, France) [ from: Bobba, Metlck & Roth, 2000 ] γ (h) = C(0) C(h) Krgng mean Krgng standard devaton Summer school SAMO B. Iooss - 26/06/14 15

16 Gaussan process metamodel (1/2) Idea: Computer code results are nterpolated wth the krgng technque Necessary hypothess: Gaussan process Stochastc process Z wth : Defnton: Y(x) = βf(x) + Z(x) Regresson stochastc part Parametrc choces: F : polynomal of degree 1 R : statonary => covarance functon Example: Gaussan covarance βf(x) = β + R(x, u) E[Z(x)] = 0 Cov(Z(x), Z(u)) = σ²r( R(x, u) where σ² s the varance and R the correlaton functon Z~N(0, σ²r) K 2 ( x - u) = exp θ x u = R = 1 Ansotropy: θ s are not equal (correlaton length of each nput varable) Summer school SAMO B. Iooss - 26/06/ K = 1 β x

17 Gaussan process metamodel (2/2) Jont dstrbuton : Gaussan process (GP) model : Y(x) = βf(x) + Z(x), x R p Learnng sample () of N smulatons : (,Y ) Condtonal GP metamodel : (1) ( N ) ( ) ( k ) ( x,..., x ), F = F( ), R = ( R( x, x ) k =, Y ~ N( βf, σ ² R ) Y ( x), ~ Y GP Mean : Yˆ( x) = Ε Covarance : Cov [ ] 1 Y ( x) = βf( x) + r( x R [ Y βf ] ), Y wth r( x) = (1) ( [ R( x, x),..., R( x x) ] N ), ( ) t 1 Y ( u), Y ( v) = σ ² ( R( u, v) + r( u) R r( v) ) Varance Mean Square Error (MSE), Y, Y Summer school SAMO B. Iooss - 26/06/14 17

18 Y Illustraton Y 2 code runs 3 code runs mean 95% confdence ntervals (from MSE) Y x * gaussan law 5 code runs Concluson: Gven a suffcent number of ponts, we obtan an accurate metamodel Summer school SAMO B. Iooss - 26/06/14 18

19 ln L Hyperparameters estmaton Maxmum lkelhood method Lkelhood maxmsaton on the learnng bass (( s,y s ): wth t 1 ( Y, β,θ, σ) = ln( 2πσ² ) ln( detr ) σ² [ Y βf ] R [ Y βf ] * * * ( β,θ,σ ) N 2 = Argmax ( β,θ,σ ) 1 2 ln L Y (, β,θ, σ) 1 2 Jont estmaton of β and σ : * β = * 1 σ² = N [ ] t 1 F R F t 1 t 1 [ ] [ ] * 1 * Y β F R Y β F F R Y Estmaton of correlaton parameters θ : ( θ ) Argmn ψ ( θ) * = ( ) * θ wth 1 ψ θ = N σ ² R Summer school SAMO B. Iooss - 26/06/14 19

20 Estmaton and valdaton Hyperparameters (θ ι ) =1 K estmated by lkelhood maxmzaton K 2 ( u - v) = exp u v R( u, v) = R θ = 1 Smplex method, stochastc algorthms Problems n hgh dmensonal context (K > 10), can be solved by sequental fttng algorthms [ Marrel et al ] Predctor valdaton: Predctvty coeffcent Q ( Y, Yˆ ) - Test sample - or leave-one-out - or k-fold cross valdaton 2 n = 1 = 1 n = 1 ( Y Yˆ ) ( Y Y ) code run (test pont) 3 code runs MSE valdaton: Percentage of predcted values nsde confdence bounds Summer school SAMO B. Iooss - 26/06/14 20

21 Effects of the hyperparameters θ and σ f ( x) = sn( 4πx) 2 σ = 1; θ = σ = 1; θ = σ = 4; θ = σ = 4; θ = 10 Summer school SAMO B. Iooss - 26/06/ [ Le Gratet, 2011 ]

22 Effects of the covarance structure [ Chevaler, 2011 ] Summer school SAMO B. Iooss - 26/06/14 22

23 GP metamodel n summary [ Le Gratet, 2014 ] The unkown functon smulatons condtonal smulatons Man hypothess: z(x) s the realzaton of a Gaussan process (defned by ts mean and cov) mean observatons 95%-confdence nterval GP metamodel s gven by the the probablty law of Z(x) condtonally to the observatons Its mean gves the GP predctor Its varance gves the error Summer school SAMO B. Iooss - 26/06/14 23

24 The best way to buld GP: Model-based adaptve desgns Y Example: crteron of the Gaussan process MSE (Mean Square Error) MSE( x) = σ ² + x new r( x) R u( x) = βf( x) x D t t 1 r( x) + u( x)( k( x) R = arg max MSE( x) 1 βf t ( βf Y ) R 1 βf t ) u( x) 2 code runs 3 code runs x new x new Remark: other crtera are possble (e.g. focusng to actve varables) Concluson: Model-based adaptve desgns are the most effcent ones, but are not always applcable In practce, we need to ntate the process wth a space-fllng desgn Summer school SAMO B. Iooss - 26/06/14 24

25 Estmaton of rare events probablty usng GP [ Bect et al ] Industral problems: safety analyss wth computer code (nuclear, transport, ) Problem: fnd P f = Prob [ f ( ) > T ] wth = random nputs ; T = treshold Reasonable varance everywhere Large errors n the target regon Τ New adaptve desgn * IMSET = MSE( x)1 ( x) dx T s a small tube around T T = arg mn( IMSE T ) [ from: Pcheny et al ] Τ Large varance n non-target regon Good accuracy n target regon Summer school SAMO B. Iooss - 26/06/14 25

26 Sobol ndces estmaton usng Gaussan process metamodel Summer school SAMO B. Iooss - 26/06/14 26

27 Sobol ndces Defntons for a determnstc functon Y = f ( 1, K, K ) Frst order Total : S T Notaton : : = V Var [ Ε( f ( 1, K, S = = V V V Var [ Ε( f (,, K ) 1 K = V V K ) )] )] [,,,, ] V =, Var( Y ) ; = 1 K K K Hypothess: ndependent nputs Classcal approach estmaton: replace f () by Ŷ (GP mean) + Computatonally cheap (possble to perform analytcal calculatons) - Do not nfer from the metamodel error Summer school SAMO B. Iooss - 26/06/14 27

28 Senstvty analyss wth GP model : 2 analytcal approaches Gp model condtonally to ponts : Computaton of Sobol ndces : From predctor formula [ Chen et al ] ˆ [ Y (, ω) ] Y ( ) = EΩ, Y a( ) = Var S = E Do not nfer from metamodel error Y( x, ω), Y [ Yˆ ( ) / ] [ E ˆ ( Y ( Var ( Yˆ ) ) / )] ~ GP ˆ From full GP model [ ] Y ( x, ω) Y ( x) = EΩ, Y ( Y ( u, ω), Y ( v, ω ) Cov Ω ), Y, Y a(, ω) = E V ( ω) = Var ~ S Y (, ω), Y a(,ω)) : stochastc process of V : random varables [ ] Y(, ω) /, Y [ ( )] E Y(, ω) /, Y V = µ = E Var Y (, ω) Ω, Y E Ω ~ ( S ) Summer school SAMO B. Iooss - 26/06/14 28

29 Analytcal senstvty ndces from full GP model Sobol ndces: µ = E Ω ( [ ( )] Var E Y (, ω) ) E Ω Var, Y Y (, ω), Y [ Oakley & O Hagan 2004; Marrel et al ] σ ι = Var Ω ( [ ( )] Var ) Ε Y (, ω ), Y [ ] E Var Y (, ω ) 2 2 Ω, Y Infer from the metamodel error Computaton: Analytcal calculatons Smple and double numercal ntegrals => computatonally expensve Other lmts: Addtonal hypothess : GP covarance s product of one-dmensonal covarance Ths estmator s not the true expectaton of the Sobol ndex Not possble to estmate the total Sobol ndces Summer school SAMO B. Iooss - 26/06/14 29

30 S ω GP-smulaton based Sobol ndces (1/2) Var [ ( )] E Y (, ω) = s a random varable Var, Y Y (, ω), Y [ Le Gratet et al ] We want to buld an unbased Monte Carlo estmator the number of Monte Carlo partcles ˆ S, m where m represents and an estmator of the varance of ˆ S, m Algorthm: 1. Generate 2 m -sze matrces ( 1, 2 ) and compute the pck-freeze matrx (the one whch used n Sobol ndces estmaton formula) 2. For k=1,, nsm Perform a condtonal smulaton y * k of the GP at ponts n Compute ˆ S, m, k Bootstrap procedure (b = 1,, B replcas) gves ˆ S, m, k, b Summer school SAMO B. Iooss - 26/06/14 30

31 GP-smulaton based Sobol ndces (2/2) [ Le Gratet et al ] The estmator of Sobol ndex: Sˆ, m = 1 nsm B nsm B k = 1 b = 1 Sˆ, m, k, b The varance of ths estmator: σˆ 1 nsm B ( Sˆ, m, k, b S, m ) 2 = ˆ nsm ( B 1) k = 1 b = 1 It ntegrates the metamodelng error + the Monte Carlo ntegraton error σˆ 1 B nsm 1 ( S, m, k, b S, m b ) 2 (PG) = ˆ, B b = 1 nsm 1 k = 1 σˆ 1 nsm k = 1 B 1 b = 1, nsm B 1 ( S, m, k, b S, m k ) 2 (MC) = ˆ Remark: Same algorthm for total Sobol ndces Summer school SAMO B. Iooss - 26/06/14 31

32 Example: Ishgam functon (K = 3) Learnng sample: space fllng desgn (n = 40 to 200) Gaussan process wth 5/2 Matérn cov. m = nsm = 500 B = 300 GP predctvty coef In practce, n s fxed and m can be calbrated n order to balance the error Summer school SAMO B. Iooss - 26/06/14 32

33 Spatal output Summer school SAMO B. Iooss - 26/06/14 33

34 Senstvty analyss when model ouputs are functons Thermal-hydraulc example Hydrogeologcal applcaton 2 elementary cases: senstvty analyss on each scalar output (q pxels): Very small CPU tme consumng model Lnear regresson model => use of standardzed regresson coef. (SRC) Dffcult case: Complex/Non lnear model need of Sobol ndces (for example) and CPU tme expensve model need of metamodel Summer school SAMO B. Iooss - 26/06/14 34

35 Senstvty analyss for spatal outputs: Methodology Computer code f (.) : Input: = ( 1,, K ) random vector Output for nput x * : y = f (x *, z ), z D z R 2 [ Marrel et al ] In practce, D z s dscretzed n q ponts (here: 64 x 64 = 4096 ponts) (,Y (,z) ) = nput-output sample of sze N Decomposton of Y (z) on a functonal bass, for example: Prncpal component analyss PCA) gves functons ftted on the data, A wavelet bass s well-suted f there are dscontnutes Modelng of the decomposton coeffcents by a GP metamodel Selecton procedure of the most mportant coeffcents Predcton: x* => predcton of coeff. => spatal output map reconstructon Functonal metamodel Senstvty analyss : Spatal maps of senstvty ndces Summer school SAMO B. Iooss - 26/06/14 35

36 Metamodel fttng: Methodology for spatal output (map) Step 1 : Decomposton n bass functons of each map Step 2 : Krgng metamodelng of the man coeffcents (the most varables) n foncton of ; constant for the other coef. Computatonal challenge: we have to ft as many GP as man coeffcents Step 3 : Predcton for a new nput x* x* => predcton of the coeffcents => spatal output map reconstructon Senstvty analyss (spatal map of Sobol ndces) The estmaton of Sobol ndces requre a large number of smulatons computaton of the Sobol ndces by the way of the metamodel Computatonal challenge to manage thousand of calls of the functonal metamodel Summer school SAMO B. Iooss - 26/06/14 36

37 Descrpton of CERES-MITHRA test case Complex spato-temporal dynamcs analyss by model reducton and senstvty analyss ( ) Assessment of the consequences on human health of radonuclde accdental or routne releases (atmospherc releases) Modellng of radonuclde atmospherc dsperson based on Gaussan puff model Developpement of an atmospherc dsperson code : CERES-MITHRA (C-M) mpact calculatons relatve to CEA facltes [ CEA applcaton: Marrel & Perot, 2012 ] Summer school SAMO B. Iooss - 26/06/14 37

38 Descrpton of the scenaro C-M INPUTS Release parameters 2 release locatons (uncertan heghts) Radonuclde quantty (uncertan) Deposton velocty (uncertan) Release duraton (1h) Radonuclede: Cesum 137 C-M OUTPUTS 2D maps of tme ntegrated actvty concentratons (Bqsm -3 ) for dfferent nstants (=> 2 hours) Meteorologcal parameters Wnd speed & drecton (uncertan) Atmospherc stablty Ran Temperature Objectves C-M Smulator CPU tme from 30s to 30mn Identfy the nfluence of each uncertan parameters Cs137+ concentraton map after 20 mnutes Problems CPU tme requred for each smulaton lmted number of avalable smulatons Spato-temporal outputs (very large dmenson of model outputs) Summer school SAMO B. Iooss - 26/06/14 Cs137+ concentraton map at the end 38 of the release

39 Uncertanty quantfcaton Uncertan parameters (K = 6) Parameters Reference Value Varaton Interval Probablty dstrbuton Release hegth (m) Deposton velocty (m.s -1 ) Quantty released (Bq) A 15 [7.5 ; 22.5] Unform B 45 [22.5 ; 67.5] Unform Cs [ ; ] Log-Unform Cs [10 8 ; ] Log-Unform Wnd drecton (WD) (degree azmuth) Wnd speed (WS) (m.s -1 ) 291 [249 ; 333] 5.3 [0 ; 12.5] Truncated AR(1) process Truncated AR(1) process Summer school SAMO B. Iooss - 26/06/14 39

40 Expermental desgn and learnng sample Numercal expermental desgn : Space fllng desgn n order to have a good coverage of the nput space => Optmzed Latn hypercube Samplng (LHS) Number of C-M smulatons : n = 200 smulatons Compromse between CPU tme and nvestgaton of uncertan parameter doman Smulatons wth C-M : Learnng sample creaton Notatons ( ) s = ( x ) = 1, L, n Y ( ) ( ) ( z) = y( x, z) = 1, Ln learnng sample C-M outputs (q = 4000 pxels) Examples of Cs137+ ntegrated actvty maps (logarthmc scale) Summer school SAMO B. Iooss - 26/06/14 40

41 Approxmaton by a metamodel Spatal metamodel : Proper Orthogonal Decomposton + Gaussan Process Step A : Spatal decomposton on a functonal bass (Chatterjee( [2000]) Selecton of the h man coeffcents of the decomposton Input parameters = [ 1,... K ] Y (, z) = µ ( z) + h h α Step B : Modelng of the k man PCA coeffcents n functon of Gaussan Process metamodel Computer code ( ) φ ( z) wth α ( ) = ( Y(, z) µ ( z) ) φ ( z) j j j j= 1 D Smulated maps POD decomposton Selecton of man coeffcents GP Metamodel j dz Modelng of coefs n functon of Step C : Predcton for any new nput x* New parameters * = [* 1,... * K ] GP Metamodel Predcton of the h man coeffcents POD reconstructon Predcted map Spatal surrogate model Summer school SAMO B. Iooss - 26/06/14 41

42 Basc prncple of prncpal component analyss (PCA) Transformaton of dependent numercal varables to ndependent varables nformaton reducton PCA: sequental search of new varables = lnear combnatons of Y wth a maxmal dsperson (varance or nerta) on each axs V= t Y c Y c varance-covarance matrx of Y c (matrx of centered functons) Dagonalsaton of the varance-covarance matrx to obtan ordered egen vectors (nerta axes) Egen values are prncpal component varances and gve the total nerta (percentage of explaned varance) The matrx (N x q ) of prncpal component s H = Y c L wth L the egen vector matrx Remark: data adaptve bass whle the wavelet bases are fxed Summer school SAMO B. Iooss - 26/06/14 42

43 Selecton of PCA coeffcents Applcaton to C-M data & results : h = 15 selected PCA components = 99% of nformaton explaned 15 coeffcents to be modelled by GP metamodels Summer school SAMO B. Iooss - 26/06/14 43

44 Summer school SAMO B. Iooss - 26/06/14 44 Valdaton of the spatal metamodel accuracy Predctvty coeffcent : -Q² estmated by cross valdaton - Average value of Q 2 on the map 92 % Global and local analyss of the metamodel predctvty ( ) [ ] = = = = n n n z Y n z Y z Y z x Y z Q ) ( ) ( 1 2 ) ( ) ( ) ( 1 ) ( ) (, ˆ 1 ) ²( Q² map wth 15 PCA components modelled wth GP, for t = 40mn

45 Metamodel qualty for the varous moments Summer school SAMO B. Iooss - 26/06/14 45

46 Senstvty Analyss Use of the metamodel POD+GP to perform smulatons requred for Sobol ndces estmatons (RBD-FAST method) Frst order Sobol ndces for t = 40mn Input nfluence : Predomnant : - Wnd drecton - Release quantty : located n the center of the plume - Wnd speed : located at the edges of the plume Neglgble : deposton velocty, release heghts Sum of 1 er order ndces 75% Interactons : 25% of model varabllty Summer school SAMO B. Iooss - 26/06/14 46

47 Total Sobol ndces for t=40mn Dfferences between 1 st order and total ndces : - An average of 25% of dfference for wnd drecton and speed => Strong nteracton Wnd drecton x Wnd speed? => 2 nd order ndces: (located on the edges of the dsperson plume) CONCLUSION : An average of 97% of the C-M output varance explaned by : Wnd Drecton (34%) - Wnd Drecton x Wnd Speed (23%) Cs137+ released quantty (20%) - Wnd Speed (20%) Summer school SAMO B. Iooss - 26/06/14 47

48 Tme evoluton of Sobol ndces Sgnfcant augmentaton of nteracton wnd drecton x wnd speed nfluence: Less than 10% at the release begnnng and more than 40% at the end Summer school SAMO B. Iooss - 26/06/14 48

49 Concluson: proposed methodology Uncertanty quantfcaton : nput parameters + assocated probablty dstrbutons Uncertanty quantfcaton : Constructon of a realstc smulator of weather condtons Samplng desgn : N smulatons Samplng desgn : Latn Hypercube Sample (LHS) wth optmal recoverng propertes Smulator (C-M M code) Metamodel valdaton Learnng sample Statstcal modellng : Metamodel or Surrogate model Metamodel : Spatal decomposton wth PCA + metamodellng of PCA coeffcents wth Gaussan processes Senstvty analyss Uncertanty propagaton Senstvty analyss : Computaton of Sobol ndces Interpretaton Summer school SAMO B. Iooss - 26/06/14 49

50 Senstvty analyss n R Summer school SAMO B. Iooss - 26/06/14 50

51 Uncertanty studes: several packages In R = software envronment for statstcal computng (Open Source) Samplng: randtoolbox, DceDesgn Uncertanty propagaton: mstral Senstvty analyss: senstvty, CompModSA, planor Gaussan process metamodelng: DceKrgng Optmzaton usng metamodel: DceOptm, KrgInv Functonal outputs: multsens, modelcf Summer school SAMO B. Iooss - 26/06/14 51

52 R package «senstvty» - Demo on Ishgam functon (K=3) lbrary(senstvty) Samplng method (n = boostrap rep. - N = n(k+2) = 5e4 model runs) 1 = data.frame(matrx(runf(3*n,-p,p),nrow=n)) ; 2 = sa1 = sobol2002(model = shgam.fun, 1, 2, nboot = 100); plot(sa1) GP metamodel from a Monte Carlo sample (,Y) (n = 100 model runs): lbrary(dcekrgng) x = data.frame(matrx(runf(3*n,-p,p),nrow=n)) ; y = shgam.fun(x) PG = km(formula=~1,x,y) ; prnt(pg) ; plot(pg) GP valdaton usng metamodel predctvty (Q 2 coeffcent wth ntest=1000): test = ; ytest = shgam.fun(xtest) Q2 = 1 - (mean(ytest-predct(pg,xtest,type="uk")$mean)^2)/var(ytest) Senstvty ndces computed va the GP predctor: sa2 = sobol2002(model = NULL, 1, 2, nboot = 100) Y = predct(pg,sa2$,type="uk")$mean ; tell(sa2,y) ; plot(sa2) Senstvty ndces computed va the full GP: sa3=sobolgp(model=pg,type="uk",mcmethod="sobol2002",1,2,canddate=x); plot(sa3) Summer school SAMO B. Iooss - 26/06/14 52

53 Concluson Summer school SAMO B. Iooss - 26/06/14 53

54 Stakes of uncertanty management n computer experments Modelng phase Improve the model Explore the best as possble dfferent nput combnatons Identfy the predomnant nputs and phenomena n order to prorze R&D Valdaton phase Reduce predcton uncertantes Calbrate the model parameters Practcal use of a model Safety studes: assess a rsk of falure (rare events) Concepton studes: optmze system performances and robustness Advantages of a probablstc approach to propagate the uncertantes, to perform global senstvty analyss to desgn/optmze the system takng nto account uncertantes to gve rgorous safety margns, Summer school SAMO B. Iooss - 26/06/14 54

55 Uncertanty management - The generc methodology Step C : Propagaton of uncertanty sources Step B: Quantfcaton of uncertanty sources Modelsaton wth probablty dstrbutons Input varables Uncertan :: x Fxed :: u Step A : Problem specfcaton Model (or measurement process) f(x,u) Varables of nterest Y = f(x,u) Quantty of nterest Ex: varance, probablty.... Drect methods, statstcs, expertse Step C : Senstvty analyss, Prortzaton Step B : Quantfcaton of sources Inverse methods, calbraton, assmlaton Observed varables Y obs obs Feedback Decson crteron Summer school SAMO B. Iooss - 26/06/14 process Ex: Probablty < 10 -b 55

56 Open TURNS The software mplementaton of the uncertanty methodology TURNS : Treatments of Uncertantes, Rsk n Statstcs Snce 2005 Open : Open source : LGPL (code), FDL (doc.) Envronment : Lnux, Wndows Languages : C++ (lbrares), Python (command scrpts) IHM Efcas Summer school SAMO B. Iooss - 26/06/14 56

57 Man features n OpenTURNS Use of non-ntrusve technques generc Wrappng of external codes, consdered as mathematcal functons n OpenTurns Step B: modelng uncertanty of nputs Wth data: parametrc & non-parametrc stats Assessng experts advce Dependence: defnton by margnals + copula Step C: Uncertanty propagaton Standard and advanced Monte Carlo technques (Importance samplng, drectonal samplng ) FORM-SORM method Metamodels: Polynomals, polynomal chaos expanson, Gaussan processw Step C : Senstvty analyss - Lnear & rank regresson, Sobol, polynomal chaos, relablty mportance factors Summer school SAMO B. Iooss - 26/06/14 57

58 On the Gaussan process metamodel Valuable tool when computer code s cpu-tme expensve (n ~ hundreds runs) GP model constructon s possble for moderate dmensonal cas (K < 50) Man advantage of GP: probablstc metamodel whch gves confdence bands n addton to a predctor Full nterest n senstvty analyss Fttng qualty s dependent of the ntal desgn GP model s well adapted to sequental and adaptatve desgns Caveats: t can requre a large amount of effort durng the fttng process and cases wth more than 1000 ponts begn to be dffcult (matrx nverson) Desgns for specfc objectves (optmzaton, quantle, probablty, etc.) Other hot topcs: calbraton and valdaton of computer codes Summer school SAMO B. Iooss - 26/06/14 58

59 Bblography Books: - Chlès & Delfner, Geostatstcs, Wley, 1999 Fang, L & Sudjanto, Desgn and modelng for computer experments, Chapman, 2006 Metamodels: Smpson, Peplnsk, Koc & Allen, Metamodels for computer-based engneerng desgn: Survey and recommendatons, Engneerng wth computers, 17, 2001 Storle, Swler, Helton & Salaberry, Implementaton and evaluaton of nonparametrc regresson procesures for senstvty analyss of computatonally, RESS, 94, 2009 Gaussan process: Koehler & Owen, Computer experments, In Handbook of Statstcs (Ghosh & Rao), 13, 1996 Marrel, Iooss, Van Dorpe & Volkova. An effcent methodology for modelng complex computer codes wth Gaussan processes. Computatonal Stat. and Data Analyss, 52, 2008 Le Gratet, Cannamela, Iooss. A Bayesan approach for global senstvty analyss of (multfdelty) computer codes. SIAM/ASA Journal of Uncertanty Quantfcaton, n press Functonal nput and output: Marrel et al., Global senstvty analyss for models wth spatally dependent outputs. Envronmetrcs, 22, 2011 Marrel, Iooss, da Vega & Rbatet. Global senstvty analyss of stochastc computer models wth jont metamodels. Statstcs and Computng, 22, 2012 Applcatons: Auder, de Crecy, Iooss & Marquès. Screenng and metamodelng of computer experments wth functonal outputs. Applcaton to thermal-hydraulc computatons. RESS, 107, 2012 Marrel and Perot. Development of a surrogate model and senstvty analyss for an atmospherc dsperson computer code. Proceedngs of ESREL 2012 Conf, Helsnk, 2012 Summer school SAMO B. Iooss - 26/06/14 59