Multi-fidelity co-kriging models

Transcription

1 Application to Sequential design Loic Le Gratiet 12, Claire Cannamela 3 1 EDF R&D, Chatou, France 2 UNS CNRS, 69 Sophia Antipolis, France 3 CEA, DAM, DIF, F Arpajon, France ANR CHORUS April 3, 214

2 Introduction Context Input parameters x Physical system Simulator Meta-model g exp (x) g(x) ĝ(x)

3 Motivations Motivations Objective: replace the output of a code, called g 2(x), by a metamodel. g 2(x) : x Q R d R Framework: a coarse version g 1 of g 2 is available. g 2(x) : Highfidelity code g 1(x) : Lowfidelity code. Principle: build a metamodel of g 2(x) which integrates as well observations of the coarse code output. Multi-fidelity co-kriging model

4 Recursive formulation of the model Multi-fidelity co-kriging model:[kennedy & O Hagan (2), Le Gratiet (213), Le Gratiet (214)] { Z2(x) = ρz 1(x)+δ(x) Z 1(x) δ(x) where Z 1(x) [Z 1(x) Z 1 = g 1,β 1,σ 2 1, θ 1], with g 1 = g 1(x),x D 1 ( ) ( ) and Z 1(x) GP f1(x)β t 1,σ1r 2 1(x, x;θ 1), δ(x) GP fδ(x)β t δ,σδr 2 δ (x, x;θ δ ) Parameters estimation: θ 1, θ δ,σ1, 2 σδ 2 : maximum likelihood method βδ ) β 1,( : analytical posterior distribution (Bayesian inference) ρ

5 Recursive formulation of the model Multi-fidelity co-kriging model:[kennedy & O Hagan (2), Le Gratiet (213), Le Gratiet (214)] { Z2(x) = ρz 1(x)+δ(x) Z 1(x) δ(x) where Z 1(x) [Z 1(x) Z 1 = g 1,β 1,σ 2 1, θ 1], with g 1 = g 1(x),x D 1 ( ) ( ) and Z 1(x) GP f1(x)β t 1,σ1r 2 1(x, x;θ 1), δ(x) GP fδ(x)β t δ,σδr 2 δ (x, x;θ δ ) Parameters estimation: θ 1, θ δ,σ1, 2 σδ 2 : maximum likelihood method βδ ) β 1,( : analytical posterior distribution (Bayesian inference) ρ Finally, Z 2(x) [Z 2(x) Z 2 = g 2,Z 1 = g 1] with g 2 = g 2(x),x D 2 We suppose that D 2 D 1 and θ 1, θ δ,σ 2 1, σ 2 δ are known.

6 Recursive formulation of the model Multi-fidelity co-kriging model:[kennedy & O Hagan (2), Le Gratiet (213), Le Gratiet (214)] { Z2(x) = ρz 1(x)+δ(x) Z 1(x) δ(x) where Z 1(x) [Z 1(x) Z 1 = g 1,β 1,σ 2 1, θ 1], with g 1 = g 1(x),x D 1 ( ) ( ) and Z 1(x) GP f1(x)β t 1,σ1r 2 1(x, x;θ 1), δ(x) GP fδ(x)β t δ,σδr 2 δ (x, x;θ δ ) Parameters estimation: θ 1, θ δ,σ1, 2 σδ 2 : maximum likelihood method βδ ) β 1,( : analytical posterior distribution (Bayesian inference) ρ Finally, Z 2(x) [Z 2(x) Z 2 = g 2,Z 1 = g 1] with g 2 = g 2(x),x D 2 We suppose that D 2 D 1 and θ 1, θ δ,σ 2 1, σ 2 δ are known.

7 Predictive distribution In Universal Cokriging, the predictive distribution of Z 2(x) is not Gaussian. The predictive mean and variance can be decomposed as: µ Z2 (x) = E[Z 2(x) Z 2 = g 2,Z 1 = g 1] = ˆρµ Z1 (x)+µ δ (x) σz 2 2 (x) = var(z 2(x) Z 2 = g 2,Z 1 = g 1]) = ˆσ ρσ 2 Z 2 1 (x)+σδ(x) 2 Remarks: in µ Z2 (x): β and ρ are replaced by their posterior means. in σz 2 2 (x): we infer from the posterior distributions of β and ρ.

8 Generalizations Generalizations for the AR(1) model: s > 2 levels of code. ρ(x) = f ρ(x)β ρ. Bayesian formulation. Non-nested experimental design sets (see L. Le Gratiet thesis 213). Extend the AR(1) approach (see F. Zertuche): Z 2(x) = ψ(z 1(x))+δ(x) Other Bayesian formulation with (see Qian and Wu 28): ρ(x) a Gaussian process. z 1(x) is supposed as known.

9 Sequential design Objective: we want to minimize the following generalization error: IMSE = σz 2 2 (x)dx = ˆσ ρ 2 σz 2 1 (x)dx + σδ(x)dx 2 Q Q Q Sequential strategy: we select a new point x n+1 such that : x n+1 = argmaxσz 2 x 2 (x)

10 Sequential design Objective: we want to minimize the following generalization error: IMSE = σz 2 2 (x)dx = ˆσ ρ 2 σz 2 1 (x)dx + σδ(x)dx 2 Q Q Q Sequential strategy: we select a new point x n+1 such that : x n+1 = argmaxσz 2 x 2 (x) Question : which code level should be simulated? What is the contribution of each code level to the model error? σ 2 Z 2 (x n+1 ) = ˆσ 2 ρσ 2 Z 1 (x n+1 )+σ 2 δ (x n+1) What is computational cost of each code? What is the expected reduction of the generalization error?

11 Sequential design Objective: we want to minimize the following generalization error: IMSE = σz 2 2 (x)dx = ˆσ ρ 2 σz 2 1 (x)dx + σδ(x)dx 2 Q Q Q Sequential strategy: we select a new point x n+1 such that : x n+1 = argmaxσz 2 x 2 (x) Question : which code level should be simulated? What is the contribution of each code level to the model error? σ 2 Z 2 (x n+1 ) = ˆσ 2 ρσ 2 Z 1 (x n+1 )+σ 2 δ (x n+1) What is computational cost of each code? What is the expected reduction of the generalization error?

12 Code level selection What is computational cost of each code? C : CPU time ration between g 2(x) and g 1(x). 1 run of g 1(x) and g 2(x) C +1 runs of g 1(x) (i.e. D 2 D 1) What is the expected reduction of the error? Reduction of the generalization error for Z 1(x) : ˆσ 2 ρσ 2 Z 1 (x n+1) Reduction of the generalization error for the bias δ(x) : σ 2 δ(x n+1) θ i δ θ i 1

13 Code level selection What is computational cost of each code? C : CPU time ration between g 2(x) and g 1(x). 1 run of g 1(x) and g 2(x) C +1 runs of g 1(x) (i.e. D 2 D 1) What is the expected reduction of the error? Reduction of the generalization error for Z 1(x) : ˆσ 2 ρσ 2 Z 1 (x n+1) Reduction of the generalization error for the bias δ(x) : σ 2 δ(x n+1) θ i δ θ i 1

14 Illustration of the design criterion X X 1 X 1 X 2 X X X 1 X 1

15 Code level selection Expected error reduction for the code level 2 : ˆσ ρσ 2 Z 2 1 (x n+1) θ i 1 +σδ(x 2 n+1) Expected error reduction for the code level 1 : (C +1)ˆσ ρσ 2 Z 2 1 (x n+1) θ i 1 θ i δ

16 Code level selection Expected error reduction for the code level 2 : ˆσ ρσ 2 Z 2 1 (x n+1) θ i 1 +σδ(x 2 n+1) Expected error reduction for the code level 1 : (C +1)ˆσ ρσ 2 Z 2 1 (x n+1) It is worth simulating g 2(x) if : (C +1)ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 < ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 +σδ(x 2 n+1) i.e. θ i 1 θ i δ θ i δ

17 Code level selection Expected error reduction for the code level 2 : ˆσ ρσ 2 Z 2 1 (x n+1) θ i 1 +σδ(x 2 n+1) Expected error reduction for the code level 1 : (C +1)ˆσ 2 ρσ 2 Z 1 (x n+1) It is worth simulating g 2(x) if : (C +1)ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 < ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 +σδ(x 2 n+1) i.e. θ i 1 ˆσ ρσ 2 Z 2 1 (x d n+1) θi 1 ˆσ ρσ 2 Z 2 1 (x d n+1) θi 1 +σδ 2(xn+1) < 1 d θi C +1 δ θ i δ θ i δ

18 Code level selection Expected error reduction for the code level 2 : ˆσ ρσ 2 Z 2 1 (x n+1) θ i 1 +σδ(x 2 n+1) Expected error reduction for the code level 1 : (C +1)ˆσ 2 ρσ 2 Z 1 (x n+1) It is worth simulating g 2(x) if : (C +1)ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 < ˆσ ρσ 2 2 Z 1 (x n+1) θ i 1 +σδ(x 2 n+1) i.e. θ i 1 ˆσ ρσ 2 Z 2 1 (x d n+1) θi 1 ˆσ ρσ 2 Z 2 1 (x d n+1) θi 1 +σδ 2(xn+1) < 1 d θi C +1 δ θ i δ θ i δ

19 New strategy Problem: the following criterion does not take into account the real prediction error x n+1 = argmaxσz 2 x 2 (x) New strategy: to take into account the true prediction error we consider the following criterion (adjusted variance) : x n+1 = argmax x ( ˆσ ρσ 2 Z 2 1,UK(x) 1+ n 1 +σ 2 δ,uk(x) ( 1+ n 2 ε 2 LOO,1 (x(1) ) i 1 σ LOO,1 2 x V i,1 (x(1) ) i ε 2 LOO,δ (x(2) ) i σ 2 LOO,δ (x(2) i ) ) 1 x V i,2 ) where V i,j is the Voronoi cell associated to x (j) i, j = 1,2, i = 1,...,n j.

20 New strategy Problem: the following criterion does not take into account the real prediction error x n+1 = argmaxσz 2 x 2 (x) New strategy: to take into account the true prediction error we consider the following criterion (adjusted variance) : x n+1 = argmax x ( ˆσ ρσ 2 Z 2 1,UK(x) 1+ n 1 +σ 2 δ,uk(x) ( 1+ n 2 ε 2 LOO,1 (x(1) ) i 1 σ LOO,1 2 x V i,1 (x(1) ) i ε 2 LOO,δ (x(2) ) i σ 2 LOO,δ (x(2) i ) ) 1 x V i,2 ) where V i,j is the Voronoi cell associated to x (j) i, j = 1,2, i = 1,...,n j.

21 Illustration of the new criterion f(x) x

22 Academic example: the Schubert s function g(x) = ( ( i cos (i +1)x 1 +i ) )( ( ) ) i cos (i +1)x 2 +i Max variance Kleijnen criterion X X X 1 X 1 Kleijnen, JPC and van Beers, WCM (24), Application-driven sequential designs for simulation experiments : Kriging metamodelling. Journal of the Operational Research

23 Academic example: the Schubert s function g(x) = ( ( i cos (i +1)x 1 +i ) )( ( ) ) i cos (i +1)x 2 +i Adjusted variance Shubert s function X IMSE Number of observations adjmmse IMSE KLEIJNEN X 1

24 Application : spherical tank under internal pressure High-fidelity code: g 2(x) is the von Mises stress at point 1, 2 or 3 provided by a finit elements code. x = (P,R int,t shell,t cap,e shell,e cap,σ y,shell,σ y,cap) (d = 8) P : internal pression. R int : tank internal radius. T shell : tank thickness. T cap: cap thickness. E shell : tank Young s modulus. E cap: cap Young s modulus. σ y,shell : tank yield stress. σ y,cap: cap yield stress. Low-fidelity code: g 1 is the 1D approximation of g 2 (perfectly spherical tank). g 1(x) = 3 (R int +T shell ) 3 P 2 (R int +T shell ) 3 Rint 3 3 P CAP 2 SHELL 1

25 Application : spherical tank under internal pressure High-fidelity code: g 2(x) is the von Mises stress at point 1, 2 or 3 provided by a finit elements code. x = (P,R int,t shell,t cap,e shell,e cap,σ y,shell,σ y,cap) (d = 8) P : internal pression. R int : tank internal radius. T shell : tank thickness. T cap: cap thickness. E shell : tank Young s modulus. E cap: cap Young s modulus. σ y,shell : tank yield stress. σ y,cap: cap yield stress. Low-fidelity code: g 1 is the 1D approximation of g 2 (perfectly spherical tank). g 1(x) = 3 (R int +T shell ) 3 P 2 (R int +T shell ) 3 Rint 3 3 P CAP 2 SHELL 1 Co-kriging multi-fidelity model built with n 1 = 1 and n 2 = 2. The model efficiency Q 2 is estimated from a test set of 7 points. Q 2 86%.

26 Application : spherical tank under internal pressure High-fidelity code: g 2(x) is the von Mises stress at point 1, 2 or 3 provided by a finit elements code. x = (P,R int,t shell,t cap,e shell,e cap,σ y,shell,σ y,cap) (d = 8) P : internal pression. R int : tank internal radius. T shell : tank thickness. T cap: cap thickness. E shell : tank Young s modulus. E cap: cap Young s modulus. σ y,shell : tank yield stress. σ y,cap: cap yield stress. Low-fidelity code: g 1 is the 1D approximation of g 2 (perfectly spherical tank). g 1(x) = 3 (R int +T shell ) 3 P 2 (R int +T shell ) 3 Rint 3 3 P CAP 2 SHELL 1 Co-kriging multi-fidelity model built with n 1 = 1 and n 2 = 2. The model efficiency Q 2 is estimated from a test set of 7 points. Q 2 86%.

27 Results Response 1 Response 2 Normalized RMSE.2..1 Sequential kriging Sequential co kriging Normalized RMSE Sequential kriging Sequential co kriging CPU time CPU time Response 3 Normalized RMSE Sequential kriging Sequential co kriging Proportion of accurate code runs Rep. 3 Rep. 2 Rep CPU time CPU time

28 Kennedy, M. C. & O Hagan, A. (2), Predicting the output from a complex computer code when fast approximations are available. Biometrika 87, Le Gratiet, L. (213), Bayesian analysis of hierarchical multifidelity codes. SIAM/ASA J. Uncertainty Quantification 1-1, pp Le Gratiet, L. & Garnier, J. (214), Recursive co-kriging model for Design of Computer experiments with multiple levels of fidelity with an application to hydrodynamic, Int. J. Uncertainty Quantification.DOI: Bates, R.A., Buck, R., Riccomagno, E. & Wynn, H. (1996), Experimental design of computer experiments for large systems, Journal of the Royal Statistical Society B, 8 (1): Kleijnen, J. & van Beers, W. (24), Application-driven sequential design for simulation experiments: Kriging metamodeling., Journal of the Operational Research Society, : Le Gratiet, L. & Cannamela Claire (214), Kriging-based sequential design strategies using fast cross-validation techniques with extensions to multi-fidelity computer codes, to appear in TECHNOMETRICS. R CRAN package: MuFiCokriging