[MCEN 5228]: Project report: Markov Chain Monte Carlo method with Approximate Bayesian Computation

Transcription

1 [MCEN 228]: Project report: Markov Chain Monte Carlo method with Approximate Bayesian Computation Olga Doronina December 2, 217 Contents 1 Introduction 1 2 Background Autonomic Closure (AC) Approximate Bayesian Computation (ABC) Approximate Bayesian Computation Autonomic Closure (ABC-AC) A Priori testing Computational expence and requared improvements Monte Carlo Markov Chain without likelihood ABC-MCMC algorithm Results Problems Improved Markov Chain Monte Carlo Improved MCMC algorithm Results Summary, conclusion, and future plans 17 1 Introduction Despite the increasing ubiquity of large eddy simulation (LES) in engineering research, accuracy remains limited by the fidelity with which unresolved small-scale physics are modeled. Due to the use of grid-scale filtering in nearly all practical LES, such models are commonly termed subgrid-scale (SGS) models. In order to improve upon Smagorinsky-based models, an autonomic closure approach was recently proposed by King et al [1]. In autonomic closure, test scale filtering is used to provide training data that can be used to determine unknown coefficients in the model for the unclosed stresses. In this sense, autonomic closure can be considered a data-driven turbulence closure, except that the training data are provided internally to the simulation rather than externally from experimental or higher fidelity DNS data. The resulting relation can be then applied at the LES grid scale to achieve closure. 1

2 We can consider the inverse problem of finding the model parameters given the training data at the test scale. To estimate model parameters we use the Approximate Bayesian Computation (ABC) approach [2]. ABC approximates the posterior distribution by sampling model parameters from prior distributions and compare model outcomes (or data) with observed truth data. The initial sampling technique is sampling from a uniform grid in parameter space, which is inefficient for a high number of parameters. An implementation of a Monte Carlo Markov Chain sampling algorithm can significantly reduce the number of required computations. In this paper, we provide the necessary background information of autonomic closure using the ABC method and a priori testing results for sampling from uniform grid. Then we outline the initial Markov Chain Monte Carlo (MCMC) sampling algorithm and show its a priori testing results. Finally, we consider Improved Markov Chain Monte Carlo (IMCMC) and show its a priori testing results. 2 Background Coarse-graining of the Navier Stokes equations using a low pass filter at scale gives the LES equations for an incompressible flow, which are written as [3] ũ i x i =, ũ i t + ũ ũ i j = p + ν 2 ũ i τ, (1) x j x i x j x j x j where ũ i is the resolved scale velocity, p is the resolved scale pressure normalized by density, ν is the kinematic viscosity, and τ is the unclosed SGS stress tensor given by τ = ũ i u j ũ i ũ j. (2) Note that here, and in the following, ( ) denotes a low-pass filter at the grid scale, which is also often termed the LES scale since this is the finest scale represented when using the grid discretization as an implicit LES filter. The SGS stress term τ prevents closure and, in order to solve Eq. (1), an appropriate relation for τ must be found in terms of resolved scale quantities only. 2.1 Autonomic Closure (AC) Closure of Eq. (1) can be achieved by modeling the deviatoric part of the stress tensor σ = τ 1 3 τ kkδ, which can be written in terms of an unknown, high-dimensional, nonparametric functional F that takes as its arguments only quantities that can be expressed in terms of the resolved-scale strain rate, S, and rotation rate, R, tensors; namely [4] [ σ (x, t) σ F (x, t) = F S (x + x, t t ), R ] (x + x, t t ) x and t, (3) where the resolved scale strain and rotation rate tensors are given by S = 1 ( ũi + ũ ) j, R = 1 ( ũi ũ ) j. (4) 2 x j x i 2 x j x i The same functional can be used to write down the closure relation at a test scale, where >, as ] σ (x, t) σ F (x, t) = F [ S (x + x, t t ), R (x + x, t t ) x and t. () 2

3 The deviatoric test scale stress tensor is then given by σ = T 1 3 T kkδ where T = ũ i ũ j ũ i ũj represents the test scale stress tensor. The test scale strain and rotation rate tensors are written as ( ) ( ) S = 1 ũ i + ũ j, R = 1 ũ i ũ j. (6) 2 x j x i 2 x j x i Here, and in the following, ( ) denotes a low-pass filter at test scale. The functional F in Eqs. (3) and () can be written as the fifth-order, single point, single time, nonlinear model introduced by Pope [4], which is given for generic strain and rotation rate tensors, S and R [following similar definitions to those in Eqs. (4) and (6)] as F (S, R ) = 1 λ=1 C λ G (λ), (7) where C λ are coefficients that can depend on invariants of S, R, and their products up to sixth order. The tensor bases G (λ) are formed from products of S and R up to fifth order, namely [4] G (1) = S, G (6) = S ik R kl R lj + R ik R kl S lj 2 3 δ S kl R lm R mk, G (2) = S ik R kj R ik S kj, G (7) = S ik S kl R lm R mj + R ik R kl S lm S mj 2 3 δ S kl S lm R mn R nk, G (3) = S ik S kj 1 3 δ S kl S kl, G (8) = S ik R kl S lm S mj S ik S kl R lm S mj, G (4) = R ik R kj 1 3 δ R kl R kl, G (9) = R ik S kl R lm R mj R ik R kl S lm S mj, G () = S ik S kl R lj R ik S kl S lj, G (1) = R ik S kl S lm R mn R nj R ik R kl S lm S mn R nj. In order to obtain G (λ) and Ĝ(λ) at the LES and test scales, respectively, S and R are replaced by the appropriate LES or test scale values of the strain and rotation rate tensors [as given in Eqs. (4) and (6)]. At the test scale, both the left and right hand sides of Eq. () are known, and thus the model parameters of F can be determined through an appropriate inversion technique. Once F is known at the test scale, it is then projected to the grid scale. The resulting closure approach is similar to dynamic versions of classical closures such as the Smagorinsky model [, 6] and to scale-similarity models such as that by Bardina et al.[7], except that the relation F is high dimensional and nonparametric. 2.2 Approximate Bayesian Computation (ABC) In King et al. [1], optimization and a least squares technique were used to invert Eq. () and solve for the unknown coefficients in the Volterra series formulation of F. However, the inversion process was memory intensive and only highly truncated forms of F were possible. Alternatively, ABC can be used to determine F. ABC is based on Bayes theorem, but instead of computing the likelihood function to determine the true posterior, it provides an approximation to the posterior distribution using an ad hoc distance function [2]. In general terms, ABC methods sample model parameters from prior distributions and compare model outcomes (or data) D with observed truth data D, which may come from experiments or a higher fidelity model. In order to reduce the computational expense, instead of full observed data D one can use summary statistics S(D), such as a mean, standard deviation, or full probability density function (pdf). Model statistics S (D ) can then be compared with the truth statistic S using a statistical distance ρ(s, S), where the distance function ρ may be a Kullback-Leibler divergence, Hellinger distance, or 3

4 simply a mean-square error. If S and S are similar to within some specified tolerance ɛ, then the sampled parameter values are retained and collected in an N-dimensional joint pdf, termed the posterior, where N is the number of unknown model parameters. From the maximum a posteriori probability (MAP), mean, or other characteristic statistic of the posterior distribution, a single set of parameters can be selected that gives the best agreement with the truth data. 2.3 Approximate Bayesian Computation Autonomic Closure (ABC-AC) Using concepts from autonomic closure and ABC, we can combine the following closure approach, denoted ABC-AC, at a particular time t. Here the prior time steps are neglected for simplicity; the ABC-AC approach is easily extended to include information at earlier times. Note that in the following, we assume that an appropriate relation containing a set of N unknown coefficients has already been selected for F. Moreover, we also assume that an N-dimensional prior joint pdf has been formulated as an estimate of the distribution of the unknown coefficients; typically, these priors are formulated as uniform distributions. The steps involved in application of the ABC-AC approach to closure of the LES equations are and denoted as Algorithm 1: 1. Filter resolved scale quantities ũ i and p in order to obtain test scale quantities ũ i and p. 2. Compute test scale deviatoric stresses σ. 3. Using σ, evaluate the truth statistic S; in the present case this statistic is the pdf of σ. 4. Sample the prior to obtain one possible realization of F and compute σ F.. Using σ F, evaluate the statistic SF using data from M training points. Note that M need not be (and indeed should not be) equal to the full size of the training data set. 6. If S and S F are in agreement to within some specified tolerance ɛ, retain the sample of N parameters in a new joint pdf. 7. Return to step 4 and repeat. This process is repeated a total P times until a reasonable estimate is obtained for the posterior. 8. From the estimated posterior, choose the modal value of the parameters; these parameters then give a statisticallyaccurate estimate for F. 9. Apply F at the LES scale to achieve closure. 2.4 A Priori testing A Priori testing is fundamental testing approach based on direct comparison between σ (x, t) and σ F (x, t) [3]. Such a comparison needs data at high spatial resolution that are sufficient to resolve the SGS range. σ (x, t) is evaluated based on its definition σ = τ 1 3 τ kkδ and Eq. (2), and σ F (x, t) is modeled using F with estimated parameters and filtered data. In the following, a priori tests of the ABC-AC approach are performed using pseudospectral HIT DNS data at Re λ = 433 from the Johns Hopkins Turbulence Database [8]. We outline two simplified formulations of ABC-AC where, as a validation test, we ensure that for a first-order simplification of the closure relation in Eq. (7), the approach can recover the coefficient C S used in the Smagorinsky model. second-order truncation of Eq. (7). Then we obtain coefficients from a 4

5 Both first and second order versions of the full model in Eq. (7) are tested here, giving the test-scale representations σ F = C 1 S [1st order], σ F = C 1 S + C 2 ( Sik Rkj Rik Skj ) + C 3 ( Sik Skj 1 3 δ Skl Slk ) (8) [2nd order]. (9) Note that in the second order formulation, C 4 = in order to satisfy requirements on material frame indifference [9]. Based on the formulation of the classical Smagorinsky model, it can be shown that C 1 = 2(C S ) 2 S where CS is the Smagorinsky coefficient and S ) 1/2 (2 S S. In order to facilitate comparisons with prior work, we similarly rewrite Eqs. (8) and (9) in terms of C S as σ F = 2(C S ) 2 S S [1st order], σ F = 2(C S ) 2 S S + C 2 2 ( Sik Rkj Rik Skj ) + C 3 2 ( Sik Skj 1 ) 3 δ Skl Slk (1) [2nd order]. (11) The test filtering required in these a priori tests is accomplished using spectrally sharp filtering. Figure 1 shows example velocity fields at the full DNS resolution, as well as at filtered LES and test scales. This figure also shows deviatoric SGS stress fields at the LES scale, denoted σ, for this data. It should be noted that the true expected value of C S from the data can be obtained from the relation C 2 S = [ ε S 3 ], where ε = 2νS S is the true kinetic energy dissipation rate. Using this relation, the data give C S =.22. In order for the ABC-AC approach to be deemed successful, it should recover this value for C S. The statistics used in the ABC analysis are pdfs of the test scale deviatoric stresses σ (as shown in Figure 2), which are denoted S since there is a pdf for each (i, j) component of the stress tensor. The prior distribution for C S is taken to be a uniform distribution between and.4. The sampled parameter value of C S is accepted with tolerance ɛ if i,j j i ρ(s F, S ) ɛ, where the summation is over all i and j such that j i, giving six independent terms in the summation; this summation approach is necessary since the stress tensors are symmetric and the terms with i j should not be double-counted in the combined distance metric. The statistical distance ρ(s F, S ) can be calculated many different ways. Here we show results using the Kullback- Leibler (KL) divergence. The KL divergence is given by ρ(s F, S ) = S ln S ln S F. It is emphasized that, in the present tests, S represents the pdf of σ and S F which are computed at the test scale. represents the pdf of σf, both of For the first order model, Figure 3 shows how the distance ρ(s F, S ) depends on C S for P = 1 samples of C S from its uniform prior, using a tolerance of ɛ = 2. The parameter value resulting in the minimum distance is close to the true value C S =.22. More specifically, the values obtained are C S.216. This value of C S determined at

6 6 v ṽ ṽ y x x x (a) Velocity fields at the full DNS resolution, and at LES and test scales (left to right). 6 σ 11 σ 12 σ y x x x.2 (b) SGS deviatoric stress (σ ) fields at the LES scale. Figure 1: Example velocity fields and SGS stresses σ at the LES scale for pseudospectral HIT data [8] used in a priori testing LES test pdf σ 11, σ 11 σ 12, σ 12 σ 13, σ 13 Figure 2: Probability density functions of the deviatoric stresses at LES (red lines) and test (green lines) scales, denoted σ and σ, respectively. the test filter scale can then be applied at the LES (i.e., grid) scale to approximate the unclosed stresses σ, namely σ F = 2(C S ) 2 S S. Figure 4 shows the resulting pdfs of σ F and σ, revealing reasonable agreement between the modeled and true stresses at the LES scale. A similar analysis for the second order model in Eq. (11) gives the posterior pdfs shown in Figure. In order to facilitate visualization of the three dimensional posterior, marginalized joint pdfs over one and two parameters are shown for each of C S, C 2 and C 3. Both the mode and mean of the marginal pdf for C S show that, once again, the 6

7 2 i,j ρ(sf, S ) C s Figure 3: Accepted values for the C S parameter in the first order model and corresponding statistical distances. 1 1 true modeled dist 1 1 pdf σ 11 σ 12 σ 13 Figure 4: Probability density functions of stresses σ F from the one-parameter first order model in Eq. (1) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). ABC approach provides a value of C S close to.2. The marginal pdf for C 2 is bimodal with peaks at C 2 = ±.1; this indicates that the sign of C 2 is of no importance compared to the magnitude. Finally, the marginal pdf of C 3 peaks near, indicating that this term is of negligible importance compared to the other two terms in the second order closure relation. Using the values of C i that give the minimum distance in the statistics, the second order model parameters at the test scale are found to be C s =.18, C 2 =.88, C 3 =.1. These values can then be applied at the LES scale to give the pdfs of σ F at the LES scale shown in Figure 6. Once again, the second order model gives close agreement between the modeled and true stresses. 7

8 C s C mean max C 3 Figure : Marginal and joint probability density functions of accepted values of C S, C 2, and C 3 for the second order model in Eq. (11) at the test scale. Dashed green lines show the mean of each marginal pdf, and dashed red lines show the location of the MAP. 1 1 true modeled dist 1 1 pdf σ 11 σ 12 σ 13 Figure 6: Probability density functions of stresses σ F from the three-parameter second order model in Eq. (11) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). 8

9 2. Computational expence and requared improvements The current realization of the ABC-AC algorithm samples model parameters from a uniform distribution (since we do not have any prior information about unknown parameters) for each parameter. The full nonlinear model F (Eq. (7)) has fifth order and ten unknown parameters, which potentially makes the prior distribution to be a 1-dimensional uniform grid. There is one problem with large-dimensional spaces mensioned by Tarantola [1]: they tend to be terribly empty. That is the probability of sampling from uniform distribution the (maximum-size) hypersphere inscribed in a hypercube rapidly decreases to zero when the dimension of the space grows. Even having only 1 points per dimension in the prior uniform grid makes the problem prohibitively expansive. Table 1 shows the estimated time of computation on one core. Model F Computational time # sampled points Order # parameters N iteration/s time on 1 core 1st order [it/s] 1.9 [s] 2nd order (without C 4 ) [it/s] 8. [h] 2nd order [it/s] 92 [h] 3rd order [years] 4th order th order Table 1: The estimated time of computation on one core with 1 points per dimension in prior uniform grid. The number of accepted parameters, which in fact form the posterior distribution, is only a small part of sampled parameters. For example, for the second order model with three parameters (Eq. (11)) only about 2% of the sampled points were accepted in the posterior distribution (Figure ) and this number rapidly decreases as the number of parameters grows. However, this problem can be solved, and the sampling technique can be significantly improved, by using Monte Carlo Markov Chain (MCMC) algorithm. 3 Monte Carlo Markov Chain without likelihood 3.1 ABC-MCMC algorithm Since ABC does not evaluate a likelihood function, we need to use the Monte Carlo Markov Chain method without a likelihood [11], which is based on the Metropolis Hastings algorithm, however, no likelihoods are used or estimated. [11] provided the following steps in the algorithm (instead of steps 4-7 in Algorithm 1): 4. If now at θ, propose a move to θ according to a transition kernel q(θ θ ).. Generate σ F using model F with parameters θ and evaluate the statistic S F using data from M training points. 6. If ρ(s, S F ) < ε, go to 7, and otherwise stay at θ and return to Calculate h = h(θ, θ ) = min ( 1, π(θ )q(θ ) θ) π(θ)q(θ θ ) Accept θ with probability h otherwise stay at θ, then return to 4. 9

10 The algorithm also can be effectively parallelized on large number of processors [12]. Choosing transitional kernel q(θ θ ) to be a gaussian with prescribed constant variance and current parameter θ as mean value, we have q(θ θ ) = q(θ θ). Therefore h, in step 4, depends only on the prior distribution.in our case, we do not have any prior information about the possible distribution of parameters, thus π(θ) is uniform and equal to π(θ ) for any θ. This leads to h = 1 for any θ and the algorithm reduces to a rejection method with correlated outputs. But since all new samples appear around accepted parameters, almost all of them are accepted, whereas for ABC algorithm without MCMC and 3 parameters model only about 1% of the sampled parameters were accepted. The percent of accepted parameters in ABC-MCMC depends on the variance in the transition kernel, that is the smaller variance leads to a larger number of accepted parameters. Thus the ABC-MCMC requires less evaluations of modeled deviatoric stresses σ F, statistic SF on this data, and distance function ρ(s, S F ) to get the same number of accepted parameters in the posterior distribution. 3.2 Results To test our implementation of MCMC sampling algorithm we use the same a priori testing with data described in Section 2.4 and first and second order models given by Eqs. (1) and (11) correspondingly. The transition kernel q(θ θ ) used below is a gaussian distribution with constant prescribed variance for each parameter. The variance is set to be 1/2 of initial parameters range, that is.7, for parameter C S. First order model. For the first order model, Figure 7 shows how the distance ρ(s F, S ) depends on C S for P = 1 accepted samples of C S using MCMC sampling and tolerance of ɛ = 2. The parameter value resulting in the minimum distance is the same as parameter recovered using uniform grid sampling in Section 2.4. More specifically, the estimated value is C S.216. Figure 8 shows the resulting pdfs of σ F and σ, revealing reasonable agreement between the modeled and true stresses at the LES scale. The advantage of the ABC-MCMC algorithm is it s efficiency. The number of accepted parameters is 1, which is 94% of the total number of evaluations of modeled stresses σ F, statistic SF, and distance function ρ(s, S F ). 2 2 i,j ρ(sf, S ) C S Figure 7: Accepted values for the C S parameter in the first order model and corresponding statistical distances. 1

11 1 1 true modeled dist 1 1 pdf σ 11 σ 12 σ 13 Figure 8: Probability density functions of stresses σ F from the one-parameter first order model in Eq. (1) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). Second order model with 3 parameters. A similar analysis for the second order model in Eq. (11) gives the posterior pdfs shown in Figure 9. For better visualization of the three dimensional posterior, the marginal pdfs shown on the diagonal subplots and marginalized joint pdfs over one parameter are on off-diagonal subplots. The MAP values of marginal pdfs are C S =.21, C 2 =.73, C 3 =.4. Using the values of C i that give the minimum distance in the statistics, the second order model parameters at the test scale are found to be C s =.18, C 2 =.88, C 3 =.2. These values can then be applied at the LES scale to give the pdfs of σ F at the LES scale shown in Figure 1. Figure 1 also shows the pdf resulting from parameters estimated as MAP of posterior pdf, C s =.22, C 2 =.37, C 3 =.3. Once again, the second order model gives close agreement between the modeled and true stresses. The number of accepted parameters was set to be 2 1, the same as for ABC with uniform grid sampling, which is 76% of the total number of samples. Thus, the total number of evaluations of modeled stresses σ F, statistic S F, and distance function ρ(s, S F ) is four times smaller than for ABC with uniform grid sampling. Table 2 provides time of computation on one core using the MCMC sampling algorithm. We also implemented the ABC-MCMC algorithm in parallel using separate Markov Chain for each processor. Model F Computational time # sampled # accepted Order # parameters iteration/s time on 1 core 1st order (94%) [it/s] 2[s] 2nd order (79%) 2 [it/s] 2 [h] Table 2: Time of computation on one core using MCMC sampling algorithm. 11

12 C S C mean max C 3 Figure 9: Marginal and joint probability density functions of accepted values of C S, C 2, and C 3 for the second order model in Eq. (11) at the test scale. Dashed green lines show the mean of each marginal pdf, and dashed red lines show the location of the MAP true modeled joint modeled dist pdf σ 11 σ 12 σ 13 Figure 1: Probability density functions of stresses σ F from the three-parameter second order model in Eq. (11) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). 12

13 3.3 Problems Some problems were noticed (see [13]) in MCMC algorithm described above. One of them is the fixed acceptance threshold value ɛ. The choice of ɛ is important as too large a tolerance interval results in a chain being dominated by the prior π(θ). On the other hand, too small a value leads to a very small acceptance rate. The other problem is the initializing step. To start the chain, the algorithm need to find the initial accepted parameter θ. Since the area of accepted parameters tends to be smaller as dimensionality grows, the time the initialization step takes can increase dramatically with the number of parameters. To solve these problems Wagman et al. [13] suggested an improved version of the MCMC algorithm. 4 Improved Markov Chain Monte Carlo 4.1 Improved MCMC algorithm The improved MCMC (IMCMC) algorithm proposed by Wagman et al. [13] has a calibration step. In this step it performs a series of n simulations, where parameters at each time are randomly sampled from their prior π(θ) to obtain p n (ρ θ, S F ), an approximation of the distances distribution π(ρ θ, S F ). Using this calibration step, we can define a tolerance level x and a threshold distance ɛ such that P (ρ < ɛ) = x. We should then be able to use any simulation for which ρ < ɛ as a starting point for the Markov chain. These n simulations are also used to adjust the transition kernel q(θ θ ). In our case, q is set to be a gaussian distribution with standart deviation equal to the standard deviation of the retained parameters. We did not consider the statistics modification suggested by Wagman et al. [13], therefore we have a slightly simplified version of the algorithm provided in [11], that is 1. Perform n simulations with parameters θ randomly drawn from their priors π(θ ), and each time compute their associated statistics S F. 2. Fix tolerance level x and estimate threshold distance ɛ from p n (ρ θ, S F ) suth that P (ρ < ɛ) = x. Set the proposal range of the parameters for the transition kernel q(θ θ ) on the basis of the parameters variability among the xn retained simulations. 3. Start an MCMC chain of total length P from a position θ randomly chosen from the xn retained simulations. Set i =. 4. If now at θ, propose a move to θ according to a transition kernel q(θ θ ). Increment i.. Generate σ F using model F with parameters θ and evaluate the statistic S F using data from M training points. 6. If ρ(s, S F ) < ε, go to 7, and otherwise stay at θ and return to Calculate h = h(θ, θ ) = min ( 1, π(θ )q(θ ) θ) π(θ)q(θ θ ) Accept θ with probability h otherwise stay at θ, then return to If i < s go to 4. Other possible improvement is modification in parameter estimation. Wagman et al. [13] proposed to estimate parameters on a subsample of size t with the smallest distances ρ generated by the Markov chain and to perform a 13

14 local regression adjustment where samples are weighted by their associated ρ i -values. These steps can greatly improve the quality of the estimation. This approach allows the chain to have a large acceptance rate, while making the final estimation not too sensitive to the prior due to the regression adjustment step. 4.2 Results First order model. The calibration step used n = 1 simulations and x =.1 and, as result, it set threshold value ɛ 1 (see Figure 11). The Markov chain used P = 9 steps and got the same parameter value as parameter recovered using uniform grid sampling in Section 2.4 and the MCMC algorithm in Section 3.2. More specifically, the estimated value is C S.216. Figure 12 shows the resulting pdfs of σ F and σ, revealing reasonable agreement between the modeled and true stresses at the LES scale. 2 2 i,j ρ(sf, S ) C S Figure 11: Accepted values for the C S parameter in the first order model and corresponding statistical distances. 1 1 true modeled dist 1 1 pdf σ 11 σ 12 σ 13 Figure 12: Probability density functions of stresses σ F from the one-parameter first order model in Eq. (1) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). Second order model with 3 parameters. The calibration step used n = 1 4 simulations with tolerance level x =.1 and, as result, it set threshold value ɛ 21. The Markov chain used P = 1 steps and all of them were accepted. Therefore, 1/11 = 9% of all evaluations of modeled σ, statistics S F and distance ρ led to accepted parameters, which makes it more efficient then even MCMC. 14

15 F, S ) depends on each of three parameters C, C and C. Using the values Figure 13 shows how the distance ρ(s 1 2 S of Ci that give the minimum distance in the statistics, the second order model parameters at the test scale are found to be Cs =.183, C2 =.86, C3 =.29. which is close to the parameters estimated earlier. The marginal posterior pdfs are shown on Figure 14. The MAP values of marginal pdfs are CS =.217, C2 =.79, C3 =.22. We can see on Figure 14 that the marginal pdf of C2 is not symmetric. This could happen because of too small a standard deviation in the kernel transition for the second parameter, so the algorithm could not find the second maximum.however this fact did not influence our resulting pdfs because, as was seen in the previous results, the sign of C2 is of no importance compared to the magnitude. F at the LES scale shown in Figure 1. These values can then be applied at the LES scale to give the pdfs of σ Figure 1 also shows the pdf resulting from the parameters estimated as the MAP of the posterior pdf, Cs =.21, C2 =.67, C3 =.27. We can see that these estimated parameters give a better agreement to the true stresses calculated from the DNS data CS.3 1 P i,j 1 P i,j 1 P i,j 1 F ρ(s, S ) 2 F ρ(s, S ) 2 F ρ(s, S ) 2.1. C C3.1 Figure 13: Accepted values for the CS, C1, C2 parameters in the second order model and corresponding statistical distances. 1

16 C S C mean max C 3 Figure 14: Marginal and joint probability density functions of accepted values of C S, C 2, and C 3 for the second order model in Eq. (11) at the test scale. Dashed green lines show the mean of each marginal pdf, and dashed red lines show the location of the MAP true modeled joint modeled dist pdf σ 11 σ 12 σ 13 Figure 1: Probability density functions of stresses σ F from the three-parameter second order model in Eq. (11) at the LES scale (green lines) and the true LES scale stresses σ from the DNS (red lines). 16

17 Summary, conclusion, and future plans.in this project we implemented the ABC-MCMC algorithm, which reduced the number of function evaluations and the time of computation by a factor of four without introducing any additional bias in the estimated parameters. We also implemented an improved MCMC algorithm, which allows one to automatically adjust the initial treshold ɛ and transitional kernel q(θ θ ). This improvement reduced the number of function evaluations even more and improved the agreement between the truth and modeled pdfs. Moreover, both algorithms were validated using the results of ABC with sampling from a uniform grid. In the future these two algorithms can be considered more deeply. We can adjust the algorithms to better find both peaks of the pdf of the second parameter, find the best acceptance level for x, and to find the number of simulations needed in calibration step for n. We also can improve the estimation technique using a subsample of size t with the smallest distances ρ generated by the Markov chain. References [1] King, R. N., Hamlington, P. E., and Dahm, W. J., Autonomic closure for turbulence simulations, Physical Review E, Vol. 93, No. 3, 216, pp [2] Sunnaker, M., Busetto, A. G., Numminen, E., Corander, J., Foll, M., and Dessimoz, C., Approximate Bayesian Computation, PLoS Computational Biology, Vol. 9, No. 1, Jan. 213, pp. e1283. [3] Meneveau, C. and Katz, J., Scale-invariance and turbulence models for large-eddy simulation, Annu. Rev. Fluid Mech., Vol. 32, 2, pp [4] Pope, S. B., A more general effective-viscosity hypothesis, Journal of Fluid Mechanics, Vol. 72, No. 2, 197, pp [] Germano, M., Piomelli, U., Moin, P., and Cabot, W. H., A dynamic subgridscale eddy viscosity model, Phys. Fluids A, Vol. 3, 1991, pp [6] Lilly, D. K., A proposed modification of the Germano subgrid-scale closure method, Phys. Fluids A, Vol. 4, No. 3, 1992, pp [7] Bardina, J., Ferziger, J., and Reynolds, W. C., Improved subgrid scale models for Large Eddy Simulation, AIAA paper, Vol. No , 198. [8] Li, Y., Perlman, E., Wan, M., Yang, Y., Meneveau, C., Burns, R., Chen, S., Szalay, A., and Eyink, G., A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence, Journal of Turbulence,, No. 9, 28, pp. N31. [9] Speziale, C. G., Comments on the material frame-indifferencecontroversy, Physical Review A, Vol. 36, No. 9, 1987, pp [1] Tarantola, A., Inverse problem theory and methods for model parameter estimation, SIAM, 2. [11] Marjoram, P., Molitor, J., Plagnol, V., and Tavar, S., Markov chain Monte Carlo without likelihoods, Proceedings of the National Academy of Sciences, Vol. 1, No. 26, 23, pp [12] Sadegh, M. and Vrugt, J. A., Approximate bayesian computation using markov chain monte carlo simulation: Dream (abc), Water Resources Research, Vol., No. 8, 214, pp

18 [13] Wegmann, D., Leuenberger, C., and Excoffier, L., Efficient approximate Bayesian computation coupled with Markov chain Monte Carlo without likelihood, Genetics, Vol. 182, No. 4, 29, pp [14] Beaumont, M. A., Cornuet, J.-M., Marin, J.-M., and Robert, C. P., Adaptive approximate Bayesian computation, Biometrika, Vol. 96, No. 4, 29, pp [1] Ishida, E., Vitenti, S., Penna-Lima, M., Cisewski, J., de Souza, R., Trindade, A., Cameron, E., Busti, V., collaboration, C., et al., cosmoabc: likelihood-free inference via population Monte Carlo approximate Bayesian computation, Astronomy and Computing, Vol. 13, 21, pp