Challenges in Uncertainty Quantification in Computational Models

Size: px
Start display at page:

Download "Challenges in Uncertainty Quantification in Computational Models"

Transcription

1 Introduction UQ Basics Challenges Closure Challenges in Uncertaint Quantification in Computational Models Habib N. Najm Sandia National Laboratories, Livermore, CA Computational and Theoretical Challenges in Interdisciplinar Predictive Modeling Over Random Fields 12th Annual Red Raider Mini-Smposium Department of Mathematics and Statistics Teas Tech Universit, Lubbock, TX October 26, 2012 SNL Najm Challenges in UQ 1 / 52

2 Introduction UQ Basics Challenges Closure Acknowledgement B.J. Debusschere, R.D. Berr, K. Sargsan, C. Safta, K. Chowdhar Sandia National Laboratories, CA R.G. Ghanem U. South. California, Los Angeles, CA O.M. Knio Duke Univ., Durham, NC O.P. Le Maître CNRS, Paris, France Y.M. Marzouk Mass. Inst. of Tech., Cambridge, MA This work was supported b: DOE Office of Basic Energ Sciences, Div. of Chem. Sci., Geosci., & Biosci. DOE Office of Advanced Scientific Computing Research (ASCR), Scientific Discover through Advanced Computing (SciDAC) DOE ASCR Applied Mathematics program American Recover and Reinvesment Act. DOE Office of Biological and Environmental Research Sandia National Laboratories is a multiprogram laborator operated b Sandia Corporation, a Lockheed Martin Compan, for the United States Department of Energ under contract DE-AC04-94-AL SNL Najm Challenges in UQ 2 / 52

3 Introduction UQ Basics Challenges Closure Outline 1 Introduction 2 Uncertaint Quantification Basics Forward Uncertaint Propagation Statistical Inverse Problems 3 Uncertaint Quantification Challenges a Selection Characterization of Uncertain Inputs High-dimensionalit Discontinuities Oscillator Dnamics 4 Closure SNL Najm Challenges in UQ 3 / 52

4 Introduction UQ Basics Challenges Closure The Case for Uncertaint Quantification (UQ) UQ is needed in: Assessment of confidence in computational predictions Validation and comparison of scientific/engineering models Design optimization Use of computational predictions for decision-support Assimilation of observational data and model construction Multiscale and multiphsics model coupling SNL Najm Challenges in UQ 4 / 52

5 Introduction UQ Basics Challenges Closure Overview of UQ Methods Estimation of model/parametric uncertaint Epert opinion, data collection Regression analsis, fitting, parameter estimation Baesian inference of uncertain models/parameters Forward propagation of uncertaint in models Local sensitivit analsis (SA) and error propagation Fuzz logic; Evidence theor interval math Probabilistic framework Global SA / stochastic UQ Random sampling, statistical methods Galerkin methods Polnomial Chaos (PC) intrusive/non-intrusive Collocation, interpolants, regression, fitting... PC/other SNL Najm Challenges in UQ 5 / 52

6 Introduction UQ Basics Challenges Closure Forward Inverse Probabilistic Forward UQ & Polnomial Chaos Representation of Random Variables With = f(), a random variable, estimate the RV Can describe a RV in terms of its densit, moments, characteristic function, or most fundamentall as a function on a probabilit space Constraining the analsis to RVs with finite variance, enables the representation of a RV as a spectral epansion in terms of orthogonal functions of standard RVs. Polnomial Chaos Enables the use of available functional analsis methods for forward UQ SNL Najm Challenges in UQ 6 / 52

7 Introduction UQ Basics Challenges Closure Forward Inverse Polnomial Chaos Epansion (PCE) Model uncertain quantities as random variables (RVs) Given a germ ξ(ω) = {ξ 1,,ξ n } a set of i.i.d. RVs where p(ξ) is uniquel determined b its moments An RV in L 2 (Ω,S(ξ), P) can be written as a PCE: u(, t,ω) = f(, t,ξ) P u k (, t)ψ k (ξ(ω)) k=0 u k (, t) are mode strengths Ψ k () are functions orthogonal w.r.t. p(ξ) With dimension n and order p: P+1 = (n+p)! n!p! SNL Najm Challenges in UQ 7 / 52

8 Introduction UQ Basics Challenges Closure Forward Inverse Orthogonalit B construction, the functions Ψ k () are orthogonal with respect to the densit of ξ u k (, t) = uψ k Ψ 2 k = 1 Ψ 2 k u(, t;λ(ξ))ψ k (ξ)p ξ (ξ)dξ Eamples: Hermite polnomials with Gaussian basis Legendre polnomials with Uniform basis,... Global versus Local PC methods Adaptive domain decomposition of the support of ξ SNL Najm Challenges in UQ 8 / 52

9 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV Lognormal; WH PC order = 1 Wiener-Hermite PCE constructed for a Lognormal RV PCE-sampled PDF superposed on true PDF Order = P u = u k Ψ k (ξ) k=0 = u 0 + u 1 ξ SNL Najm Challenges in UQ 9 / 52

10 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV Lognormal; WH PC order = 2 Wiener-Hermite PCE constructed for a Lognormal RV PCE-sampled PDF superposed on true PDF Order = P u = u k Ψ k (ξ) k=0 = u 0 + u 1 ξ + u 2 (ξ 2 1) SNL Najm Challenges in UQ 9 / 52

11 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV Lognormal; WH PC order = 3 Wiener-Hermite PCE constructed for a Lognormal RV PCE-sampled PDF superposed on true PDF Order = P u = u k Ψ k (ξ) k=0 = u 0 + u 1 ξ + u 2 (ξ 2 1)+u 3 (ξ 3 3ξ) SNL Najm Challenges in UQ 9 / 52

12 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV Lognormal; WH PC order = 4 Wiener-Hermite PCE constructed for a Lognormal RV PCE-sampled PDF superposed on true PDF Order = P u = u k Ψ k (ξ) k= = u 0 + u 1 ξ + u 2 (ξ 2 1)+u 3 (ξ 3 3ξ)+u 4 (ξ 4 6ξ 2 + 3) SNL Najm Challenges in UQ 9 / 52

13 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV Lognormal; WH PC order = 5 Wiener-Hermite PCE constructed for a Lognormal RV PCE-sampled PDF superposed on true PDF Order = P u = u k Ψ k (ξ) k= = u 0 + u 1 ξ + u 2 (ξ 2 1)+u 3 (ξ 3 3ξ)+u 4 (ξ 4 6ξ 2 + 3) + u 5 (ξ 5 10ξ ξ) SNL Najm Challenges in UQ 9 / 52

14 Introduction UQ Basics Challenges Closure Forward Inverse PC Illustration: WH PCE for a Lognormal RV 1.4 Lognormal RV WH PC order WH PC order Lognormal RV PC mode amplitudes u 0 u PC function u(ξ) Order 1 5 Fifth-order Wiener-Hermite PCE represents the given Lognormal well Higher order terms are negligible SNL Najm Challenges in UQ 10 / 52

15 Introduction UQ Basics Challenges Closure Forward Inverse Random Fields KLE Smooth random fields can be repesented with a small no. of stochastic degrees of freedom Karhunen-Loeve Epansion (KLE) for a RF with a continuous covariance function M(,ω) = µ()+ λi η i (ω)φ i () µ() is the mean of M(,ω) at λ i and φ i () are the eigenvalues and eigenfunctions of the covariance C( 1, 2 ) = [M( 1,ω) µ( 1 )][M( 2,ω) µ( 2 )] The η i are uncorrelated zero-mean unit-variance RVs i=1 KLE representation of random fields using PC SNL Najm Challenges in UQ 11 / 52

16 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: KL of 2D Gaussian Process δ = 0.1 δ = δ = 0.5 2D Gaussian Process with covariance: Cov( 1, 2 ) = ep( /δ 2 ) Realizations smoother as covariance length δ increases SNL Najm Challenges in UQ 12 / 52

17 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - Modes for δ = 0.1 λ1 φ 1 λ2 φ 2 λ3 φ 3 λ4 φ 4 λ5 φ 5 λ6 φ 6 λ7 φ 7 λ8 φ 8 SNL Najm Challenges in UQ 13 / 52

18 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - Modes for δ = λ1 φ 1 λ2 φ 2 λ3 φ 3 λ4 φ 4 λ5 φ 5 λ6 φ 6 λ7 φ 7 λ8 φ 8 SNL Najm Challenges in UQ 14 / 52

19 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - Modes for δ = 0.5 λ1 φ 1 λ2 φ 2 λ3 φ 3 λ4 φ 4 λ5 φ 5 λ6 φ 6 λ7 φ 7 λ8 φ 8 SNL Najm Challenges in UQ 15 / 52

20 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - eigenvalue spectrum δ = terms 16 terms Eigenvalue Magnitude δ =0.1 δ = δ = Eigenvalue # 32 terms 64 terms SNL Najm Challenges in UQ 16 / 52

21 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - eigenvalue spectrum δ = 4 terms 16 terms Eigenvalue Magnitude δ =0.1 δ = δ = Eigenvalue # 32 terms 64 terms SNL Najm Challenges in UQ 17 / 52

22 Introduction UQ Basics Challenges Closure Forward Inverse RF Illustration: 2D KL - eigenvalue spectrum δ = terms 16 terms Eigenvalue Magnitude δ =0.1 δ = δ = Eigenvalue # 32 terms 64 terms SNL Najm Challenges in UQ 18 / 52

23 Introduction UQ Basics Challenges Closure Forward Inverse Essential Use of PC in UQ Strateg: Represent model parameters/solution as random variables Construct PCEs for uncertain parameters Evaluate PCEs for model outputs Advantages: Computational efficienc Sensitivit information Requirement: Random variables in L 2, i.e. with finite variance SNL Najm Challenges in UQ 19 / 52

24 Introduction UQ Basics Challenges Closure Forward Inverse Intrusive PC UQ: A direct non-sampling method Given model equations: M(u(, t);λ) = 0 Epress uncertain parameters/variables using PCEs P P u = u k Ψ k ; λ = λ k Ψ k k=0 k=0 Substitute in model equations; appl Galerkin projection G(U(, t),λ) = 0 New set of equations: with U = [u 0,...,u P ] T, Λ = [λ 0,...,λ P ] T Solving this sstem once provides the full specification of uncertain model ouputs SNL Najm Challenges in UQ 20 / 52

25 Introduction UQ Basics Challenges Closure Forward Inverse Laminar 2D Channel Flow with Uncertain Viscosit Incompressible flow Viscosit PCE ν = ν 0 +ν 1 ξ Streamwise velocit P v = v i Ψ i i=0 v 0 : mean v i : i-th order mode P σ 2 = v 2 v0 v1 v2 v3 sd i Ψ 2 i v 0 v 1 v 2 v 3 σ i=1 SNL Najm Challenges in UQ 21 / 52

26 Introduction UQ Basics Challenges Closure Forward Inverse Non-intrusive Spectral Projection (NISP) PC UQ Sampling-based Relies on black-bo utilization of the computational model Evaluate projection integrals numericall For an quantit of interest φ(, t;λ) = P k=0 φ k(, t)ψ k (ξ) φ k (, t) = 1 φ(, t;λ(ξ))ψ Ψ 2 k (ξ)p ξ (ξ)dξ, k = 0,..., P k Integrals can be evaluated using A variet of (Quasi) Monte Carlo methods Slow convergence; indep. of dimensionalit Quadrature/Sparse-Quadrature methods Fast convergence; depends on dimensionalit SNL Najm Challenges in UQ 22 / 52

27 Introduction UQ Basics Challenges Closure Forward Inverse 1D H 2 -O 2 SCWO Flame NISP UQ/Chemkin-Premi σ σ 20 Rns 1,6 All modes Rns 5,8,1,6 Y OH 5 σ (OH) 10 Rns 8,1,6 All 1st order (cm) (cm) Fast growth in OH uncertaint in the primar reaction zone Constant uncertaint and mean of OH in post-flame region Uncertaint in pre-eponential of Rn.5 (H 2O 2+OH=H 2O+HO 2) has largest contribution to uncertaint in predicted OH SNL Najm Challenges in UQ 23 / 52

28 Introduction UQ Basics Challenges Closure Forward Inverse Other non-intrusive methods Response surface emploing PC or other functional basis Collocation: Fit interpolant to samples Oscillation concern Regression: Estimate best-fit response surface Least-squares Baesian inference Useful when quadrature methods are infeasible, e.g. when Can t choose sample locations; samples given a priori Can t take enough samples Forward model is nois SNL Najm Challenges in UQ 24 / 52

29 Introduction UQ Basics Challenges Closure Forward Inverse Inverse UQ Estimation of Uncertain Inputs Forward UQ requires specification of uncertain inputs Probabilistic setting Require joint PDF on input space Baesian setting PDF on uncertain inputs can be found given data Probabilistic inference an inverse problem Uncertaint in computational predictions can depend strongl on detailed structure of the parametric PDF SNL Najm Challenges in UQ 25 / 52

30 Introduction UQ Basics Challenges Closure Forward Inverse The strong role of detailed input PDF structure Simple nonlinear algebraic model (u, v) = ( 2 2, 2) Two input PDFs, p(, ) same nominals/bounds different correlation structure Drasticall different output PDFs different nominals and bounds SNL Najm Challenges in UQ 26 / 52

31 Introduction UQ Basics Challenges Closure Forward Inverse Baes formula for Parameter Inference Data Model (fit model + noise model): Baes Formula: p(λ, ) = p(λ )p() = p( λ)p(λ) p(λ ) Posterior = Likelihood p( λ) Prior: knowledge of λ prior to data p() Evidence Prior p(λ) = f(λ) g(ǫ) Likelihood: forward model and measurement noise Posterior: combines information from prior and data Evidence: normalizing constant for present contet SNL Najm Challenges in UQ 27 / 52

32 Introduction UQ Basics Challenges Closure Forward Inverse Eploring the Posterior Given an sample λ, the un-normalized posterior probabilit can be easil computed p(λ ) p( λ)p(λ) Eplore posterior w/ Markov Chain Monte Carlo (MCMC) Metropolis-Hastings algorithm: Random walk with proposal PDF & rejection rules Computationall intensive, O(10 5 ) samples Each sample: evaluation of the forward model Surrogate models Evaluate moments/marginals from the MCMC statistics SNL Najm Challenges in UQ 28 / 52

33 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc UQ Challenges - Characterization of Uncertain Inputs Computational Model M(u, λ) = 0 Uncertain input parameter λ Eperimental Measurement F(, λ) = 0 Uncertain model inputs can be estimated from data on Regression Baesian inference Quite frequentl, we have partial data/information Partial missing data, e.g. failed measurements Full data loss No data, but have summar information, e.g. moments and/or quantiles on data processed data products fitted parameters SNL Najm Challenges in UQ 29 / 52

34 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Dealing with Partial Data Imputation Baesian Multiple Imputation (Rubin, 1987) Use the posterior predictive conditioned on observed data to generate replicates of the missing data z t mis, t = 1,..., m For each full data set, (z obs, z t mis ) appl Baesian inference to get a posterior densit on the model parameters Marginalize over the missing data to get the observed data posterior p(β z obs ) p(β z obs ) = q(β z mis, z obs )f(z mis z obs )dz mis SNL Najm Challenges in UQ 30 / 52

35 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Parameter Estimation in the Absence of Data Frequentl: we know summar statistics about data or parameters from previous work the raw data used to arrive at these statistics is not available How can we construct a joint PDF on the parameters? In the absence of data, the structure of the fit model, combined with the summar statistics, implicitl inform the joint PDF on the parameters Goal: Make available information eplicit in the joint PDF Data Free Inference (DFI) Discover a consensus joint PDF on the parameters consistent with given information in the absence of data SNL Najm Challenges in UQ 31 / 52

36 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Generate ignition "data" using a detailed model+noise Ignition using a detailed chemical model for methane-air chemistr Ignition time versus Initial Temperature Multiplicative noise error model 11 data points: Ignition time (sec) GRI+noise GRI d i = tig,i GRI (1+σǫ i ) ǫ N(0, 1) Initial Temperature (K) SNL Najm Challenges in UQ 32 / 52

37 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Fitting with a simple chemical model Fit a global single-step irreversible chemical model CH 4 + 2O 2 CO 2 + 2H 2 O R = [CH 4 ][O 2 ]k f k f = A ep( E/R o T) Infer 3-D parameter vector (ln A, ln E, lnσ) Use an adaptive MCMC procedure Start at the maimum likelihood estimate SNL Najm Challenges in UQ 33 / 52

38 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Baesian Inference Posterior and Nominal Prediction Ignition time (sec) GRI+noise GRI GRI GRI+noise Fit Model Marginal joint posterior on (ln A, ln E) ehibits strong correlation Initial Temperature (K) Nominal fit model is consistent with the true model SNL Najm Challenges in UQ 34 / 52

39 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc DFI Challenge Chemical Model Problem Discarding initial data, reconstruct marginal (ln A, ln E) posterior using the following information Form of fit model Range of initial temperature Nominal fit parameter values of ln A and ln E Marginal 5% and 95% quantiles on ln A and ln E Further, for now, presume Multiplicative Gaussian errors N = 8 data points SNL Najm Challenges in UQ 35 / 52

40 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc DFI Algorithm Structure Basic idea: Eplore the space of hpothetical data sets MCMC chain on the data Each state defines a data set For each data set: MCMC chain on the parameters Evaluate statistics on resulting posterior Accept data set if posterior is consistent with given information Evaluate pooled posterior from all acceptable posteriors Logarithmic pooling: p(λ ) = [ K ] 1/K p(λ i ) i=1 SNL Najm Challenges in UQ 36 / 52

41 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc 2D Marginal Pooled Posteriors vs. Reference Posterior ln(e) ln(e) ln(e) ln(a) ln(a) ln(a) ln(e) ln(e) ln(e) ln(a) ln(a) ln(a) SNL Najm Challenges in UQ 37 / 52

42 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Empirical Convergence of 3D Pooled Posteriors Kullback-Leibler divergence between posteriors of successivel increased data volumes Combinatorial choices of chains pooled at each stage statistical scatter of KLdiv KLdiv Overall convergence evident 1/N c 2-4c 4-8c 8-16c 16-32c SNL Najm Challenges in UQ 38 / 52

43 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Challenges in PC UQ High-Dimensionalit Dimensionalit n of the PC basis: ξ = {ξ 1,...,ξ n } number of degrees of freedom P+1 = (n+p)!/n!p! grows fast with n Impacts: Size of intrusive sstem # non-intrusive (sparse) quadrature samples Generall n number of uncertain parameters Reduction of n: Sensitivit analsis Dependencies/correlations among parameters Dominant eigenmodes of random fields Manifold learning: Isomap, Diffusion maps Sparsification: Compressed Sensing, LASSO SNL Najm Challenges in UQ 39 / 52

44 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc PC Quadrature in hid Full quadrature: N = (N 1D ) n Sparse Quadrature Wide range of methods Nested & hierarchical Clenshaw-Curtis: N = O(n p ) Adaptive greed algorithms Number of points can still be ecessive in hi-d Large no. of terms Reduction/sparsit Number 1e+07 1e+06 1e D Surrogate No. of Sparse Quadrature Points No. of PC Terms PC Order SNL Najm Challenges in UQ 40 / 52

45 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc PC coefficients via sparse regression PCE: K 1 = f() c k Ψ k () k=0 with R n, Ψ k ma order p, and K = (p+n)!/p!/n! N samples ( 1, 1 ),...,( N, N ) Estimate K terms c 0,..., c K 1, s.t. min Ac 2 2 where R N, c R K, A ik = Ψ k ( i ), A R N K With N << K under-determined Need some form of regularization SNL Najm Challenges in UQ 41 / 52

46 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Regularization Compressive Sensing (CS) l 2 -norm Tikhonov regularization; Ridge regression: min { Ac c 2 2} l 1 -norm Compressive Sensing; LASSO; basis pursuit min { Ac c 1} min { Ac 2 2 } subject to c 1 ǫ min { c 1 } subject to Ac 2 2 ǫ discover of sparse signals SNL Najm Challenges in UQ 42 / 52

47 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Baesian Regression Baes formula p(c D) p(d c)π(c) Baesian regression: prior as a regularizer, e.g. Log Likelihood Ac 2 2 Log Prior c p p Laplace sparsit priors π(c k α) = 1 2α e c k /α LASSO (Tibshirani 1996)... formall: min { Ac 2 2 +λ c 1 } Solution the posterior mode of c in the Baesian model N(Ac, I N ), c k 1 2α e c k /α Baesian LASSO (Park & Casella 2008) SNL Najm Challenges in UQ 43 / 52

48 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Baesian Compressive Sensing (BCS) BCS (Ji 2008; Babacan 2010) hierarchical priors: Gaussian priors N(0,σ 2 k ) on the c k Gamma priors on the σ 2 k Laplace sparsit priors on the c k Evidence maimization establishes ML estimates of the σ k man of which are found 0 c k 0 iterativel include terms that lead to the largest increase in the evidence iterative BCS (ibcs) (Sargsan 2012): adaptive iterative order growth BCS on order-p Legendre-Uniform PC repeat with order-p+1 terms added to surviving p-th order terms SNL Najm Challenges in UQ 44 / 52

49 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc CS and BCS Oscillator Genz function f() = cos(2πr + n i=1 a i i ); a i 1/i 2 ; r = 0 Legendre-Uniform PC, 10 th -order in 5d; (5, 6) th -order in 10d Baesian CS l 1 minimization BCS 5 th order CS 5 th order BCS 6 th order L 2 error 1 L 2 error Number of measurements 5d Number of measurements 10d SNL Najm Challenges in UQ 45 / 52

50 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Oscillator function BCS number of terms (n,p)=(10,6) (n,p)={(5,10),(10,5)} 3003 Number of Terms dim=5, ord=10 dim=10, ord=5 dim=10, ord= Number of measurements SNL Najm Challenges in UQ 46 / 52

51 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Challenges in PC UQ Non-Linearit Bifurcative response at critical parameter values Raleigh-Bénard convection Transition to turbulence Chemical ignition Discontinuous u(λ(ξ)) Failure of global PCEs in terms of smooth Ψ k () failure of Fourier series in representing a step function Local PC methods Subdivide support of λ(ξ) into regions of smooth u λ(ξ) Emplo PC with compact support basis on each region A spectral-element vs. spectral construction Domain mapping SNL Najm Challenges in UQ 47 / 52

52 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Multi-Block Multiwavelet PC UQ in Ignition 2.0e 12 OH, MRA, full F_1 F_7 Mean +/ 1σ envelope OH Concentration (mole/cm 3 sec) 1.5e 12 e e 13 e time (sec) H 2 -O 2 supercritical water oidation model Empiricall-based uncertaint in all 7 reactions Adaptive refinement of MW block decomposition (Le Maître, 2004, 2007) Challenges in hi-dimensional contets SNL Najm Challenges in UQ 48 / 52

53 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Uncertaint in Discontinuous Climate Response Climate sensitivit [K] Recover No Recover Rate of CO 2 increase [%] Atlantic meridional ocean circulation (AMOC) Predicted response to increasing CO 2 (Webster, 2007) Circulation ON/OFF response over parameter space Rate of CO 2 increase Climate sensitivit SNL Najm Challenges in UQ 49 / 52

54 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Domain Mapping for Discontinuous Response (c) Input λ Computational Model PC Epansion Input λ MOC ON pdf(z) 4 MOC OFF η η 1 η η 1 Initial set of computational samples z Discover uncertain discontinuit with Baesian inference Map sub-domains to unit hpercubes; Rosenblatt transform PC quadrature in mapped domains; map back Marginalize over uncertain curve (Sargsan, 2012) SNL Najm Challenges in UQ 50 / 52

55 Introduction UQ Basics Challenges Closure Characterization hi-d Discont Osc Challenges in PC UQ Time Dnamics Sstems with limit-ccle or chaotic dnamics Large amplification of phase errors over long time horizon PC order needs to be increased in time to retain accurac Time shifting/scaling remedies Futile to attempt representation of detailed turbulent velocit field v(, t;λ(ξ)) as a PCE Fast loss of correlation due to energ cascade Problem studied in 60 s and 70 s Focus on flow statistics, e.g. Mean/RMS quantities Well behaved Argues for non-intrusive methods with DNS/LES of turbulent flow SNL Najm Challenges in UQ 51 / 52

56 Introduction UQ Basics Challenges Closure Closure Probabilistic UQ framework Forward UQ Polnomial Chaos representation of random variables Intrusive and non-intrusive forward PC UQ methods Inverse UQ Baesian methods Highlighted UQ Challenges Missing data Imputation / DFI High dimensionalit Non-linearit Long term oscillator dnamics SNL Najm Challenges in UQ 52 / 52

Uncertainty Quantification in Computational Models

Uncertainty Quantification in Computational Models Uncertainty Quantification in Computational Models Habib N. Najm Sandia National Laboratories, Livermore, CA, USA Workshop on Understanding Climate Change from Data (UCC11) University of Minnesota, Minneapolis,

More information

Uncertainty Quantification

Uncertainty Quantification Uncertaint Quantification in Computational Models Habib N. Najm hnnajm@sandia.gov Sandia National Laboratories Livermore, CA, USA Computational Infrastructure for Geodnamics (CIG) Webinar, UC Davis, CA

More information

Stochastic Spectral Approaches to Bayesian Inference

Stochastic Spectral Approaches to Bayesian Inference Stochastic Spectral Approaches to Bayesian Inference Prof. Nathan L. Gibson Department of Mathematics Applied Mathematics and Computation Seminar March 4, 2011 Prof. Gibson (OSU) Spectral Approaches to

More information

Intro BCS/Low Rank Model Inference/Comparison Summary References. UQTk. A Flexible Python/C++ Toolkit for Uncertainty Quantification

Intro BCS/Low Rank Model Inference/Comparison Summary References. UQTk. A Flexible Python/C++ Toolkit for Uncertainty Quantification A Flexible Python/C++ Toolkit for Uncertainty Quantification Bert Debusschere, Khachik Sargsyan, Cosmin Safta, Prashant Rai, Kenny Chowdhary bjdebus@sandia.gov Sandia National Laboratories, Livermore,

More information

Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications

Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Dongbin Xiu Department of Mathematics, Purdue University Support: AFOSR FA955-8-1-353 (Computational Math) SF CAREER DMS-64535

More information

Stochastic Dimension Reduction

Stochastic Dimension Reduction Stochastic Dimension Reduction Roger Ghanem University of Southern California Los Angeles, CA, USA Computational and Theoretical Challenges in Interdisciplinary Predictive Modeling Over Random Fields 12th

More information

PC EXPANSION FOR GLOBAL SENSITIVITY ANALYSIS OF NON-SMOOTH FUNCTIONALS OF UNCERTAIN STOCHASTIC DIFFERENTIAL EQUATIONS SOLUTIONS

PC EXPANSION FOR GLOBAL SENSITIVITY ANALYSIS OF NON-SMOOTH FUNCTIONALS OF UNCERTAIN STOCHASTIC DIFFERENTIAL EQUATIONS SOLUTIONS PC EXPANSION FOR GLOBAL SENSITIVITY ANALYSIS OF NON-SMOOTH FUNCTIONALS OF UNCERTAIN STOCHASTIC DIFFERENTIAL EQUATIONS SOLUTIONS M. Navarro, O.P. Le Maître,2, O.M. Knio,3 mariaisabel.navarrojimenez@kaust.edu.sa

More information

Predictability of Chemical Systems

Predictability of Chemical Systems Predictability of Chemical Systems Habib N. Najm Sandia National Laboratories, Livermore, CA, USA Collaborators: M.T. Reagan & B.J. Debusschere O.M. Knio, A. Matta, & R.G. Ghanem O.P. Le Maitre Sandia

More information

6. Vector Random Variables

6. Vector Random Variables 6. Vector Random Variables In the previous chapter we presented methods for dealing with two random variables. In this chapter we etend these methods to the case of n random variables in the following

More information

Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems

Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Krzysztof Fidkowski, David Galbally*, Karen Willcox* (*MIT) Computational Aerospace Sciences Seminar Aerospace Engineering

More information

Polynomial Chaos and Karhunen-Loeve Expansion

Polynomial Chaos and Karhunen-Loeve Expansion Polynomial Chaos and Karhunen-Loeve Expansion 1) Random Variables Consider a system that is modeled by R = M(x, t, X) where X is a random variable. We are interested in determining the probability of the

More information

SENSITIVITY ANALYSIS IN NUMERICAL SIMULATION OF MULTIPHASE FLOW FOR CO 2 STORAGE IN SALINE AQUIFERS USING THE PROBABILISTIC COLLOCATION APPROACH

SENSITIVITY ANALYSIS IN NUMERICAL SIMULATION OF MULTIPHASE FLOW FOR CO 2 STORAGE IN SALINE AQUIFERS USING THE PROBABILISTIC COLLOCATION APPROACH XIX International Conference on Water Resources CMWR 2012 University of Illinois at Urbana-Champaign June 17-22,2012 SENSITIVITY ANALYSIS IN NUMERICAL SIMULATION OF MULTIPHASE FLOW FOR CO 2 STORAGE IN

More information

Uncertainty Propagation in Puff-based Dispersion Models using Polynomial Chaos

Uncertainty Propagation in Puff-based Dispersion Models using Polynomial Chaos Uncertaint Propagation in Puff-based Dispersion Models using Polnomial Chaos Umamaheswara Konda Dept. of MAE Universit at Buffalo venkatar@buffalo.edu Tarunraj Singh Dept. of MAE Universit at Buffalo tsingh@eng.buffalo.edu

More information

Perturbation Theory for Variational Inference

Perturbation Theory for Variational Inference Perturbation heor for Variational Inference Manfred Opper U Berlin Marco Fraccaro echnical Universit of Denmark Ulrich Paquet Apple Ale Susemihl U Berlin Ole Winther echnical Universit of Denmark Abstract

More information

Uncertainty Propagation in Puff-based Dispersion Models using Polynomial Chaos

Uncertainty Propagation in Puff-based Dispersion Models using Polynomial Chaos Uncertaint Propagation in Puff-based Dispersion Models using Polnomial Chaos Umamaheswara Konda Dept. of MAE Universit at Buffalo venkatar@buffalo.edu Tarunraj Singh Dept. of MAE Universit at Buffalo tsingh@eng.buffalo.edu

More information

Error Budgets: A Path from Uncertainty Quantification to Model Validation

Error Budgets: A Path from Uncertainty Quantification to Model Validation Error Budgets: A Path from Uncertainty Quantification to Model Validation Roger Ghanem Aerospace and Mechanical Engineering Civil Engineering University of Southern California Los Angeles Advanced Simulation

More information

Polynomial Chaos Based Uncertainty Propagation

Polynomial Chaos Based Uncertainty Propagation Polynomial Chaos Based Uncertainty Propagation Lecture 2: Forward Propagation: Intrusive and Non-Intrusive Bert Debusschere bjdebus@sandia.gov Sandia National Laboratories, Livermore, CA Aug 12, 2013 Summer

More information

Sparse polynomial chaos expansions in engineering applications

Sparse polynomial chaos expansions in engineering applications DEPARTMENT OF CIVIL, ENVIRONMENTAL AND GEOMATIC ENGINEERING CHAIR OF RISK, SAFETY & UNCERTAINTY QUANTIFICATION Sparse polynomial chaos expansions in engineering applications B. Sudret G. Blatman (EDF R&D,

More information

Polynomial chaos expansions for sensitivity analysis

Polynomial chaos expansions for sensitivity analysis c DEPARTMENT OF CIVIL, ENVIRONMENTAL AND GEOMATIC ENGINEERING CHAIR OF RISK, SAFETY & UNCERTAINTY QUANTIFICATION Polynomial chaos expansions for sensitivity analysis B. Sudret Chair of Risk, Safety & Uncertainty

More information

Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient

Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient F. Nobile, L. Tamellini, R. Tempone, F. Tesei CSQI - MATHICSE, EPFL, Switzerland Dipartimento di

More information

MCMC Sampling for Bayesian Inference using L1-type Priors

MCMC Sampling for Bayesian Inference using L1-type Priors MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling

More information

Introduction to Uncertainty Quantification in Computational Science Handout #3

Introduction to Uncertainty Quantification in Computational Science Handout #3 Introduction to Uncertainty Quantification in Computational Science Handout #3 Gianluca Iaccarino Department of Mechanical Engineering Stanford University June 29 - July 1, 2009 Scuola di Dottorato di

More information

A Stochastic Collocation based. for Data Assimilation

A Stochastic Collocation based. for Data Assimilation A Stochastic Collocation based Kalman Filter (SCKF) for Data Assimilation Lingzao Zeng and Dongxiao Zhang University of Southern California August 11, 2009 Los Angeles Outline Introduction SCKF Algorithm

More information

Utilizing Adjoint-Based Techniques to Improve the Accuracy and Reliability in Uncertainty Quantification

Utilizing Adjoint-Based Techniques to Improve the Accuracy and Reliability in Uncertainty Quantification Utilizing Adjoint-Based Techniques to Improve the Accuracy and Reliability in Uncertainty Quantification Tim Wildey Sandia National Laboratories Center for Computing Research (CCR) Collaborators: E. Cyr,

More information

Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems

Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems The MIT Faculty has made this article openly available. Please share how this access benefits you. Your

More information

Multiscale stochastic preconditioners in non-intrusive spectral projection

Multiscale stochastic preconditioners in non-intrusive spectral projection Multiscale stochastic preconditioners in non-intrusive spectral projection Alen Alenxanderian a, Olivier P. Le Maître b, Habib N. Najm c, Mohamed Iskandarani d, Omar M. Knio a, a Department of Mechanical

More information

Application and validation of polynomial chaos methods to quantify uncertainties in simulating the Gulf of Mexico circulation using HYCOM.

Application and validation of polynomial chaos methods to quantify uncertainties in simulating the Gulf of Mexico circulation using HYCOM. Application and validation of polynomial chaos methods to quantify uncertainties in simulating the Gulf of Mexico circulation using HYCOM. Mohamed Iskandarani Matthieu Le Hénaff Carlisle Thacker University

More information

Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs

Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs Roman Andreev ETH ZÜRICH / 29 JAN 29 TOC of the Talk Motivation & Set-Up Model Problem Stochastic Galerkin FEM Conclusions & Outlook Motivation

More information

Estimating functional uncertainty using polynomial chaos and adjoint equations

Estimating functional uncertainty using polynomial chaos and adjoint equations 0. Estimating functional uncertainty using polynomial chaos and adjoint equations February 24, 2011 1 Florida State University, Tallahassee, Florida, Usa 2 Moscow Institute of Physics and Technology, Moscow,

More information

Uncertainty Quantification in Multiscale Models

Uncertainty Quantification in Multiscale Models Uncertainty Quantification in Multiscale Models H.N. Najm Sandia National Laboratories Livermore, CA Workshop on Stochastic Multiscale Methods: Mathematical Analysis and Algorithms Univ. Southern California,

More information

Dinesh Kumar, Mehrdad Raisee and Chris Lacor

Dinesh Kumar, Mehrdad Raisee and Chris Lacor Dinesh Kumar, Mehrdad Raisee and Chris Lacor Fluid Mechanics and Thermodynamics Research Group Vrije Universiteit Brussel, BELGIUM dkumar@vub.ac.be; m_raisee@yahoo.com; chris.lacor@vub.ac.be October, 2014

More information

Evaluation of Non-Intrusive Approaches for Wiener-Askey Generalized Polynomial Chaos

Evaluation of Non-Intrusive Approaches for Wiener-Askey Generalized Polynomial Chaos 49th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference 6t 7 - April 28, Schaumburg, IL AIAA 28-892 Evaluation of Non-Intrusive Approaches for Wiener-Askey Generalized

More information

Gaussian Process Approximations of Stochastic Differential Equations

Gaussian Process Approximations of Stochastic Differential Equations Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Centre for Computational Statistics and Machine Learning University College London c.archambeau@cs.ucl.ac.uk CSML

More information

STA 4273H: Sta-s-cal Machine Learning

STA 4273H: Sta-s-cal Machine Learning STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our

More information

Collocation based high dimensional model representation for stochastic partial differential equations

Collocation based high dimensional model representation for stochastic partial differential equations Collocation based high dimensional model representation for stochastic partial differential equations S Adhikari 1 1 Swansea University, UK ECCM 2010: IV European Conference on Computational Mechanics,

More information

Monte Carlo integration

Monte Carlo integration Monte Carlo integration Eample of a Monte Carlo sampler in D: imagine a circle radius L/ within a square of LL. If points are randoml generated over the square, what s the probabilit to hit within circle?

More information

Uncertainty Quantification and Validation Using RAVEN. A. Alfonsi, C. Rabiti. Risk-Informed Safety Margin Characterization. https://lwrs.inl.

Uncertainty Quantification and Validation Using RAVEN. A. Alfonsi, C. Rabiti. Risk-Informed Safety Margin Characterization. https://lwrs.inl. Risk-Informed Safety Margin Characterization Uncertainty Quantification and Validation Using RAVEN https://lwrs.inl.gov A. Alfonsi, C. Rabiti North Carolina State University, Raleigh 06/28/2017 Assumptions

More information

Uncertainty Evolution In Stochastic Dynamic Models Using Polynomial Chaos

Uncertainty Evolution In Stochastic Dynamic Models Using Polynomial Chaos Noname manuscript No. (will be inserted by the editor) Uncertainty Evolution In Stochastic Dynamic Models Using Polynomial Chaos Umamaheswara Konda Puneet Singla Tarunraj Singh Peter Scott Received: date

More information

Solving the steady state diffusion equation with uncertainty Final Presentation

Solving the steady state diffusion equation with uncertainty Final Presentation Solving the steady state diffusion equation with uncertainty Final Presentation Virginia Forstall vhfors@gmail.com Advisor: Howard Elman elman@cs.umd.edu Department of Computer Science May 6, 2012 Problem

More information

Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems

Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems John Bardsley, University of Montana Collaborators: H. Haario, J. Kaipio, M. Laine, Y. Marzouk, A. Seppänen, A. Solonen, Z.

More information

Bayesian Inference. Sudhir Shankar Raman Translational Neuromodeling Unit, UZH & ETH. The Reverend Thomas Bayes ( )

Bayesian Inference. Sudhir Shankar Raman Translational Neuromodeling Unit, UZH & ETH. The Reverend Thomas Bayes ( ) The Reverend Thomas Baes (1702-1761) Baesian Inference Sudhir Shankar Raman Translational Neuromodeling Unit, UZH & ETH With man thanks for some slides to: Klaas Enno Stephan & Ka H. Brodersen Wh do I

More information

Uncertainty quantification for flow in highly heterogeneous porous media

Uncertainty quantification for flow in highly heterogeneous porous media 695 Uncertainty quantification for flow in highly heterogeneous porous media D. Xiu and D.M. Tartakovsky a a Theoretical Division, Los Alamos National Laboratory, Mathematical Modeling and Analysis Group

More information

STA414/2104 Statistical Methods for Machine Learning II

STA414/2104 Statistical Methods for Machine Learning II STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements

More information

Regression with Input-Dependent Noise: A Bayesian Treatment

Regression with Input-Dependent Noise: A Bayesian Treatment Regression with Input-Dependent oise: A Bayesian Treatment Christopher M. Bishop C.M.BishopGaston.ac.uk Cazhaow S. Qazaz qazazcsgaston.ac.uk eural Computing Research Group Aston University, Birmingham,

More information

Solving the stochastic steady-state diffusion problem using multigrid

Solving the stochastic steady-state diffusion problem using multigrid IMA Journal of Numerical Analysis (2007) 27, 675 688 doi:10.1093/imanum/drm006 Advance Access publication on April 9, 2007 Solving the stochastic steady-state diffusion problem using multigrid HOWARD ELMAN

More information

Sobol-Hoeffding Decomposition with Application to Global Sensitivity Analysis

Sobol-Hoeffding Decomposition with Application to Global Sensitivity Analysis Sobol-Hoeffding decomposition Application to Global SA Computation of the SI Sobol-Hoeffding Decomposition with Application to Global Sensitivity Analysis Olivier Le Maître with Colleague & Friend Omar

More information

Uncertainty Propagation in Puff-based Dispersion Models Using Polynomial Chaos

Uncertainty Propagation in Puff-based Dispersion Models Using Polynomial Chaos Uncertaint Propagation in Puff-based Dispersion Models Using Polnomial Chaos Abstract Atmospheric dispersion is a comple nonlinear phsical process with numerous uncertainties in model parameters, inputs,

More information

Uncertainty quantification for sparse solutions of random PDEs

Uncertainty quantification for sparse solutions of random PDEs Uncertainty quantification for sparse solutions of random PDEs L. Mathelin 1 K.A. Gallivan 2 1 LIMSI - CNRS Orsay, France 2 Mathematics Dpt., Florida State University Tallahassee, FL, USA SIAM 10 July

More information

Fault Detection and Diagnosis Using Information Measures

Fault Detection and Diagnosis Using Information Measures Fault Detection and Diagnosis Using Information Measures Rudolf Kulhavý Honewell Technolog Center & Institute of Information Theor and Automation Prague, Cech Republic Outline Probabilit-based inference

More information

Stochastic Spectral Methods for Uncertainty Quantification

Stochastic Spectral Methods for Uncertainty Quantification Stochastic Spectral Methods for Uncertainty Quantification Olivier Le Maître 1,2,3, Omar Knio 1,2 1- Duke University, Durham, North Carolina 2- KAUST, Saudi-Arabia 3- LIMSI CNRS, Orsay, France 38th Conf.

More information

On a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model

On a Data Assimilation Method coupling Kalman Filtering, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model On a Data Assimilation Method coupling, MCRE Concept and PGD Model Reduction for Real-Time Updating of Structural Mechanics Model 2016 SIAM Conference on Uncertainty Quantification Basile Marchand 1, Ludovic

More information

Probabilistic Structural Dynamics: Parametric vs. Nonparametric Approach

Probabilistic Structural Dynamics: Parametric vs. Nonparametric Approach Probabilistic Structural Dynamics: Parametric vs. Nonparametric Approach S Adhikari School of Engineering, Swansea University, Swansea, UK Email: S.Adhikari@swansea.ac.uk URL: http://engweb.swan.ac.uk/

More information

Quadrature for Uncertainty Analysis Stochastic Collocation. What does quadrature have to do with uncertainty?

Quadrature for Uncertainty Analysis Stochastic Collocation. What does quadrature have to do with uncertainty? Quadrature for Uncertainty Analysis Stochastic Collocation What does quadrature have to do with uncertainty? Quadrature for Uncertainty Analysis Stochastic Collocation What does quadrature have to do with

More information

Polynomial chaos expansions for structural reliability analysis

Polynomial chaos expansions for structural reliability analysis DEPARTMENT OF CIVIL, ENVIRONMENTAL AND GEOMATIC ENGINEERING CHAIR OF RISK, SAFETY & UNCERTAINTY QUANTIFICATION Polynomial chaos expansions for structural reliability analysis B. Sudret & S. Marelli Incl.

More information

Sequential Importance Sampling for Rare Event Estimation with Computer Experiments

Sequential Importance Sampling for Rare Event Estimation with Computer Experiments Sequential Importance Sampling for Rare Event Estimation with Computer Experiments Brian Williams and Rick Picard LA-UR-12-22467 Statistical Sciences Group, Los Alamos National Laboratory Abstract Importance

More information

Fast Numerical Methods for Stochastic Computations

Fast Numerical Methods for Stochastic Computations Fast AreviewbyDongbinXiu May 16 th,2013 Outline Motivation 1 Motivation 2 3 4 5 Example: Burgers Equation Let us consider the Burger s equation: u t + uu x = νu xx, x [ 1, 1] u( 1) =1 u(1) = 1 Example:

More information

Gaussian Filtering Strategies for Nonlinear Systems

Gaussian Filtering Strategies for Nonlinear Systems Gaussian Filtering Strategies for Nonlinear Systems Canonical Nonlinear Filtering Problem ~u m+1 = ~ f (~u m )+~ m+1 ~v m+1 = ~g(~u m+1 )+~ o m+1 I ~ f and ~g are nonlinear & deterministic I Noise/Errors

More information

A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems

A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems COMMUNICATIONS IN COMPUTATIONAL PHYSICS Vol. 6, No. 4, pp. 86-847 Commun. Comput. Phys. October 9 A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems Youssef Marzouk, and Dongbin

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

Forward Problems and their Inverse Solutions

Forward Problems and their Inverse Solutions Forward Problems and their Inverse Solutions Sarah Zedler 1,2 1 King Abdullah University of Science and Technology 2 University of Texas at Austin February, 2013 Outline 1 Forward Problem Example Weather

More information

Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques

Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Institut für Numerische Mathematik und Optimierung Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Oliver Ernst Computational Methods with Applications Harrachov, CR,

More information

Spatial smoothing using Gaussian processes

Spatial smoothing using Gaussian processes Spatial smoothing using Gaussian processes Chris Paciorek paciorek@hsph.harvard.edu August 5, 2004 1 OUTLINE Spatial smoothing and Gaussian processes Covariance modelling Nonstationary covariance modelling

More information

SAMSI Astrostatistics Tutorial. More Markov chain Monte Carlo & Demo of Mathematica software

SAMSI Astrostatistics Tutorial. More Markov chain Monte Carlo & Demo of Mathematica software SAMSI Astrostatistics Tutorial More Markov chain Monte Carlo & Demo of Mathematica software Phil Gregory University of British Columbia 26 Bayesian Logical Data Analysis for the Physical Sciences Contents:

More information

Introduction to Linear Models

Introduction to Linear Models Introduction to Linear Models STA721 Linear Models Duke Universit Merlise Clde August 25, 2015 Coordinates Instructor: Merlise Clde 214 Old Chemistr Office Hours MWF 1:00-2:0 or right after class (or b

More information

Parametric Problems, Stochastics, and Identification

Parametric Problems, Stochastics, and Identification Parametric Problems, Stochastics, and Identification Hermann G. Matthies a B. Rosić ab, O. Pajonk ac, A. Litvinenko a a, b University of Kragujevac c SPT Group, Hamburg wire@tu-bs.de http://www.wire.tu-bs.de

More information

CERTAIN THOUGHTS ON UNCERTAINTY ANALYSIS FOR DYNAMICAL SYSTEMS

CERTAIN THOUGHTS ON UNCERTAINTY ANALYSIS FOR DYNAMICAL SYSTEMS CERTAIN THOUGHTS ON UNCERTAINTY ANALYSIS FOR DYNAMICAL SYSTEMS Puneet Singla Assistant Professor Department of Mechanical & Aerospace Engineering University at Buffalo, Buffalo, NY-1426 Probabilistic Analysis

More information

Integrated Non-Factorized Variational Inference

Integrated Non-Factorized Variational Inference Integrated Non-Factorized Variational Inference Shaobo Han, Xuejun Liao and Lawrence Carin Duke University February 27, 2014 S. Han et al. Integrated Non-Factorized Variational Inference February 27, 2014

More information

SUPPLEMENT TO MARKET ENTRY COSTS, PRODUCER HETEROGENEITY, AND EXPORT DYNAMICS (Econometrica, Vol. 75, No. 3, May 2007, )

SUPPLEMENT TO MARKET ENTRY COSTS, PRODUCER HETEROGENEITY, AND EXPORT DYNAMICS (Econometrica, Vol. 75, No. 3, May 2007, ) Econometrica Supplementary Material SUPPLEMENT TO MARKET ENTRY COSTS, PRODUCER HETEROGENEITY, AND EXPORT DYNAMICS (Econometrica, Vol. 75, No. 3, May 2007, 653 710) BY SANGHAMITRA DAS, MARK ROBERTS, AND

More information

A reduced-order stochastic finite element analysis for structures with uncertainties

A reduced-order stochastic finite element analysis for structures with uncertainties A reduced-order stochastic finite element analysis for structures with uncertainties Ji Yang 1, Béatrice Faverjon 1,2, Herwig Peters 1, icole Kessissoglou 1 1 School of Mechanical and Manufacturing Engineering,

More information

Stochastic structural dynamic analysis with random damping parameters

Stochastic structural dynamic analysis with random damping parameters Stochastic structural dynamic analysis with random damping parameters K. Sepahvand 1, F. Saati Khosroshahi, C. A. Geweth and S. Marburg Chair of Vibroacoustics of Vehicles and Machines Department of Mechanical

More information

An Integrated Prognostics Method under Time-Varying Operating Conditions

An Integrated Prognostics Method under Time-Varying Operating Conditions An Integrated Prognostics Method under Time-Varying Operating Conditions Fuqiong Zhao, ZhigangTian, Eric Bechhoefer, Yong Zeng Abstract In this paper, we develop an integrated prognostics method considering

More information

Variational Principal Components

Variational Principal Components Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings

More information

Dynamic response of structures with uncertain properties

Dynamic response of structures with uncertain properties Dynamic response of structures with uncertain properties S. Adhikari 1 1 Chair of Aerospace Engineering, College of Engineering, Swansea University, Bay Campus, Fabian Way, Swansea, SA1 8EN, UK International

More information

Bayesian Regression Linear and Logistic Regression

Bayesian Regression Linear and Logistic Regression When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we

More information

Algorithms for Uncertainty Quantification

Algorithms for Uncertainty Quantification Algorithms for Uncertainty Quantification Lecture 9: Sensitivity Analysis ST 2018 Tobias Neckel Scientific Computing in Computer Science TUM Repetition of Previous Lecture Sparse grids in Uncertainty Quantification

More information

Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.

Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we

More information

Part 1: Expectation Propagation

Part 1: Expectation Propagation Chalmers Machine Learning Summer School Approximate message passing and biomedicine Part 1: Expectation Propagation Tom Heskes Machine Learning Group, Institute for Computing and Information Sciences Radboud

More information

MCMC algorithms for fitting Bayesian models

MCMC algorithms for fitting Bayesian models MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models

More information

Stochastic representation of random positive-definite tensor-valued properties: application to 3D anisotropic permeability random fields

Stochastic representation of random positive-definite tensor-valued properties: application to 3D anisotropic permeability random fields Sous la co-tutelle de : LABORATOIRE DE MODÉLISATION ET SIMULATION MULTI ÉCHELLE CNRS UPEC UNIVERSITÉ PARIS-EST CRÉTEIL UPEM UNIVERSITÉ PARIS-EST MARNE-LA-VALLÉE Stochastic representation of random positive-definite

More information

A Non-Intrusive Polynomial Chaos Method For Uncertainty Propagation in CFD Simulations

A Non-Intrusive Polynomial Chaos Method For Uncertainty Propagation in CFD Simulations An Extended Abstract submitted for the 44th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada January 26 Preferred Session Topic: Uncertainty quantification and stochastic methods for CFD A Non-Intrusive

More information

Overview. Bayesian assimilation of experimental data into simulation (for Goland wing flutter) Why not uncertainty quantification?

Overview. Bayesian assimilation of experimental data into simulation (for Goland wing flutter) Why not uncertainty quantification? Delft University of Technology Overview Bayesian assimilation of experimental data into simulation (for Goland wing flutter), Simao Marques 1. Why not uncertainty quantification? 2. Why uncertainty quantification?

More information

16 : Markov Chain Monte Carlo (MCMC)

16 : Markov Chain Monte Carlo (MCMC) 10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions

More information

EFFICIENT SHAPE OPTIMIZATION USING POLYNOMIAL CHAOS EXPANSION AND LOCAL SENSITIVITIES

EFFICIENT SHAPE OPTIMIZATION USING POLYNOMIAL CHAOS EXPANSION AND LOCAL SENSITIVITIES 9 th ASCE Specialty Conference on Probabilistic Mechanics and Structural Reliability EFFICIENT SHAPE OPTIMIZATION USING POLYNOMIAL CHAOS EXPANSION AND LOCAL SENSITIVITIES Nam H. Kim and Haoyu Wang University

More information

Dimension-Independent likelihood-informed (DILI) MCMC

Dimension-Independent likelihood-informed (DILI) MCMC Dimension-Independent likelihood-informed (DILI) MCMC Tiangang Cui, Kody Law 2, Youssef Marzouk Massachusetts Institute of Technology 2 Oak Ridge National Laboratory 2 August 25 TC, KL, YM DILI MCMC USC

More information

arxiv: v1 [math.na] 14 Sep 2017

arxiv: v1 [math.na] 14 Sep 2017 Stochastic collocation approach with adaptive mesh refinement for parametric uncertainty analysis arxiv:1709.04584v1 [math.na] 14 Sep 2017 Anindya Bhaduri a, Yanyan He 1b, Michael D. Shields a, Lori Graham-Brady

More information

Overview. IAML: Linear Regression. Examples of regression problems. The Regression Problem

Overview. IAML: Linear Regression. Examples of regression problems. The Regression Problem 3 / 38 Overview 4 / 38 IAML: Linear Regression Nigel Goddard School of Informatics Semester The linear model Fitting the linear model to data Probabilistic interpretation of the error function Eamples

More information

Steps in Uncertainty Quantification

Steps in Uncertainty Quantification Steps in Uncertainty Quantification Challenge: How do we do uncertainty quantification for computationally expensive models? Example: We have a computational budget of 5 model evaluations. Bayesian inference

More information

Introduction to Probability Theory for Graduate Economics Fall 2008

Introduction to Probability Theory for Graduate Economics Fall 2008 Introduction to Probability Theory for Graduate Economics Fall 008 Yiğit Sağlam October 10, 008 CHAPTER - RANDOM VARIABLES AND EXPECTATION 1 1 Random Variables A random variable (RV) is a real-valued function

More information

Efficient Sampling for Non-Intrusive Polynomial Chaos Applications with Multiple Uncertain Input Variables

Efficient Sampling for Non-Intrusive Polynomial Chaos Applications with Multiple Uncertain Input Variables Missouri University of Science and Technology Scholars' Mine Mechanical and Aerospace Engineering Faculty Research & Creative Works Mechanical and Aerospace Engineering 4-1-2007 Efficient Sampling for

More information

INF Introduction to classifiction Anne Solberg Based on Chapter 2 ( ) in Duda and Hart: Pattern Classification

INF Introduction to classifiction Anne Solberg Based on Chapter 2 ( ) in Duda and Hart: Pattern Classification INF 4300 151014 Introduction to classifiction Anne Solberg anne@ifiuiono Based on Chapter 1-6 in Duda and Hart: Pattern Classification 151014 INF 4300 1 Introduction to classification One of the most challenging

More information

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu

Lecture: Gaussian Process Regression. STAT 6474 Instructor: Hongxiao Zhu Lecture: Gaussian Process Regression STAT 6474 Instructor: Hongxiao Zhu Motivation Reference: Marc Deisenroth s tutorial on Robot Learning. 2 Fast Learning for Autonomous Robots with Gaussian Processes

More information

ST 740: Markov Chain Monte Carlo

ST 740: Markov Chain Monte Carlo ST 740: Markov Chain Monte Carlo Alyson Wilson Department of Statistics North Carolina State University October 14, 2012 A. Wilson (NCSU Stsatistics) MCMC October 14, 2012 1 / 20 Convergence Diagnostics:

More information

Elsevier Editorial System(tm) for Journal Of Computational Physics Manuscript Draft

Elsevier Editorial System(tm) for Journal Of Computational Physics Manuscript Draft Elsevier Editorial System(tm) for Journal Of Computational Physics Manuscript Draft Manuscript Number: JCOMP-D-11-756 Title: New evolution equations for the joint response-ecitation probability density

More information

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

More information

Data-Driven Bayesian Model Selection: Parameter Space Dimension Reduction using Automatic Relevance Determination Priors

Data-Driven Bayesian Model Selection: Parameter Space Dimension Reduction using Automatic Relevance Determination Priors Data-Driven : Parameter Space Dimension Reduction using Priors Mohammad Khalil mkhalil@sandia.gov, Livermore, CA Workshop on Uncertainty Quantification and Data-Driven Modeling Austin, Texas March 23-24,

More information

Introduction to Computational Stochastic Differential Equations

Introduction to Computational Stochastic Differential Equations Introduction to Computational Stochastic Differential Equations Gabriel J. Lord Catherine E. Powell Tony Shardlow Preface Techniques for solving many of the differential equations traditionally used by

More information

Variational inference

Variational inference Simon Leglaive Télécom ParisTech, CNRS LTCI, Université Paris Saclay November 18, 2016, Télécom ParisTech, Paris, France. Outline Introduction Probabilistic model Problem Log-likelihood decomposition EM

More information

Dynamic System Identification using HDMR-Bayesian Technique

Dynamic System Identification using HDMR-Bayesian Technique Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in

More information

Uncertainty Quantification for multiscale kinetic equations with high dimensional random inputs with sparse grids

Uncertainty Quantification for multiscale kinetic equations with high dimensional random inputs with sparse grids Uncertainty Quantification for multiscale kinetic equations with high dimensional random inputs with sparse grids Shi Jin University of Wisconsin-Madison, USA Kinetic equations Different Q Boltmann Landau

More information

Nonparametric Bayesian Methods (Gaussian Processes)

Nonparametric Bayesian Methods (Gaussian Processes) [70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent

More information