Parameter Selection Techniques and Surrogate Models

Similar documents
Uncertainty Propagation

Steps in Uncertainty Quantification

Concepts in Global Sensitivity Analysis IMA UQ Short Course, June 23, 2015

Spectral Representation of Random Processes

ACTIVE SUBSPACES for dimension reduction in parameter studies

Organization. I MCMC discussion. I project talks. I Lecture.

Applied Linear Algebra in Geoscience Using MATLAB

ECE295, Data Assimila0on and Inverse Problems, Spring 2015

7. Symmetric Matrices and Quadratic Forms

Chapter 3 Transformations

Stochastic Spectral Approaches to Bayesian Inference

Algorithms for Uncertainty Quantification

Notes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T.

Applications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices

MATH 350: Introduction to Computational Mathematics

Review of Some Concepts from Linear Algebra: Part 2

MA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2

PRACTICE FINAL EXAM. why. If they are dependent, exhibit a linear dependence relation among them.

Introduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

Lecture 6. Numerical methods. Approximation of functions

Linear algebra for MATH2601: Theory

DS-GA 1002 Lecture notes 10 November 23, Linear models

OPTIMAL CONTROL AND ESTIMATION

Properties of Matrices and Operations on Matrices

Linear Systems. Class 27. c 2008 Ron Buckmire. TITLE Projection Matrices and Orthogonal Diagonalization CURRENT READING Poole 5.4

Background Mathematics (2/2) 1. David Barber

Gaussian Filtering Strategies for Nonlinear Systems

for Complex Environmental Models

Review problems for MA 54, Fall 2004.

Math 1060 Linear Algebra Homework Exercises 1 1. Find the complete solutions (if any!) to each of the following systems of simultaneous equations:

Quadratic forms. Here. Thus symmetric matrices are diagonalizable, and the diagonalization can be performed by means of an orthogonal matrix.

Outline Introduction OLS Design of experiments Regression. Metamodeling. ME598/494 Lecture. Max Yi Ren

Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications

Previously Monte Carlo Integration

Probabilistic Graphical Models

Preliminary Examination, Numerical Analysis, August 2016

Chap 3. Linear Algebra

DATA ASSIMILATION FOR FLOOD FORECASTING

Orthonormal Bases; Gram-Schmidt Process; QR-Decomposition

Utilizing Adjoint-Based Techniques to Improve the Accuracy and Reliability in Uncertainty Quantification

Computer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo

Polynomial Chaos and Karhunen-Loeve Expansion

Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2

Math 322. Spring 2015 Review Problems for Midterm 2

Chapter 7: Symmetric Matrices and Quadratic Forms

Stat 451 Lecture Notes Numerical Integration

The Singular Value Decomposition

Probing the covariance matrix

TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY

1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det

Linear Algebra Practice Problems

LINES IN P 3. Date: December 12,

randomized block krylov methods for stronger and faster approximate svd

6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities

CS 450 Numerical Analysis. Chapter 8: Numerical Integration and Differentiation

Bayesian Methods and Uncertainty Quantification for Nonlinear Inverse Problems

Least Squares Approximation

Stochastic Complexities of Reduced Rank Regression in Bayesian Estimation

LINEAR ALGEBRA (PMTH213) Tutorial Questions

MAT Linear Algebra Collection of sample exams

I. Multiple Choice Questions (Answer any eight)

Geometric Modeling Summer Semester 2012 Linear Algebra & Function Spaces

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

STA414/2104 Statistical Methods for Machine Learning II

= 2e t e 2t + ( e 2t )e 3t = 2e t e t = e t. Math 20D Final Review

9. Model Selection. statistical models. overview of model selection. information criteria. goodness-of-fit measures

ECE521 week 3: 23/26 January 2017

Maths for Signals and Systems Linear Algebra in Engineering

Computational Methods. Eigenvalues and Singular Values

Scalable kernel methods and their use in black-box optimization

A Note on the Particle Filter with Posterior Gaussian Resampling

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017

MATH 56A: STOCHASTIC PROCESSES CHAPTER 0

Lecture 9: Low Rank Approximation

June 2011 PURDUE UNIVERSITY Study Guide for the Credit Exam in (MA 262) Linear Algebra and Differential Equations

arxiv: v2 [math.na] 5 Dec 2013

Learning gradients: prescriptive models

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

1. Select the unique answer (choice) for each problem. Write only the answer.

Linear Models 1. Isfahan University of Technology Fall Semester, 2014

Proposition 42. Let M be an m n matrix. Then (32) N (M M)=N (M) (33) R(MM )=R(M)

STA 4273H: Sta-s-cal Machine Learning

Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson

3. Identify and find the general solution of each of the following first order differential equations.

Singular value decomposition. If only the first p singular values are nonzero we write. U T o U p =0

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Incomplete Cholesky preconditioners that exploit the low-rank property

Theorems. Least squares regression

A Tour of the Lanczos Algorithm and its Convergence Guarantees through the Decades

Lecture 4 and 5 Controllability and Observability: Kalman decompositions

A NEW SOLUTION OF SIR MODEL BY USING THE DIFFERENTIAL FRACTIONAL TRANSFORMATION METHOD

Applied Linear Algebra

Kernel-based Approximation. Methods using MATLAB. Gregory Fasshauer. Interdisciplinary Mathematical Sciences. Michael McCourt.

DS-GA 1002 Lecture notes 12 Fall Linear regression

MCMC Sampling for Bayesian Inference using L1-type Priors

CS 572: Information Retrieval

Positive Definite Matrix

Preface to Second Edition... vii. Preface to First Edition...

Lecture 6: Bayesian Inference in SDE Models

Transcription:

Parameter Selection Techniques and Surrogate Models Model Reduction: Will discuss two forms Parameter space reduction Surrogate models to reduce model complexity Input Representation Local Sensitivity Analysis Global Sensitivity Analysis Parameter Selection Model Discrepancy Surrogate Models Sparse Grids Model Calibration Stochastic Spectral Methods Uncertainty Propagation Sparse Grids

Parameter Selection Techniques and Surrogate Models Parameter Space Reduction: SIR Model ds = N - S - kis, S() =S dt di dt = kis - (r + )I, I() =I dr = ri - R, R() =R dt Susceptible Infectious Recovered Parameters: Response: : Infection coefficient k: Interaction coefficient r: Recovery rate : Birth/death rate y = Z 5 R(t, q)dt Note: Parameters q =[, k, r, ] not uniquely determined by data

Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements

Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, Solution: Reformulate problem as d 2 z dt 2 + C dz dt + Kz = F cos(! F t) z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements dz dt () =z 1 where C = c m, K = k m and F = f m Techniques for General Models: Linear algebra analysis; e.g., SVD or QR algorithms Identifiability analysis: M. Eisenberg Sensitivity analysis: M. Hyman Active Subspaces: Present discussion Surrogates: Present and later talks

Parameter Selection Techniques and Surrogate Models Second Issue: Coupled compartment can have 1 s of nonidentifiable parameters Example: Metabolism Techniques for General Models: Identifiability and sensitivity analysis: M. Eisenberg, M. Hyman Active Subspaces: Present discussion

Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1

Parameter Space Reduction Techniques: Linear Problems Second Issue: Models depends on very large number of parameters e.g., millions but only a few are significant. Linear Algebra Techniques: Linearly parameterized problems y = Aq, q 2 R p,y2 R m Singular Value Decomposition (SVD): Rank Revealing QR Decomposition: Problem: Neither is directly applicable when m or p are very large; e.g., millions. Solution: Random range finding algorithms.

Random Range Finding Algorithms: Linear Problems Algorithm: Halko, Martinsson and Tropp, SIAM Review, 211 Example: y i = px q k sin(2 kt i ),i=1,,m k=1

Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1: Analytic value for rank is 49 Column Entries of A 1.5.5 A(:,1) A(:,11) A(:,21) A(:,91) 1.2.4.6.8 1 Time (s) Column Entries of A Aliasing 1.5.5 A(:,1) A(:,51) 1.2.4.6 Time (s).8 1 Singular Values 35 3 25 2 15 1 5 2 4 6 8 1 Singular Values Deterministic Algorithm Random Algorithm Absolute Difference in Singular Values 1 12 1 13 1 14 1 15 1 16 2 4 6 8 1 Absolute Difference Between Singular Values Example: m = 11, p = 1,,: Random algorithm still viable

Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1 Column Entries of A 1.5.5 A(:,1) A(:,11) A(:,21) A(:,91) 1.2.4.6.8 1 Time (s) Column Entries of A 1.5.5 A(:,1) A(:,51) 1.2.4.6 Time (s).8 1 Aliasing Later: Reproduce these results using the code alias.m. You can experiment with the size of the random sample. See how far you can increase the dimension. Singular Values 35 3 25 2 15 1 5 Deterministic Algorithm Random Algorithm 2 4 6 8 1 Singular Values Absolute Difference in Singular Values 1 12 1 13 1 14 1 15 1 16 2 4 6 8 1 Absolute Difference Between Singular Values

Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1

Gradient-Based Active Subspace Construction Active Subspace: See [Constantine, SIAM 215]. Consider f = f (q), q 2 Q R p and r q f (q) = Construct outer product C = apple @f @q 1,, @f @q p Z (r q f )(r q f ) T dq T (q): Distribution of input parameters q Partition eigenvalues: C = W W T = apple 1 Rotated Coordinates: 2, W =[W 1 W 2 ] y = W T 1 q 2 Rn and z = W T 2 q 2 Rp-n Active Variables Active Subspace: Range of eigenvectors in W 1

Gradient-Based Active Subspace Construction Active Subspace: Construction based on random sampling 1. Draw M independent samples {q j } from 2. Evaluate r q f j = r q f (q j ) 3. Approximate outer product C e C = 1 M MX (r q f j )(r q f j ) T j=1 Note: e C = GG T where G = 1 p M [r q f 1,..., r q f M ] 4. Take SVD of G = W p V T Monte Carlo Quadrature: More on Wednesday Active subspace of dimension n is first n columns of W Research: Develop algorithms for codes that do not have derivative capabilities Note: Finite-difference approximations tempting but not very effective Strategy: Algorithms based on linearization and sampling in active directions.

Gradient-Based Active Subspace Construction Example: y = exp(.7q 1 +.3q 2 ) Active Subspace: For gradient matrix G, form SVD G = U V T Eigenvalue spectrum indicates 1-D active subspace with basis U(:, 1) = [.91,.39] 7 6 5 4 3 2 1 g(y) Response Surface Testing Points -1-2 -1 1 2 y Exercise: Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] Now take the SVD to determine a basis for the active subspace. How might you construct a response surface g(y) on the 1-D subspace y?

Later Talk: C. Webster Steps in Uncertainty Quantification

Surrogate Models: Motivation Sinko-Streifer Model Notation and Assumptions: u (t, x ): Number of fish of size x at time t µ: Death rate Growth rate of same-sized individuals is same and denoted by g (t, x ) dx = g (t, x ) dt Flux Balance: u (t, x ) is a density and rate is (t, x ) = g (t, x )u (t, x ) @u @(gu ) @u @ ) + = -µu + = - deaths @t @x @t @x Model: @u @(gu ) + = -µu @t @x Z g (t, x )u (t, x ) u (, x ) = x =x (x ) = x1 x k (t, )u (t, )d

Example: Consider the model Surrogate Models: Motivation @u @t + @2 u @x 2 + @2 u @y 2 + @2 u @z 2 = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 u(t, x, y, z; q)dxdydzdt t 1 x, y, z Notes: Requires approximation of PDE in 3-D What would be a simple surrogate?

Recall: Consider the model @u @t + @2 u @x 2 + @2 u @y 2 + @2 u @z 2 = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) 2 +.5 Question: How do you construct a polynomial surrogate? Regression Interpolation

Recall: Consider the model @u @t + @2 u @x 2 + @2 u @y 2 + @2 u @z 2 = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) 2 +.5 Question: How do you construct a polynomial surrogate? Interpolation Regression Statistical Model: y s (q): Emulator for y(q) y m = y s (q m )+" m,m=1,...,m M=7 k=2

Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) BIC = k log(m) - 2 log[ (y q)] Likelihood: 1 2 (y q) = Maximize (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 Minimize m=1 Later Talks: M. Eisenberg, B. Fitzpatrick, R. Smith M=7 k=2

Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) Example: Exercise: BIC = k log(m) - 2 log[ (y q)] Likelihood: 1 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 m=1 y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate? Maximize Minimize q2 q 1 M=7 k=2

Example: m = 11, p = 1 y i = Code: Computer Tutorials px q k sin(2 kt i ),i=1,,m k=1 Reproduce these results using the code alias.m. You can experiment with the size of the random sample and tol. See how far you can increase the dimension. Column Entries of A 1.5.5 A(:,1) A(:,51) 1.2.4.6 Time (s).8 1 Example: y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate. Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] q2 Take SVD to determine basis for the active subspace. q 1

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback