Parameter Selection Techniques and Surrogate Models

Transcription

1 Parameter Selection Techniques and Surrogate Models Model Reduction: Will discuss two forms Parameter space reduction Surrogate models to reduce model complexity Input Representation Local Sensitivity Analysis Global Sensitivity Analysis Parameter Selection Model Discrepancy Surrogate Models Sparse Grids Model Calibration Stochastic Spectral Methods Uncertainty Propagation Sparse Grids

2 Parameter Selection Techniques and Surrogate Models Parameter Space Reduction: SIR Model ds = N - S - kis, S() =S dt di dt = kis - (r + )I, I() =I dr = ri - R, R() =R dt Susceptible Infectious Recovered Parameters: Response: : Infection coefficient k: Interaction coefficient r: Recovery rate : Birth/death rate y = Z 5 R(t, q)dt Note: Parameters q =[, k, r, ] not uniquely determined by data

3 Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements

4 Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, Solution: Reformulate problem as d 2 z dt 2 + C dz dt + Kz = F cos(! F t) z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements dz dt () =z 1 where C = c m, K = k m and F = f m Techniques for General Models: Linear algebra analysis; e.g., SVD or QR algorithms Identifiability analysis: M. Eisenberg Sensitivity analysis: M. Hyman Active Subspaces: Present discussion Surrogates: Present and later talks

5 Parameter Selection Techniques and Surrogate Models Second Issue: Coupled compartment can have 1 s of nonidentifiable parameters Example: Metabolism Techniques for General Models: Identifiability and sensitivity analysis: M. Eisenberg, M. Hyman Active Subspaces: Present discussion

6 Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1

7 Parameter Space Reduction Techniques: Linear Problems Second Issue: Models depends on very large number of parameters e.g., millions but only a few are significant. Linear Algebra Techniques: Linearly parameterized problems y = Aq, q 2 R p,y2 R m Singular Value Decomposition (SVD): Rank Revealing QR Decomposition: Problem: Neither is directly applicable when m or p are very large; e.g., millions. Solution: Random range finding algorithms.

8 Random Range Finding Algorithms: Linear Problems Algorithm: Halko, Martinsson and Tropp, SIAM Review, 211 Example: y i = px q k sin(2 kt i ),i=1,,m k=1

9 Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1: Analytic value for rank is 49 Column Entries of A A(:,1) A(:,11) A(:,21) A(:,91) Time (s) Column Entries of A Aliasing A(:,1) A(:,51) Time (s).8 1 Singular Values Singular Values Deterministic Algorithm Random Algorithm Absolute Difference in Singular Values Absolute Difference Between Singular Values Example: m = 11, p = 1,,: Random algorithm still viable

10 Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1 Column Entries of A A(:,1) A(:,11) A(:,21) A(:,91) Time (s) Column Entries of A A(:,1) A(:,51) Time (s).8 1 Aliasing Later: Reproduce these results using the code alias.m. You can experiment with the size of the random sample. See how far you can increase the dimension. Singular Values Deterministic Algorithm Random Algorithm Singular Values Absolute Difference in Singular Values Absolute Difference Between Singular Values

11 Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1

12 Gradient-Based Active Subspace Construction Active Subspace: See [Constantine, SIAM 215]. Consider f = f (q), q 2 Q R p and r q f (q) = Construct outer product C @q p Z (r q f )(r q f ) T dq T (q): Distribution of input parameters q Partition eigenvalues: C = W W T = apple 1 Rotated Coordinates: 2, W =[W 1 W 2 ] y = W T 1 q 2 Rn and z = W T 2 q 2 Rp-n Active Variables Active Subspace: Range of eigenvectors in W 1

13 Gradient-Based Active Subspace Construction Active Subspace: Construction based on random sampling 1. Draw M independent samples {q j } from 2. Evaluate r q f j = r q f (q j ) 3. Approximate outer product C e C = 1 M MX (r q f j )(r q f j ) T j=1 Note: e C = GG T where G = 1 p M [r q f 1,..., r q f M ] 4. Take SVD of G = W p V T Monte Carlo Quadrature: More on Wednesday Active subspace of dimension n is first n columns of W Research: Develop algorithms for codes that do not have derivative capabilities Note: Finite-difference approximations tempting but not very effective Strategy: Algorithms based on linearization and sampling in active directions.

14 Gradient-Based Active Subspace Construction Example: y = exp(.7q 1 +.3q 2 ) Active Subspace: For gradient matrix G, form SVD G = U V T Eigenvalue spectrum indicates 1-D active subspace with basis U(:, 1) = [.91,.39] g(y) Response Surface Testing Points y Exercise: Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] Now take the SVD to determine a basis for the active subspace. How might you construct a response surface g(y) on the 1-D subspace y?

15 Later Talk: C. Webster Steps in Uncertainty Quantification

16 Surrogate Models: Motivation Sinko-Streifer Model Notation and Assumptions: u (t, x ): Number of fish of size x at time t µ: Death rate Growth rate of same-sized individuals is same and denoted by g (t, x ) dx = g (t, x ) dt Flux Balance: u (t, x ) is a density and rate is (t, x ) = g (t, x )u (t, @ ) + = -µu + = Model: ) + Z g (t, x )u (t, x ) u (, x ) = x =x (x ) = x1 x k (t, )u (t, )d

17 Example: Consider the model Surrogate = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 u(t, x, y, z; q)dxdydzdt t 1 x, y, z Notes: Requires approximation of PDE in 3-D What would be a simple surrogate?

18 Recall: Consider = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) Question: How do you construct a polynomial surrogate? Regression Interpolation

19 Recall: Consider = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) Question: How do you construct a polynomial surrogate? Interpolation Regression Statistical Model: y s (q): Emulator for y(q) y m = y s (q m )+" m,m=1,...,m M=7 k=2

20 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) BIC = k log(m) - 2 log[ (y q)] Likelihood: 1 2 (y q) = Maximize (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 Minimize m=1 Later Talks: M. Eisenberg, B. Fitzpatrick, R. Smith M=7 k=2

21 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k - 2 log[ (y q)] Bayesian Information Criterion (AIC) Example: Exercise: BIC = k log(m) - 2 log[ (y q)] Likelihood: 1 2 (y q) = (2 2 ) M/2 e-ss q/2 MX SS q = [y m - y s (q m )] 2 m=1 y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate? Maximize Minimize q2 q 1 M=7 k=2

22 Example: m = 11, p = 1 y i = Code: Computer Tutorials px q k sin(2 kt i ),i=1,,m k=1 Reproduce these results using the code alias.m. You can experiment with the size of the random sample and tol. See how far you can increase the dimension. Column Entries of A A(:,1) A(:,51) Time (s).8 1 Example: y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate. Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] q2 Take SVD to determine basis for the active subspace. q 1