Experiences with Model Reduction and Interpolation

Transcription

1 Experiences with Model Reduction and Interpolation Paul Constantine Stanford University, Sandia National Laboratories Qiqi Wang (MIT) David Gleich (Purdue) Emory University March 7, 2012 Joe Ruthruff (SNL) Jeremy Templeton (SNL)

2 PA = LU = PAU 1 L 1 = I Education & Positions B.A. Math (U North Texas) M.S. & Ph.D. CME (Stanford) von Neumann Fellow (Sandia) Postdoc (Stanford, ME) Recent work Sparse Fourier transforms via Smolyak s algorithm Input dimension reduction for multivariate approximation Polynomial approximation for composite functions (via Lanczos) Stanford President John Hennessy Tony The Tiger me Application areas Turbulent flows Climate models Coupled multiphysics systems Paul G. Constantine

3 (inspiration for) Just ask a good question! Paul G. Constantine

4 Paul G. Constantine Motivation Uncertainty quantification Parameterized models and model reduction Two approaches: Optimization Interpolation A few examples

5 How can we test systems without testing them? Simulation! How do we get measures of confidence from simulations? Uncertainty quantification! 1996 Nuclear Test Ban Treaty Paul G. Constantine 1

6 differential operators / forcings / boundaries space L(u; x, t, ω) =0 randomness solution time Can we compute statistics of the solution with respect to the randomness? Paul G. Constantine 2

7 differential operators / forcings / boundaries space L(u; x, t, s) =0 parameters solution time Can we compute statistics of the solution with respect to the parameters? Each realization corresponds to a point in the parameter space, where we solve the differential equation. Paul G. Constantine 3

8 space time input parameters solution u(x, t, s) Given input parameters, a physical simulation approximates a space/time dependent solution. Each solution is computationally expensive, so exhaustive parameter studies (e.g., UQ) are infeasible. Cheaper reduced order models enable guesstimates for parameter inquiries. Paul G. Constantine 4

9 discretized differential operator discretized solution A(s)x(s) =b(s) forcing term Paul G. Constantine 5

10 discretized differential operator discretized solution A(s)x(s) =b(s) forcing term Suppose we have solved this system for some parameter values. Can we use those runs to approximate the solution at other points in the parameter space (more cheaply or efficiently)? For each we compute x j = x(s j ) that solves A(s j )x j = b(s j ). Then we approximate x = x(s) n a j x j j=1 How do we compute the coefficients? = Xa Paul G. Constantine 5

11 minimize a Xa x 2 = X T Xa = X T x It s a lot like Kriging Gaussian process emulation Radial basis functions where X T X approximates the correlation model. But what s the problem here? Paul G. Constantine 6

12 Paul G. Constantine Motivation Uncertainty quantification Parameterized models and model reduction Two approaches: Optimization Interpolation A few examples

13 For A = A(s) and b = b(s) minimize a AXa b 2 Funny story about this And then there s Galerkin X T AXa = X T b *Ahem* What about nonlinear problems? Paul G. Constantine 7

14 Full system: f(x,s)=0 Interpolatory system: minimize a f(xa,s) 2 A nonlinear least-squares problem! And time dependence? Paul G. Constantine 8

15 Full system: dx dt = f(x,s) minimize a minimize a Interpolatory system: T 2 d (Xa) f(xa,s) 0 dt dt 2 2 d dt (X ia) f(x i a,s) i 2 A bigger nonlinear least-squares problem (when discretized in time)! Constantine and Wang. Residual Minimizing Model Interpolation for Parameterized Nonlinear Dynamical Systems. In review. Paul G. Constantine 9

16 Point-wise optimality by construction. Norm of residual is natural measure of point-wise misfit. Implementation is independent of dimension of parameter space. Interpolates data with continuous basis functions. Yields a straightforward strategy for adaptively adding bases. Full reuse of previously computed solutions. Each additional basis enhances the approximation globally. Monotonic decrease of the maximum residual (over parameter space). Exact point-wise convergence (in exact arithmetic) after finite model evaluations (although this is a little ridiculous). A rank-revealing property. Easy to implement using only residuals. Paul G. Constantine 10

17 Sandia Research Project Questions: What is the probability of failure? Which input values cause failure? Challenges: Each simulation is expensive (30 minutes on 16 cores) Large scale nonlinear, time dependent heat transfer problem nodes, 10 3 time steps Approach: Construct an interpolating reduced order model from a budgetconstrained ensemble of runs for uncertainty and optimization studies. Paul G. Constantine 11

18 Glory: 4,608 cores 32 GB RAM/core MapReduce cluster: 256 cores 2 TB storage/core Researcher: Infinite curiosity! Each multi-day HPC simulation generates gigabytes of data. A MapReduce cluster can hold hundreds or thousands of simulations enabling engineers to query and analyze simulation data for statistical studies and uncertainty quantification. Paul G. Constantine 12

19 1. Getting the residual out of existing codes was no easy task. 2. For some simple numerical examples, solving the nonlinear least squares problem was as costly as solving the full model. Approach: Reformulate the interpolation scheme as a non-intrusive method, sacrificing accuracy for an increase in performance. Paul G. Constantine 13

20 Paul G. Constantine Motivation Uncertainty quantification Parameterized models and model reduction Two approaches: Optimization Interpolation A few examples

21 solution space/time bases u(x, t, s) = singular values φ i (x, t) σ i v i (s) parameter bases i=1 Exponential decay of singular values for smooth functions. Uniform convergence. See the work of P.C. Hansen on integral operator kernels. See also the Karhunen-Loeve expansion. Paul G. Constantine 14

22 fixed space/time discretization x(s) r u i σ i v i (s) i=1 Paul G. Constantine 15

23 fixed space/time discretization x(s) r u i σ i v i (s) i=1 experimental design SVD x(s1 ) x(s n ) = X = UΣV T Paul G. Constantine 15

24 fixed space/time discretization x(s) r u i σ i v i (s) i=1 experimental design SVD x(s1 ) x(s n ) = X = UΣV T Paul G. Constantine 15

25 fixed space/time discretization x(s) r u i σ i v i (s) i=1 experimental design SVD x(s1 ) x(s n ) = X = UΣV T Paul G. Constantine 15

26 fixed space/time discretization x(s) r u i σ i v i (s) i=1 experimental design SVD x(s1 ) x(s n ) = X = UΣV T Paul G. Constantine 15

27 fixed space/time discretization x(s) r u i σ i v i (s) i=1 experimental design SVD x(s1 ) x(s n ) = X = UΣV T treat each right singular vector as samples of the unknown basis functions! V ij = v i (s j ) Paul G. Constantine 15

28 vi (s j ) ṽ i (s) polynomials radial basis functions gaussian processes piecewise linear splines Paul G. Constantine 16

29 Which section would you rather try to interpolate, A or B? A B Paul G. Constantine 17

30 Folk Theorem (O Leary, 2011) The singular vectors of a matrix (with smooth data) become more oscillatory as the index increases. Paul G. Constantine 18

31 Folk Theorem (O Leary, 2011) The singular vectors of a matrix (with smooth data) become more oscillatory as the index increases. This implies The gradient of the singular vectors increases as the index increases. Paul G. Constantine 18

32 Folk Theorem (O Leary, 2011) The singular vectors of a matrix (with smooth data) become more oscillatory as the index increases. This implies The gradient of the singular vectors increases as the index increases. PREDICTABLE v 1 (s),..., v t (s) UNPREDICTABLE v t+1 (s),..., v r (s) Paul G. Constantine 18

33 predictable unpredictable x(s) t(s) u i σ i ṽ i (s) + r u i σ i η i t(s) i=1 i=t(s)+1 η N(0, 1) Space/time dependent prediction variance at each interpolation: var[x] = diag r u i σi 2 u T i i=t(s)+1 Paul G. Constantine 19

34 1 σ 1 t(s) τ i=1 is the largest σ i v i s τ such that < threshold This local truncation can be used as an importance sampling measure to adaptively refine the approximation. Paul G. Constantine 20

35 Paul G. Constantine Motivation Parameterized models and model reduction Two approaches: Optimization Interpolation A few examples

36 Model: d a(x, s) du dx dx Boundary conditions: u(0) = u(1) = 1 =1 Coefficients: a(x, s) =1+4s (x 2 x) Parameter space: s [0, 1 γ], γ > 0 There s a singularity at x =0.5 s =1 Solution: u(x, s) = 1 8s log 1+4s (x 2 x) Paul G. Constantine 21

37 x(s1 ) x(s n ) = UΣV T Paul G. Constantine 22

38 Paul G. Constantine 23

39 Paul G. Constantine 24

40 Paul G. Constantine 25

41 Heat equation: ρ t (c pt )= (k T ) Material properties: Paul G. Constantine 26

42 Heat equation: ρ t (c pt )= (k T ) Material properties: input parameters Paul G. Constantine 26

43 Heat equation: ρ t (c pt )= (k T ) Material properties: input parameters Paul G. Constantine 26

44 x(s1 ) x(s n ) = UΣV T 20 point Latin hypercube design Paul G. Constantine 27

45 Model Reduction x(s1 ) x(s n ) = UΣV T 20 point Latin hypercube design Make sure to ask me about higher dimensions. Paul G. Constantine 27

46 Model Reduction x(s1 ) x(s n ) = UΣV T Paul G. Constantine 28

47 1 σ 1 t i=1 σ i v i ξ There is probably a better way to do this. 10,000 samples of the weighted sum of the norm of the gradient Paul G. Constantine 29

48 1 σ 1 t i=1 σ i v i ξ There is probably a better way to do this. Paul G. Constantine 29

49 1 σ 1 t i=1 σ i v i ξ < threshold There is probably a better way to do this. Paul G. Constantine 29

50 1,585 randomly sampled cross-validation tests Paul G. Constantine 30

51 truth Where is the error the worst? ROM Paul G. Constantine 31

52 prediction variance Where is the error the worst? error Paul G. Constantine 31

53 Qualitatively compare the parameter distribution of the crossvalidation error to the metric that determines truncation. Paul G. Constantine 32

54 Use the truncation metric as a measure for importance sampling to refine the ROM in areas of large error. Paul G. Constantine 32

55 1,585 Cross-validation Tests Original Design (20 LHS) Refined Design (20 LHS Importance) Mean Relative Error Max Relative Error Paul G. Constantine 33

56 Full Model Reduced Model Prediction Variance Nonlinear heat transfer model 80k nodes, 300 time-steps 104 basis runs SVD of 24m x 104 data matrix 500x reduction in wall clock time Paul G. Constantine 34

57 A framework for model reduction Construct the time/space bases with SVD of sampled outputs. Interpolate the parameter bases. Use the gradient to determine if the interpolant is predictive. Use unpredictability as an importance sampling measure for refinement. Some interesting questions When will this work? (Rapid decay of singular values.) What s the relationship between the predictability measure and the error? When do the singular vectors behave like that? Spatial/temporal refinement? How the heck do you scale the SVD computation? Paul G. Constantine 35

58 Exascale computing requires exascale data management! Supercomputer MapReduce cluster Researcher Paul G. Constantine 36

59 QUESTIONS? Paul G. Constantine 37

60 We have implemented a QR factorization for tall and skinny matrices in MapReduce. A 1 A 2 A 3 A 4 = Q 1 Q 2 Q 4 8n 4n =Q R 1 R 2 R 3 R 4 4n n Q 1 = Q 2 Q 3 Q R github.com/dgleich/mrtsqr! Q 4n n n n 4 8n 4n The Tall, Skinny QR gives us the R factor. We compute Q = XR 1 with another MapReduce call. Demmel, et al. Communication-avoiding parallel and sequential QR factorizations Constantine and Gleich. Tall and Skinny QR factorizations in MapReduce Architectures. MAPREDUCE, '11. Paul G. Constantine 38

61 Reality Prediction/ Validation Modeling Computer Simulation Mathematical Model Verification Paul G. Constantine