CME 345: MODEL REDUCTION

Transcription

1 CME 345: MODEL REDUCTION Proper Orthogonal Decomposition (POD) Charbel Farhat & David Amsallem Stanford University 1 / 43

2 Outline 1 Time-continuous Formulation 2 Method of Snapshots for a Single Parametric Configuration 3 The POD Method in the Frequency Domain 4 Connection with SVD 5 Error Analysis 6 Extension to Multiple Parametric Configurations 7 Applications 2 / 43

3 Time-continuous Formulation Nonlinear High-Dimensional Model d w(t) dt = f(w(t), t) y(t) = g(w(t), t) w(0) = w 0 w R N : vector of state variables y R q : vector of output variables (typically q N) f(, ) R N : together with d w(t), defines the high-dimensional dt system of equations 3 / 43

4 Time-continuous Formulation POD Minimization Problem Consider a fixed initial condition w 0 R N Denote the associated state trajectory in the time-interval [0, T ] by T w = {w(t)} 0 t T The Proper Orthogonal Decomposition (POD) method seeks an orthogonal projector Π V,V of fixed rank k that minimizes the integrated projection error T 0 T w(t) Π V,V w(t) 2 2 dt = E V (t) 2 2dt = E V 2 = J(Π V,V ) 0 4 / 43

5 Time-continuous Formulation Solution to the POD Minimization Problem Theorem Let K R N N be the real, symmetric, positive, semi-definite matrix defined as follows T K = w(t)w(t) T dt 0 Let ˆλ 1 ˆλ 2 ˆλ N 0 denote the ordered eigenvalues of K, and φ i R N, i = 1,, N, denote their associated eigenvectors which are also referred to as the POD modes K φ i = ˆλ i φi, i = 1,, N The subspace V = range ( V) of dimension k that minimizes J(Π V,V ) is the invariant subspace of K associated with the eigenvalues ˆλ 1 ˆλ 2 ˆλ k 5 / 43

6 Method of Snapshots for a Single Parametric Configuration Discretization of POD by the Method of Snapshots Solving the eigenvalue problem K φ i = ˆλ i φi is in general computationally intractable because: (1) the dimension N of the matrix K is usually large, (2) this matrix is usually dense However, the state data is typically available under the form of discrete snapshot vectors In this case, T quadrature rule as follows 0 {w(t i )} Nsnap i=1 w(t)w(t) T dt can be approximated using a N snap K = α i w(t i )w(t i ) T i=1 where α i, i = 1,, N snap are the quadrature weights 6 / 43

7 Method of Snapshots for a Single Parametric Configuration Discretization of POD by the Method of Snapshots Let S R N Nsnap denote the snapshot matrix defined as follows S = [ α1 w(t 1 )... αnsnap w(t Nsnap ) ] It follows that K = SS T where K is still a large-scale (N N) matrix 7 / 43

8 Method of Snapshots for a Single Parametric Configuration Discretization of POD by the Method of Snapshots Note that the non-zero eigenvalues of the matrix K = SS T R N N are the same as those of the matrix R = S T S R Nsnap Nsnap Since usually N snap N, it is more economical to solve instead the symmetric eigenvalue problem Rψ i = λ i ψ i, i = 1,, N snap However, if S is ill-conditioned, R is worse conditioned κ 2 (S) = κ 2 (S T S) κ 2 (R) = κ 2 (S) 2 8 / 43

9 Method of Snapshots for a Single Parametric Configuration Discretization of POD by the Method of Snapshots If rank(r) = r, then the first r POD modes φ i are given by φ i = 1 λi Sψ i, i = 1,, r Let Φ = [ ] [ ] φ 1... φ r and Ψ = ψ1... ψ r with Ψ T Ψ = I r = Φ = SΨΛ 1 2 where λ 1 (0) Λ =... (0) λ r Rψ i = λ i ψ i, i = 1,, N snap Ψ T RΨ = Ψ T S T SΨ = Λ Hence Φ T KΦ = Λ 1 2 Ψ T S T SS T SΨΛ 1 2 = Λ 1 2 ΛΨ T ΨΛΛ 1 2 = Λ Since the columns of Φ are the eigenvectors of K ordered by decreasing eigenvalues, the optimal orthogonal basis of size k r is V = [ ] [ ] I Φ k Φ k r k = Φ 0 k 9 / 43

10 The POD Method in the Frequency Domain Fourier Analysis Parseval s theorem 1 (the Fourier transform is unitary) T 1 2 lim T T T 2 V T w(t) 2 2 dt = lim T, Ω 1 2πT Ω where F[w(t)] = W(ω) is the Fourier transform of w(t) Consequence V T ( = V T ( lim T lim T, Ω 1 T 1 2πT T 2 T 2 Ω Ω w(t)w(t) T dt Ω (Proof: see Homework assignment #2) F [ V T w(t) ] 2 2 dω ) V W(ω)W(ω) dω ) V 1 Rayleigh s energy theorem, Plancherel s theorem 10 / 43

11 The POD Method in the Frequency Domain Snapshots in the Frequency Domain Let K denote the analog to K in the frequency domain K = Ω Ω W(ω)W(ω) dω N C snap i= N C snap α i W(ω i )W(ω i ) where ω i = ω i is The corresponding snapshot matrix is [ α0 S = W(ω 0 ) ( ) 2α 1 Re (W(ω 1 ))... 2α N C snap Re W(ω N C snap ) ( ) ] 2α1 Im (W(ω 1 ))... 2α N C snap Im W(ω N C snap ) It follows that K = S S T R = S T S T = Ψ Λ Ψ Φ = S Ψ Λ ] 1 2 Ṽ = [ Φk [ ] I k ΦN r = 0 Φ k 11 / 43

12 The POD Method in the Frequency Domain Case of Linear-Time Invariant Systems f(w(t), t) = Aw(t) + Bu(t) g(w(t), t) = Cw(t) + Du(t) Single input case: p = 1 B R N Time trajectory t w(t) = e At w 0 + e A(t τ) Bu(τ)dτ 0 Snapshots in the time-domain for an impulse input u(t) = δ(t) and zero initial condition w(t i ) = e At i B, t i 0 In the frequency domain, the LTI system can be written as jω l W = AW + B, ω l 0 and the associated snapshots are W(ω l ) = (jω l I A) 1 B 12 / 43

13 The POD Method in the Frequency Domain Case of Linear-Time Invariant Systems How to sample the frequency domain? approximate time trajectory for a zero initial condition t ΠṼ, Ṽ w(t) = ṼṼT e A(t τ) Bu(τ)dτ low-dimensional solution is accurate if the corresponding error is small that is w(t) ΠṼ, Ṽ w(t) = (I ṼṼT ) 0 t 0 e A(t τ) Bu(τ)dτ is small, which depends on the frequency content of u(τ) = the sampled frequency band should contain the dominant frequencies of u(τ) 13 / 43

14 Connection with SVD Definition Given A R N M, there exist two orthogonal matrices U R N N (U T U = I N ) and Z R M M (Z T Z = I M ) such that A = UΣZ T where Σ R N M has diagonal entries Σ ii = σ i satisfying σ 1 σ 2 σ min(n,m) 0 and zero entries everywhere else {σ i } min(n,m) i=1 are the singular values of A, and the columns of U and Z are the left and right singular vectors of A, respectively U = [u 1 u N ], Z = [z 1 z M ] 14 / 43

15 Connection with SVD Properties The SVD of a matrix provides many useful information about it (rank, range, null space, norm,...) {σi 2 } min(n,m) i=1 are the eigenvalues of the symmetric positive, semi-definite matrices AA T and A T A Az i = σ i u i, i = 1,, min(n, M) rank(a) = r, where r is the index of the smallest non-zero singular value If U r = [u 1 u r ] and Z r = [z 1 z r ] denote the singular vectors associated with the non-zero singular values and U N r = [u r+1 u N ] and Z M r = [z r+1 z M ], then A = σ 1 u 1 z T σr ur zt r = r σ i u i z T i i=1 range (A) = range (U r ) range (A T ) = range (Z r ) null (A) = range (Z M r ) null (A T ) = range (U N r ) 15 / 43

16 Connection with SVD Application of the SVD to Optimality Problems Given A R N M with N M, which matrix X R N M with rank(x) = k < r = rank(a) M minimizes A X 2? Theorem (Schmidt-Eckart-Young-Mirsky) min A X 2 = σ k+1 (A), if σ k (A) > σ k+1 (A) X, rank(x)=k Hence, X = k σ i u i z T i, where A = UΣZ T, minimizes A X 2 i=1 This minimizer is also the unique solution of the related problem (Eckart-Young theorem) min X, rank(x)=k A X F 16 / 43

17 Connection with SVD Application to Image Compression Consider a color image in RGB representation made of M N pixels, where M < N (i.e., a landscape image) this image can be represented by an M N 3 real matrix A 1 A 1 can be converted to a 3N M matrix A 3 as follows A 1 2 R M N 3 A 2 2 R M (N 3) A 3 = A T 2 2 R (3N) M finally, A 3 can be approximated using the SVD as follows r A 3 = σ 1u 1z T σ r u r z T r = σ i u i z T i i=1 17 / 43

18 CME 345: MODEL REDUCTION - Proper Orthogonal Decomposition Connection with SVD Application to Image Compression Example: A3 R (a) rank 1 (b) rank 2 (c) rank 3 (d) rank 4 (e) rank 5 (f) rank 6 18 / 43

19 CME 345: MODEL REDUCTION - Proper Orthogonal Decomposition Connection with SVD Application to Image Compression (g) rank 10 (h) rank 20 (i) rank 50 (j) rank 75 (k) rank 100 (l) rank 285 = the SVD can be used for data compression 19 / 43

20 Connection with SVD Discretization of POD by the Method of Snapshots and the SVD The discretization of the POD by the method of snapshots requires computing the eigenspectrum of K = SS T Φ T KΦ = Φ T SS T Φ = Λ corresponding to its non-zero eigenvalues Link with the SVD of S [ S = UΣZ T Σr 0 = [U r U N r ] 0 0 ] Z T = K = UΣ 2 U T and U T KU = Σ 2 = Φ = U r and Λ 1 2 = Σr = U r R N r is to be identified with X R N M, N M r Computing the SVD of S is usually preferred to computing the eigendecomposition of R = S T S because, as noted earlier κ 2 (R) = κ 2 (S) 2 20 / 43

21 Error Analysis Reduction Criterion How to choose the size k of the Reduced-Order Basis (ROB) V obtained using the POD method start from the property of the Frobenius norm of S S F = r σi 2(S) consider the error measured with the Frobenius norm induced by the truncation of the POD basis r (I N VV T )S F = σi 2(S) i=1 i=k+1 the square of the relative error gives an indication of the magnitude of the missing information k r σi 2 (S) σi 2 (S) i=1 i=k+1 E POD (k) = 1 E r POD (k) = σi 2(S) r σi 2(S) i=1 i=1 21 / 43

22 Error Analysis Reduction Criterion How to choose the size k of the ROB V obtained using the POD method (continue) k σi 2(S) i=1 E POD (k) = r σi 2(S) i=1 E POD (k) represents the relative energy of the snapshots captured by the k first POD basis vectors k is usually chosen as the minimum integer for which 1 E POD (k) ɛ for a given tolerance 0 < ɛ < 1 (for instance ɛ = 0.1%) this criterion originates from turbulence applications 22 / 43

23 Error Analysis Reduction Criterion Recall the model reduction error components E ROM (t) = E V (t) + E V (t) = (I N Π V,V ) w(t) + V ( V T w(t) q(t) ) denote E snap ROM = [E ROM(t 1) E ROM (t Nsnap )] r [E V (t 1) E V (t Nsnap )] F = σi 2(S) hence and i=k+1 1 E POD (k) = [E V (t1) E V (t N snap )] 2 F r σi 2(S) 1 E POD (k) i=1 E snap ROM 2 F r σi 2(S) note that the energy criterion is valid only for the sampled snapshots i=1 23 / 43

24 Extension to Multiple Parametric Configurations The Steady-State Case Consider the parametrized steady-state high-dimensional system of equations f(w; µ) = 0, µ D R d, µ = [µ 1,, µ d ] T Consider the goal of constructing a ROB and the associated projection-based ROM for computing the approximate solution w(µ) Vq(µ), µ D Question: How do we build a global ROB V that can capture the solution in the entire parameter domain D? 24 / 43

25 Extension to Multiple Parametric Configurations Choice of Snapshots Lagrange basis V span { ( w µ (1)) (,, w µ (s))} N snap = s Hermite basis { ( V span w µ (1)), w ( µ (1)) (,, w µ (s)), w ( µ (s))} µ 1 µ d Taylor basis N snap = s (d + 1) V span w ( µ (1)), w µ 1 ( µ (1)), 2 w µ 2 1 ( µ (1)),, q w µ q 1 ( µ (1)),, w µ d ( µ (1)),, q w µ q d (µ (1)) N snap = 1+d+ d(d + 1) (d + q 1)! (d 1)!q! = 1+ q i=1 (d + i 1)! (d 1)!i! 25 / 43

26 Extension to Multiple Parametric Configurations Design of Numerical Experiments How one chooses the s parameter samples µ (1),, µ (s) where to compute the snapshots { w ( µ (1)),, w ( µ (s))}? the samples location in the parameter space will determine the accuracy of the resulting global ROM in the entire parameter domain D R d Possible approaches Uniform sampling for parameter spaces of moderate dimensions (d 5) Latin Hypercube sampling for higher-dimensional parameter spaces Goal-oriented greedy sampling that exploits an error indicator to focus on the ROM accuracy 26 / 43

27 Extension to Multiple Parametric Configurations A Greedy Approach Ideally, one can build a ROM progressively and update it (increase its dimension) by considering additional samples µ (i) and corresponding solution snapshots at the locations of the parameter space where the current ROM is the most inaccurate that is, µ (i) = argmax µ D E ROM (µ) = argmax w(µ) Vq(µ) µ D q(µ) can be efficiently computed but the cost of obtaining w(µ) can be high eventually an intractable approach Idea: rely on an economical a posteriori error estimator option 1: error bound E ROM (µ) (µ) option 2: error indicator based on the norm of the (affordable) residual r(µ) = f(vq(µ); µ) For this purpose, D is{ usually replaced by a large discrete set of candidate parameters µ (1),, µ (c)} D 27 / 43

28 Extension to Multiple Parametric Configurations A Greedy Approach Greedy procedure based on the norm of the residual as an error indicator Algorithm (given a termination criterion) 1 randomly select a first sample µ (1) 2 solve the HDM-based problem ( f w(µ (1) ); µ (1)) = 0 3 build a corresponding ROB V 4 for i = 2, 5 solve µ (i) = argmax { µ µ (1),,µ (c) } r(µ) 6 solve the HDM-based problem ( f w(µ (i) ); µ (i)) = 0 7 { build a ROB V based } on the snapshots (or in this case samples) w(µ (1) ),, w(µ (i) ) 28 / 43

29 Extension to Multiple Parametric Configurations The Unsteady Case Parameterized HDM Lagrange basis d w(t; µ) = f (w(t; µ), t; µ) dt { ( V span w t 1 ; µ (1)) ( (,, w t N t ; µ(1)),, w t 1 ; µ (s)) (,, w t N t ; µ(s))} Nsnap = s Nt A posteriori error estimators option 1: error bound ( T 1/2 E ROM (µ) = E ROM (t; µ) dt) 2 (µ) 0 option 2: error indicator based on the norm of the (affordable) residual ( T 1/2 T r(µ) = r(t; µ) dt) 2 = d 2 w(t; µ) f(vq(t; µ), t; µ) 0 dt dt 0 29 / 43

30 Extension to Multiple Parametric Configurations The Unsteady Case Greedy procedure based on the residual norm as an error indicator Algorithm (given a termination criterion) 1 randomly select a first sample µ (1) 2 solve the HDM-based problem d dt w(t; µ(1) ) = f (w(t; µ (1) ), t; µ (1)) 3 build a ROB V based on the snapshots { } w(t 1; µ (1) ),, w(t Nt ; µ (1) ) 4 for i = 2, 5 solve µ (i) = argmax { µ µ (1),,µ (c) } r(µ) 6 solve the HDM-based problem d dt w(t; µ(i) ) = f (w(t; µ (i) ), t; µ (i)) 7 build a ROB V based on the snapshots { } w(t 1; µ (1) ),, w(t Nt ; µ (i) ) 30 / 43

31 CME 345: MODEL REDUCTION - Proper Orthogonal Decomposition Applications Image Compression Recall = 1 EPOD (m) < 10 1 rank 2 (n) < 10 2 rank 47 (o) < 10 3 rank 138 (p) < 10 4 rank 210 (q) < 10 5 rank 249 (r) < 10 6 rank / 43

32 Applications Image Compression E POD (k) k 32 / 43

33 Applications Second-Order Dynamical System N u = 48 masses N = 96 degrees of freedom in state space form Model reduction by the POD method in the frequency domain 33 / 43

34 Applications Second-Order Dynamical System Nyquist plots this leads to the choice of a ROM of size k = / 43

35 Applications Second-Order Dynamical System Bode diagram for a ROM of size k = / 43

36 Applications Fluid System - Advection-Diffusion High-dimensional model (N = 5, 402) 36 / 43

37 Applications Fluid System - Advection-Diffusion POD modes 37 / 43

38 Applications Fluid System - Advection-Diffusion Projection error (singular values decay) 10 2 ɛ(k) k 38 / 43

39 Applications Fluid System - Advection-Diffusion POD-based ROM (k = 1 and k = 2) 39 / 43

40 Applications Fluid System - Advection-Diffusion POD-based ROM (k = 3 and k = 4) 40 / 43

41 Applications Fluid System - Advection-Diffusion POD-based ROM (k = 5 and k = 6) 41 / 43

42 Applications Fluid System - Advection-Diffusion Model reduction error E ROM (t) 10 2 Relative Error k 42 / 43

43 Applications Fluid System - Advection-Diffusion Model reduction error E ROM (t) and projection error E V (t) 10 2 Projection error Total error Error k for this problem, E V (t) dominates E V (t) 43 / 43