Proper Orthogonal Decomposition (POD)

Transcription

1 Intro Results Problem etras Proper Orthogonal Decomposition () Advisor: Dr. Sorensen CAAM699 Department of Computational and Applied Mathematics Rice University September 5, 28

2 Outline Intro Results Problem etras 1 Introduction/Motivation Low Rank Approimation and Model Reduction for ODEs and PDEs 2 Proper Orthogonal Decomposition() Introduction to in Euclidean Space in General Hilbert Space 3 Numerical Results Solutions from Full and Reduced Systems of PDE: 1D Burgers Equation 4 Problem: for PDEs with nonlinearities Nonlinear Approimation Error and Computational Time

3 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs Problem in finite dimensional space Given y 1,..., y n R m. Let Y = span{y 1, y 2,..., y n } R m ; r = rank(y ) y 1,..., y n R m possible (almost) linearly dependent NOT a good basis for Y Goal: Find orthonormal basis vectors {φ 1,..., φ k } that best approimates Y, for given k < r Solution: Use Low Rank Approimation Form a matri of known data: Y = y 1... y n R m n.

4 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs Low Rank Approimation Singular Value Decomposition (SVD) Let Y R m n, r = rank(y ), and k < r. Problem: Low Rank Approimation min{ Y Ŷ 2 F : rank(ŷ ) = k} Ŷ Solution Ŷ = U k Σ k V T k with min error Y Ŷ 2 F = r i=k+1 σ2 i where Y = UΣV T is SVD of Y ; Σ = diag(σ 1,..., σ r ) R r r with σ 1 σ 2... σ r > Optimal orthonormal basis of rank k =: basis Ŷ = k i=1 σ iu i v T i {φ} k i=1 = {u i} k i=1

5 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs EX: Image Compression via SVD 1 % Low Rank Approimation source:

6 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs EX: Image Compression via SVD 5 % Low Rank Approimation source:

7 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs Model Reduction for ODEs Full-order system (dim = N) d y(t) = Ky(t) + g(t) + N(y(t)) y(t) dt Reduced-order system (dim = k < N) Let y = U k ỹ, for U k R N k, with orthonormal columns: U T k U k = I R k k. U k d dt ỹ(t) = KU k ỹ(t) + g(t) + N(U k y(t)) d dt ỹ(t) = UT k KU k }{{} ỹ(t) + UT k g(t) }{{} +UT k N(U k y(t)) d dt ỹ(t) = Kỹ(t) + g(t) + U T k N(U k ỹ(t)) ỹ(t) How to construct U k?...use!

8 Intro Results Problem etras Low Rank Appro and Model Reduction for ODEs and PDEs Model Reduction for PDEs E. Unsteady 1D Burgers Equation t 2 y(, t) ν y(, t) + 2 ( ) y(, t) 2 = [, 1], t 2 y(, t) = y(1, t) =, t, y(, ) = y (), [, 1], Discretized system (Galerkin) Full-order system (dim = N): FE basis {ϕ i } N i=1 M h d dt y(t) + νk hy(t) N h (y(t)) = y h (, t) = N i=1 ϕ i()y i (t) Reduced-order system (dim = k < N): orthonormal basis {φ i } k i=1 M d dt ỹ(t) + ν Kỹ(t) Ñ(ỹ(t)) = ỹ h(, t) = k i=1 φ i()ỹ i (t) How to construct {φ i } k i=1?...use!

9 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space y Proper Orthogonal Decomposition() is a method for finding a low-dimensional approimate representation of: large-scale dynamical systems, e.g. signal analysis, turbulent fluid flow large data set, e.g. image processing SVD in Euclidean space. Etracts basis functions containing characteristics from the system of interest Generally gives a good approimation with substantially lower dimension 1 basis # basis # basis # Sol of Full System (FE):dim = 1 basis # t

10 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space Definition of Let X be Hilbert Space(I.e. Complete Inner Product Space) with inner product, and norm =, Given y 1,..., y n X. Define y i snapshot i, i. Let Y span{y 1, y 2,..., y n } X. Given k n, generates a set of orthonormal basis of dimension k, which minimizes the error from approimating the snapshots: basis Optimal solution of: n y j ŷ j 2, s.t. φ i, φ j = δ ij min {φ} k i=1 j=1 where ŷ j () = k i=1 y j, φ i φ i (), an approimation of y j using {φ} k i=1. Solve by SVD

11 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space Derivation for in Euclidean Space: E.g. X = R m min {φ} k i=1 s.t. n j=1 y j k i=1 (y T j φ i )φ i 2 2 E(φ 1,..., φ k ) φ T i φ j = δ ij = { 1 if i = j if i j i, j = 1,..., k Lagrange Function: k L(φ 1,..., φ k, λ 11,..., λ ij,..., λ kk ) = E(φ 1,..., φ k ) + λ ij (φ T i φ j δ ij ) i,j=1 KKT Necessary Conditions: φ i L = { n j=1 y j(y T j φ i ) = λ ii φ i λ ij =, ifi j }. φ T i φ j = δ ij

12 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space Optimal Condition: Symmetric m-by-m eigenvalue problem YY T φ i = λ i φ i where λ i = λ ii, Y = y 1... y n for i = 1,..., k Error for basis: n y i j=1 k (yj T φ i )φ i 2 2 = i=1 r i=k+1 λ i φ i = YY T φ i = n j=1 y j(yj T φ i ) λ i = n j=1 (y j T φ i ) 2 Since φ T i φ j = δ ij and span(y ) = span{φ 1,..., φ r } (from SVD), then y j = r i=1 (y j T φ i )φ i, forj = 1,..., n, and n y j j=1 k n (yj T φ i )φ i 2 2 = i=1 j=1 i=k+1 λ i r (yj T φ i ) 2 = r i=k+1 λ i

13 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space SOLUTION for basis in R m Recall Optimality Condition and the Error: YY T φ i = λ i φ i, i = 1,..., k n j=1 y j k i=1 (y j T φ i )φ i 2 2 = r i=k+1 λ i Optimal solution: basis: {φ i }k i=1 = {u i} k i=1 Lagrange multiplier: λ i = σ 2 i, i = 1,..., k, can be obtained by the SVD of Y R m n or EVD of YY T R m m, i.e., Y = UΣV T YY T u i = σ 2 i u i, i = 1,..., r, where Σ = diag(σ 1,..., σ r ) R r r with σ 1 σ 2... σ r > ; U = [u 1,..., u r ] R m r and V = [v 1,..., v r ] R n r have orthonormal columns. This is equivalent to SVD solution for Low Rank Approimation

14 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space SOLUTION for basis in a Hilbert Space X Two approaches: 1 Define linear symmetric operator F (w) = n j=1 w, y j y j, w X. Find eigenfunction u i X: EVD: F (u i ) = σ 2 i u i n j=1 u i, y j y j = σ 2 i u i (cf. YY T u i = σ 2 i u i ) Sol: φ i = u i, λ i = σ 2 i, i = 1,..., k 2 Define linear symmetric operator L = [ y i, y j ] R n n. Find eigenvector v i R n : EVD: L v i = σ 2 i v i [ y i, y j ]v i = σ 2 i v i (cf. Y T Yv i = σ 2 i v i ) Sol: φ i = 1 σ i n j=1 (v i) j y j, λ i = σ 2 i U k = YV k Σ 1 k NOTE: v i R n,but u i, y j X

15 Intro Results Problem etras Intro in Euclidean Space in General Hilbert Space Remark: Practical way for computing the basis for discretized PDEs Let {ϕ i } N i=1 X H1 (Ω) be finite element(fe) basis. FE snapshots(solutions): {y j } n i=j X at time {t j } n j=1. y j = y(, t j ) = N Ŷ ij (t j )ϕ i (). i=1 m L = [ y i, y j ] = [ Ŷ ik Ŷ jl ϕ i, ϕ j ] = Ŷ T MŶ Rn n, k,l=1 where M = [ ϕ i, ϕ j ] R N N. basis: φ i = 1 n (v i ) j y j, σ i where L v i = σi 2 v i, with v 1, v 2,..., v k corresponding to σ 1 σ 2... σ k >. j=1

16 y Intro Results Problem etras Solutions from Full and Reduced Systems of PDE: 1D Burgers Equation y Numerical Results for PDEs Recall the 1D Burgers Equation( [,) 1], t : 2 t y(, t) ν 2 y(, t) + y(,t) 2 2 =, y(, t) = y(1, t) =, t, y(, ) = y (), [, 1]. 1 Singular values of the Snapshots Sol of Reduced System (Direct ):dim = 6 Sol of Full System (Finite Element):dim = y(,t) 1.5 Sol of Full System (Finite Element) :dim = 1 t= t=.229 t= Sol of Reduced System (Direct ):dim = 2 y(,t) Sol of Reduced System (Direct ):dim = 1 t= Sol of Reduced System (Direct ):dim = 3 t=.229 t= t t.5 1 y(,t) Sol of Reduced System (Direct ):dim = 4 t= t=.229 t= y(,t) t= Sol of Reduced System (Direct ):dim = 5 t=.229 t= y(,t) t= t=.229 t= y(,t) t= t=.229 t=

17 Intro Results Problem etras Nonlinear Approimation Error and CPU Time Problem: for PDEs with nonlinearities If we apply the basis directly to construct a discretized system, the original system of order N: become a system of order k N: where the nonlinear term : M h d dt y(t) + νk hy(t) N h (y(t)) = M d ỹ(t) + ν Kỹ(t) Ñ(ỹ(t)) =, dt Ñ(ỹ(t)) = U T k }{{} k N N h (U k ỹ(t)) }{{} N 1 Computational Compleity still depends on N!!

18 Intro Results Problem etras Nonlinear Approimation Error and CPU Time Nonlinear Approimation Recall the problem from bf Direct : Ñ(ỹ(t)) = U T N }{{} h (Uỹ(t)) }{{} k N N 1 WANT: Ñ(ỹ(t)) }{{} C ˆN(ỹ(t)) }{{} k n m n m 1 k, n m N Independent of N 2 Approaches: Precomputing Technique: Simple nonlinear Empirical Interpolation Method (EIM): General nonlinear

19 Intro Results Problem etras Nonlinear Approimation Error and CPU Time ACCURACY vs. COMPLEXITY Relative Error:= sum y FE (t) y(t) / y FE (t) of Reconstruct Snapshots using t 1 1 Direct Precompute 1 EIM.35.3 CPU TIME for solving reduced order ODE system Direct Precompute EIM Average Relative Error cpu time (sec) Number of basis used Number of basis used Figure: LEFT: Error E avg = 1 nt ỹ h (,t i ) y h (,t i ) X n t i=1 y h (,t i ) X.RIGHT: CPU time (sec)

20 Intro Results Problem etras Nonlinear Approimation Error and CPU Time QUESTION?

21 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F Empirical Interpolation Method (EIM) [Patera; 24] Approimates nonlinear parametrized functions Let s(; µ) be a parametrized function with spatial variable Ω and parameter µ D. Function approimation from EIM: ŝ(; µ) = n m m=1 q m ()β m (µ), where span{q m } nm m=1 M s {s( ; µ) : µ D} and β m (µ) is specified from the interpolation points {z m } nm m=1, in Ω: for i = 1,..., n m. s(z i ; µ) = n m m=1 q m (z i )β m (µ),

22 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F EIM: Numerical Eample s(; µ) = (1 )cos(3πµ( + 1))e (1+)µ, where [ 1, 1] and µ [1, π]. Plot of Approimate Functions (dim = 1) with Eact Functions (in black solid line) 2 µ= µ= µ= µ= s(;µ)

23 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F Plots of Numerical Solutions from 3 Approaches 1 5 Direct Precomputed EIM- 1 Singular values of the Snapshots y(,t) y(,t) Sol of Full System (Finite Element) :dim = 1 t= t=.229 t= Sol of Reduced System (Direct ):dim = Sol of Reduced System (Direct ):dim = 4 t= t=.229 t= y(,t) y(,t) Sol of Reduced System (Direct ):dim = 1 t= Sol of Reduced System (Direct ):dim = 3 t=.229 t= t= Sol of Reduced System (Direct ):dim = 5 t=.229 t= y(,t) t= t=.229 t= y(,t) t= t=.229 t= Figure: SVD Figure: Direct

24 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F y(,t) Sol of Full System (Finite Element) :dim = 1 1 t= t=.229 t= y(,t) Sol of Reduced System (Precompute ):dim = 1 t=.8 t=.229 t= y(,t) Sol of Full System (Finite Element) :dim = 1 1 t= t=.229 t= y(,t) Sol of Reduced System (EIM ):dim = 1 t=.8 t=.229 t= Sol of Reduced System (Precompute ):dim = Sol of Reduced System (Precompute ):dim = Sol of Reduced System (EIM ):dim = Sol of Reduced System (EIM ):dim = 3 y(,t) t= t=.229 t= Sol of Reduced System (Precompute ):dim = 4 y(,t) t= t=.229 t= Sol of Reduced System (Precompute ):dim = 5 y(,t) t= t=.229 t= Sol of Reduced System (EIM ):dim = 4 y(,t) t= t=.229 t= Sol of Reduced System (EIM ):dim = 5 y(,t) t= t=.229 t= y(,t) t= t=.229 t= y(,t) t= t=.229 t= y(,t) t= t=.229 t= Figure: Precomputed Figure: EIM-

25 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F Conclusions Unique Contribution: Clear description of the EIM Successful Implementation of EIM with EIM is comparable to widely accepted methods, such as precomputing technique The results suggest that EIM with basis is a promising model reduction technique for more general nonlinear PDEs. Future Work Etend to higher dimensions Etend to PDEs with more general nonlinearities Apply to practical problem, e.g. optimal control problem CPU time (sec) Average Relative Error n Relative Error:=(1/n )sum t ( y FE (t) y(t) / y FE (t) ) 2 of t i=1 Reconstructed Snapshots using different dim of basis from EIM EIM dim=1 EIM dim=2 EIM dim=4 EIM dim=6 EIM dim=1 EIM dim=2 Direct Precompute Number of basis used CPU TIME for solving reduced order ODE system Number of basis used EIM dim=1 EIM dim=2 EIM dim=4 EIM dim=6 EIM dim=1 EIM dim=2 Direct Precompute Figure: Plots of the first 4 basis

26 Intro Results Problem etras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and F Overview Nonlinear PDE FE-Galerkin FULL Discretized System (ODE) dim = N Solve ODE SNAPSHOTS: {y(, t l )} ns l=1 basis: {φ i } k i=1 -Galerkin REDUCED Discretized System (ODE) Linear Term: dim = k N Non-Linear Term: dim N Nonlinear Appro -EIM -Precomputing REDUCED Discretized System (ODE) Linear Term: dim = k N Non-Linear Term: dim = n m N (i) Memory Saving (ii) Accuracy (iii) Efficiency (Time Saving)