Model Order Reduction Techniques

Transcription

1 Model Order Reduction Techniques SVD & POD M. Grepl a & K. Veroy-Grepl b a Institut für Geometrie und Praktische Mathematik b Aachen Institute for Advanced Study in Computational Engineering Science (AICES) RWTH Aachen Sommersemester / 24

2 Singular Value Decomposition Satz For each matrix Y R m n there exists orthogonal matrices U R m m, V R n n and a diagonal matrix where such that Here: Σ := diag(σ 1,..., σ p ) R m n, Singular values of Y σ 1 σ 2... σ p 0, Y = U Σ V T. : σ i, i = 1,..., p p = min(m, n), Left singular vectors : columns of U = [u 1 u 2... u m ] Right singular vectors : columns of V = [v 1 v 2... v n ] 2 / 24

3 Singular Value Decomposition Case: m = 4, n = 2, rank(y ) = 2 Full SVD 0 = Y U Σ Reduced SVD = Y Û [ ] 0 0 ˆΣ [ ] V T [ ] V T 3 / 24

4 Singular Value Decomposition Assumption: m n Case m < n: consider SVD of Y T Y T = U Σ V T Y = V Σ U T Singular values are real and non-negative Convention σmax = σ 1 largest singular value σmin = σ n smallest singular values ordered according to magnitude σ 1 σ 2... σ n 0 Numerical computation of SVD is backward stable Y + Y = Ũ Σ Ṽ T with Y 2 = O(eps) 4 / 24

5 Singular Value Decomposition Geometry Quelle: Trefethen & Bau Mapping of unit sphere S = {x : x 2 = 1} under Y Singular values: lengths σ 1, σ 2 of principal semiaxes of Y S. Left singular vectors: unit vectors {u 1, u 2 } in direction of principal semiaxes of Y S Right singular vectors: unit vectors {v 1, v 2 } S, preimages of principal semiaxes of Y S, so that Y v j = σ j u j. 5 / 24

6 Singular Value Decomposition Geometry Quelle: Trefethen & Bau Mapping of unit sphere S = {x : x 2 = 1} under Y Singular values: lengths σ 1, σ 2 of principal semiaxes of Y S. Left singular vectors: unit vectors {u 1, u 2 } in direction of principal semiaxes of Y S Right singular vectors: unit vectors {v 1, v 2 } S, preimages of principal semiaxes of Y S, so that Y v j = σ j u j. 6 / 24

7 Singular Value Decomposition Geometry Quelle: Trefethen & Bau Mapping of unit sphere S = {x : x 2 = 1} under Y Singular values: lengths σ 1, σ 2 of principal semiaxes of Y S. Left singular vectors: unit vectors {u 1, u 2 } in direction of principal semiaxes of Y S Right singular vectors: unit vectors {v 1, v 2 } S, preimages of principal semiaxes of Y S, so that Y v j = σ j u j. 7 / 24

8 Singular Value Decomposition Geometry Quelle: Trefethen & Bau Mapping of unit sphere S = {x : x 2 = 1} under Y Singular values: lengths σ 1, σ 2 of principal semiaxes of Y S. Left singular vectors: unit vectors {u 1, u 2 } in direction of principal semiaxes of Y S Right singular vectors: unit vectors {v 1, v 2 } S, preimages of principal semiaxes of Y S, so that Y v j = σ j u j. 8 / 24

9 Singular Value Decomposition Geometry 9 / 24

10 Let Y = U Σ V T be an SVD of Y R m n with singular values σ 1... σ d σ d+1 =... = σ n = 0. We then have: Y v i = σ i u i, Y T u i = σ i v i, i = 1,..., n. Rang(Y ) = d = number of singular values not equal to zero. Y 2 = σ 1 = σ max Y F = σ σ2 n Y 1 2 = 1 σ n, if Y R n n regular κ 2 (Y ) = Y 2 Y 1 2 = σ 1 σ n, if Y R n n regular κ 2 (Y ) = Y 2 Y + 2 = σ 1 σ n = σmax σ min, if Y regular 10 / 24

11 Let Y = U Σ V T be an SVD of Y R m n with singular values σ 1... σ d σ d+1 =... = σ n = 0. We then have: The strictly positiv singular values are the roots of the strictly positiv eigenvalues of Y T Y : {σ i i = 1,..., d} = { λ i (Y T Y ) i = 1,..., d} The singular values are equal to the absolute values of the eigenvalues of Y if Y = Y T. For Y R n n it holds det(y ) = m σ i i=1 The pseudo inverse Y + R n m is defined as Y + = V Σ + U T where Σ + = diag(σ 1 1,..., σ 1 d, 0,..., 0) Rn m. 11 / 24

12 Low-Rank Approximation Let Y = U Σ V T be an SVD of Y R m n with singular values σ 1... σ d σ d+1 =... = σ n = 0. For k d = Rang(Y ) define Y k = U Σ k V T where Σ k = diag(σ 1,..., σ k, 0,..., 0) R m n. We then have: Rang(Y ) = k. The distance between Y k and Y in the 2-norm is Y Y k 2 = σ k+1. Y k is the best approximation of Y of rank k Y Y k 2 = min Y B 2. Rang(B) k 12 / 24

13 Image Compression Singular values decomposition of Y can be written as p Y = σ i u i vi T i=1 where p = min(m, n). Y = U Σ V T = σ 1 u 1 v T 1 + σ 2 u 2 v T σ p u p v T p Approximation of matrix Y of rank k, k p, is and Y Y k 2 = σ k+1. Y k = k i=1 σ i u i v T i Data compression: Memory requirement of Y k is k(m + n) vs. mn for Y. 13 / 24

14 Image Compression Consider black and white picture I(x, y) as a matrix Y : matrix entries correspond to the gray-level of the pixels Singular Values, Melencolia I Depending on the area of application: Principal Component Analysis (PCA), (POD), Karhunen-Loéve Decomposition. 14 / 24

15 Image Compression k = 1, compression = k = 20, compression = 0.11 k = 40, compression = Original: A. Duerer, Melencolia I 15 / 24

16 Image Compression Gatlinburg Conference: J.H. Wilkinson, W. Givens, G. Forsythe, A. Householder, G. Henrici, F.L. Bauer (von links nach rechts) 16 / 24

17 Preliminaries Parametrized Problems SVD & POD We consider the problem of approximating all spatial vectors {y j } n j=1 of Y simultaneously by a single, normalized vector as well as possible. This problem can be expressed as n (P 1 ) max (y j, u) R m 2 subject to (s.t.) u 2 u R m R = 1 m j=1 Solution: u 1 solves (P 1 ) and arg max(p 1 ) = σ 2 1 = λ 1. Find second vector, orthogonal to u 1, that describes {y j } n j=1 as well as possible (P 2 ) max u R m n (y j, u) R m 2 j=1 s.t. u 2 R m = 1, (u, u 1 ) R m = 0 Solution: u 2 solves (P 2 ) and arg max(p 2 ) = σ 2 2 = λ / 24

18 Preliminaries Parametrized Problems SVD & POD We consider the problem of approximating all spatial vectors {y j } n j=1 of Y simultaneously by a single, normalized vector as well as possible. This problem can be expressed as n (P 1 ) max (y j, u) R m 2 subject to (s.t.) u 2 u R m R = 1 m j=1 Solution: u 1 solves (P 1 ) and arg max(p 1 ) = σ 2 1 = λ 1. Find second vector, orthogonal to u 1, that describes {y j } n j=1 as well as possible (P 2 ) max u R m n (y j, u) R m 2 j=1 s.t. u 2 R m = 1, (u, u 1 ) R m = 0 Solution: u 2 solves (P 2 ) and arg max(p 2 ) = σ 2 2 = λ / 24

19 Preliminaries Parametrized Problems SVD & POD Theorem Let Y = [y 1,..., y n ] R m n be a given matrix with rank d min{m, n}. Further, let Y = UΣV T be the singular value decomposition of Y, where U = [u 1,..., u m ] R m m, V = [v 1,..., v n ] R n n are orthogonal matrices and the matrix Σ R m n contains the singular values. Then, for any l {1,..., d} the solution to (P l ) max ũ 1,...,ũ l R m l n i=1 j=1 (y j, ũ i ) R m 2 s.t. (ũ i, ũ j ) R m = δ ij, 1 i, j l is given by the singular vector {u i } l i=1, i.e., by the first l columns of U. Moreover, arg max(p l ) = l i=1 σ 2 i = l i=1 λ i. 19 / 24

20 Preliminaries Parametrized Problems Optimality of the POD basis Theorem Suppose that Û d R m d denotes a matrix with pairwise orthonormal vectors û i and that the expansion of the columns of Y in the basis {û i } d i=1 be given by Y = Û d C d, with C d ij = (û i, y j ) R m, 1 i d, 1 j n. Then for every l {1,..., d} we have Y U l B l F Y Û l C l F where F denotes the Frobenius norm, the matrix U l denotes the first l columns of U, B l the first l rows of B and similarly for Û l and C l. S. Volkwein. Model Reduction Using. Skript Universität Konstanz, / 24

21 Preliminaries Parametrized Problems A POD Approach (POD) (or Karhunen-Loève expansion) approach: POD Spaces X POD N = arg inf spaces X N span{u(µ) µ Ξ train } u Π X N u L 2 (Ξ train ;X) "Best" approximation error Note: X POD N ε POD N are hierarchical, Weaker norm over Ξ train, u Π X POD u L N 2 (Ξ train ;X) Optimization can be solved using "method of snapshosts." 21 / 24

22 Preliminaries Parametrized Problems A POD Approach Method of Snapshots 1. Form correlation matrix C POD R n train n train given by C POD ij = 1 n train (u(µ i train ), u(µj train )) X, 1 i, j n train. 2. Solve for eigenpairs (ψ POD,k R n train, λ POD,k R +0 ), 1 k n train, from C POD ψ POD,k = λ POD,k ψ POD,k, (ψ POD,k ) T X ψ POD,k = Arrange eigenvalues in descending order λ POD,1 λ POD,1... λ POD,n train / 24

23 Preliminaries Parametrized Problems A POD Approach Method of Snapshots 4. POD Basis functions Ψ POD,k n train ψm POD,k u(µ m train ), 1 k n train. m=1 POD reduced basis space X POD N = span{ψ POD,n, 1 n N}, 1 N N max Error bound: define N max as the smallest N such that ( ε POD N ) n train k=n+1 λ POD,k ε tol,min. 23 / 24

24 Preliminaries Parametrized Problems A POD Approach Method of Snapshots Since (Ψ POD,n, Ψ POD,n ) X = δ nm, 1 n, m n train, it follows that (Ψ POD,n )ξ n = ζ n, 1 n N max, and hence Z R N N is given by Note: Z = [Ψ POD,1 Ψ POD,2... Ψ POD,N ]. n train A N -solves and n 2 train X-inner products to form C Solve for largest N max eigenvalues/eigenvectors of C higher parameter dimensions prohibitive because of large n train. 24 / 24