Computation and Application of Balanced Model Order Reduction

Transcription

1 Computation and Application of Balanced Model Order Reduction D.C. Sorensen Collaborators: M. Heinkenschloss, A.C. Antoulas and K. Sun Support: NSF and AFOSR MIT Nov 2007

2 Projection Methods : Large Scale Problems Brief Intro to Model Reduction Gramian Based Model Reduction: Solving Large Lyapunov Equations: Balanced Reduction of Oseen Eqns: Domain Decomposition: Balanced Reduction Approximate Balancing 1M vars now possible Extension to Descriptor System Couple Linear with Nonlinear Domains Neural Modeling: Local Reduction Many Interactions

3 LTI Systems and Model Reduction ẋ = Ax + Bu y = Cx + Du A R n n, B R n m, C R p n, D R p m n >> m, p Construct LOW dimensional system ˆx = ŷ = ˆx + ˆBu Ĉˆx + Du Goal: ŷ should approximate y Want the small system response to be the same as the original system response to the same input D.C. Sorensen 3

4 Application Examples 1. Passive devices VLSI circuits Thermal issues 2. Data assimilation North sea forecast Air quality forecast Sensor placement 3. Biological/Molecular systems Honeycomb vibrations MD simulations Heat capacity 4. CVD reactor Bifurcations 5. Mechanical systems: Windscreen vibrations Buildings 6. Optimal cooling Steel profile 7. MEMS: Micro Elec-Mech Systems Elf sensor D.C. Sorensen 4

5 Passive Devices: VLSI circuits 1960 s: IC 1971: Intel : Intel Pentium IV 10µ details 0.18µ details 2300 components 42M components 64KHz speed 2GHz speed 2km interconnect 7 layers D.C. Sorensen 5

6 LTI Systems and Model Reduction Time Domain ẋ = Ax + Bu y = Cx A R n n, B R n m, C R p n, n >> m, p Frequency Domain sx = Ax + Bu y = Cx Transfer Function H(s) C(sI A) 1 B, y(s) = H(s)u(s) This is an p m matrix with rational functions as entries D.C. Sorensen 6

7 Model Reduction Construct a new system {Â, ˆB, Ĉ} with LOW dimension k << n ˆx = ˆx + ˆBu ŷ = Ĉˆx Goal: Preserve system response ŷ should approximate y Rational Approximation : y ŷ = (H(s) H(s))u(s) ˆ D.C. Sorensen 7

8 Model Reduction by Projection Approximate x S V = Range(V) k-diml. subspace i.e. Put x = Vˆx, and then force W T [V ˆx (AVˆx + Bu)] = 0 ŷ = CVˆx If W T V = I k, then the k dimensional reduced model is ˆx = ˆx + ˆBu ŷ = Ĉˆx where  = WT AV, ˆB = W T B, Ĉ = CV. D.C. Sorensen 8

9 Moment Matching Krylov Subspace Projection Based on Lanczos, Arnoldi, Rational Krylov methods Padé via Lanczos (PVL) Freund, Feldmann Bai Multipoint Rational Interpolation Grimme Gallivan, Grimme, Van Dooren Recent: Optimal H 2 approximation via interpolation Gugercin, Antoulas, Beattie D.C. Sorensen 9

10 Gramian Based Model Reduction Proper Orthogonal Decomposition (POD) Principal Component Analysis (PCA) The Gramian ẋ(t) = f(x(t), u(t)), y = g(x(t), u(t)) Eigenvectors of P Orthogonal Basis P = o x(τ)x(τ) T dτ P = VS 2 V T x(t) = VSw(t) D.C. Sorensen 10

11 PCA or POD Reduced Basis Low Rank Approximation x V kˆx k (t) Galerkin condition Global Basis ˆx k = Vk T f(v kˆx k (t), u(t)) Global Approximation Error (H 2 bound for LTI) Snapshot Approximation to P x V kˆx k 2 σ k+1 P 1 m m x(t j )x(t j ) T = XX T j=1 Truncate SVD : X = VSU T V k S k U T k

12 SVD Compression n 100 SVD of Clown m k ( m + n) v.s. m x n Storage k 20 Advantage of SVD Compression D.C. Sorensen 12

13 Image Compression - Feature Detection original rank = 10 rank = 30 rank = 50

14 POD in CFD Extensive Literature Karhunen-Loéve, L. Sirovich Burns, King Kunisch and Volkwein Gunzburger Many, many others Incorporating Observations Balancing Lall, Marsden and Glavaski K. Willcox and J. Peraire D.C. Sorensen 14

15 POD for LTI systems Impulse Response: H(t) = C(tI A) 1 B, t 0 Input to State Map: Controllability Gramian: x(t) = e At B P = o x(τ)x(τ) T dτ = o e Aτ BB T e AT τ dτ State to Output Map: Observability Gramian: y(t) = Ce At x(0) Q = o e AT τ C T Ce Aτ dτ D.C. Sorensen 15

16 Balanced Reduction (Moore 81) Lyapunov Equations for system Gramians AP + PA T + BB T = 0 A T Q + QA + C T C = 0 With P = Q = S : Want Gramians Diagonal and Equal States Difficult to Reach are also Difficult to Observe Reduced Model A k = W T k AV k, B k = W T k B, C k = C k V k PVk = W k S k QW k = V k S k Reduced Model Gramians Pk = S k and Q k = S k. D.C. Sorensen 16

17 Hankel Norm Error estimate (Glover 84) Why Balanced Truncation? Hankel singular values = λ(pq) Model reduction H error (Glover) y ŷ 2 2 (sum neglected singular values) u 2 Extends to MIMO Preserves Stability Key Challenge Approximately solve large scale Lyapunov Equations in Low Rank Factored Form D.C. Sorensen 17

18 CD Player Frequency Response 10 2 Freq Response CD Player : τ = 0.01, n = 120, k = 9 Original Reduced G(jω) Frequency ω

19 CD Player Frequency Response Freq Response CD Player : τ = 0.001, n = 120, k = Original Reduced G(jω) Frequency ω

20 CD Player Frequency Response Freq Response CD Player : τ = 1e 005, n = 120, k = Original Reduced G(jω) Frequency ω

21 CD Player - Hankel Singular Values λ(pq) 10 2 Hankel Singular Values

22 When are Low Rank Solutions Expected = Eigenvalue Decay in P, Q Evals of P, N= Evals σ 1 *1e

23 Approximate Balancing AP + PA T + BB T = 0 A T Q + QA + C T C = 0 Sparse Case: Iteratively Solve in Low Rank Factored Form, P U k U T k, Q L kl T k [X, S, Y] = svd(u T k L k) W k = LY k S 1/2 k and V k = UX k S 1/2 k. Now: PW k V k S k and QV k W k S k D.C. Sorensen 23

24 Balanced Reduction via Projection Reduced model of order k: A k = W T k AV k, B k = W T k B, C k = CV k. 0 = W T k (AP + PAT + BB T )W k = A k S k + S k A T k + B kb T k 0 = V T k (AT Q + QA + C T C)V k = A T k S k + S k A k + C T k C k Reduced model is balanced and asymptotically stable for every k. D.C. Sorensen 24

25 Low Rank Smith = ADI Convert to Stein Equation: AP + PA T + BB T = 0 P = A µ PA T µ + B µ B T µ, where A µ = (A µi)(a + µi) 1, B µ = 2 µ (A + µi) 1 B. Solution: P = A j µb µ B T µ (A j µ) T = LL T, j=0 where L = [B µ, A µ B µ, A 2 µb µ,... ] Factored Form D.C. Sorensen 25

26 Multi-Shift (Modified) Low Rank Smith LR - Smith: Update Factored Form P m = L m L T m: (Penzl, Li, White) L m+1 = [A µ L m, B µ ] = [A m+1 µ B µ, L m ] Multi-Shift LR - Smith: (Gugercin, Antoulas, and S.) Update and Truncate SVD Re-Order and Aggregate Shift Applications Much Faster and Far Less Storage B A µ B; [V, S, Q] = svd([a µ B, L m ]); L m+1 V k S k ; (σ k+1 < tol σ 1 ) D.C. Sorensen 26

27 Approximate Power Method (Hodel) APU + PA T U + BB T U = 0 APU + PUU T A T U + BB T U + P(I UU T )A T U = 0 Thus APU + PUH T + BB T U 0 where H = U T AU Solving AZ + ZH T + BB T U = 0 gives approximation to Z PU Iterate Approximate Power Method Z j US with PU = US (also see Vasilyev and White 05)

28 A Parameter Free Synthesis (P US 2 U T ) Step 1: Solve the reduced order Lyapunov equation Solve H ˆP + ˆPH T + ˆBˆB T = 0. with H = U k T AU k, ˆB = U k T B. Step 2: (APM step) Solve a projected Sylvester equation AZ + ZH T + BˆB T = 0, Step 3: Modify B Update B (I Z ˆP 1 U T )B. Step 4: (ADI step) Update factorization and basis U k Re-scale Z Z ˆP 1/2. Update (and truncate) [U, S] svd[us, Z]. U k U(:, 1 : k), basis for dominant subspace. D.C. Sorensen 28

29 Automatic Shift Selection - Placement? SUPG discretization advection-diffusion operator on square grid k = 32, m = 59, n = 32*32, Thanks Embree Comparison Eigenvalues A vs H eigenvalues A eigenvalues small H eigenvalues full H

30 ɛ-pseudospectra for A from SUPG, n=32*

31 Convergence History, Supg, n = 32, N = 1024 Laptop Iter P + P P + B j ˆB j 1 2.7e-1 1.6e+0 4.7e e-2 1.6e-1 1.5e e-3 1.1e-2 1.3e e-4 3.5e-7 1.3e e-9 1.4e e-7 P f is rank k = 59 Comptime(P f ) = 16.2 secs P P f P = 8.8e 9 Comptime(P) = 810 secs D.C. Sorensen 31

32 Convergence History, Supg, n = 800, N = 640,000 CaamPC Iter P + P P + B j ˆB j 6 1.3e e e e e e e e e e e e e e e e e e-07 P f is rank k = 120 Comptime(P f ) = 157 mins = 2.6 hrs D.C. Sorensen 32

33 A Descriptor System d E 11 dt v(t) = A 11v(t) + A 12 p(t) + B 1 g(t), 0 = A T 12v(t), v(0) = v 0, y(t) = C 1 v(t) + C 2 p(t) + Dg(t). E d v(t) = Av(t) + Bu(t), dt y(t) = Cv(t) + Du(t) Note E is singular index 2

34 Notation Put Ẽ = ΠE 11 Π T, à = ΠA 11 Π T, B = ΠB1, C = CΠ T. With this notation, where à P Ẽ + Ẽ P ÃT + B B T = 0, à T Q Ẽ + Ẽ Q à + C T C = 0. Π = I A 12 (A T 12E 1 11 A 12) 1 A T 12E 1 11 = Θ l Θ T r Π T Projector onto Null(A T 12 )

35 ADI Derivation Step 1 Begin with and derive T ) ÃP (Ẽ + µ Ã) = [(Ẽ µ Ã PÃT + B B ] T. T ) T (Ẽ + µ Ã) P (Ẽ + µ Ã) = (Ẽ µ Ã P (Ẽ µ Ã) 2 Re(µ) B BT. Problem: (Ẽ + µ Ã) is Singular

36 Key Pseudo-Inverse Lemma Suppose Θ T r E 11 Θ r + µ Θ T r A 11 Θ r is invertible. Then the matrix (Ẽ + µ Ã) I Θr ( Θ T r E 11 Θ r + µ Θ T r A 11 Θ r ) 1 Θ T r satisfies ) I (Ẽ + µ Ã (Ẽ + µ Ã) = Π T and ) ) I (Ẽ + µ Ã (Ẽ + µ Ã = Π.

37 Projected Stein Equation P = õPà µ 2 Re( µ) B µ B µ. where I ) à µ (Ẽ + µ Ã) (Ẽ µ Ã, and B I µ (Ẽ + µ Ã) B Solution: P = 2 Re(µ) à j B ) j µ µ B µ (à µ. j=0 Convergent for stable pencil with Real(µ) < 0

38 Key Implementation Lemma If M = Π T M, then the computation I ) Z = (Ẽ + µ Ã) (Ẽ µ Ã M may be accomplished with the following steps. 1. Put F = (E 11 µ A 11 ) M. 2. Solve ( E11 + µ A 11 A 12 A T 12 0 ) ( Z Λ ) = ( F 0 ). Note that Z satisfies Z = Π T Z Similar result holds for computing B µ

39 Algorithm:Single Shift ADI ( E11 + µ A 1. Solve 11 A 12 A T U = Z; 3. while ( not converged ) ) ( Z Λ ) = ( B 0 ) ; 3.1 Z (E 11 µ A 11 () Z; ) ( ) ( E11 + µ A 3.2 Solve (in place) 11 A 12 Z Z A T = 12 0 Λ U [U, Z] ; end 4. U 2 Re(µ) U. ) ;

40 Derivation Multi-Shift ADI Easy to see (P P k ) = Ãk µp (à µ ) k. Hence ) ) Ã(P P k )Ẽ + Ẽ(P Pk)à = ÃÃk µp (Ãk µ Ẽ + ẼÃk µp (Ãk µ à = Âk µ (ÃP Ẽ + ẼPà ) (  k µ ). To get à (P P k ) Ẽ + Ẽ (P P k) à = Âk µ ( B B ) (  k µ ). Where  µ (Ẽ µ Ã) (Ẽ + µ Ã) I.

41 Algorithm:Multi-Shift ADI 1. U = [ ]; 2. while ( not converged ) for i = 1:m, ( E11 + µ 2.1 Solve i A 11 A 12 end A T 12 0 ) ( Z Λ 2.2 U 0 = Z; 2.3 for j = 1:k-1, Z (E 11 µ i A 11) Z; E11 + µ i A 11 A Solve (in place) A T U 0 [U 0, Z] ; end 2.4 U [U, ] 2 Re(µ i ) U 0 ; % Update and truncate SVD(U); 2.5 B (E 11 µ i A 11 ) Z. end ) ( B = 0 Z Λ ) ; = Z 0 ;

42 Model Problem: Oseen Equations v(x, t) + (a(x) )v(x, t) ν v(x, t) + p(x, t) t = χ Ωg (x)g Ω (x, t) in Ω (0, T ), v(x, t) = 0 in Ω (0, T ), ( p(x, t)i + ν v(x, t))n(x) = 0 on Γ n (0, T ), v(x, t) = 0 on Γ d (0, T ), v(x, t) = g Γ (x, t) on Γ g (0, T ), v(x, 0) = v 0 (x) in Ω,

43 Channel Geometry and Grid Figure: The channel geometry and coarse grid EXAMPLE 1. y(t) = x2 v 1 (x, t) + x1 v 2 (x, t)dx Ω obs over the subdomain Ω obs = (1, 3) (0, 1/2).

44 Model Reduction Results n v n p k Table: Number n v of semidiscrete velocities v(t), number n p of semidiscrete pressures p(t), and and size k of the reduced order velocities v(t) for various uniform refinements of the coarse grid.

45 Hankel S-vals and Convergence ADI P Q The largest Hankel singular values Residual (Normalized) Iteration Convergence of the multishift ADI Algorithm Figure: The left plot shows the largest Hankel singular values and the threshold τσ 1. The right plot shows the normalized residuals B k 2 generated by the multishift ADI Algorithm. for the approximate solution of the controllability Lyapunov equation ( ) and of the observability Lyapunov equation ( ).

46 Time (left) and Frequency (right) Responses full reduced full reduced Time Frequency Figure: Time response (left) and frequency response (right) for the full order model (circles) and for the reduced order model (solid line).

47 Velocity Profile full 22K dof reduced 15 dof Figure: Velocities generated with the full order model (left column) and with the reduced order model (right column) at t = , , , , (top to bottom).

48 Domain Decomposition Model Reduction Systems with Local Nonlinearities Linear Model in Large Domain Ω 1 coupled to Non-Linear Model in Small Domain Ω 2 Linear Model provides B.C. s to Non-Linear Model with R. Glowinski D.C. Sorensen 48

49 Simple 1-D Model Problem Ω 1 Ω 2 Ω 3 Ω 1 = ( 10, 1), Ω 2 = ( 1, 1), Ω 3 = (1, 10) Ω 1 : Convection-Diffusion Ω 2 : Burgers Equation Ω 3 : Convection-Diffusion D.C. Sorensen 49

50 Equations y k ρ k t (x, t) µ 2 y k k x 2 (x, t) = S k(x, t), (x, t) Ω k (0, T ), y k (x, 0) = y k0 (x), x Ω k, k = 1, 3, y 1 x ( 10, t) = 0, y 3 (10, t) = 0 x t (0, T ), y 2 ρ 2 t (x, t) µ 2 y 2 2 x 2 + y y 2 2 x (x, t) = 0, (x, t) Ω 2 (0, T ), with appropriate interface conditions y 2 (x, 0) = y 20 (x), x Ω 2,

51 Semi-Discrete Equations M II 2 d dt yi 1 + A II 1 y1 I + M I 1 Γ d dt yγ 12 + A I 1 Γ y12 Γ = B I 1u 1, 2 y2 I + M I 2 Γ d dt yγ + A I 2 Γ y Γ + N I (y2, I y Γ ) = 0, d dt yi 3 + A II 3 y3 I + M I 3 Γ d dt yγ 23 + A I 3 Γ y23 Γ = B I 3u 3. M II 1 d dt yi 2 + A II M II 3 D.C. Sorensen 51

52 Equations to Reduce Inputs to System 1: Outputs of System 1: Apply Model Reduction to M I Γ 1 d dt yγ 12, AI Γ 1 yγ 12 and BI 1 u 1 C ΓI 1 yi 1, MΓI 1 d dt yi 1 + AΓI 1 yi 1 M II 1 d dt yi 1 = A II 1 y1 I A I 1 Γ y12 Γ + B I 1u 1 z I 1 = C I 1y1, I z Γ 1 = A ΓI 1 y1. I D.C. Sorensen 52

53 Dimension Reduction Table: Dimension of the full and of the reduced order models for various discretization parameters N 1, N 2, N 3 and τ = N 1 = N 3 N 2 size of full model size of ROM D.C. Sorensen 53

54 Time Response t Figure: Outputs 1, 2, 3 of the full order system corresponding to the discretization N 1 = N 2 = N 3 = 10 are given by dotted, dashed and solid lines, respectively. Outputs 1, 2, 3 of the reduced order system are given by, and, respecitively. D.C. Sorensen 54

55 State Approximation !5 15! t 5 0!10 0!5 x 5 10 t 0!10 0! x Figure: Solution of the reduced order discretized PDE (left) and error between the solution of the discretized PDE and the reduced order system (right) for discretization N1 = N2 = N3 = 10. D.C. Sorensen 55

56 Neuron Modeling Balanced Truncation on a Compartmental Neuron Model Steve Cox Tony Kellems Undergrads Nan Xiao and Derrick Roos Quasi-active integrate and fire model. Complex model dimension 6000 Faithfully approximated with variable ROM D.C. Sorensen 56

57 90µm Neuron Cell

58 Neuron Model Full Non-Linear Model I j,syn is the synaptic input into branch j a j 2R i xx v j = C m t v j + G Na m 3 j h j (v j E Na ) + G K n 4 j (v j E K ) + G l (v j E l ) + I j,syn (x, t) Kinetics of the potassium (n) and sodium (h, m) channels t m j = α m (v j )(1 m j ) β m (v j )m j t h j = α h (v j )(1 h j ) β h (v j )h j t n j = α n (v j )(1 n j ) β n (v j )n j. D.C. Sorensen 58

59 Linearized Neuron Model Quasi-Active Approximation a j xx ṽ j 2R i = C m t ṽ j + G Na {m 3 hṽ j (3 m j m 2 h + m 3 hj )E Na } +G K {n 4 ṽ j 4ñ j n 3 E K } + G l ṽ j + I j,syn t m j = t h j = σ m ṽ j m j /τ m σ h ṽ j h j /τ h t ñ j = σ n ṽ j ñ j /τ n where τ g 1/(α g (0) + β g (0)) σ g α g (0)(1 g) β g (0)g, g = m, h, n.

60 v Cell Response - Lin and Non-Lin Morphology: RC B.asc h n Lin HH BT Full HH m t (ms)

61 v Cell Response - Linear Morphology: RC B.asc h n Lin HH BT Full HH m t (ms)

62 v Cell Response - Near Threshold 2 h n 1.5 Morphology: RC B.asc Lin HH BT Full HH m t (ms)

63 Hankel Singular Value Decay 10 0 Morphology: RC B.asc Normalized Hankel Singular Values Hankel Singular Value Index of singular value

64 Error Morphology: RC B.asc 5 Stimulated Dendrites Max L 2 Rel. Error Max. Error 25 75% Errorbars 10 2 Error (v lin vs. v BT ) Number of Singular Values used (100 runs/value)

65 Neural ROM Results Interesting example of many-input, single-output system Simulation time single cell 14 sec Full vs.01 sec Reduced Ultimate goal is to simulate a few-million neuron system over a minute of brain-time Currently limited to a 10K neuron system over a few brain-seconds Parallel computing required

66 Summary CAAM TR07-02, M. Heinkenschloss, D. C. S., & K. Sun CAAM TR07-14, K. Sun, R. Glowinski, M. Heinkenschloss, DCS. Gramian Based Model Reduction: Solving Large Lyapunov Equations: Balanced Reduction of Oseen Eqns: Balanced Reduction Approximate Balancing 1M vars now possible Parameter Free Extension to Descriptor System Multi-Shift ADI Without Explicit Projectors: Only need Saddle Point Solver Sparse Direct or Iterative Domain Decomposition - Systems with Local Nonlinearities Neural Modeling - Single Cell ROM Many Interactions