New Directions in the Application of Model Order Reduction

Transcription

1 New Directions in the Application of Model Order Reduction D.C. Sorensen Collaborators: M. Heinkenschloss, K. Willcox S. Chaturantabut, R. Nong Support: AFOSR and NSF MTNS 08 Blacksburg, VA July 2008

2 Projection Methods for MOR Impossible Calculations Made Possible with ROM Experiments with many instances of same reduced model Brief Intro to Gramian Based Model Reduction Proper Orthogonal Decomposition (POD) Balanced Reduction Neural Modeling: Nonlinear MOR: Process/Design Variation: Local Reduction Many Interactions Application of Empirical Interpolation (EIM) Monte-Carlo via ROM

3 LTI Model Reduction by Projection ẋ = Ax + Bu y = Cx Approximate x S V = Range(V), a k-diml. subspace i.e. Put x = Vˆx, and then force W T [V ˆx (AVˆx + Bu)] = 0 ŷ = CVˆx If W T V = I k, then the k dimensional reduced model is ˆx = ˆx + ˆBu ŷ = Ĉˆx where  = W T AV, ˆB = W T B, Ĉ = CV.

4 Moment Matching Krylov Subspace Projection Based on Lanczos, Arnoldi, Rational Krylov methods Padé via Lanczos (PVL) Freund, Feldmann Bai Multipoint Rational Interpolation Grimme Gallivan, Grimme, Van Dooren Recent: Optimal H 2 approximation via interpolation Gugercin, Antoulas, Beattie

5 Gramian Based Model Reduction Proper Orthogonal Decomposition (POD) Principal Component Analysis (PCA) ẋ(t) = f(x(t), u(t)), y = g(x(t), u(t)) The Gramian Eigenvectors of P P = o x(τ)x(τ) T dτ P = VS 2 V T Orthogonal Basis x(t) = VSw(t)

6 PCA or POD Reduced Basis Low Rank Approximation x V kˆx k (t) Galerkin condition Global Basis ˆx k = Vk T f(v kˆx k (t), u(t)) Global Approximation Error? (H 2 bound for LTI) Snapshot Approximation to P x V kˆx k 2 σ k+1 P 1 m m x(t j )x(t j ) T = XX T j=1 Truncate SVD : X = VSU T V k S k U T k

7 SVD Compression n 100 SVD of Clown m k ( m + n) v.s. m x n Storage k 20 Advantage of SVD Compression

8 Image Compression - Feature Detection original rank = 10 rank = 30 rank = 50

9 POD in CFD Extensive Literature Karhunen-Loéve, L. Sirovich Burns, King Kunisch and Volkwein Gunzburger Many, many others Incorporating Observations Balancing Lall, Marsden and Glavaski K. Willcox and J. Peraire

10 POD vs. FEM Both are Galerkin Projection POD - Global Basis fns. vs FEM - Local Basis fns. FEM - Complex Behavior via Mesh Refinement/ Higher Order High Dimension - Sparse Matrices POD - Complex Behavior contained in Global Basis Low Dimension - Dense Matrices Caveat: POD is ad hoc: Must sample rich set of inputs and parameter settings Qx: How to automate parameter/input sampling for POD

11 POD for LTI systems Impulse Response: H(s) = C(sI A) 1 B, s 0 Input to State Map: Controllability Gramian: x(t) = e At B P = o x(τ)x(τ) T dτ = o e Aτ BB T e AT τ dτ State to Output Map: Observability Gramian: y(t) = Ce At x(0) Q = o e AT τ C T Ce Aτ dτ

12 Balanced Reduction (Moore 81) Lyapunov Equations for system Gramians AP + PA T + BB T = 0 A T Q + QA + C T C = 0 With P = Q = S : Want Gramians Diagonal and Equal States Difficult to Reach are also Difficult to Observe Reduced Model A k = W T k AV k, B k = W T k B, C k = C k V k PVk = W k S k QW k = V k S k Reduced Model Gramians Pk = S k and Q k = S k.

13 Hankel Norm Error estimate (Glover 84) Why Balanced Truncation? Hankel singular values = λ(pq) Model reduction H error (Glover) y ŷ 2 2 (sum neglected singular values) u 2 Extends to MIMO Preserves Stability Key Challenge Approximately solve large scale Lyapunov Equations in Low Rank Factored Form

14 CD Player Frequency Response y ŷ 2 2 (sum neglected singular values) u 2 Freq Response CD Player : τ = 0.001, n = 120, k = Original Reduced G(jω) Frequency ω

15 CD Player Frequency Response y ŷ 2 2 (sum neglected singular values) u 2 Freq Response CD Player : τ = 1e 005, n = 120, k = Original Reduced G(jω) Frequency ω

16 CD Player - Hankel Singular Values λ(pq) y ŷ 2 2 (sum neglected singular values) u Hankel Singular Values

17 Approximate Balancing AP + PA T + BB T = 0 A T Q + QA + C T C = 0 Sparse Case: Iteratively Solve in Low Rank Factored Form, P U k U T k, Q L kl T k [X, S, Y] = svd(u T k L k) W k = LY k S 1/2 k and V k = UX k S 1/2 k. Now: PW k V k S k and QV k W k S k

18 Recent Progress LTI MOR Low Rank Approximate Solutions to Lyapunov Eqns n = 1M Now Possible Large Scale BTMOR Possible Optimal H2 reduction IRKA Promising for Large no. Inputs Descriptor Systems e.g. Stykel (LAA 06) general theory and approach

19 Balanced Truncation MOR of Oseen Eqn. Semi-Discrete Oseen Equations: A Descriptor System E d v(t) = Av(t) + Bu(t), dt y(t) = Cv(t) + Du(t) 150 full reduced full reduced Time Frequency Figure: Time response (left) and frequency response (right) for the full order model (circles) and for the reduced order model (solid line).

20 Velocity Profile full 22K dof reduced 15 dof Figure: Velocities generated with the full order model (left column) and with the reduced order model (right column) at t = , , , , (top to bottom). Heinkenschloss, Sun, S., SISSC 08

21 NOTICE State Variables will be y for remainder

22 Reduced Order Neural Modeling Steve Cox Tony Kellems LINEAR MODELS Balanced Truncation Optimal H 2 Nan Xiao and Derrick Roos Ryan Nong Complex Model (Dim 160 K) 20 variable ROM NONLINEAR MODELS Empirical Interpolation (EIM) - Patera Saifon Chaturantabut T. Kellems Nonlinear H-H Neuron Model (Dim 1198) 30 variable ROM Complex nonlinear behavior well approximated

23 Neuron Image AR A Image from Neuromart J.O. Martinez, Rice-Baylor Archive of Neuronal Morphology, cox/neuromart/, accessed 29 July 08 Developed using Neurolucida and NeuroExplorer

24 90µm Neuron Cell

25 Hodgkin-Huxley Neuron Model Full Non-Linear Model I j,syn is the synaptic input into branch j a j 2R i xx v j = C m t v j + G Na m 3 j h j (v j E Na ) + G K n 4 j (v j E K ) + G l (v j E l ) + I j,syn (x, t) Kinetics of the potassium (n) and sodium (h, m) channels t m j = α m (v j )(1 m j ) β m (v j )m j t h j = α h (v j )(1 h j ) β h (v j )h j t n j = α n (v j )(1 n j ) β n (v j )n j.

26 Ultimate Goal for Neural Modeling Ultimate goal is to simulate a few-million neuron system over a minute of brain-time: Feasibility Demonstrated Currently limited to a 10K neuron system over a few brain-seconds without new technology Single Cell Simulation Time Reduction: times ROM computation time: seconds - few minutes neuron types sufficient for Cortex Parallel computing required - under development(kellems)

27 v Cell Response - Lin and Non-Lin Morphology: RC B.asc h n Lin HH BT Full HH m t (ms)

28 v Cell Response - Linear Morphology: RC B.asc h n Lin HH BT Full HH m t (ms)

29 v Cell Response - Near Threshold 2 h n 1.5 Morphology: RC B.asc Lin HH BT Full HH m t (ms)

30 Optimal H 2 Methods: IRKA PROBLEM: Many inputs Controllability Gramian Not Low Rank Kellems and Nong Using Variant of IRKA Gugercin, Antoulas, Beattie (2008) Reduced 160K Neuron Model to a ROM of order 20. Solve times to construct ROM are under 2 mins High End Workstation(Sun Ultra 20) in MATLAB Parameter Study Experiment 158 hrs (full) 3.4 hrs (ROM)

31 ROM Results on Realistic Neuron AR A (Rosenkranz lab) Full system size 6726 Reduced system size 25 B 60µm Soma Potential w.r.t. rest (mv) Comparison of Soma Voltages Nonlinear Linearized Reduced (k = 25) Soma Voltage Error (mv) 10 2 Errors in Soma Voltage (Lin. vs. Reduced) Abs. Error Rel. Error Time (ms) Time (ms)

32 Error vs HS-Values: AR A Error follows Hankel singular value decay: AR A, 35 Stimuli A 0 Max. Abs. Error 25 75% Error Bars Normalized HSV s log 10 of Error (v lin vs. v BT ) Number of Singular Values used (10 runs/value)

33 Highly Branched Neuron n408 (Pyapali et al., 1998) Full system size 41, , 330 Reduced system C size µm A Seconds Time to compute reduced model using IRKA 1000 h = 2 µm (N = 41364) 800 h = 0.5 µ m (N = ) B log 10 of Max. Abs. Error Output error vs. full system k (reduced system size) k (reduced system size)

34 Model Reduction of Nonlinear Terms Saifon Chaturantabut Implemenation of EIM method of Patera et.al. (2004) Test Case: FitzHugh-Nagumo equations εv t (x, t) = ε 2 v xx (x, t) + f (v(x, t)) w(x, t) w t (x, t) = βv(x, t) γw(x, t), f (v) = v(v 0.1)(1 v) IC s and BC s v(x, 0) = 0, w(x, 0) = 0 x [0, L] v x (0, t) = i 0 (t), v x (L, t) = 0 t 0 After FEM discretization, Ey t = Ay + g(t) + N (y), y(0) = 0

35 Problem with Direct POD If we apply the POD basis directly to construct a discretized system, the original system of order N: become a system of order k N: where the nonlinear term : E d dt y(t) = Ay(t) + g(t) + N (y(t)) Ẽ d dt ỹ(t) = Ãỹ(t) + g(t) + Ñ (ỹ(t)), Ñ (ỹ(t)) = }{{} U T N (Uỹ(t)) }{{} k N N 1 Computational Complexity still depends on N!!

36 Nonlinear Approximation via EIM WANT: Ñ (ỹ(t)) }{{} C ˆN (ỹ(t)) }{{} k n m n m 1 k, n m N Complexity k Independent of N

37 EIM Steps Ey t = Ay + g(t) + N (y), y(0) = 0 1) Run trajectory and collect snapshots Y = [y 1, y 2,... y m ] perhaps with several different inputs and parameter values. 2) Truncate SVD of snapshots to get a POD basis for Trajectory 3) Collect nonlinear snapshots s j = f (y j ) = [f (y 1,j ), f (y 2,j ),..., f (y N,j )] T 4) Truncate SVD of nonlinear snapshots to get another POD basis for the nonlinear term 5) Select EIM interpolation points and approximate nonlinear term via collocation in the non-linear POD basis 6) Construct the nonlinear ROM from the reduced linear and nonlinear terms

38 Nonlinear Approximation via EIM Contd. HOW: C ˆN (ỹ(t)) = (U T ) [Ψ(x)] T Q(x)dx } Ω {{ } C ( Q 1 z f z (t) ) } {{ } ˆN (ỹ(t)) where fz (t) = [f (Φ(z 1 )ỹ(t)), f (Φ(z 2 )ỹ(t)),... f (Φ(z nm )ỹ(t))] T {zj } Empirical Interpolation Points and where Ψ(x) = [ψ1 (x), ψ 2 (x),... ψ N (x)] - FEM basis Φ(x) = [φ1 (x), φ 2 (x),... φ k (x)] - POD basis - From Snapshots Ψ(x)y(t l ) Q(x) = [q1 (x), q 2 (x),... q nm (x)] - Nonlinear POD basis - From Snapshots s l = f (Ψ(x)y(t l ))

39 EIM: Numerical Example f (y; µ) = (1 y)cos(3πµ(y + 1))e (1+y)µ, 6 POD basis fns from 50 snapshots µ [1, π] uniform Plot of Approximate Functions (dim = 10) with Exact Functions (in black solid line) 2 µ= µ= µ= µ= s(x;µ) x

40 EIM Reduction of FitzHugh-Nagumo Fiber click figure for movie

41 EIM Reduction of FitzHugh-Nagumo Fiber Nonlinear Reduction N = 1024 k = 40 click figure for movie

42 EIM for Hodgkin-Huxley Equations Kellems After FEM or FD discretization HH-equations yield ODE system a(x)c mi I I I v t m t h t n t = H 2R i succinctly written as Ey t = Ay + g(t) + N (y) Implementation Issues v m h n g 0 (t) ) Choose input stimuli to generate snapshots over full wave 2) NL-snapshots generated for nonlinear term N v (v, m, h, n) N v (v, m, h, n) N m (v, m) N h (v, h) N n (v, n) 3) Terms N m (v, m), N h (v, h), N n (v, n) evaluated only at EIM points 3 7 5

43 EIM Reduction of Hodgkin-Huxley Fiber Three Inputs Forked neuron: voltages at node 10 Full (N = 1198) Reduced (k,nm = 30) Voltage (mv) Time (ms)

44 EIM Reduction of HH : 3 inputs Voltage Profile at Various Times (Full vs ROM)

45 Blade Variation Turbine Reliability/Performance Siemens press picture Small variations among blades large impact on forced response Determines high-cycle fatigue properties

46 Parametric ROM for Geometric Shape Variations T. Bui-Thanh,, K. Willcox, and O. Ghattas 2007 Efficient reduced order models for probabilistic analysis of the effects of blade geometry 2D problem governed by the Euler equations Three orders of magnitude reduction in number of states Accurately reproduce CFD Monte Carlo simulation results at fraction of computational cost. Results impossible without ROM

47 Euler Equations DG Discretization Mathematical Model governed by 2D Euler Equations DG + BC s E(w) dy + R(y, u; w) = 0 dt R nonlinear function of y, u; w u R m m-external forcing inputs via BC s w - random vector associated with geometric variability

48 Parametrically Dependent Linear Model Linearize about steady state y ss (g(w)) corresponding to geometry g(w) y = y ss + ỹ E(w) dỹ dt = A(w)ỹ + B(w)u z = C(ỹ) Linearize E, A, B, C : e.g. A(w) A 0 + A 1 w 1 + A 2 w A m w m One time off-line cost: nominal geometry base

49 Construct POD Basis ROM Adaptive Sampling Method Starting with existing POD basis Φ, construct ROM Then iterate: 1) Select new parameter value w j via w j = argmax w z(w) z r (w) s.t. PDE Constraints 2) Collect additional snapshots y(t; w j ) for w = w j 3) Update POD basis Φ 5) Project A i, B i, C i via e.g. Â i = Φ T A i Φ

50 ROM vs Full Linear Response Outputs: z(1) = C L Lift Coefficient z(2) = C M Moment Coefficient

51 ROM Preservation of Statistics Linearized CFD (left) ROM predictions of work per cycle (WPC) Monte Carlo simulation results for 10,000 blade geometries

52 Prediction and Cost CFD (Full) Reduced Model Size 103, Number nonzeros 2,846,056 84,100 Offline cost hrs Online cost hrs 1.10 hrs Blade 1 WPC mean Blade 1 WPC variance Blade 2 WPC mean Blade 2 WPC variance random parameters Computations on 64 bit PC with 3.2GH Pentium Processor WPC = integral of the blade motion times the lift force over one unsteady cycle

53 Summary Gramian Based Model Reduction: POD, Balanced Reduction Optimal H 2 Reduction via IRKA Neural Modeling - Single Cell ROM Many Interactions Example of important class of problems (including Monte Carlo of Stochastic Systems) Nonlinear MOR via EIM Demonstrated effective reduction and feasibility Process Variation - Blade Geometry Effects Example of important future application of MOR