Multilevel Methods for Eigenspace Computations in Structural Dynamics Ulrich Hetmaniuk & Rich Lehoucq Sandia National Laboratories, Computational Math and Algorithms, Albuquerque, NM Joint work with Peter Arbenz (ETH), Jeff Bennighof (UT Austin), Mark Muller (UT Austin), Bill Cochran (UIUC), Heidi Thornquist & Ray Tuminaro (Sandia)
Eigenvalue problem in structural dynamics Who/What is the problem? Multilevel approach 1 (use multilevel preconditioners) Multilevel approach 2 (component mode synthesis) Conclusions and future directions Thanks to the organizers
Eigen-gang Jeff Bennighof, University of Texas Ray Tuminaro, Sandia Mark Muller, University of Texas Heidi Thornquist Sandia (Jeff s Ph.D student) Peter Arbenz, ETH Bill Cochran, Sandia summer intern; Ph.d. student UIUC Ulrich Hetmaniuk, Sandia
Salinas: Implicit structural dynamics Salinas is a massively parallel implementation of structural dynamics finite element analysis This capability is required for high fidelity, validated models used in modal, vibration, static and shock analysis of weapons systems. 2002 Gordon Bell Prize winner (C++ code).
Salinas and linear algebra Robust scalable iterative linear and eigensolvers are required Fundamental mode of an aircraft carrier
PDE of interest Consider the hyperbolic PDE on a domain u E( u) = f ( t) tt where E( u) is a linear elliptic coercive operator and is the mass density. Assume that appropriate initial and boundary conditions are specified and that is a two or three dimensional domain that represents a structure.
Problems of interest Frequency (or harmonic) response how does the structure respond to a prescribed load? Transient analysis how does the structure evolve in time? My assumption is that 100 plus eigenpairs are needed for large-scale 3D FEM discretizations the eigenpairs effect a reduced order model (ROM).
Helmholtz equation The Fourier transform of the hyperbolic PDE gives 2 u E( u ) = f The direct approach (Helmholtz) is to solve numerous linear PDEs as varies over the frequency range of interest. This approach is typically not undertaken when there are a large number of load terms or the frequency range of interest is large.
Modal approach (ROM) Compute the free vibrations or modal subspace of E ( z ) = z i i i over the domain with the same boundary conditions as the ROM. This standard approach computes numerous modes and then projects the PDE onto the modal subspace. This approach is useful when there are many load terms or the frequency range of interest is large.
Some details The i are the squares of the natural frequencies i Assume the ordering 0 1 2 3 The eigenvectors z i are orthogonal
Discrete Modal Analysis A finite element discretization leads to the generalized eigenvalue problem K Kz M n h h h = Mz i i i where and are the stiffness and mass matrices of order respectively. Both are symmetric and semi-positive definite matrices and sparse. As in the continuous problem, the eigenvectors are used to project the PDE onto the span of the modal subspace.
Project with the Eigenvectors 2 u E( u ) = f h T h ˆ 2 h T h ˆ = h T m m m m m ( Z ) KZ u ( Z ) MZ u ( Z ) f Diagonal matrix containing the m eigenvalues Identity matrix of order m The solution is approximated with u h m Z uˆ
Why only m modes? Eigenvectors associated with the smallest eigenvalues have a mechanical significance they dominate the low frequency response FEM discretization only approximates a small number of fundamental modes of the PDE why compute more? (Error increases as the frequency increases) Keep in mind that thousands of eigenvectors may be needed when the modal density is large many eigenvalues within the desired frequency range.
Eigenvalue problem in structural dynamics Who/What is the problem? Multilevel approach 1 (use multilevel preconditioners) Multilevel approach 2 (component mode synthesis) Conclusions and future directions
Requirements of an algorithm and ensuing implementation Robust (vary data, parameters and still compute the same answers) Reliable (don t miss eigenvalues, compute a basis of eigenvectors) Return orthogonal eigenvectors Ability to efficiently compute 100-10,000 modes (remember we to generate a ROM).
Standard computational approach Large-scale eigenvalue problem needs to be solved for 100 s to 1000 s of modes; the state of the art industrial solution is to run a block Lanczos algorithm (Boeing code Grimes, Lewis & Simon Simax 1994) on 1 h h h 1 i i i ( K M) Mz = z ( ) using an inner product. This is the approach traditionally used by MSC.NASTRAN and almost every engineering analysis package.
Key computational issue Solving the linear system Kx = Mv at every Lanczos iteration The Boeing code uses a sparse direct solver (and shifts into the spectrum) Salinas uses parallel ARPACK along with a FETI domain decomposition solver
Overview of multilevel approach 1 Kz h h h = Mz i i i 1. Shift-invert Lanczos method with a scalable preconditioned inner iteration 2. Preconditioned eigensolver (no inner iteration necessary)
Shift-invert Lanczos with a preconditioned inner iteration K m { m1 } ( T, x) = Span x, Tx,, T x T -1 K M Apply via a preconditioned conjugate iteration use a scalable preconditioner (multilevel/multigrid)
Preconditioned Eigensolver Only require an application of a preconditioner per (outer) iteration N D x m+ 1 { m m m } ( x ) = Span x,, x, N ( M K) x (0) (0) ( ) 1 ( ) ( ) x Kx ( x ) Kx = x Mx ( x ) Mx T ( m) min ( m) and T xd m ( m) T ( m) ( m) T ( m)
Preconditioned Eigensolver Replace shift-invert Lanczos attempt to minimize the Rayleigh-Quotient Newton s method (Davidson methods) (Davidson, Morgan, Scott, van der Vorst, Sleijpen, Stathopoulos, Saad, Notay) Nonlinear conjugate gradient iteration (Longsine, McCormick, Knyazev, Bergamaschi, Pini)
Separation of linear solver and the eigensolver Boeing Lanczos uses the sparse direct solver as a kernel. Preconditioned eigensolver replaces the eigenvalue algorithm and a preconditioner for the sparse direct solve. Quite often a good preconditioner is already available because a PDE required it. (See Knyazev Oxymoron paper (ETNA 1998))
Potential limitations of these methods Repeated use of stiffness, mass matrices and repeated preconditioning of the large-scale problem and cost of maintaining orthogonality. Clearly not an issue for a small number of modes or a limited frequency response (small frequency range). What about a large number of eigenvectors?
A Comparison of Eigensolvers for Largescale 3D Modal Analysis using AMG Preconditioned Iterative Methods Finished the revision of a paper for publication in IJNME (Arbenz, Hetmaniuk, Lehoucq, Tuminaro) Massively parallel modal analysis combine ML (multigrid), Anasazi (suite of codes implementing several of the previous algorithms including block extensions) and parallel ARPACK Extremely meticulous implementation, timings and verification on 3D FEM (2 elasticity) problems.
One conclusion from our comparison The cost of all the algorithms is asymptotic with the cost of maintaining orthogonality of the eigenvectors as the number of eigenpairs requested is increased. Are there alternate approaches?
Eigenvalue problem in structural dynamics Who/What is the problem? Multilevel approach 1 (use multilevel preconditioners) Multilevel approach 2 (component mode synthesis ideas) Conclusions and future directions
Component Mode Synthesis Component Mode Synthesis (CMS) techniques originated in the aerospace engineering community (Hurty 1965, Craig-Bampton 1968). The idea is simple. 1. Decompose the structure into component parts 2. Determine the modes of the parts 3. Synthesize these component modes to say something about the structure
Matrix view of CMS One way dissection on the union of the graphs of the mass and stiffness matrices produces K 0 K 0 K K K K K 1 2 T, 1 1 T, 2 2,, and M 0 M 0 M M M M M 1 2 T, 1 1 T, 2 2,,
Component Mode Synthesis (CMS) K Z = M Z 1 1 1 1 1 Fixed-interface modes K Z = M Z 2 2 2 2 2 1 2 K Z = Coupling modes M Z K = K K K K 1 T,, i i i i M = M K K M + M K K K K M K K 1 T 1 T 1 1 T ( ), i i, i, i i, i, i i i i, i i
Theorem The interface eigenvalue problem is equivalent to: 1/ 2 Find ( u, ) H ( ) such that 00 Su, v = Mu, v v H ( ) 1/2 00 where the Steklov-Poincaré and mass operators 1/ 2 act on H ( ) : 00 S & M 2 S = ( ( Ei ) i ) i= 1 2 M = ( ( Gi ( Ei )) i ) i= 1 Gi ( ) is the Green s function for the Dirichlet problem on and extends a trace function to. i E i i
CMS as a Rayleigh-Ritz process Project the global eigenvalue problem onto the span of Rectangular not square I 0 Z 0 1 0 I Z 0 2 K K K K 2 2 Z I 1 T 1, 1 1 T, Modal truncation only retain the low order modes These three spaces are K orthogonal and so the projection of the stiffness matrix is diagonal but the mass matrix is NOT diagonal.
Previous/related work Bourquin (1992, 1993) casts CMS in a variational setting Asymptotic error analysis for 1,2,3 dimensional elliptic equations and their discretization Error is sum of modal truncation and discretization errors Can also consider free-interface methods (introduce a Lagrange multiplier at the interface)
0,1 1,1 1,2 AMLS Don t stop at one level! (Bennighof and graduate students, AIAA proceedings papers early 1990 s) 2,1 1,1 0,1 2,3 1,2 2,5 2,4 Recursively apply above procedures to obtain may small fixed interface eigenvalue problems. 2,2 Nested dissection automates the recursive reordering of the stiffness and mass matrices and so partitions the structure
Reference An Automated Multilevel Substructuring Method for Eigenspace Computation in Linear Elastodynamics SISC, 2004, by Bennighof & L. AMLS cast in a multilevel variational formulation (extension of the work by Bourquin) Showed that discrete AMLS represents a matrix decomposition of the stiffness matrix. The resulting decomposition performs a change of coordinates AMLS is a Rayleigh-Ritz method that efficiently computes an approximation to the modal subspace reduced order modeling
Example AMLS Calculation Recent paper Efficient Broadband Vibro- Acoustic Analysis of Passenger Car bodies Using an FE-based Component Mode Synthesis approach (Kropp & Heiserer, proceedings of the World Congress in Computational Mechanics, Vienna, 2002). Compared the industry standard Block Lanczos as (embedded in MSC.Nastran) approach against AMLS on modal analysis on the BMW 3 series.
BMW comparison result 1600 Frequency range (Hz) Direct (Helmholtz) AMLS Boeing Lanczos 100 5 10 DOFS (Matrix order) mesh refinement 6 10 7 10
Cost of the computations These computations were performed on an HP 9000 (800 MHz) with 2 gigabytes of memory. The largest calculation took just under 3 days (2,500 eigenvectors for a problem with 13.5 million DOFs!). A 2.3 million DOF computation up through 400 Hz took 4 days of cputime and over a week turnaround time on a CRAY SV1 using the block Lanczos code. This calculation is not even feasible on the HP 9000.
Some stats on the BMW problem Order 13.5 million problem substructured into 38 levels 46,767 substructures Order 37,848 coarse problem During the past three years, AMLS has replaced Boeing Lanczos within the automotive industry. Cray supercomputers have been replaced by PC/workstations
Limitations of AMLS Not a good technique for 3D problems because of the size of the interface eigenvalue problem. AMLS assumes that the interface matrix operators are formed and factored. Important to point out that AMLS has the same complexity of one sparse direct factorization of the stiffness matrix.
Mass interface operator M = M K K M + M K K K K M K K 1 T 1 T 1 1 T ( ), i i, i, i i, i, i i i i, i i How can we approximate the above matrix operator? It s expensive to apply because of the amount of data.
Multilevel approach 2 alternatives CMS technique where the interface operators are not formed and instead preconditioned eigensolvers are employed. For example, use the BDDC (Dohrmann) preconditioner at the interface. Problem: Multilevel approach 1 issues come to bear. Algebraic multilevel?
Influence of the number of coupling modes Pencil (S, Mcplt) - 160,380 nodes - 16 subdomains Influence of the number of coupling modes Pencil (S, Mint) - 160,380 nodes - 16 subdomains 1.E+00 1 10 100 1000 1.E+00 1 10 100 1000 1.E-01 1.E-01 1.E-02 1.E-02 1.E-03 1.E-03 Relative error 1.E-04 EV 1 EV 5 EV 10 EV 50 EV 100 Relative error 1.E-04 EV 1 EV 5 EV 10 EV 50 EV 100 1.E-05 1.E-05 1.E-06 1.E-06 1.E-07 1.E-07 1.E-08 Number of coupling eigenmodes 1.E-08 Number of coupling modes M = M K K M + M K K K K M K K 1 T 1 T 1 1 T ( ), i i, i, i i, i, i i i i, i i How can we approximate the above matrix operator? It s expensive to apply because of the amount of data.
Algebraic Multilevel Start with RQMG (Mandel and McCormick 1989) that approximately minimizes the Rayleigh Quotient over a sequence of grids. We re replacing the geometric information and smoothers of RQMG with algebraic info and better smoothers resulting in RQAMG. Goal of RQAMG is to overcome the cost of maintaining numerical orthogonality of the Ritz vectors associated with multilevel approach 1. Work in progress, Hetmaniuk & L.
Summary of multilevel approaches Approach 1 separates the preconditioner from the eigensolver Approach 2 interleaves the preconditioner and the eigensolver Our view is that the decision to use an approach depends upon how dominant the cost of orthogonality is for the modal analysis at hand c n 2 (nev)
Future/ongoing work Error estimation and stopping criterion (joint work with Hetmaniuk, Knyazev and Ovtchinnikov) for multilevel aproach 1 Modal truncation criterion (recent reports by others) Regularity issues or the effect of partitions on the approximation of global modes Approximation of the mass interface operator RQAMG, preconditioned CMS Packaging the preconditioned eigensolvers for public release into a the Anasazi subpackage of Trilinos (joint with Hetmaniuk and Thornquist) DD16 proceedings paper