APPLIED NUMERICAL LINEAR ALGEBRA

Transcription

1 APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia

2 Contents Preface 1 Introduction Basic Notation Standard Problems of Numerical Linear Algebra General Techniques Matrix Factorizations Perturbation Theory and Condition Numbers Effects of Roundoff Error on Algorithms Analyzing the Speed of Algorithms Engineering Numerical Software Example: Polynomial Evaluation Floating Point Arithmetic Further Details Polynomial Evaluation Revisited Vector and Matrix Norms References and Other Topics for Chapter Questions for Chapter Linear Equation Solving Introduction Perturbation Theory Relative Perturbation Theory Gaussian Elimination Error Analysis The Need for Pivoting Formal Error Analysis of Gaussian Elimination Estimating Condition Numbers Practical Error Bounds Improving the Accuracy of a Solution Single Precision Iterative Refinement Equilibration Blocking Algorithms for Higher Performance Basic Linear Algebra Subroutines (BLAS) How to Optimize Matrix Multiplication Reorganizing Gaussian Elimination to Use Level 3 BLAS More About Parallelism and Other Performance Issues. 75 ix

3 vi Contents 2.7 Special Linear Systems Real Symmetric Positive Definite Matrices Symmetric Indefinite Matrices Band Matrices General Sparse Matrices Dense Matrices Depending on Fewer Than O(n 2 ) Parameters References and Other Topics for Chapter Questions for Chapter Linear Least Squares Problems Introduction Matrix Factorizations That Solve the Linear Least Squares Problem Normal Equations QR Decomposition Singular Value Decomposition Perturbation Theory for the Least Squares Problem Orthogonal Matrices Householder Transformations Givens Rotations Roundoff Error Analysis for Orthogonal Matrices Why Orthogonal Matrices? Rank-Deficient Least Squares Problems Solving Rank-Deficient Least Squares Problems Using the SVD Solving Rank-Deficient Least Squares Problems Using QR with Pivoting Performance Comparison of Methods for Solving Least Squares Problems References and Other Topics for Chapter Questions for Chapter Nonsymmetric Eigenvalue Problems Introduction Canonical Forms Computing Eigenvectors from the Schur Form Perturbation Theory Algorithms for the Nonsymmetric Eigenproblem Power Method Inverse Iteration Orthogonal Iteration QR Iteration Making QR Iteration Practical Hessenberg Reduction 164

4 Contents vii Tridiagonal and Bidiagonal Reduction QR Iteration with Implicit Shifts Other Nonsymmetric Eigenvalue Problems Regular Matrix Pencils and Weierstrass Canonical Form Singular Matrix Pencils and the Kronecker Canonical Form Nonlinear Eigenvalue Problems Summary References and Other Topics for Chapter Questions for Chapter The Symmetric Eigenproblem and Singular Value Decomposition Introduction Perturbation Theory Relative Perturbation Theory Algorithms for the Symmetric Eigenproblem Tridiagonal QR Iteration Rayleigh Quotient Iteration Divide-and-Conquer Bisection and Inverse Iteration Jacobi's Method Performance Comparison Algorithms for the Singular Value Decomposition QR Iteration and Its Variations for the Bidiagonal SVD Computing the Bidiagonal SVD to High Relative Accuracy Jacobi's Method for the SVD Differential Equations and Eigenvalue Problems The Toda Lattice The Connection to Partial Differential Equations References and Other Topics for Chapter Questions for Chapter Iterative Methods for Linear Systems Introduction On-line Help for Iterative Methods Poisson's Equation Poisson's Equation in One Dimension Poisson's Equation in Two Dimensions Expressing Poisson's Equation with Kronecker Products Summary of Methods for Solving Poisson's Equation Basic Iterative Methods Jacobi's Method Gauss-Seidel Method Successive Overrelaxation 283

5 viii Contents Convergence of Jacobi's, Gauss-Seidel, and SOR(u;) Methods on the Model Problem Detailed Convergence Criteria for Jacobi's, Gauss-Seidel, and SOR(o;) Methods Chebyshev Acceleration and Symmetric SOR (SSOR) Krylov Subspace Methods Extracting Information about A via Matrix-Vector Multiplication Solving Ax = b Using the Krylov Subspace /C^ Conjugate Gradient Method Convergence Analysis of the Conjugate Gradient Method Preconditioning Other Krylov Subspace Algorithms for Solving Ax = b Fast Fourier Transform The Discrete Fourier Transform Solving the Continuous Model Problem Using Fourier Series Convolutions Computing the Fast Fourier Transform Block Cyclic Reduction Multigrid Overview of Multigrid on the Two-Dimensional Poisson's Equation Detailed Description of Multigrid on the One-Dimensional Poisson's Equation Domain Decomposition Nonoverlapping Methods Overlapping Methods References and Other Topics for Chapter Questions for Chapter Iterative Methods for Eigenvalue Problems Introduction ^ The Rayleigh-Ritz Method The Lanczos Algorithm in Exact Arithmetic The Lanczos Algorithm in Floating Point Arithmetic The Lanczos Algorithm with Selective Orthogonalization Beyond Selective Orthogonalization Iterative Algorithms for the Nonsymmetric Eigenproblem References and Other Topics for Chapter Questions for Chapter Bibliography 389 Index 409