DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an Imprint of the Taylor & Francis Croup, an Informa business A CHAPMAN & HALL BOOK
Contents Preface xv About the author xxi 1 Some problems posed on vector spaces 1 11 Linear equations 1 111 Systems of linear algebraic equations 1 112 Linear ordinary differential equations 4 113 Some interpretation: The structure of the solution set to a linear equation 5 114 Finite fields and applications in discrete mathematics 7 12 Best approximation 8 121 Overdetermined linear systems 8 122 Best approximation by a polynomial 11 13 Diagonalization 13 14 Summary 17 2 Fields and vector spaces 19 21 Fields 19 211 Definition and examples 19 212 Basic properties of fields 21 22 Vector spaces 29 221 Examples of vector spaces 31 23 Subspaces 38 24 Linear combinations and spanning sets 43 25 Linear independence 50 26 Basis and dimension 57 27 Properties of bases 66 28 Polynomial interpolation and the Lagrange basis 73 281 Secret sharing 77 29 Continuous piecewise polynomial functions 82 291 Continuous piecewise linear functions 84 292 Continuous piecewise quadratic functions 87 293 Error in polynomial interpolation 90 ix
x Contents 3 Linear operators 93 31 Linear operators 93 311 Matrix operators 95 32 More properties of linear operators 101 321 Vector spaces of operators 101 322 The matrix of a linear operator on Euclidean spaces 101 323 Derivative and differential operators 103 324 Representing spanning sets and bases using matrices 103 325 The transpose of a matrix 104 33 Isomorphic vector spaces 107 331 Injective and surjective functions; inverses 108 332 The matrix of a linear operator on general vector spaces 111 34 Linear operator equations 116 341 Homogeneous linear equations 117 342 Inhomogeneous linear equations 118 343 General solutions 120 35 Existence and uniqueness of solutions 124 351 The kernel of a linear operator and injectivity 124 352 The rank of a linear operator and surjectivity 126 353 Existence and uniqueness 128 36 The fundamental theorem; inverse operators 131 361 The inverse of a linear operator 133 362 The inverse of a matrix 134 37 Gaussian elimination 142 371 Computing A'1 148 372 Fields other than R 149 38 Newton's method 153 39 Linear ordinary differential equations 158 391 The dimension of ker(l) 158 392 Finding a basis for ker(l) 161 3921 The easy case: Distinct real roots 162 3922 The case of repeated real roots 162 3923 The case of complex roots 163 393 The Wronskian test for linear independence 163 394 The Vandermonde matrix 166 310 Graph theory 168 3101 The incidence matrix of a graph 168 3102 Walks and matrix multiplication 169 3103 Graph isomorphisms 171 311 Coding theory 175 3111 Generator matrices; encoding and decoding 177 3112 Error correction 179 3113 The probability of errors 181 312 Linear programming 183 3121 Specification of linear programming problems 184
Contents xi 3122 Basic theory 186 3123 The simplex method 191 31231 Finding an initial BPS 196 31232 Unbounded LPs 199 31233 Degeneracy and cycling 200 3124 Variations on the standard LPs 202 4 Determinants and eigenvalues 205 41 The determinant function 206 411 Permutations 210 412 The complete expansion of the determinant 212 42 Further properties of the determinant function 217 43 Practical computation of det(a) 221 431 A recursive formula for det(a) 224 432 Cramer's rule 226 44 A note about polynomials 230 45 Eigenvalues and the characteristic polynomial 232 451 Eigenvalues of real matrix 235 46 Diagonalization 241 47 Eigenvalues of linear operators 251 48 Systems of linear ODEs 257 481 Complex eigenvalues 259 482 Solving the initial value problem 260 483 Linear systems in matrix form 261 49 Integer programming 265 491 Totally unimodular matrices 265 492 Transportation problems 268 5 The Jordan canonical form 273 51 Invariant subspaces 273 511 Direct sums 276 512 Eigenspaces and generalized eigenspaces 277 52 Generalized eigenspaces 283 521 Appendix: Beyond generalized eigenspaces 290 522 The Cayley-Hamilton theorem 294 53 Nilpotent operators 300 54 The Jordan canonical form of a matrix 309 55 The matrix exponential 318 551 Definition of the matrix exponential 319 552 Computing the matrix exponential 319 56 Graphs and eigenvalues 325 561 Cospectral graphs 325 562 Bipartite graphs and eigenvalues 326 563 Regular graphs 328 564 Distinct eigenvalues of a graph 330
xii Contents 6 Orthogonality and best approximation 333 61 Norms and inner products 333 611 Examples of norms and inner products 337 62 The adjoint of a linear operator 342 621 The adjoint of a linear operator 343 63 Orthogonal vectors and bases 350 631 Orthogonal bases 351 64 The projection theorem 357 641 Overdetermined linear systems 361 65 The Gram-Schmidt process 368 651 Least-squares polynomial approximation 371 66 Orthogonal complements 377 661 The fundamental theorem of linear algebra revisited 381 67 Complex inner product spaces 386 671 Examples of complex inner product spaces 388 672 Orthogonality in complex inner product spaces 389 673 The adjoint of a linear operator 390 68 More on polynomial approximation 394 681 A weighted L2 inner product 397 69 The energy inner product and Galerkin's method 401 691 Piecewise polynomials 404 692 Continuous piecewise quadratic functions 407 693 Higher degree finite element spaces 409 610 Gaussian quadrature 411 6101 The trapezoidal rule and Simpson's rule 412 6102 Gaussian quadrature 413 6103 Orthogonal polynomials 415 6104 Weighted Gaussian quadrature 419 611 The Helmholtz decomposition 420 6111 The divergence theorem 421 6112 Stokes's theorem 422 6113 The Helmholtz decomposition 423 7 The spectral theory of symmetric matrices 425 71 The spectral theorem for symmetric matrices 425 711 Symmetric positive definite matrices 428 712 Hermitian matrices 430 72 The spectral theorem for normal matrices 434 721 Outer products and the spectral decomposition 437 73 Optimization and the Hessian matrix 440 731 Background 440 732 Optimization of quadratic functions 441 733 Taylor's theorem 443 734 First- and second-order optimality conditions 444 735 Local quadratic approximations 446
Contents xiii 74 Lagrange multipliers 448 75 Spectral methods for differential equations 453 751 Eigenpairs of the differential operator 454 752 Solving the BVP using eigenfunctions 456 The singular value decomposition 463 81 Introduction to the SVD 463 811 The SVD for singular matrices 467 82 The SVD for general matrices 470 83 Solving least-squares problems using the SVD 476 84 The SVD and linear inverse problems 483 841 Resolving inverse problems through regularization 489 495 842 The truncated SVD method 489 843 Tikhonov regularization 490 85 The Smith normal form of a matrix 494 851 An algorithm to compute the Smith normal form 852 Applications of the Smith normal form 501 Matrix factorizations and numerical linear algebra 507 91 The LU factorization 507 911 Operation counts 512 912 Solving Ax = b using the LU factorization 514 92 Partial pivoting 516 921 Finite-precision arithmetic 517 922 Examples of errors in Gaussian elimination 518 923 Partial pivoting 519 924 The PLU factorization 522 93 The Cholesky factorization 524 94 Matrix norms 530 941 Examples of induced matrix norms 534 95 The sensitivity of linear systems to errors 537 96 Numerical stability 542 961 Backward error analysis 543 962 Analysis of Gaussian elimination with partial pivoting 545 97 The sensitivity of the least-squares problem 548 98 The QR factorization 554 981 Solving the least-squares problem 556 982 Computing the QR factorization 556 983 Backward stability of the Householder QR algorithm 561 984 Solving a linear system 562 99 Eigenvalues and simultaneous iteration 564 991 Reduction to triangular form 564 992 The power method 566 993 Simultaneous iteration 567 910 The QR algorithm 572
xiv Contents 9101 A practical QR algorithm 573 91011 Reduction to upper Hessenberg form 574 91012 The explicitly shifted QR algorithm 576 91013 The implicitly shifted QR algorithm 579 LO Analysis in vector spaces 581 101 Analysis 1011 Convergence and continuity in Rn 582 1012 Compactness 584 in R" 581 1013 Completeness of R" 586 1014 Equivalence of norms on R 586 102 Infinite-dimensional vector spaces 590 1021 Banach and Hubert spaces 592 103 Functional analysis 596 1031 The dual of a Hilbert space 600 104 Weak convergence 605 1041 Convexity 611 A The Euclidean algorithm 617 A0-1 Computing multiplicative inverses in Zp 618 A02 Related results 619 B Permutations 621 C Polynomials 625 Cl Rings of polynomials 625 C2 Polynomial functions 630 C 21 Factorization of polynomials 632 D Summary of analysis in R 633 D 01 Convergence 633 D02 Completeness of R 634 D03 Open and closed sets 635 D04 Continuous functions 636 Bibliography 637 Index 641