Preface vii 1. R Preliminaries 1 1.1 Matrix Defined, Deeper Understanding Using Software.. 1 1.2 Introduction, Why R?.................... 2 1.3 Obtaining R.......................... 4 1.4 Reference Manuals in R................... 5 1.5 Basic R Language Tips................... 6 1.6 Packages within R...................... 12 1.7 R Object Types and Their Attributes........... 17 1.7.1 Dataframe Matrix and Its Summary........ 18 2. Elementary Geometry and Algebra Using R 21 2.1 Mathematical Functions................... 21 2.2 Introductory Geometry and R Graphics.......... 22 2.2.1 Graphs for Simple Mathematical Functions and Equations...................... 25 2.3 Solving Linear Equation by Finding Roots......... 27 2.4 Polyroot Function in R................... 29 2.5 Bivariate Second Degree Equations and Their Plots... 32 3. Vector Spaces 41 3.1 Vectors............................ 41 3.1.1 Inner or Dot Product and Euclidean Length or Norm 42 3.1.2 Angle Between Two Vectors, Orthogonal Vectors 43 3.2 Vector Spaces and Linear Operations............ 46 xi
xii Hands-on Matrix Algebra Using R 3.2.1 Linear Independence, Spanning and Basis..... 47 3.2.2 Vector Space Defined................ 49 3.3 Sum of Vectors in Vector Spaces.............. 50 3.3.1 Laws of Vector Algebra............... 52 3.3.2 Column Space, Range Space and Null Space... 52 3.4 Transformations of Euclidean Plane Using Matrices.... 52 3.4.1 Shrinkage and Expansion Maps.......... 52 3.4.2 Rotation Map.................... 53 3.4.3 Reflexion Maps................... 53 3.4.4 Shifting the Origin or Translation Map...... 54 3.4.5 Matrix to Compute Deviations from the Mean.. 54 3.4.6 Projection in Euclidean Space........... 55 4. Matrix Basics and R Software 57 4.1 Matrix Notation....................... 57 4.1.1 Square Matrix.................... 60 4.2 Matrices Involving Complex Numbers........... 60 4.3 Sum or Difference of Matrices................ 61 4.4 Matrix Multiplication.................... 63 4.5 Transpose of a Matrix and Symmetric Matrices...... 66 4.5.1 Reflexive Transpose................. 66 4.5.2 Transpose of a Sum or Difference of Two Matrices 67 4.5.3 Transpose of a Product of Two or More Matrices 67 4.5.4 Symmetric Matrix.................. 68 4.5.5 Skew-symmetric Matrix............... 69 4.5.6 Inner and Outer Products of Matrices....... 71 4.6 Multiplication of a Matrix by a Scalar........... 72 4.7 Multiplication of a Matrix by a Vector........... 73 4.8 Further Rules for Sum and Product of Matrices...... 74 4.9 Elementary Matrix Transformations............ 76 4.9.1 Row Echelon Form................. 80 4.10 LU Decomposition...................... 80 5. Decision Applications: Payoff Matrix 83 5.1 Payoff Matrix and Tools for Practical Decisions...... 83 5.2 Maximax Solution...................... 85 5.3 Maximin Solution....................... 86 5.4 Minimax Regret Solution.................. 87
xiii 5.5 Digression: Mathematical Expectation from Vector Multiplication............................ 89 5.6 Maximum Expected Value Principle............ 90 5.7 General R Function payoff.all for Decisions........ 92 5.8 Payoff Matrix in Job Search................. 95 6. Determinant and Singularity of a Square Matrix 99 6.1 Cofactor of a Matrix..................... 101 6.2 Properties of Determinants................. 103 6.3 Cramer s Rule and Ratios of Determinants........ 108 6.4 Zero Determinant and Singularity............. 110 6.4.1 Nonsingularity.................... 113 7. The Norm, Rank and Trace of a Matrix 115 7.1 Norm of a Vector....................... 115 7.1.1 Cauchy-Schwartz Inequality............ 116 7.2 Rank of a Matrix....................... 116 7.3 Properties of the Rank of a Matrix............. 118 7.4 Trace of a Matrix....................... 121 7.5 Norm of a Matrix....................... 123 8. Matrix Inverse and Solution of Linear Equations 127 8.1 Adjoint of a Matrix...................... 127 8.2 Matrix Inverse and Properties................ 128 8.3 Matrix Inverse by Recursion................. 132 8.4 Matrix Inversion When Two Terms Are Involved..... 132 8.5 Solution of a Set of Linear Equations Ax = b....... 133 8.6 Matrices in Solution of Difference Equations........ 135 8.7 Matrix Inverse in Input-output Analysis.......... 136 8.7.1 Non-negativity in Matrix Algebra and Economics 140 8.7.2 Diagonal Dominance................ 141 8.8 Partitioned Matrices..................... 142 8.8.1 Sum and Product of Partitioned Matrices..... 143 8.8.2 Block Triangular Matrix and Partitioned Matrix Determinant and Inverse.............. 143 8.9 Applications in Statistics and Econometrics........ 147 8.9.1 Estimation of Heteroscedastic Variances...... 149 8.9.2 MINQUE Estimator of Heteroscedastic Variances 151
xiv Hands-on Matrix Algebra Using R 8.9.3 Simultaneous Equation Models........... 151 8.9.4 Haavelmo Model in Matrices............ 153 8.9.5 Population Growth Model from Demography... 154 9. Eigenvalues and Eigenvectors 155 9.1 Characteristic Equation................... 155 9.1.1 Eigenvectors..................... 157 9.1.2 n Eigenvalues.................... 158 9.1.3 n Eigenvectors.................... 158 9.2 Eigenvalues and Eigenvectors of Correlation Matrix.... 159 9.3 Eigenvalue Properties.................... 161 9.4 Definite Matrices....................... 163 9.5 Eigenvalue-eigenvector Decomposition........... 164 9.5.1 Orthogonal Matrix................. 166 9.6 Idempotent Matrices..................... 168 9.7 Nilpotent and Tripotent matrices.............. 172 10. Similar Matrices, Quadratic and Jordan Canonical Forms 173 10.1 Quadratic Forms Implying Maxima and Minima..... 173 10.1.1 Positive, Negative and Other Definite Quadratic Forms......................... 176 10.2 Constrained Optimization and Bordered Matrices..... 178 10.3 Bilinear Form......................... 179 10.4 Similar Matrices....................... 179 10.4.1 Diagonalizable Matrix................ 180 10.5 Identity Matrix and Canonical Basis............ 180 10.6 Generalized Eigenvectors and Chains............ 181 10.7 Jordan Canonical Form................... 182 11. Hermitian, Normal and Positive Definite Matrices 189 11.1 Inner Product Admitting Complex Numbers........ 189 11.2 Normal and Hermitian Matrices............... 191 11.3 Real Symmetric and Positive Definite Matrices...... 197 11.3.1 Square Root of a Matrix.............. 200 11.3.2 Positive Definite Hermitian Matrices....... 200 11.3.3 Statistical Analysis of Variance and Quadratic Forms......................... 201 11.3.4 Second Degree Equation and Conic Sections... 204
xv 11.4 Cholesky Decomposition................... 205 11.5 Inequalities for Positive Definite Matrices......... 207 11.6 Hadamard Product...................... 207 11.6.1 Frobenius Product of Matrices........... 208 11.7 Stochastic Matrices...................... 209 11.8 Ratios of Quadratic Forms, Rayleigh Quotient....... 209 12. Kronecker Products and Singular Value Decomposition 213 12.1 Kronecker Product of Matrices............... 213 12.1.1 Eigenvalues of Kronecker Products........ 220 12.1.2 Eigenvectors of Kronecker Products........ 221 12.1.3 Direct Sum of Matrices............... 222 12.2 Singular Value Decomposition (SVD)............ 222 12.2.1 SVD for Complex Number Matrices........ 226 12.3 Condition Number of a Matrix............... 228 12.3.1 Rule of Thumb for a Large Condition Number.. 228 12.3.2 Pascal Matrix is Ill-conditioned.......... 229 12.4 Hilbert Matrix is Ill-conditioned.............. 230 13. Simultaneous Reduction and Vec Stacking 233 13.1 Simultaneous Reduction of Two Matrices to a Diagonal Form.............................. 233 13.2 Commuting Matrices..................... 234 13.3 Converting Matrices Into (Long) Vectors.......... 239 13.3.1 Vec of ABC..................... 241 13.3.2 Vec of (A + B).................... 243 13.3.3 Trace of AB In Terms of Vec............ 244 13.3.4 Trace of ABC In Terms of Vec........... 245 13.4 Vech for Symmetric Matrices................ 247 14. Vector and Matrix Differentiation 249 14.1 Basics of Vector and Matrix Differentiation........ 249 14.2 Chain Rule in Matrix Differentiation............ 254 14.2.1 Chain Rule for Second Order Partials wrt θ... 254 14.2.2 Hessian Matrices in R................ 255 14.2.3 Bordered Hessian for Utility Maximization.... 255 14.3 Derivatives of Bilinear and Quadratic Forms........ 256 14.4 Second Derivative of a Quadratic Form.......... 257
xvi Hands-on Matrix Algebra Using R 14.4.1 Derivatives of a Quadratic Form wrt θ...... 257 14.4.2 Derivatives of a Symmetric Quadratic Form wrt θ 258 14.4.3 Derivative of a Bilinear form wrt the Middle Matrix........................ 258 14.4.4 Derivative of a Quadratic Form wrt the Middle Matrix........................ 258 14.5 Differentiation of the Trace of a Matrix.......... 258 14.6 Derivatives of tr(ab), tr(abc).............. 259 14.6.1 Derivative tr(a n ) wrt A is na 1......... 261 14.7 Differentiation of Determinants............... 261 14.7.1 Derivative of log(det A) wrt A is (A 1 )...... 261 14.8 Further Derivative Formulas for Vec and A 1....... 262 14.8.1 Derivative of Matrix Inverse wrt Its Elements.. 262 14.9 Optimization in Portfolio Choice Problem......... 262 15. Matrix Results for Statistics 267 15.1 Multivariate Normal Variables............... 267 15.1.1 Bivariate Normal, Conditional Density and Regression.................... 272 15.1.2 Score Vector and Fisher Information Matrix... 273 15.2 Moments of Quadratic Forms in Normals......... 274 15.2.1 Independence of Quadratic Forms......... 276 15.3 Regression Applications of Quadratic Forms........ 276 15.4 Vector Autoregression or VAR Models........... 276 15.4.1 Canonical Correlations............... 277 15.5 Taylor Series in Matrix Notation.............. 278 16. Generalized Inverse and Patterned Matrices 281 16.1 Defining Generalized Inverse................. 281 16.2 Properties of Moore-Penrose g-inverse........... 283 16.2.1 Computation of g-inverse.............. 284 16.3 System of Linear Equations and Conditional Inverse... 287 16.3.1 Approximate Solutions to Inconsistent Systems. 288 16.3.2 Restricted Least Squares.............. 289 16.4 Vandermonde and Fourier Patterned Matrices....... 290 16.4.1 Fourier Matrix.................... 292 16.4.2 Permutation Matrix................. 293 16.4.3 Reducible matrix.................. 293
xvii 16.4.4 Nonnegative Indecomposable Matrices...... 293 16.4.5 Perron-Frobenius Theorem............. 294 16.5 Diagonal Band and Toeplitz Matrices........... 294 16.5.1 Toeplitz Matrices.................. 295 16.5.2 Circulant Matrices.................. 296 16.5.3 Hankel Matrices................... 297 16.5.4 Hadamard Matrices................. 298 16.6 Mathematical Programming and Matrix Algebra..... 299 16.7 Control Theory Applications of Matrix Algebra...... 300 16.7.1 Brief Introduction to State Space Models..... 300 16.7.2 Linear Quadratic Gaussian Problems....... 301 16.8 Smoothing Applications of Matrix Algebra........ 303 17. Numerical Accuracy and QR Decomposition 307 17.1 Rounding Numbers...................... 307 17.1.1 Binary Arithmetic and Computer Bits...... 308 17.1.2 Floating Point Arithmetic............. 308 17.1.3 Fibonacci Numbers Using Matrices and Digital Computers...................... 309 17.2 Numerically More Reliable Algorithms........... 312 17.3 Gram-Schmidt Orthogonalization.............. 313 17.4 The QR Modification of Gram-Schmidt.......... 313 17.4.1 QR Decomposition................. 314 17.4.2 QR Algorithm.................... 314 17.5 Schur Decomposition..................... 318 Bibliography 321 Index 325