Linear Algebra Review CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Linear Algebra Review 1 / 16
Midterm Exam Tuesday Feb 13, in class Covers linalg (up to and incl SVD). Closed book exam. Can use 1 page of notes (two-sided, human readable) No calculators or computers allowed (or needed) SCPD: Arrange proctor for exam at similar time. Previous midterm exams on website (similar material) Simple HW this week short question. CS 205A: Mathematical Methods Linear Algebra Review 2 / 16
What Have We Done? A x = b CS 205A: Mathematical Methods Linear Algebra Review 3 / 16
Gaussian Elimination Codifies the typical approach taken on paper Phases: Forward substitution, back substitution (pivoting) Elimination matrices: Notational convenience, algorithmically slow! CS 205A: Mathematical Methods Linear Algebra Review 4 / 16
LU Factorization O(n 3 ) time to compute Allows for solving linear systems via forward/backward substitution (O(n 2 ) time) Might not exist need pivots (e.g. LUP) CS 205A: Mathematical Methods Linear Algebra Review 5 / 16
Cholesky Factorization: LL For symmetric, positive definite matrices CS 205A: Mathematical Methods Linear Algebra Review 6 / 16
QR Factorization R is upper triangular Q has orthonormal columns Many algorithms: Gram-Schmidt, Householder, Givens Least-squares w/o squaring condition # CS 205A: Mathematical Methods Linear Algebra Review 7 / 16
Diagonalizability: D = X 1 AX Diagonalizable iff there is a full eigenspace Spectral theorem: symmetric/hermitian = full, orthogonal eigenbasis Computation: Variations of power method Note: AX = XD (usually AX DX!!) CS 205A: Mathematical Methods Linear Algebra Review 8 / 16
Singular Value Decomposition A = UΣV CS 205A: Mathematical Methods Linear Algebra Review 9 / 16
Variational Approach Define energy measuring something desirable and minimize it. CS 205A: Mathematical Methods Linear Algebra Review 10 / 16
Variational Approach Define energy measuring something desirable and minimize it. E( x) = A x b 2 2 CS 205A: Mathematical Methods Linear Algebra Review 10 / 16
Variational Approach Define energy measuring something desirable and minimize it. E( x) = A x b 2 2 Lagrange multipliers! CS 205A: Mathematical Methods Linear Algebra Review 10 / 16
Regularization Improves conditioning of ill-posed problems E.g., Tikhonov regularization. E( x) = A x b 2 2 + α x 2 2 Multiple formulations. Connection to truncated SVD. CS 205A: Mathematical Methods Linear Algebra Review 11 / 16
Look for Special Structure Symmetric Positive definite Sparse Normal equations Square Full rank Block Triangular CS 205A: Mathematical Methods Linear Algebra Review 12 / 16
Reduce to Known Algorithm Show that a specific problem is equivalent to: Least squares (curve fitting) Eigenvectors (ODEs, embedding) Factorization (metric learning) SVD (principal components analysis) CS 205A: Mathematical Methods Linear Algebra Review 13 / 16
Stability and Conditioning Complement algorithmic analysis with understanding quality of output CS 205A: Mathematical Methods Linear Algebra Review 14 / 16
Julia Not on written midterm, but useful for study. CS 205A: Mathematical Methods Linear Algebra Review 15 / 16
Advice Draw matrix pictures. CS 205A: Mathematical Methods Linear Algebra Review 16 / 16
Advice Draw matrix pictures. Experiment. CS 205A: Mathematical Methods Linear Algebra Review 16 / 16
Advice Draw matrix pictures. Experiment. Ask for help. CS 205A: Mathematical Methods Linear Algebra Review 16 / 16 Next