# Math 225 Linear Algebra II Lecture Notes. John C. Bowman University of Alberta Edmonton, Canada

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Math 225 Linear Algebra II Lecture Notes John C Bowman University of Alberta Edmonton, Canada March 23, 2017

2 c 2010 John C Bowman ALL RIGHTS RESERVED Reproduction of these lecture notes in any form, in whole or in part, is permitted only for nonprofit, educational use

3 Contents 1 Vectors 4 2 Linear Equations 6 3 Matrix Algebra 8 4 Determinants 11 5 Eigenvalues and Eigenvectors 13 6 Linear Transformations 16 7 Dimension 17 8 Similarity and Diagonalizability 18 9 Complex Numbers Projection Theorem Gram-Schmidt Orthonormalization QR Factorization Least Squares Approximation Orthogonal (Unitary) Diagonalizability Systems of Differential Equations Quadratic Forms Vector Spaces: Inner Product Spaces General linear transformations Singular Value Decomposition The Pseudoinverse 57 Index 58

4 1 Vectors Vectors in R n : Parallelogram law: u 1 v 1 u =, v =, 0 = 0 u n v n 0 u 1 + v 1 u + v = u n + v n Multiplication by scalar c R: cv 1 cv = cv n Dot (inner) product: Length (norm): Unit vector: u v = u 1 v u n v n v = v v = v v 2 n 1 v v Distance between (endpoints of) u and v (positioned at 0): d(u, v) = u v Law of cosines: u v 2 = u 2 + v 2 2 u v cos θ Angle θ between u and v: u v cos θ = 1 2 ( u 2 + v 2 u v 2 ) = 1 2 ( u 2 + v 2 [u v] [u v]) = 1 2 ( u 2 + v 2 [u u 2u v + v v]) = u v 4

5 Orthogonal vectors u and v: u v = 0 Pythagoras theorem for orthogonal vectors u and v: u + v 2 = u 2 + v 2 Cauchy-Schwarz inequality ( cos θ 1): u v u v Triangle inequality: u + v = (u + v) (u + v) = u 2 +2u v + v 2 u 2 +2 u v + v 2 u 2 +2 u v + v 2 = ( u + v ) 2 = u + v Component of v in direction u: v u u = v cos θ Equation of line: Ax + By = C Parametric equation of line through p and q: v = (1 t)p + tq Equation of plane with unit normal (A, B, C) a distance D from origin: Ax + By + Cz = D Equation of plane with normal (A, B, C) through (x 0, y 0, z 0 ): A(x x 0 ) + B(y y 0 ) + C(z z 0 ) = 0 Parametric equation of plane spanned by u and w through v 0 : v = v 0 + tu + sw 5

6 2 Linear Equations Linear equation: a 1 x 1 + a 2 x a n x n = b Homogeneous linear equation: a 1 x 1 + a 2 x a n x n = 0 System of linear equations: a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a m1 x 1 + a m2 x a mn x n = b m Systems of linear equations may have 0, 1, or an infinite number of solutions [Anton & Busby, p 41] Matrix formulation: a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a }{{ mn } n m coefficient matrix x 1 x 2 x n = b 1 b 2 b m Augmented matrix: a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a m1 a m2 a mn b m Elementary row operations: Multiply a row by a nonzero constant Interchange two rows Add a multiple of one row to another 6

7 Remark: Elementary row operations do not change the solution! Row echelon form (Gaussian elimination): Reduced row echelon form (Gauss Jordan elimination): For example: 2x 1 + x 2 = 5, 7x 1 + 4x 2 = 17 A diagonal matrix is a square matrix whose nonzero values appear only as entries a ii along the diagonal For example, the following matrix is diagonal: An upper triangular matrix has zero entries everywhere below the diagonal (a ij = 0 for i > j) A lower triangular matrix has zero entries everywhere above the diagonal (a ij = 0 for i < j) Problem 21: Show that the product of two upper triangular matrices of the same size is an upper triangle matrix Problem 22: If L is a lower triangular matrix, and U is an upper triangular matrix, show that LU is a lower triangular matrix and UL is an upper triangular matrix 7

8 3 Matrix Algebra Element a ij appears in row i and column j of the m n matrix A = [a ij ] m n Addition of matrices of the same size: [a ij ] m n + [b ij ] m n = [a ij + b ij ] m n Multiplication of a matrix by a scalar c R: c[a ij ] m n = [ca ij ] m n Multiplication of matrices: [ n ] [a ij ] m n [b jk ] n l = a ij b jk j=1 m l Matrix multiplication is linear: A(αB + βc) = αab + βac for all scalars α and β Two matrices A and B are said to commute if AB = BA Problem 31: Give examples of 2 2 matrices that commute and ones that don t Transpose of a matrix: [a ij ] T m n = [a ji ] n m A symmetric matrix A is equal to its transpose Problem 32: Does a matrix necessarily compute with its transpose? provide a counterexample Prove or Problem 33: Let A be an m n matrix and B be an n l matrix Prove that (AB) T = B T A T 8

9 Problem 34: Show that the dot product can be expressed as a matrix multiplication: u v = u T v Also, since u v = v u, we see equivalently that u v = v T u Trace of a square matrix: Tr [a ij ] n n = n a ii i=1 Problem 35: Let A and B be square matrices of the same size Prove that Tr(AB) = Tr(BA) Identity matrix: I n = [δ ij ] n n Inverse A 1 of an invertible matrix A: AA 1 = A 1 A = I Problem 36: If Ax = b is a linear system of n equations, and the coefficient matrix A is invertible, prove that the system has the unique solution x = A 1 b Problem 37: Prove that (AB) 1 = B 1 A 1 Inverse of a 2 2 matrix: [ a b c d [ ] 1 a b = c d 1 ad bc ][ ] [ ] d b 1 0 = (ad bc) c a 0 1 [ d ] b c a exists if and only if ad bc 0 Determinant of a 2 2 matrix: [ ] a b det = c d a c b d = ad bc 9

10 An elementary matrix is obtained on applying a single elementary row operation to the identity matrix Two matrices are row equivalent if one can be obtained from the other by elementary row operations An elementary matrix is row equivalent to the identity matrix A subspace is closed under scalar multiplication and addition span{v 1, v 2,, v n } is the subspace formed by the set of linear combinations c 1 v 1 + c 2 v c n v n generated by all possible scalar multipliers c 1, c 2,, c n The subspaces in R 3 are the origin, lines, planes, and all of R 3 The solution space of the homogeneous linear system Ax = 0 with n unknowns is always a subspace of R n A set of vectors {v 1, v 2,, v n } in R n are linearly independent if the equation c 1 v 1 + c 2 v c n v n = 0 has only the trivial solution c 1 = c 2 = = c n = 0 If this were not the case, we say that the set is linearly dependent; for n 2, this means we can express at least one of the vectors in the set as a linear combination of the others The following are equivalent for an n n matrix A: (a) A is invertible; (b) The reduced row echelon form of A is I n ; (c) A can be expressed as a product of elementary matrices; (d) A is row equivalent to I n ; (e) Ax = 0 has only the trivial solution; (f) Ax = b has exactly one solution for all b R n ; (g) The rows of A are linearly independent; (h) The columns of A are linearly independent; (i) det A 0; (j) 0 is not an eigenvalue of A 10

11 The elementary row operations that reduce an invertible matrix A to the identity matrix can be applied directly to the identity matrix to yield A 1 Remark: If BA = I then the system of equations Ax = 0 has the unique solution x = Ix = BAx = B(Ax) = B0 = 0 Hence A is invertible and B = A 1 4 Determinants Determinant of a 2 2 matrix: a c b d = ad bc Determinant of a 3 3 matrix: a b c d e f g h i = a e h f i b d g f i + c d g e h Determinant of a square matrix A = [a ij ] n n can be formally defined as: det A = sgn(j 1, j 2,, j n ) a 1j1 a 2j2 a njn permutations (j 1, j 2,, j n ) of (1, 2,, n) Properties of Determinants: Multiplying a single row or column by a scalar c scales the determinant by c Exchanging two rows or columns swaps the sign of the determinant Adding a multiple of one row (column) to another row (column) leaves the determinant invariant Given an n n matrix A = [a ij ], the minor M ij of the element a ij is the determinant of the submatrix obtained by deleting the ith row and jth column from A The signed minor ( 1) i+j M ij is called the cofactor of the element a ij Determinants can be evaluated by cofactor expansion (signed minors), either along some row i or some column j: n n det[a ij ] n n = a ik M ik = a kj M kj k=1 k=1 11

12 In general, evaluating a determinant by cofactor expansion is inefficient However, cofactor expansion allows us to see easily that the determinant of a triangular matrix is simply the product of the diagonal elements! Problem 41: Prove that det A T = det A Problem 42: Let A and B be square matrices of the same size Prove that det(ab) = det(a) det(b) Problem 43: Prove for an invertible matrix A that det(a 1 ) = 1 det A Determinants can be used to solve linear systems of equations like [ ][ ] [ ] a b x u = c d y v Cramer s rule: x = u v a c b d b, y = d a c a c u v b d For matrices larger than 3 3, row reduction is more efficient than Cramer s rule An upper (or lower) triangular system of equations can be solved directly by back substitution: 2x 1 + x 2 = 5, 4x 2 = 17 The transpose of the matrix of cofactors, adj A = [( 1) i+j M ji ], is called the adjugate (formerly sometimes known as the adjoint) of A The inverse A 1 of a matrix A may be expressed compactly in terms of its adjugate matrix: A 1 = 1 adj A det A However, this is an inefficient way to compute the inverse of matrices larger than 3 3 A better way to compute the inverse of a matrix is to row reduce [A I] to [I A 1 ] 12

13 The cross product of two vectors u = (u x, u y, u z ) and v = (v x, v y, v z ) in R 3 can be expressed as the determinant i j k u v = u x u y u z v x v y v z = (u yv z u z v y )i + (u z v x u x v z )j + (u x v y u y v x )k The magnitude u v = u v sin θ (where θ is the angle between the vectors u and v) represents the area of the parallelogram formed by u and v 5 Eigenvalues and Eigenvectors A fixed point x of a matrix A satisfies Ax = x The zero vector is a fixed point of any matrix An eigenvector x of a matrix A is a nonzero vector x that satisfies Ax = λx for a particular value of λ, known as an eigenvalue (characteristic value) An eigenvector x is a nontrivial solution to (A λi)x = Ax λx = 0 This requires that (A λi) be noninvertible; that is, λ must be a root of the characteristic polynomial det(λi A): it must satisfy the characteristic equation det(λi A) = 0 If [a ij ] n n is a triangular matrix, each of the n diagonal entries a ii are eigenvalues of A: det(λi A) = (λ a 11 )(λ a 22 ) (λ a nn ) If A has eigenvalues λ 1, λ 2,, λ n then det(a λi) = (λ 1 λ)(λ 2 λ) (λ n λ) On setting λ = 0, we see that det A is the product λ 1 λ 2 λ n of the eigenvalues of A Given an n n matrix A, the terms in det(λi A) that involve λ n and λ n 1 must come from the main diagonal: But (λ a 11 )(λ a 22 ) (λ a nn ) = λ n (a 11 + a a nn )λ n 1 + det(λi A) = (λ λ 1 )(λ λ 2 ) (λ λ n ) = λ n (λ 1 +λ 2 + +λ n )λ n 1 + +( 1) n λ 1 λ 2 λ n Thus, Tr A is the sum λ 1 +λ 2 + +λ n of the eigenvalues of A and the characteristic polynomial of A always has the form λ n Tr Aλ n ( 1) n det A 13

14 Problem 51: Show that the eigenvalues and corresponding eigenvectors of the matrix [ ] 1 2 A = 3 2 are 1, with eigenvector [1, 1], and 4, with eigenvector [2, 3] Note that the trace (3) equals the sum of the eigenvalues and the determinant ( 4) equals the product of eigenvalues (Any nonzero multiple of the given eigenvectors are also acceptable eigenvectors) Problem 52: Compute Av where A is the matrix in Problem 51 and v = [7, 3] T, both directly and by expressing v as a linear combination of the eigenvectors of A: [7, 3] = 3[1, 1] + 2[2, 3] Problem 53: Show that the trace and determinant of a 5 5 matrix whose characteristic polynomial is det(λi A) = λ 5 + 2λ 4 + 3λ + 4λ + 5λ + 6 are given by 2 and 6, respectively The algebraic multiplicity of an eigenvalue λ 0 of a matrix A is the number of factors of (λ λ 0 ) in the characteristic polynomial det(λi A) The eigenspace of A corresponding to an eigenvalue λ is the linear space spanned by all eigenvectors of A associated with λ This is the null space of A λi The geometric multiplicity of an eigenvalue λ of a matrix A is the dimension of the eigenspace of A corresponding to λ The sum of the algebraic multiplicities of all eigenvalues of an n n matrix is equal to n The geometric multiplicity of an eigenvalue is always less than or equal to its algebraic multiplicity Problem 54: Show that the 2 2 identity matrix [ ] has a double eigenvalue of 1 associated with two linearly independent eigenvectors, say [1, 0] T and [0, 1] T The eigenvalue 1 thus has algebraic multiplicity two and geometric multiplicity two 14

15 Problem 55: Show that the matrix [ ] has a double eigenvalue of 1 but only a single eigenvector [1, 0] T The eigenvalue 1 thus has algebraic multiplicity two, while its geometric multiplicity is only one Problem 56: Show that the matrix has [Anton and Busby, p 458]: (a) an eigenvalue 2 with algebraic and geometric multiplicity one; (b) an eigenvalue 3 with algebraic multiplicity two and geometric multiplicity one 15

16 6 Linear Transformations A mapping T : R n R m is a linear transformation from R n to R m if for all vectors u and v in R n and all scalars c: (a) T (u + v) = T (u) + T (v); (b) T (cu) = ct (u) If m = n, we say that T is a linear operator on R n The action of a linear transformation on a vector can be represented by matrix multiplication: T (x) = Ax An orthogonal linear operator T preserves lengths: T (x) = x for all vectors xb A square matrix A with real entries is orthogonal if A T A = I; that is, A 1 = A T This implies that det A = ±1 and preserves lengths: Ax 2 = Ax Ax = (Ax) T Ax = x T A T Ax = x T A T Ax = x T x = x x = x 2 Since the columns of an orthogonal matrix are unit vectors in addition to being mutually orthogonal, they can also be called orthonormal matrices The matrix that describes rotation about an angle θ is orthogonal: [ ] cos θ sin θ sin θ cos θ The kernel ker T of a linear transformation T is the subspace consisting of all vectors u that are mapped to 0 by T The null space null A (also known as the kernel ker A) of a matrix A is the subspace consisting of all vectors u such that Au = 0 Linear transformations map subspaces to subspaces The range ran T of a linear transformation T : R n R m is the image of the domain R n under T A linear transformation T is one-to-one if T (u) = T (v) u = v 16

17 Problem 61: Show that a linear transformation is one-to-one if and only if ker T = {0} A linear operator T on R n is one-to-one if and only if it is onto A set {v 1, v 2,, v n } of linearly independent vectors is called a basis for the n- dimensional subspace span{v 1, v 2,, v n } = {c 1 v 1 + c 2 v c n v n : (c 1, c 2,, c n ) R n } If w = a 1 v 1 + a 2 v v n is a vector in span{v 1, v 2,, v n } with basis B = {v 1, v 2,, v n } we say that [a 1, a 2,, a n ] T are the coordinates [w] B of the vector w with respect to the basis B If w is a vector in R n the transition matrix P B B that changes coordinates [w] B with respect to the basis B = {v 1, v 2,, v n } for R n to coordinates [w] B = P B B[w] B with respect to the basis B = {v 1, v 2,, v n} for R n is P B B = [[v 1 ] B [v 2 ] B [v n ] B ] An easy way to find the transition matrix P B B from a basis B to B is to use elementary row operations to reduce the matrix [B B] to [I P B B] Note that the matrices P B B and P B B are inverses of each other If T : R n R n is a linear operator and if B = {v 1, v 2,, v n } and B = {v 1, v 2,, v n} are bases for R n then the matrix representation [T ] B with respect to the bases B is related to the matrix representation [T ] B with respect to the bases B according to [T ] B = P B B[T ] B P B B 7 Dimension The number of linearly independent rows (or columns) of a matrix A is known as its rank and written rank A That is, the rank of a matrix is the dimension dim(row(a)) of the span of its rows Equivalently, the rank of a matrix is the dimension dim(col(a)) of the span of its columns The dimension of the null space of a matrix A is known as its nullity and written dim(null(a)) or nullity A 17

18 The dimension theorem states that if A is an m n matrix then rank A + nullity A = n If A is an m n matrix then dim(row(a)) = rank A dim(null(a)) = n rank A, dim(col(a)) = rank A dim(null(a T )) = m rank A Problem 71: Find dim(row) and dim(null) for both A and A T : A = While the row space of a matrix is invariant to (unchanged by) elementary row operations, the column space is not For example, consider the row-equivalent matrices [ ] [ ] A = and B = Note that row(a) = span([1, 0]) = row(b) However col(a) = span([0, 1] T ) but col(b) = span([1, 0] T ) The following are equivalent for an m n matrix A: (a) A has rank k; (b) A has nullity n k; (c) Every row echelon form of A has k nonzero rows and m k zero rows; (d) The homogeneous system Ax = 0 has k pivot variables and n k free variables 8 Similarity and Diagonalizability If A and B are square matrices of the same size and B = P 1 AP for some invertible matrix P, we say that B is similar to A Problem 81: If B is similar to A, show that A is also similar to B 18

19 Problem 82: Suppose B = P 1 AP Multiplying A by the matrix P 1 on the left certainly cannot increase its rank (the number of linearly independent rows or columns), so rank P 1 A rank A Likewise rank B = rank P 1 AP rank P 1 A rank A But since we also know that A is similar to B, we see that rank A rank B Hence rank A = rank B Problem 83: Show that similar matrices also have the same nullity Similar matrices represent the same linear transformation under the change of bases described by the matrix P Problem 84: Prove that similar matrices have the following similarity invariants: (a) rank; (b) nullity; (c) characteristic polynomial and eigenvalues, including their algebraic and geometric multiplicities; (d) determinant; (e) trace Proof of (c): Suppose B = P 1 AP We first note that λi B = λi P 1 AP = P 1 λip P 1 AP = P 1 (λi A)P The matrices λi B and λi A are thus similar and share the same nullity (and hence geometric multiplicity) for each eigenvalue λ Moreover, this implies that the matrices A and B share the same characteristic polynomial (and hence eigenvalues and algebraic multiplicities): det(λi B) = det(p 1 ) det(λi A) det(p) = 1 det(λi A) det(p) = det(λi A) det(p) A matrix is said to be diagonalizable if it is similar to a diagonal matrix 19

20 An n n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors Suppose that P 1 AP = D, where p 11 p 12 p 1n λ p P = 21 p 22 p 2n and D = 0 λ 2 0 p n1 p n2 p nn 0 0 λ n Since P is invertible, its columns are nonzero and linearly independent Moreover, λ 1 p 11 λ 2 p 12 λ n p 1n λ AP = PD = 1 p 21 λ 2 p 22 λ n p 2n λ 1 p n1 λ 2 p n2 λ n p nn That is, for j = 1, 2,, n: A p 1j p 2j p nj = λ j p 1j p 2j p nj This says, for each j, that the nonzero column vector [p 1j, p 2j,, p nj ] T is an eigenvector of A with eigenvalue λ j Moreover, as mentioned above, these n column vectors are linearly independent This established one direction of the claim On reversing this argument, we see that if A has n linearly independent eigenvectors, we can use them to form an eigenvector matrix P such that AP = PD Equivalently, an n n matrix is diagonalizable if and only if the geometric multiplicity of each eigenvalue is the same as its algebraic multiplicity (so that the sum of the geometric multiplicities of all eigenvalues is n) Problem 85: Show that the matrix [ ] λ 1 0 λ has a double eigenvalue of λ but only one eigenvector [1, 0] T (ie the eigenvalue λ has algebraic multiplicity two but geometric multiplicity one) and consequently is not diagonalizable 20

21 Eigenvectors of a matrix A associated with distinct eigenvalues are linearly independent If not, then one of them would be expressible as a linear combination of the others Let us order the eigenvectors so that v k+1 is the first eigenvector that is expressible as a linear combination of the others: v k+1 = c 1 v 1 + c 2 v c k v k, (1) where the coefficients c i are not all zero The condition that v k+1 is the first such vector guarantees that the vectors on the right-hand side are linearly independent If each eigenvector v i corresponds to the eigenvalue λ i, then on multiplying by A on the left, we find that λ k+1 v k+1 = Av k+1 = c 1 Av 1 + c 2 Av c k Av k = c 1 λ 1 v 1 + c 2 λ 2 v c k λ k v k If we multiply Eq(1) by λ k+1 we obtain: λ k+1 v k+1 = c 1 λ k+1 v 1 + c 2 λ k+1 v c k λ k+1 v k, The difference of the previous two equations yields 0 = c 1 (λ 1 λ k+1 )v 1 + (λ 2 λ k+1 )c 2 v (λ k λ k+1 )c k v k, Since the vectors on the right-hand side are linearly independent, we know that each of the coefficients c i (λ i λ k+1 ) must vanish But this is not possible since the coefficients c i are nonzero and eigenvalues are distinct The only way to escape this glaring contradiction is that all of the eigenvectors of A corresponding to distinct eigenvalues must in fact be independent! An n n matrix A with n distinct eigenvalues is diagonalizable However, the converse is not true (consider the identity matrix) In the words of Anton & Busby, the key to diagonalizability rests with the dimensions of the eigenspaces, not with the distinctness of the eigenvalues Problem 86: Show that the matrix A = [ ] from Problem 51 is diagonalizable by evaluating [ ][ ][ What do you notice about the resulting product and its diagonal elements? What is the significance of each of the above matrices and the factor 1/5 in front? Show that A may be decomposed as [ ][ ] [ ] A = ] 21

22 Problem 87: Show that the characteristic polynomial of the matrix that describes rotation in the x y plane by θ = 90 : A = [ ] is λ = 0 This equation has no real roots, so A is not diagonalizable over the real numbers However, it is in fact diagonalizable over a broader number system known as the complex numbers C: we will see that the characteristic equation λ = 0 admits two distinct complex eigenvalues, namely i and i, where i is an imaginary number that satisfies i = 0 Remark: An efficient way to compute high powers and fractional (or even negative powers) of a matrix A is to first diagonalize it; that is, using the eigenvector matrix P to express A = PDP 1 where D is the diagonal matrix of eigenvalues Problem 88: To find the 5th power of A note that A 5 = ( PDP 1)( PDP 1)( PDP 1)( PDP 1)( PDP 1) = PDP 1 PDP 1 PDP 1 PDP 1 PDP 1 = PD [ 5 P 1 ][ ] [ ] 1 2 ( 1) = [ ][ ][ ] = [ ][ ] = = 1 [ ] [ ] = Problem 89: Check the result of the previous problem by manually computing the product A 5 Which way is easier for computing high powers of a matrix? 22

23 Remark: We can use the same technique to find the square root A 1 2 of A (ie a matrix B such that B 2 = A, we will again need to work in the complex number system C, where i 2 = 1: A = PD 2 P 1 = 1 [ ][ ][ ] 1 2 i = 1 [ ][ ] i i = 1 [ ] 4 + 3i 4 2i 5 6 3i 6 + 2i Then )( ) A A 2 = (PD 1 2 P 1 PD 1 2 P 1 = PD D 2 P 1 = PDP 1 = A, as desired Problem 810: Verify explicitly that [ ][ ] i 4 2i 4 + 3i 4 2i = i 6 + 2i 6 3i 6 + 2i 9 Complex Numbers [ ] 1 2 = A 3 2 In order to diagonalize a matrix A with linearly independent eigenvectors (such as a matrix with distinct eigenvalues), we first need to solve for the roots of the characteristic polynomial det(λi A) = 0 These roots form the n elements of the diagonal matrix that A is similar to However, as we have already seen, a polynomial of degree n does not necessarily have n real roots Recall that z is a root of the polynomial P (x) if P (z) = 0 Q Do all polynomials have at least one root z R? A No, consider P (x) = x It has no real roots: P (x) 1 for all x The complex numbers C are introduced precisely to circumvent this problem If we replace z R by z C, we can answer the above question affirmatively The complex numbers consist of ordered pairs (x, y) together with the usual component-by-component addition rule (eg which one has in a vector space) (x, y) + (u, v) = (x + u, y + v), 23

24 but with the unusual multiplication rule (x, y) (u, v) = (xu yv, xv + yu) Note that this multiplication rule is associative, commutative, and distributive Since (x, 0) + (u, 0) = (x + u, 0) and (x, 0) (u, 0) = (xu, 0), we see that (x, 0) and (u, 0) behave just like the real numbers x and u In fact, we can map (x, 0) C to x R: (x, 0) x Hence R C Remark: We see that the complex number z = (0, 1) satisfies the equation z 2 +1 = 0 That is, (0, 1) (0, 1) = ( 1, 0) Denote (0, 1) by the letter i Then any complex number (x, y) can be represented as (x, 0) + (0, 1)(y, 0) = x + iy Remark: Unlike R, the set C = {(x, y) : x R, y R} is not ordered; there is no notion of positive and negative (greater than or less than) on the complex plane For example, if i were positive or zero, then i 2 = 1 would have to be positive or zero If i were negative, then i would be positive, which would imply that ( i) 2 = i 2 = 1 is positive It is thus not possible to divide the complex numbers into three classes of negative, zero, and positive numbers Remark: The frequently appearing notation 1 for i is misleading and should be avoided, because the rule xy = x y (which one might anticipate) does not hold for negative x and y, as the following contradiction illustrates: 1 = 1 = ( 1)( 1) = 1 1 = i 2 = 1 Furthermore, by definition x 0, but one cannot write i 0 since C is not ordered Remark: We may write (x, 0) = x + i0 = x since i0 = (0, 1) (0, 0) = (0, 0) = 0 The complex conjugate (x, y) of (x, y) is (x, y) That is, x + iy = x iy 24

25 The complex modulus z of z = x + iy is given by x 2 + y 2 Remark: If z R then z = z 2 is just the absolute value of z We now establish some important properties of the complex conjugate Let z = x + iy and w = u + iv be elements of C Then (i) (ii) (iii) zz = (x, y)(x, y) = (x 2 + y 2, yx xy) = (x 2 + y 2, 0) = x 2 + y 2 = z 2, z + w = z + w, zw = z w Problem 91: Prove properties (ii) and (iii) Remark: Property (i) provides an easy way to compute reciprocals of complex numbers: 1 z = z zz = z z 2 Remark: Properties (i) and (iii) imply that Thus zw = z w for all z, w C zw 2 = zwzw = zzww = z 2 w 2 The dot product of vectors u = [u 1, u 2,, u n ] and v = [v 1, v 2,, v n ] in C n is given by u v = u T v = u 1 v u n v n Remark: Note that this definition of the dot product implies that u v = v u = v T u Remark: Note however that u u can be written either as u T u or as u T u Problem 92: Prove for k C that (ku) v = k(u v) but u (kv) = k(u v) 25

26 In view of the definition of the complex modulus, it is sensible to define the length of a vector v = [v 1, v 2,, v n ] in C n to be v = v v = v T v = v 1 v v n v n = v v n 2 0 Lemma 1 (Complex Conjugate Roots): Let P be a polynomial with real coefficients If z is a root of P, then so is z Proof: Suppose P (z) = n k=0 a kz k = 0, where each of the coefficients a k are real Then P (z) = n a k (z) k = n a k z k = n a k z k = n a k z k = P (z) = 0 = 0 k=0 k=0 k=0 k=0 Thus, complex roots of real polynomials occur in conjugate pairs, z and z Remark: Lemma 1 implies that eigenvalues of a matrix A with real coefficients also occur in conjugate pairs: if λ is an eigenvalue of A, then so if λ Moreover, if x is an eigenvector corresponding to λ, then x is an eigenvector corresponding to λ: Ax = λx Ax = λx Ax = λx Ax = λx A real matrix consists only of real entries Problem 93: Find the eigenvalues and eigenvectors of the real matrix [Anton & Busby, p 528] [ ] If A is a real symmetric matrix, it must have real eigenvalues This follows from the fact that an eigenvalue λ and its eigenvector x 0 must satisfy Ax = λx x T Ax = x T λx = λx x = λ x 2 λ = xt Ax x 2 = (AT x) T x x 2 = (Ax)T x x 2 = (λx)t x x 2 = λx x x 2 = λ 26

27 Remark: There is a remarkable similarity between the complex multiplication rule (x, y) (u, v) = (xu yv, xv + yu) and the trigonometric angle sum formulae Notice that (cos θ, sin θ) (cos φ, sin φ) = (cos θ cos φ sin θ sin φ, cos θ sin φ + sin θ cos φ) = (cos(θ + φ), sin(θ + φ)) That is, multiplication of 2 complex numbers on the unit circle x 2 + y 2 = 1 corresponds to addition of their angles of inclination to the x axis In particular, the mapping f(z) = z 2 doubles the angle of z = (x, y) and f(z) = z n multiplies the angle of z by n These statements hold even if z lies on a circle of radius r 1, (r cos θ, r sin θ) n = r n (cos nθ, sin nθ); this is known as demoivre s Theorem The power of complex numbers comes from the following important theorem Theorem 1 (Fundamental Theorem of Algebra): Any non-constant polynomial P (z) with complex coefficients has a root in C Lemma 2 (Polynomial Factors): If z 0 is a root of a polynomial P (z) then P (z) is divisible by (z z 0 ) Proof: Suppose z 0 is a root of a polynomial P (z) = n k=0 a kz k of degree n Consider P (z) = P (z) P (z 0 ) = = n a k z k k=0 n k=0 n a k z0 k = k=0 n a k (z k z0) k k=0 a k (z z 0 )(z k 1 + z k 2 z z k 1 0 ) = (z z 0 )Q(z), where Q(z) = n k=0 a k(z k 1 + z k 2 z z k 1 0 ) is a polynomial of degree n 1 Corollary 11 (Polynomial Factorization): Every complex polynomial P (z) of degree n 0 has exactly n complex roots z 1, z 2,, z n and can be factorized as P (z) = A(z z 1 )(z z 2 ) (z z n ), where A C Proof: Apply Theorem 1 and Lemma 2 recursively n times (It is conventional to define the degree of the zero polynomial, which has infinitely many roots, to be ) 27

28 10 Projection Theorem The component of a vector v in the direction û is given by v û = v cos θ The orthogonal projection (parallel component) v of v onto a unit vector û is v = (v û)û = v u u 2 u The perpendicular component of v relative to the direction û is v = v (v û)û The decomposition v = v + v is unique Notice that v û = v û v û = 0 In fact, for a general n-dimensional subspace W of R m, every vector v has a unique decomposition as a sum of a vector v in W and a vector v in W (this is the set of vectors that are perpendicular to each of the vectors in W ) First, find a basis for W We can write these basis vectors as the columns of an m n matrix A with full column rank (linearly independent columns) Then W is the null space for A T The projection v of v = v + v onto W satisfies v = Ax for some vector x and hence 0 = A T v = A T (v Ax) The vector x thus satisfies the normal equation A T Ax = A T v, (2) so that x = (A T A) 1 A T v We know from Question 4 of Assignment 1 that A T A shares the same null space as A But the null space of A is {0} since the columns of A, being basis vectors, are linearly independent Hence the square matrix A T A is invertible, and there is indeed a unique solution for x The orthogonal projection v of v onto the subspace W is thus given by the formula v = A(A T A) 1 A T v Remark: This complicated projection formula becomes much easier if we first go to the trouble to construct an orthonormal basis for W This is a basis of unit vectors that are mutual orthogonal to one another, so that A T A = I In this case the projection of v onto W reduces to multiplication by the symmetric matrix AA T : v = AA T v 28

29 Problem 101: Find the orthogonal projection of v = [1, 1, 1] onto the plane W spanned by the orthonormal vectors [0, 1, 0] and [ 4 5, 0, 3 5] Compute 0 4 [ ] 16 v = AA T 5 1 v = = = As desired, we note that v v = [ ] , 0, 25 = 7 25 [3, 0, 4] is parallel to the normal [ 3 5, 0, 4 ] 5 to the plane computed from the cross product of the given orthonormal vectors 11 Gram-Schmidt Orthonormalization The Gram-Schmidt process provides a systematic procedure for constructing an orthonormal basis for an n-dimensional subspace span{a 1, a 2,, a n } of R m : 1 Start with the first vector: q 1 = a 1 2 Remove the component of the second vector parallel to the first: q 2 = a 2 a 2 q 1 q 1 2 q 1 3 Remove the components of the third vector parallel to the first and second: q 3 = a 3 a 3 q 1 q 1 2 q 1 a 3 q 2 q 2 2 q 2 4 Continue this procedure by successively defining, for k = 4, 5,, n: q k = a k a k q 1 q 1 2 q 1 a k q 2 q 2 2 q 2 a k q k 1 q k 1 2 q k 1 (3) 5 Finally normalize each of the vectors to obtain the orthogonal basis { ˆq 1, ˆq 2,, ˆq n } Remark: Notice that q 2 is orthogonal to q 1 : q 2 q 1 = a 2 q 1 a 2 q 1 q 1 2 q 1 2 = 0 29

30 Remark: At the kth stage of the orthonormalization, if all of the previously created vectors were orthogonal to each other, then so is the newly created vector: q k q j = a k q j a k q j q j 2 q j 2 = 0, j = 1, 2, k 1 Remark: The last two remarks imply that, at each stage of the orthonormalization, all of the vectors created so far are orthogonal to each other! Remark: Notice that q 1 = a 1 is a linear combination of the original a 1, a 2,, a n Remark: At the kth stage of the orthonormalization, if all of the previously created vectors were a linear combination of the original vectors a 1, a 2,, a n, then by Eq (3), we see that q k is as well Remark: The last two remarks imply that, at each stage of the orthonormalization, all of the vectors created so far are orthogonal to each vector q k can be written as a linear combination of the original basis vectors a 1, a 2,, a n Since these vectors are given to be linearly independent, we thus know that q k 0 This is what allows us to normalize q k in Step 5 It also implies that {q 1, q 2,, q n } spans the same space as {a 1, a 2,, a n } Problem 111: What would happen if we were to apply the Gram-Schmidt procedure to a set of vectors that is not linearly independent? Problem 112: Use the Gram-Schmidt process to find an orthonormal basis for the plane x + y + z = 0 In terms of the two parameters (say) y = s and z = t, we see that each point on the plane can be expressed as x = s t, y = s, z = t The parameter values (s, t) = (1, 0) and (s, t) = (0, 1) then yield two linearly independent vectors on the plane: a 1 = [ 1, 1, 0] and a 2 = [ 1, 0, 1] The Gram-Schmidt process then yields q 1 = [ 1, 1, 0], q 2 = [ 1, 0, 1] = [ 1, 0, 1] 1 [ 1, 1, 0] 2 = [ 12 ], 12, 1 [ 1, 0, 1] [ 1, 1, 0] [ 1, 1, 0] 2 [ 1, 1, 0] We note that q 2 q 1 = 0, as desired Finally, we normalize the vectors to obtain ˆq 1 = [ 1 1 2, 2, 0] and ˆq 2 = [ 1 6, 1 2 6, 6 ] 30

31 12 QR Factorization We may rewrite the kth step (Eq 3) of the Gram-Schmidt orthonormalization process more simply in terms of the unit normal vectors ˆq j : q k = a k (a k ˆq 1 ) ˆq 1 (a k ˆq 2 ) ˆq 2 (a k ˆq k 1 ) ˆq k 1 (4) On taking the dot product with ˆq k we find, using the orthogonality of the vectors ˆq j, 0 q k ˆq k = a k ˆq k (5) so that q k = (q k ˆq k ) ˆq k = (a k ˆq k ) ˆq k Equation 4 may then be rewritten as a k = ˆq 1 (a k ˆq 1 ) + ˆq 2 (a k ˆq 2 ) + + ˆq k 1 (a k ˆq k 1 ) + ˆq k (a k ˆq k ) On varying k from 1 to n we obtain the following set of n equations: a 1 ˆq 1 a 2 ˆq 1 a n ˆq 1 0 a [ a 1 a 2 a n ] = [ ˆq 1 ˆq 2 ˆq n ] 2 ˆq 2 a n ˆq a n ˆq n If we denote the upper triangular n n matrix on the right-hand-side by R, and the matrices composed of the column vectors a k and ˆq k by the m n matrices A and Q, respectively, we can express this result as the so-called QR factorization of A: A = QR (6) Remark: Every matrix m n matrix A with full column rank (linearly independent columns) thus has a QR factorization Remark: Equation 5 implies that each of the diagonal elements of R is nonzero This guarantees that R is invertible Remark: If the columns of Q are orthonormal, then Q T Q = I An efficient way to find R is to premultiply both sides of Eq 6 by Q T : Q T A = R Problem 121: Find the QR factorization of [Anton & Busby, p 418]

32 13 Least Squares Approximation Suppose that a set of data (x i, y i ) for i = 1, 2, n is measured in an experiment and that we wish to test how well it fits an affine relationship of the form y = α + βx Here α and β are parameters that we wish to vary to achieve the best fit If each data point (x i, y i ) happens to fall on the line y = α+βx, then the unknown parameters α and β can be determined from the matrix equation 1 x 1 [ ] 1 x 2 α = β 1 x n If the x i s are distinct, then the n 2 coefficient matrix A on the left-hand side, known as the design matrix, has full column rank (two) Also note in the case of exact agreement between the experimental data and the theoretical model y = α + βx that the vector b on the right-hand side is in the column space of A Recall that if b is in the column space of A, the linear system of equations Ax = b is consistent, that is, it has at least one solution x Recall that this solution is unique iff A has full column rank y 1 y 2 y n (7) In practical applications, however, there may be measurement errors in the entries of A and b so that b does not lie exactly in the column space of A The least squares approximation to an inconsistent system of equations Ax = b is the solution to Ax = b where b denotes the projection of b onto the column space of A From Eq (2) we see that the solution vector x satisfies the normal equation A T Ax = A T b Remark: For the normal equation Eq (7), we see that 1 x [ ] 1 [ ] 1 1 A T A = 1 x 2 x 1 x n = n xi xi x 2 i 1 x n 32

33 and [ ] 1 1 A T b = x 1 x n y n where the sums are computed from i = 1 to n Thus the solution to the leastsquares fit of Eq (7) is given by [ ] α = β y 1 y 2 = [ yi xi y i ], [ ] 1 [ n xi xi x 2 yi ] i xi y i Problem 131: Show that the least squares line of best fit to the measured data (0, 1), (1, 3), (2, 4), and (3, 4) is y = 15 + x The difference b = b b is called the least squares error vector The least squares method is sometimes called linear regression and the line y = α + βx can either be called the least squares line of best fit or the regression line For each data pair (x i, y i ), the difference y i (α + βx i ) is called the residual Remark: Since the least-squares solution x = [α, β] minimizes the least squares error vector b = b Ax = [y 1 (α + βx 1 ),, y n (α + βx n )], n the method effectively minimizes the sum [y i (α + βx i )] 2 of the squares of the residuals A numerically robust way to solve the normal equation A T Ax = A T b is to factor A = QR: R T Q T QRx = R T Q T b which, since R T is invertible, simplifies to i=1 x = R 1 Q T b Problem 132: Show that for the same measured data as before, (0, 1), (1, 3), (2, 4), and (3, 4), that [ ] 2 3 R = 0 5 and from this that the least squares solution again is y = 15 + x 33

34 Remark: The least squares method can be extended to fit a polynomial y = a 0 + a 1 x + + a m x m as close as possible to n given data points (x i, y i ): 1 x 1 x 2 1 x m 1 1 x 2 x 2 2 x m 2 1 x n x 2 n x m n a 0 a 1 a m = Here, the design matrix takes the form of a Vandermonde matrix, which has special properties For example, in the case where m = n 1, its determinant is given n by (x i x j ) If the n x i values are distinct, this determinant is nonzero The i,j=1 i>j system of equations is consistent and has a unique solution, known as the Lagrange interpolating polynomial This polynomial passes exactly through the given data points However, if m < n 1, it will usually not be possible to find a polynomial that passes through the given data points and we must find the least squares solution by solving the normal equation for this problem [cf Example 7 on p 403 of Anton & Busby] 14 Orthogonal (Unitary) Diagonalizability A matrix is said to be Hermitian if A T = A y 1 y 2 y n For real matrices, being Hermitian is equivalent to being symmetric On defining the Hermitian transpose A = A T, we see that a Hermitian matrix A obeys A = A For real matrices, the Hermitian transpose is equivalent to the usual transpose A complex matrix P is orthogonal (or orthonormal) if P P = I (Most books use the term unitary here but in view of how the dot product of two complex vectors is defined, there is really no need to introduce new terminology) For real matrices, an orthogonal matrix A obeys A T A = I 34

35 If A and B are square matrices of the same size and B = P AP for some orthogonal matrix P, we say that B is orthogonally similar to A Problem 141: Show that two orthogonally similar matrices A and B are similar Orthogonally similar matrices represent the same linear transformation under the orthogonal change of bases described by the matrix P A matrix is said to be orthogonally diagonalizable if it is orthogonally similar to a diagonal matrix Remark: An n n matrix A is orthogonally diagonalizable if and only if it has an orthonormal set of n eigenvectors But which matrices have this property? The following definition will help us answer this important question A matrix A that commutes with its Hermitian transpose, so that A A = AA, is said to be normal Problem 142: Show that every diagonal matrix is normal Problem 143: Show that every Hermitian matrix is normal An orthogonally diagonalizable matrix must be normal Suppose D = P AP for some diagonal matrix D and orthogonal matrix P Then and A = PDP, A = (PDP ) = PD P Then AA = PDP PD P = PDD P and A A = PD P PDP = PD DP But D D = DD since D is diagonal Therefore A A = AA If A is a normal matrix and is orthogonally similar to U, then U is also normal: U U = ( P AP ) P AP = P A PP AP = P A AP = P AA P = P APP A P = UU 35

36 if A is an orthogonally diagonalizable matrix with real eigenvalues, it must be Hermitian: if A = PDP, then A = (PDP ) = PD P = PDP = A Moreover, if the entries in A are also real, then A must be symmetric Q Are all normal matrices orthogonally diagonalizable? A Yes The following important matrix factorization guarantees this An n n matrix A has a Schur factorization A = PUP, where P is an orthogonal matrix and U is an upper triangle matrix This can be easily seen from the fact that the characteristic polynomial always has at least one complex root, so that A has at least one eigenvalue λ 1 corresponding to some unit eigenvector x 1 Now use the Gram-Schmidt process to construct some orthonormal basis {x 1, b 2,, b n } for R n Note here that the first basis vector is x 1 itself In this coordinate system, the eigenvector x 1 has the components [1, 0,, 0] T, so that 1 A 0 0 = λ This requires that A have the form λ We can repeat this argument for the n 1 n 1 submatrix obtained by deleting the first row and column from A This submatrix must have an eigenvalue λ 2 corresponding to some eigenvector x 2 Now construct an orthogonal basis {x 2, b 3,, b n } for R n 1 In this new coordinate system x 2 = [λ 2, 0,, 0] T and the first column of the submatrix appears as λ

37 This means that in the coordinate system {x 1, x 2, b 3,, b n }, A has the form λ 1 0 λ Continuing in this manner we thus construct an orthogonal basis {x 1, x 2, x 3,, x n } in which A appears as an upper triangle matrix U whose diagonal values are just the n (not necessarily distinct) eigenvalues of A Therefore, A is orthogonally similar to an upper triangle matrix, as claimed Remark: Given a normal matrix A with Schur factorization A = PUP, we have seen that U is also normal Problem 144: Show that every normal n n upper triangular matrix U is a diagonal matrix Hint: letting U ij for i j be the nonzero elements of U, we can write out for each i = 1,, n the diagonal elements i k=1 U ik U ki = i k=1 U kiu ki of U U = UU : i U ki 2 = k=1 n U ik 2 k=i Thus A is orthogonally diagonalizable if and only if it is normal If A is a Hermitian matrix, it must have real eigenvalues This follows from the fact that an eigenvalue λ and its eigenvector x 0 must satisfy Ax = λx x Ax = x λx = λx x = λ x 2 λ = x Ax x 2 = (A x) x x 2 = (Ax) x x 2 = (λx) x x 2 = λx x x 2 = λ Problem 145: Show that the Hermitian matrix [ 0 ] i i 0 is similar to a diagonal matrix with real eigenvalues and find the eigenvalues 37

38 Problem 146: Show that the anti-hermitian matrix [ ] 0 i i 0 is similar to a diagonal matrix with complex eigenvalues and find the eigenvalues If A is an n n normal matrix, then Ax 2 = x A Ax = x AA x = A x 2 for all vectors x R n Problem 147: If A is a normal matrix, show that A λi is also normal If A is a normal matrix and Ax = λx, then 0 = (A λi)x = (A λi)x, so that A x = λx If A is a normal matrix, the eigenvectors associated with distinct eigenvalues are orthogonal: if Ax = λx and Ay = µy, then 0 = (x Ay) T x Ay = y T A x x Ay = y T λx x µy = λx y µx y = (λ µ)x y, so that x y = 0 whenever λ µ An n n normal matrix A can therefore be orthogonally diagonalized by applying the Gram-Schmidt process to each of its distinct eigenspaces to obtain a set of n mutually orthogonal eigenvectors that form the columns of an orthonormal matrix P such that AP = PD, where the diagonal matrix D contains the eigenvalues of A Problem 148: Find a matrix P that orthogonally diagonalizes Remark: If a normal matrix has eigenvalues λ 1, λ 2,, λ n corresponding to the orthonormal matrix [ x 1 x 2 x n ] then A has the spectral decomposition λ x 1 A = [ x 1 x 2 x n ] 0 λ 2 0 x λ n x n = [ λ 1 x 1 λ 2 x 2 λ n x n ] x 1 x 2 x n = λ 1 x 1 x 1 + λ 2 x 2 x λ n x n x n 38

39 As we have seen earlier, powers of an orthogonally diagonalizable matrix A = PDP are easy to compute once P and U are known The Caley-Hamilton Theorem makes a remarkable connection between the powers of an n n matrix A and its characteristic equation λ n +c n 1 λ n 1 + c 1 λ+c 0 = 0 Specifically, the matrix A satisfies its characteristic equation in the sense that A n + c n 1 A n c 1 A + c 0 I = 0 (8) Problem 149: The characteristic polynomial of [ ] 1 2 A = 3 2 is λ 2 3λ 4 = 0 Verify that the Caley-Hamilton theorem A 2 3A 4I = 0 holds by showing that A 2 = 3A+4I Then use the Caley-Hamilton theorem to show that [ ] A 3 = A 2 A = (3A + 4I)A = 3A 2 + 4A = 3(3A + 4I) + 4A = 13A + 12I = Remark: The Caley-Hamilton Theorem can also be used to compute negative powers (such as the inverse) of a matrix For example, we can rewrite 8 as ( A 1 A n 1 c n 1 A n 2 c ) 1 A = I c 0 c 0 c 0 We can also easily compute other functions of diagonalizable matrices Give an analytic function f with a Taylor series f(x) = f(0) + f (0)x + f (0) x2 2! + f (0) x3 3! +, and a diagonalizable matrix A = PDP 1, where D is diagonal and P is invertible, one defines f(a) = f(0)i + f (0)A + f (0) A2 2! + f (0) A3 3! + On recalling that A n = PD n P 1 and A 0 = I, one can rewrite this relation as (9) f(a) = f(0)pd 0 P 1 + f (0)PDP 1 + f (0) PD2 P 1 + f (0) PD3 P 1 3! ] 3! = P [f(0)d 0 + f (0)D + f (0) D2 3! + f (0) D3 3! + P 1 = Pf(D)P 1 Here f(d) is simply the diagonal matrix with elements f(d ii ) + (10) 39

40 Problem 1410: Compute e A for the diagonalizable matrix [ ] 1 2 A = Systems of Differential Equations Consider the first-order linear differential equation y = ay, where y = y(t), a is a constant, and y denotes the derivative of y with respect to t For any constant c the function y = ce at is a solution to the differential equation If we specify an initial condition then the appropriate value of c is y 0 y(0) = y 0, Now consider a system of first-order linear differential equations: y 1 = a 11 y 1 + a 12 y a 1n y n, y 2 = a 21 y 1 + a 22 y a 2n y n, y n = a n1 y 1 + a n2 y a nn y n, which in matrix notation appears as y 1 a 11 a 12 a 1n y 2 = a 21 a 22 a 2n y n a n1 a n2 a nn or equivalently as y = Ay Problem 151: Solve the system [Anton & Busby p 543]: y 1 y 2 = y 1 y 2 y y 3 for the initial conditions y 1 (0) = 1, y 2 (0) = 4, y 3 (0) = 2 y 1 y 2 y n, 40

41 Remark: A linear combination of solutions to an ordinary differential equation is also a solution If A is an n n matrix, then y = Ay has a set of n linearly independent solutions All other solutions can be expressed as linear combinations of such a fundamental set of solutions If x is an eigenvector of A corresponding to the eigenvalue λ, then y = ɛ λt x is a solution of y = Ay: y = d dt eλt x = e λt λx = e λt Ax = Ae λt x = Ay If x 1, x 2,, x k, are linearly independent eigenvectors of A corresponding to the (not necessarily distinct) eigenvalues λ 1, λ 2,, λ k, then y = e λ 1t x 1, y = e λ 2t x 2,, y = e λ kt x k are linearly independent solutions of y = Ay: if then at t = 0 we find 0 = c 1 e λ 1t x 1 + c 2 e λ 2t x c k e λ kt x k 0 = c 1 x 1 + c 2 x c k x k ; the linear independence of the k eigenvectors then implies that c 1 = c 2 = = c k = 0 If x 1, x 2,, x n, are linearly independent eigenvectors of an n n matrix A corresponding to the (not necessarily distinct) eigenvalues λ 1, λ 2,, λ n, then every solution to y = Ay can be expressed in the form y = c 1 e λ 1t x 1 + c 2 e λ 2t x c n e λnt x n, which is known as the general solution to y = Ay On introducing the matrix of eigenvectors P = [ x 1 x 2 x n ], we may write the general solution as e λ 1t 0 0 c 1 y = c 1 e λ1t x 1 + c 2 e λ2t x c n e λnt x n = P 0 e λ 2t 0 c e λnt c n 41

42 Moreover, if y = y 0 at t = 0 we find that y 0 = P If the n eigenvectors x 1, x 2,, x n are linearly independent, then c 1 c 2 c n c 1 c 2 c n = P 1 y 0 The solution to y = Ay for the initial condition y 0 may thus be expressed as e λ 1t 0 0 y = P 0 e λ 2t 0 P 1 y 0 = e At y 0, 0 0 e λnt on making use of Eq (10) with f(x) = e x and A replaced by At Problem 152: Find the solution to y = Ay corresponding to the initial condition y 0 = [1, 1], where [ ] 1 2 A = 3 2 Remark: The solution y = e At y 0 to the system of ordinary differential equations y = Ay y(0) = y 0 actually holds even when the matrix A isn t diagonalizable (that is, when A doesn t have a set of n linearly independent eigenvectors) In this case we cannot use Eq (10) to find e At Instead we must find e At from the infinite series in Eq (9): e At = I + At + A2 t 2 2! + A3 t 3 3! + Problem 153: Show for the matrix A = that the series for e At simplifies to I + At [ ]

43 16 Quadratic Forms Linear combinations of x 1, x 2,, x n such as a 1 x 1 + a 2 x a n x n = n a i x i, where a 1, a 2,, a n are fixed but arbitrary constants, are called linear forms on R n Expressions of the form n a ij x i x j, i,j=1 where the a ij are fixed but arbitrary constants, are called quadratic forms on R n Without loss of generality, we restrict the constants a ij to be the components of a symmetric matrix A and define x = [x 1, x 2 x n ], so that i=1 x T Ax = i,j x i a ij x j = i<j a ij x i x j + i=j a ij x i x j + i>j a ij x i x j = i=j a ij x i x j + i<j a ij x i x j + i<j a ji x j x i = i=j a ii x 2 i + 2 i<j a ij x i x j Remark: Note that x T Ax = x Ax = Ax x (Principle Axes Theorem) Consider the change of variable x = Py, where P orthogonally diagonalizes A Then in terms of the eigenvalues λ 1, λ 2,, λ n of A: x T Ax = (Py) T APy = y T (P T AP)y = λ 1 y λ 2 y λ n y 2 n (Constrained Extremum Theorem) For a symmetric n n matrix A, the minimum (maximum) value of {x T Ax : x 2 = 1} is given by the smallest (largest) eigenvalue of A and occurs at the corresponding unit eigenvector To see this, let x = Py, where P orthogonally diagonalizes A, and list the eigenvalues in ascending order: λ 1 λ 2 λ n Then λ 1 = λ 1 (y1 2 +y yn) 2 λ 1 y1 2 + λ 2 y λ n yn 2 λ }{{} n (y1 2 +y yn) 2 = λ n x T Ax 43

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam

Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system

### HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe

### c c c c c c c c c c a 3x3 matrix C= has a determinant determined by

Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### and let s calculate the image of some vectors under the transformation T.

Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

### Topic 2 Quiz 2. choice C implies B and B implies C. correct-choice C implies B, but B does not imply C

Topic 1 Quiz 1 text A reduced row-echelon form of a 3 by 4 matrix can have how many leading one s? choice must have 3 choice may have 1, 2, or 3 correct-choice may have 0, 1, 2, or 3 choice may have 0,

### Math 25a Practice Final #1 Solutions

Math 25a Practice Final #1 Solutions Problem 1. Suppose U and W are subspaces of V such that V = U W. Suppose also that u 1,..., u m is a basis of U and w 1,..., w n is a basis of W. Prove that is a basis

### 5.6. PSEUDOINVERSES 101. A H w.

5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and

### Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

### 1 Determinants. 1.1 Determinant

1 Determinants [SB], Chapter 9, p.188-196. [SB], Chapter 26, p.719-739. Bellow w ll study the central question: which additional conditions must satisfy a quadratic matrix A to be invertible, that is to

### Chapter 8. Rigid transformations

Chapter 8. Rigid transformations We are about to start drawing figures in 3D. There are no built-in routines for this purpose in PostScript, and we shall have to start more or less from scratch in extending

### 18.06 Problem Set 8 - Solutions Due Wednesday, 14 November 2007 at 4 pm in

806 Problem Set 8 - Solutions Due Wednesday, 4 November 2007 at 4 pm in 2-06 08 03 Problem : 205+5+5+5 Consider the matrix A 02 07 a Check that A is a positive Markov matrix, and find its steady state

### CHAPTER 8: Matrices and Determinants

(Exercises for Chapter 8: Matrices and Determinants) E.8.1 CHAPTER 8: Matrices and Determinants (A) means refer to Part A, (B) means refer to Part B, etc. Most of these exercises can be done without a

### September 26, 2017 EIGENVALUES AND EIGENVECTORS. APPLICATIONS

September 26, 207 EIGENVALUES AND EIGENVECTORS. APPLICATIONS RODICA D. COSTIN Contents 4. Eigenvalues and Eigenvectors 3 4.. Motivation 3 4.2. Diagonal matrices 3 4.3. Example: solving linear differential

### Determinants. Beifang Chen

Determinants Beifang Chen 1 Motivation Determinant is a function that each square real matrix A is assigned a real number, denoted det A, satisfying certain properties If A is a 3 3 matrix, writing A [u,

### MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2.

MATH 304 Linear Algebra Lecture 23: Diagonalization. Review for Test 2. Diagonalization Let L be a linear operator on a finite-dimensional vector space V. Then the following conditions are equivalent:

### Math 240 Calculus III

The Calculus III Summer 2015, Session II Wednesday, July 8, 2015 Agenda 1. of the determinant 2. determinants 3. of determinants What is the determinant? Yesterday: Ax = b has a unique solution when A

### Practice Problems for the Final Exam

Practice Problems for the Final Exam Linear Algebra. Matrix multiplication: (a) Problem 3 in Midterm One. (b) Problem 2 in Quiz. 2. Solve the linear system: (a) Problem 4 in Midterm One. (b) Problem in

### Eigenvalues and Eigenvectors

Chapter 6 Eigenvalues and Eigenvectors 6. Introduction to Eigenvalues Eigenvalues are the key to a system of n differential equations : dy=dt ay becomes dy=dt Ay. Now A is a matrix and y is a vector.y.t/;

### 2. Review of Linear Algebra

2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

### Span and Linear Independence

Span and Linear Independence It is common to confuse span and linear independence, because although they are different concepts, they are related. To see their relationship, let s revisit the previous

### Math 20F Practice Final Solutions. Jor-el Briones

Math 2F Practice Final Solutions Jor-el Briones Jor-el Briones / Math 2F Practice Problems for Final Page 2 of 6 NOTE: For the solutions to these problems, I skip all the row reduction calculations. Please

### MATH 300, Second Exam REVIEW SOLUTIONS. NOTE: You may use a calculator for this exam- You only need something that will perform basic arithmetic.

MATH 300, Second Exam REVIEW SOLUTIONS NOTE: You may use a calculator for this exam- You only need something that will perform basic arithmetic. [ ] [ ] 2 2. Let u = and v =, Let S be the parallelegram

### (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

### MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian.

MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian. Spanning set Let S be a subset of a vector space V. Definition. The span of the set S is the smallest subspace W V that contains S. If

### Matrices. Chapter What is a Matrix? We review the basic matrix operations. An array of numbers a a 1n A = a m1...

Chapter Matrices We review the basic matrix operations What is a Matrix? An array of numbers a a n A = a m a mn with m rows and n columns is a m n matrix Element a ij in located in position (i, j The elements

### Lecture 23: Trace and determinants! (1) (Final lecture)

Lecture 23: Trace and determinants! (1) (Final lecture) Travis Schedler Thurs, Dec 9, 2010 (version: Monday, Dec 13, 3:52 PM) Goals (2) Recall χ T (x) = (x λ 1 ) (x λ n ) = x n tr(t )x n 1 + +( 1) n det(t

### Linear Algebra: Linear Systems and Matrices - Quadratic Forms and Deniteness - Eigenvalues and Markov Chains

Linear Algebra: Linear Systems and Matrices - Quadratic Forms and Deniteness - Eigenvalues and Markov Chains Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 3, 3 Systems

### Linear Algebra Final Exam Study Guide Solutions Fall 2012

. Let A = Given that v = 7 7 67 5 75 78 Linear Algebra Final Exam Study Guide Solutions Fall 5 explain why it is not possible to diagonalize A. is an eigenvector for A and λ = is an eigenvalue for A diagonalize

### Math 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.

Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses

### Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim

Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its

### Ph.D. Katarína Bellová Page 1 Mathematics 2 (10-PHY-BIPMA2) EXAM - Solutions, 20 July 2017, 10:00 12:00 All answers to be justified.

PhD Katarína Bellová Page 1 Mathematics 2 (10-PHY-BIPMA2 EXAM - Solutions, 20 July 2017, 10:00 12:00 All answers to be justified Problem 1 [ points]: For which parameters λ R does the following system

### GATE Engineering Mathematics SAMPLE STUDY MATERIAL. Postal Correspondence Course GATE. Engineering. Mathematics GATE ENGINEERING MATHEMATICS

SAMPLE STUDY MATERIAL Postal Correspondence Course GATE Engineering Mathematics GATE ENGINEERING MATHEMATICS ENGINEERING MATHEMATICS GATE Syllabus CIVIL ENGINEERING CE CHEMICAL ENGINEERING CH MECHANICAL

### Matrices and Determinants

Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or

### CHAPTER 6. Direct Methods for Solving Linear Systems

CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to

### Real symmetric matrices/1. 1 Eigenvalues and eigenvectors

Real symmetric matrices 1 Eigenvalues and eigenvectors We use the convention that vectors are row vectors and matrices act on the right. Let A be a square matrix with entries in a field F; suppose that

### FINITE-DIMENSIONAL LINEAR ALGEBRA

DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup

### Lecture 3: Review of Linear Algebra

ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

### NONCOMMUTATIVE POLYNOMIAL EQUATIONS. Edward S. Letzter. Introduction

NONCOMMUTATIVE POLYNOMIAL EQUATIONS Edward S Letzter Introduction My aim in these notes is twofold: First, to briefly review some linear algebra Second, to provide you with some new tools and techniques

### 9.1 Eigenanalysis I Eigenanalysis II Advanced Topics in Linear Algebra Kepler s laws

Chapter 9 Eigenanalysis Contents 9. Eigenanalysis I.................. 49 9.2 Eigenanalysis II................. 5 9.3 Advanced Topics in Linear Algebra..... 522 9.4 Kepler s laws................... 537

### MS-A000* Matrix algebra ( Ville Turunen, Aalto University)

x a c b Figure 1: Geometry of Pythagoras equation a 2 + b 2 = c 2, with x = a b MS-A000* Matrix algebra (21112016 Ville Turunen, Aalto University) Welcome to the lecture course on linear algebra! We shall

### Linear Algebra. Carleton DeTar February 27, 2017

Linear Algebra Carleton DeTar detar@physics.utah.edu February 27, 2017 This document provides some background for various course topics in linear algebra: solving linear systems, determinants, and finding

### An Introduction To Linear Algebra. Kuttler

An Introduction To Linear Algebra Kuttler April, 7 Contents Introduction 7 F n 9 Outcomes 9 Algebra in F n Systems Of Equations Outcomes Systems Of Equations, Geometric Interpretations Systems Of Equations,

### Linear Algebra 2 Final Exam, December 7, 2015 SOLUTIONS. a + 2b = x a + 3b = y. This solves to a = 3x 2y, b = y x. Thus

Linear Algebra 2 Final Exam, December 7, 2015 SOLUTIONS 1. (5.5 points) Let T : R 2 R 4 be a linear mapping satisfying T (1, 1) = ( 1, 0, 2, 3), T (2, 3) = (2, 3, 0, 0). Determine T (x, y) for (x, y) R

### TOPIC III LINEAR ALGEBRA

[1] Linear Equations TOPIC III LINEAR ALGEBRA (1) Case of Two Endogenous Variables 1) Linear vs. Nonlinear Equations Linear equation: ax + by = c, where a, b and c are constants. 2 Nonlinear equation:

### 1 Invariant subspaces

MATH 2040 Linear Algebra II Lecture Notes by Martin Li Lecture 8 Eigenvalues, eigenvectors and invariant subspaces 1 In previous lectures we have studied linear maps T : V W from a vector space V to another

### Image Registration Lecture 2: Vectors and Matrices

Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this

### Lecture 3: QR-Factorization

Lecture 3: QR-Factorization This lecture introduces the Gram Schmidt orthonormalization process and the associated QR-factorization of matrices It also outlines some applications of this factorization

### Linear Algebra Problems

Math 504 505 Linear Algebra Problems Jerry L. Kazdan Topics 1 Basics 2 Linear Equations 3 Linear Maps 4 Rank One Matrices 5 Algebra of Matrices 6 Eigenvalues and Eigenvectors 7 Inner Products and Quadratic

### Algorithms to Compute Bases and the Rank of a Matrix

Algorithms to Compute Bases and the Rank of a Matrix Subspaces associated to a matrix Suppose that A is an m n matrix The row space of A is the subspace of R n spanned by the rows of A The column space

### Tangent spaces, normals and extrema

Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent

### SPRING OF 2008 D. DETERMINANTS

18024 SPRING OF 2008 D DETERMINANTS In many applications of linear algebra to calculus and geometry, the concept of a determinant plays an important role This chapter studies the basic properties of determinants

### Boolean Inner-Product Spaces and Boolean Matrices

Boolean Inner-Product Spaces and Boolean Matrices Stan Gudder Department of Mathematics, University of Denver, Denver CO 80208 Frédéric Latrémolière Department of Mathematics, University of Denver, Denver

### LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY

LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT 204 - FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO 1 Adjoint of a linear operator Note: In these notes, V will denote a n-dimensional euclidean vector

### Eigenvalues and Eigenvectors 7.2 Diagonalization

Eigenvalues and Eigenvectors 7.2 Diagonalization November 8 Goals Suppose A is square matrix of order n. Provide necessary and sufficient condition when there is an invertible matrix P such that P 1 AP

### Department of Biostatistics Department of Stat. and OR. Refresher course, Summer Linear Algebra. Instructor: Meilei Jiang (UNC at Chapel Hill)

Department of Biostatistics Department of Stat. and OR Refresher course, Summer 216 Linear Algebra Original Author: Oleg Mayba (UC Berkeley, 26) Modified By: Eric Lock (UNC, 21 & 211) Gen Li (UNC, 212)

### w T 1 w T 2. w T n 0 if i j 1 if i = j

Lyapunov Operator Let A F n n be given, and define a linear operator L A : C n n C n n as L A (X) := A X + XA Suppose A is diagonalizable (what follows can be generalized even if this is not possible -

### Review of Linear Algebra

Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources

### I = i 0,

Special Types of Matrices Certain matrices, such as the identity matrix 0 0 0 0 0 0 I = 0 0 0, 0 0 0 have a special shape, which endows the matrix with helpful properties The identity matrix is an example

### 18.02 Multivariable Calculus Fall 2007

MIT OpenCourseWare http://ocw.mit.edu 18.02 Multivariable Calculus Fall 2007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. M. Matrices and Linear Algebra

### Algebra I Fall 2007

MIT OpenCourseWare http://ocw.mit.edu 18.701 Algebra I Fall 007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 18.701 007 Geometry of the Special Unitary

### Lecture Notes: Eigenvalues and Eigenvectors. 1 Definitions. 2 Finding All Eigenvalues

Lecture Notes: Eigenvalues and Eigenvectors Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk 1 Definitions Let A be an n n matrix. If there

### Appendix C Vector and matrix algebra

Appendix C Vector and matrix algebra Concepts Scalars Vectors, rows and columns, matrices Adding and subtracting vectors and matrices Multiplying them by scalars Products of vectors and matrices, scalar

### Matrix Operations: Determinant

Matrix Operations: Determinant Determinants Determinants are only applicable for square matrices. Determinant of the square matrix A is denoted as: det(a) or A Recall that the absolute value of the determinant

### MAT 242 CHAPTER 4: SUBSPACES OF R n

MAT 242 CHAPTER 4: SUBSPACES OF R n JOHN QUIGG 1. Subspaces Recall that R n is the set of n 1 matrices, also called vectors, and satisfies the following properties: x + y = y + x x + (y + z) = (x + y)

### Math 121 Homework 5: Notes on Selected Problems

Math 121 Homework 5: Notes on Selected Problems 12.1.2. Let M be a module over the integral domain R. (a) Assume that M has rank n and that x 1,..., x n is any maximal set of linearly independent elements

### Definition (T -invariant subspace) Example. Example

Eigenvalues, Eigenvectors, Similarity, and Diagonalization We now turn our attention to linear transformations of the form T : V V. To better understand the effect of T on the vector space V, we begin

### Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 1 / 14 Introduction We define eigenvalues and eigenvectors. We discuss how to

### (K + L)(c x) = K(c x) + L(c x) (def of K + L) = K( x) + K( y) + L( x) + L( y) (K, L are linear) = (K L)( x) + (K L)( y).

Exercise 71 We have L( x) = x 1 L( v 1 ) + x 2 L( v 2 ) + + x n L( v n ) n = x i (a 1i w 1 + a 2i w 2 + + a mi w m ) i=1 ( n ) ( n ) ( n ) = x i a 1i w 1 + x i a 2i w 2 + + x i a mi w m i=1 Therefore y

### v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

### Math 396. An application of Gram-Schmidt to prove connectedness

Math 396. An application of Gram-Schmidt to prove connectedness 1. Motivation and background Let V be an n-dimensional vector space over R, and define GL(V ) to be the set of invertible linear maps V V

### EIGENVALUES AND EIGENVECTORS

EIGENVALUES AND EIGENVECTORS Diagonalizable linear transformations and matrices Recall, a matrix, D, is diagonal if it is square and the only non-zero entries are on the diagonal This is equivalent to

### MATH JORDAN FORM

MATH 53 JORDAN FORM Let A,, A k be square matrices of size n,, n k, respectively with entries in a field F We define the matrix A A k of size n = n + + n k as the block matrix A 0 0 0 0 A 0 0 0 0 A k It

### Problem Set # 1 Solution, 18.06

Problem Set # 1 Solution, 1.06 For grading: Each problem worths 10 points, and there is points of extra credit in problem. The total maximum is 100. 1. (10pts) In Lecture 1, Prof. Strang drew the cone

### 18.06 Problem Set 2 Solution

18.06 Problem Set 2 Solution Total: 100 points Section 2.5. Problem 24: Use Gauss-Jordan elimination on [U I] to find the upper triangular U 1 : 1 a b 1 0 UU 1 = I 0 1 c x 1 x 2 x 3 = 0 1 0. 0 0 1 0 0

MTH50 Spring 07 HW Assignment 7 {From [FIS0]}: Sec 44 #4a h 6; Sec 5 #ad ac 4ae 4 7 The due date for this assignment is 04/05/7 Sec 44 #4a h Evaluate the erminant of the following matrices by any legitimate

### Math 425 Lecture 1: Vectors in R 3, R n

Math 425 Lecture 1: Vectors in R 3, R n Motiating Questions, Problems 1. Find the coordinates of a regular tetrahedron with center at the origin and sides of length 1. 2. What is the angle between the

### 4. Linear Subspaces Addition and scaling

71 4 Linear Subspaces There are many subsets of R n which mimic R n For example, a plane L passing through the origin in R 3 actually mimics R 2 in many ways First, L contains zero vector O as R 2 does

### GRE Math Subject Test #5 Solutions.

GRE Math Subject Test #5 Solutions. 1. E (Calculus) Apply L Hôpital s Rule two times: cos(3x) 1 3 sin(3x) 9 cos(3x) lim x 0 = lim x 2 x 0 = lim 2x x 0 = 9. 2 2 2. C (Geometry) Note that a line segment

### Eigenvalues, Eigenvectors, and Diagonalization

Math 240 TA: Shuyi Weng Winter 207 February 23, 207 Eigenvalues, Eigenvectors, and Diagonalization The concepts of eigenvalues, eigenvectors, and diagonalization are best studied with examples. We will

### 3 QR factorization revisited

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 2, 2000 30 3 QR factorization revisited Now we can explain why A = QR factorization is much better when using it to solve Ax = b than the A = LU factorization

### Jordan Normal Form. Chapter Minimal Polynomials

Chapter 8 Jordan Normal Form 81 Minimal Polynomials Recall p A (x) =det(xi A) is called the characteristic polynomial of the matrix A Theorem 811 Let A M n Then there exists a unique monic polynomial q

### Elementary Linear Algebra Review for Exam 3 Exam is Friday, December 11th from 1:15-3:15

Elementary Linear Algebra Review for Exam 3 Exam is Friday, December th from :5-3:5 The exam will cover sections: 6., 6.2, 7. 7.4, and the class notes on dynamical systems. You absolutely must be able

36 CHAPER 4 Optimization of Quadratic Functions In this chapter we study the problem (4) minimize x R n x Hx + g x + β, where H R n n is symmetric, g R n, and β R. It has already been observed that we

### Linear Algebra: Graduate Level Problems and Solutions. Igor Yanovsky

Linear Algebra: Graduate Level Problems and Solutions Igor Yanovsky Linear Algebra Igor Yanovsky, 5 Disclaimer: This handbook is intended to assist graduate students with qualifying examination preparation.

### Agenda: Understand the action of A by seeing how it acts on eigenvectors.

Eigenvalues and Eigenvectors If Av=λv with v nonzero, then λ is called an eigenvalue of A and v is called an eigenvector of A corresponding to eigenvalue λ. Agenda: Understand the action of A by seeing

### L3: Review of linear algebra and MATLAB

L3: Review of linear algebra and MATLAB Vector and matrix notation Vectors Matrices Vector spaces Linear transformations Eigenvalues and eigenvectors MATLAB primer CSCE 666 Pattern Analysis Ricardo Gutierrez-Osuna

### THE EULER CHARACTERISTIC OF A LIE GROUP

THE EULER CHARACTERISTIC OF A LIE GROUP JAY TAYLOR 1 Examples of Lie Groups The following is adapted from [2] We begin with the basic definition and some core examples Definition A Lie group is a smooth

### 1. Elements of linear algebra

Elements of linear algebra Contents Solving systems of linear equations 2 Diagonal form of a square matrix 3 The Jordan normal form of a square matrix 4 The Gram-Schmidt orthogonalization process 5 The

### Generalized Eigenvectors and Jordan Form

Generalized Eigenvectors and Jordan Form We have seen that an n n matrix A is diagonalizable precisely when the dimensions of its eigenspaces sum to n. So if A is not diagonalizable, there is at least

### Math Lecture 26 : The Properties of Determinants

Math 2270 - Lecture 26 : The Properties of Determinants Dylan Zwick Fall 202 The lecture covers section 5. from the textbook. The determinant of a square matrix is a number that tells you quite a bit about

### Fall Inverse of a matrix. Institute: UC San Diego. Authors: Alexander Knop

Fall 2017 Inverse of a matrix Authors: Alexander Knop Institute: UC San Diego Row-Column Rule If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding

### 3 - Vector Spaces Definition vector space linear space u, v,

3 - Vector Spaces Vectors in R and R 3 are essentially matrices. They can be vieed either as column vectors (matrices of size and 3, respectively) or ro vectors ( and 3 matrices). The addition and scalar

### CHAPTER 6. Representations of compact groups

CHAPTER 6 Representations of compact groups Throughout this chapter, denotes a compact group. 6.1. Examples of compact groups A standard theorem in elementary analysis says that a subset of C m (m a positive

### Problem Set 1. Homeworks will graded based on content and clarity. Please show your work clearly for full credit.

CSE 151: Introduction to Machine Learning Winter 2017 Problem Set 1 Instructor: Kamalika Chaudhuri Due on: Jan 28 Instructions This is a 40 point homework Homeworks will graded based on content and clarity