22A-2 SUMMER 204 LECTURE 2 NATHANIEL GALLUP The Dot Product Continued Matrices Group Work Vectors and Linear Equations Agenda 2 Dot Product Continued Angles between vectors Given two 2-dimensional vectors v = (v, v 2 ) and w = (w, w 2 ), we want to find the angle θ between then Insert Picture The law of cosines tells us that v 2 + w 2 = v w 2 + 2 v w cos θ Let s simplify this First of all v w = (v w, v 2 w 2 ) Then v 2 + w 2 = v w 2 + 2 v w cos θ = v 2 + v 2 2 + w 2 + w 2 2 = (v w ) 2 + (v 2 w 2 ) 2 + 2 v w cos θ = v 2 + v 2 2 + w 2 + w 2 2 = v 2 2v w + w 2 + v 2 2 2v 2 w 2 + w 2 2 + 2 v w cos θ = 0 = 2v w 2v 2 w 2 + 2 v w cos θ = 2v w + 2v 2 w 2 = 2 v w cos θ = v w + v 2 w 2 = v w cos θ Therefore the angle between v and w is arccos ( ) v w + v 2 w 2 v w This tells us that if we want to know the angle between v and w, the quantity v w + v 2 w 2 is important In fact, we will define this to be the dot product Definition The dot product of two vectors v = (v,, v n ) and w = (w,, w n ) is v v n w w 2 w n v 2 v w = def = v w + v 2 w 2 + + v n w n Example The dot product of the vectors (2, 3,, 5) and (, 0,, ) is (2)( ) + (3)(0) + ()() + (5)() = 2 + 0 + + 5 = 4 With the dot product defined, we can define the angle between two vectors:
22A-2 SUMMER 204 LECTURE 2 2 Definition 2 The angle between two vectors v and w is ( ) v w arccos v w We can also express this by saying that if θ is the angle between v and w, then v w cos θ = v w Example 2 What s the angle between the vectors (3, 6) and (2, )? Insert Picture We first calculate their dot product: The dot product of the vectors (3, 6) and (2, ) is Therefore So they re perpendicular! In fact we define: [ [ 3 2 = (3)(2) + (6)( ) = 6 + ( 6) = 0 6] ] ( ) 0 θ = arccos 45 5 = arccos(0) = π/2 Definition 3 Two vectors v and w are perpendicular or orthogonal if v w = 0 Example 3 From above, (3, 6) and (2, ) are orthogonal Length and the dot product We can express the length of a vector in terms of the dot product: Proposition Proof If v = (v, v 2,, v n ), then we can compute: Properties of the dot product v = Proposition 2 For all v, w, u R n and α R, v = v v v 2 + v2 2 + + v2 n = v v + v 2 v 2 + + v n v n = v v () (Commutativity) v w = w v (2) (Distributivity) u ( v + w) = u v + u w (3) (Homogeneity) α( v w) = (α v) w = v (α w) (4) (Positivity) v v 0 and v v = 0 if and only if v = 0 Note I didn t mention Homogeneity or Positivity in class because we won t need them right now If you want another exercise, try proving these! Proof of proposition () (Commutativity) This will be assigned for homework
22A-2 SUMMER 204 LECTURE 2 3 (2) (Distributivity) Let v = (v,, v n ), w = (w,, w n ), and u = (u,, u n ) Now we just write everything out: u u n v v n w w n u ( v + w) = + u v + w = u n v n + w n = u (v + w ) + u 2 (v 2 + w 2 ) + + u n (v n + w n ) = (u v + u w ) + (u 2 v 2 + u 2 w 2 ) + + (u n v n + u n w n ) = (u v + u 2 v 2 + + u n v n ) + (u w + u 2 w 2 + + u n w n ) u u n v v n u u n w = + w n Inequalities involving the dot product Proposition 3 (The Schwartz Inequality) Proof We compute: v w v w v w = v w cos θ = v w cos θ v w where the final step holds because cos θ is between and, so cos θ is between 0 and, and therefore less than or equal to Proposition 4 (The Triangle Inequality) Proof v + w 2 = ( v + w) ( v + w) v + w v + w = v v + w v + v w + w w (Distributivity) = v v + v w + v w + w w (Commutativity) = v v + 2 v w + w w = v 2 + 2 v w + w 2 v 2 + 2 v w + w 2 = ( v + w ) 2 (Schwartz Inequality) Therefore v + w 2 ( v + w ) 2, so taking the square root of both sides, we get that v + w v + w
22A-2 SUMMER 204 LECTURE 2 4 3 Group Work Homework Question (Homework 2, Question ) Compute the following dot products: () 3 5 0 = 4 (2) 3 0 = 2 2 What does this answer mean? (3) If v and w are two n-dimensional vectors with lengths v = 3 and w = π and the angle between them is π/4, then v w = Homework Question 2 In this homework problem we will show that for all n-dimensional vectors v and w, v w = w v () Let v = (v,, v n ) and w = (w,, w n ) (2) Compute v w (3) Compute w v (4) Explain why these are the same! (5) Let s check this with concrete numbers In the previous problem, you found Now compute You should get the same answer as before 3 5 0 4 3 5 0 4 4 Matrices The whole point of matrices is to give us a better way to think about systems of linear equations Let s start with an example, and then we ll move into the general theory from there Example 4 (This example is just to give you the feel of how matrices and linear equations interact We ll define everything formally afterwards) Recall our system of linear equations from Lecture :
22A-2 SUMMER 204 LECTURE 2 5 x + 2 = 4 3x = 8 (Note that I ve changed the variable names from x and y to x and ) We ll now talk about two other equivalent ways of writing this system of equations () The second way to write this system of linear equations is using linear combinations The system above is the same as the following vector equation: [ ] [ ] [ 2 4 x + x 3 2 = 8] because when we multiply out the left-hand side of this bottom equation we get [ ] [ ] 2 x + x 3 2 = [ ] x + 2 3x (2) The third way of writing this system of equations is using matrices: [ ] [ ] [ 2 x 4 = 3 8] Some notes [ on] this particular notation: [ [ 2 2 A = is a matrix Its columns are and 3 3] ] [ ] [ ] 2 x This: denotes the product of a matrix and a vector It is defined as a linear combination 3 of the columns of A with the coefficients x and : [ ] [ ] [ ] [ ] 2 x def 2 = x 3 x + x 2 3 2 [ ] [ x x =, x 4 b = are vectors 2 8] We use the following shorthand to denote our system: A x = b This shorthand notation is at the heart of linear algebra, and we ll see it everywhere from now on Now let s define matrices in general Definition 4 An n m matrix A is an array of real numbers with n rows and m columns: a a 2 a 3 a m a 2 a 22 a 23 A = a 3 a 32 a 33 a n a nm If we want to denote that A has entries a, a 2 etc, then we write A = [a ij ] Note 2 Notice both when we say n m matrix and a ij, the number indexing the row comes first, and the number indexing the column comes second Ie in the first case there are n rows and m columns, and in the second case a ij is the entry in the ith row and the jth column Example 5 Here s a 3 4 matrix: 3 e 4 2 A = 2 3 0 2 9 9 7/3
If A = [a ij ], then a 2 = 2 a 2 = e a 34 = 7/3 Example 6 Here s a 3 3 matrix: It s called a square matrix 22A-2 SUMMER 204 LECTURE 2 6 2 34 32 3 223 0 76 036 Definition 5 A matrix is square if it is an n n matrix, ie if it has the same number of rows as columns Example 7 Here s a particularly important 4 4 matrix: It s called the 4 4 identity matrix 0 0 0 0 0 0 0 0 0 0 0 0 Definition 6 The n n identity matrix is the n n matrix with s on the diagonal, and 0 s elsewhere, such as: 0 0 0 0 0 0 0 0 Now let s define the product of a matrix and a vector: Definition 7 Given an n m matrix A and an m-dimensional vector x: a a 2 a 3 a m a 2 a 22 a 23 A = a 3 a 32 a 33 a n a nm x x m x = Their product is defined to be the m-dimensional vector obtained by taking the linear combination of the columns of A with the coefficients x,,, x m : Important points: a a 2 a 3 a m a 2 a 22 a 23 x A x = a 3 a 32 a 33 x m a n a nm a a 2 a n a 2 a 22 a n2 a 3 a 23 = x + + x 3 + + x m a n3 a m a 2m a nm
22A-2 SUMMER 204 LECTURE 2 7 We can only multiply an n m matrix with an m-dimensional vector (ie the m s have to match) The product of an n m matrix and an m-dimensional vector is an m-dimensional vector I picture taking the vector x, turning it sideways, putting it above the matrix, and then multiplying the columns of A by the corresponding entries in x and adding these vectors up Some examples Example 8 Example 9 3 0 0 2 0 3 0 3 = 2 3 0 + 3 3 + 0 + 0 0 2 9 9 2 9 9 = 2 3 + 3 + 0 + 0 2 0 + 3 3 + + 0 2 2 + 3 9 + 9 + = 9 0 4 [ ] 3 0 0 0 =? 0 3 0 This multiplication is not defined! The matrix is 2 4 and the vector is 3-dimensional Since the 4 does not match the 3, this product is not defined Example 0 If I n is the n n identity matrix, then for any n-dimensional vector v, we have For example, I n v = v 0 0 0 0 3 2 = 3 0 + 2 0 0 0 0 0 0 0 = 3 2 There is an alternate way of thinking about matrix multiplication that will give you the same answer: The product of A and x is the m-dimensional vector whose ith entry is the dot product of the ith row of A with x: a a 2 a 3 a m a 2 a 22 a 23 x A x = a 3 a 32 a 33 x m a n a nm (a, a 2, a 3,, a m ) x (a 2, a 22, a 23,, a 2m ) x = (a 3, a 32, a 33,, a 3m ) x (a n, a n2, a n3,, a nm ) x Let s redo the previous example (the one that worked!) using this way of thinking about matrix multiplication:
22A-2 SUMMER 204 LECTURE 2 8 Example Yay! It worked 3 0 0 2 0 3 0 3 (3,, 0, 0) (2, 3,, ) = (0, 3,, 0) (2, 3,, ) 2 9 9 (2, 9, 9, ) (2, 3,, ) = 2 3 + 3 + 0 + 0 2 0 + 3 3 + + 0 2 2 + 3 9 + 9 + = 9 0 4 Now let s see how to convert a general system of n equations with m variables into matrix form : Definition 8 Given a system of n linear equations with m variables: a x + a 2 + + a m x m = b a 2 x + a 22 + + a 2m x m = b 2 a 3 x + a 32 + + a 3m x m = b 3 a n x + a n2 + + a nm x m = b n (here the a ij s and b i s are real numbers and the x i s are variables), the matrix form of this system is: We often denote this with the shorthand A x = b a a 2 a 3 a m a 2 a 22 a 23 x b a 3 a 32 a 33 = b 2 x m b m a n a nm Let s practice putting systems of equations into matrix form: Example 2 The matrix form of the system of 3 linear equations with 5 variables is: 3x + 4 + 0x 3 + 3x 4 + πx 5 = 3 3 4 0 3 π 0 2 0 0 0 0 4 4 4 x + 2x 3 = 2/3 4x 3 + 4x 4 + 4x 5 = /2 x x 3 x 4 x 5 3 = 2/3 /2