Linear Algebra Alvin Lin August 207 - December 207 Linear Algebra The study of linear algebra is about two basic things. We study vector spaces and structure preserving maps between vector spaces. A vector space is set v with two binary operations, addition and scalar multiplcation. A vector space will satisfy various distributive identities. R 2, R 3,..., R n are all vector spaces. Vectors A vector is a directed line segment corresponding to the displacement from points A to B (in R 2 ). If a vector starts at the origin, we will say that the vector is in standard position. We can represent them as row vectors or column vectors depending on convenience. Each is an ordered pair. v = [ 2 3 ] The zero vector ( 0) is a special vector. v = [ ] 2 3 0 = [ 0 0 ] Let v R 2 : v + 0 = 0 + v = v
Adding Vectors in R 2 Take the following: v = v, v 2 w = w, w 2 v + w = v + w, v 2 + w 2 Example: v =, 2 w = 3, 4 v + w = + 3, 2 + 4 = 4, 6 Scalar Multiplication Let c be a scalar and let v = v, v 2 R 2. c v = c v, v 2 = cv, cv 2 Example: 2 3, 4 = 2(3), 2(4) = 6, 8 If c > 0, then c v points in the same direction as v. If c < 0, then c v points in the opposite direction as v. If c >, then c v is v stretched by a factor of c. If c <, then c v is v compressed by a factor of c. Vectors in R 3 R 3 = {(x, y, z) x, y, z R} Similarly, vectors in R n are: R n = {(x, x 2,..., x n ) all x i R} Let u, v R n, let c R: u + v = u, u 2,..., u n + v, v 2,..., v n + u + v, u 2 + v 2,..., u n + v n c v = c v, v 2,..., v n = cv, cv 2,..., cv n 2
Algebraic Properties of R n Let: u = u, u 2,..., u n R n v = v, v 2,..., v n R n w = w, w 2,..., w n R n Let c, d R (scalars):. Commutative 2. Associative 3. 4. 5. Distributive 6. 7. 8. u + v = v + u ( u + v) + w = u + ( v + w) u + 0 = 0 + u = u u + ( u) = 0 c( u + v) = c u + c v (c + d) u = c u + d u c(d u) = (cd) u u = u Proof of (): u + v = u, u 2,..., u n + v, v 2,..., v n = u + v, u 2 + v 2,..., u n + v n = v + u, v 2 + u 2,..., v n + u n = v, v 2,..., v n + u, u 2,..., u n = v + u 3
Proof of (2): ( u + v) + w = ( u,..., u n + v,..., v n ) + w,..., w n = u + v,..., u n + v n + w,..., w n = (u + v ) + w,..., (u n + v n ) + w n = u,..., u n + ( v + w,..., v n + w n ) = u + ( v,..., v n + w,..., w n ) = u + ( v + w) Proof of (3): u + 0 = u, u 2,..., u n + 0, 0,..., 0 = u + 0, u 2 + 0,..., u n + 0 = u, u 2,..., u n = u Proof of (4): u + ( u) = u, u 2,..., u n + u, u 2,..., u n = u u, u 2 u 2,..., u n u n = 0, 0,..., 0 = 0 Proof of (5): Linear Combinations c( u + v) = c( u, u 2,..., u n + v, v 2,..., v n ) = c( u + v, u 2 + v 2,..., u n + v n ) = c(u + v ), c(u 2 + v 2 ),..., c(u n + v n ) = cu + cv, cu 2 + cv 2,..., cu n + cv n = cu, cu 2,..., cu n + cv, cv 2,..., cv n = c u, u 2,..., u n + c v, v 2,..., v n = c u + c v Let v, v 2,..., v k R n. We say v is a linear combination of v,..., v k if there exists scalars c,..., c k such that: v = k c i v i = c v + c 2 v 2 + + c n v n i= 4
Example in R 3 : Let: 2 v = 2 v = 0 2 5 v 2 = 3 v 3 = 4 0 Claim: v is a linear combination of v, v 2, v 3 : We must find c, c 2, c 3 such that: v = c v + c 2 v 2 + c 3 v 3 Dot Product Let: u u 2 u n u =. v v 2 v =. v n u v = u i v i i= = u v + u 2 v 2 + + u n v n Rules for the Dot Product Let u, v, w R n, c is a scalar:. u v = v u 2. u ( v + w) = u v + u w 5
3. 4. (c u) v = c( u v) u u 0 u u = 0 iff u = 0 Proof for (): u v = = u i v i i= v i u i i= = v u Proof for (2): u ( v + w) = u ( v + w,..., v n + w n ) = u i (v i + w i ) = = i= (u i v i + u i w i ) i= u i v i + i= i= = u v + u w u i w i Proof for (3): (cu i )v i = c(u i v i ) i= = c i= u i v i i= = c( u v) 6
Proof for (4): u u = = u i u i i= (u i ) 2 i= (u ) 2 is non-negative, therefore the summation must be greater than or equal to 0. Norm or Length of a Vector Let: Example In R 2, consider v = v v 2 v N v R n v =. [ ] 3. Calculate v. 4 Properties of a vector norm v = v v = n (v ) 2 i= v = 3 2 + 4 2 = 9 + 6 = 25 = 5 v = 0 v = 0 This is true because v = 0 v v = 0 v = 0. c v = c v 7
Proof: Unit Vectors c v 2 = (c v) (c v) = c 2 ( v v) = c 2 v 2 c v 2 = c 2 v 2 v = c v A unit vector is a vector of length. In R 2 : [ ] e = 0 [ ] 0 e 2 = In R 3 : e = 0 0 0 e 2 = 0 0 e 3 = 0 In R n, there exist the unit vectors e, e 2,..., e n. e i has a in the i t h component and zeros everywhere else. Vector Normalization In R 3 with v R n and v 0, the unit vector corresponding to v is: v norm = v v 8
Example 2 Normalize v = : 3 v = 2 2 + ( ) 2 + ( 3) 2 = 4 + + 9 = 4 v norm = v v = 2 4 3 = 2 4 4 3 4 Cauchy-Schwarz Inequality Let u, v R n : u v u v The Triangle Inequality Let u, v R n : u + v u + v Proof: u + v 2 = ( u + v) ( u + v) = u u + 2 u v + u v = u 2 + 2 u v + v 2 By the Cauchy-Schwarz Inequality u 2 + 2 u v + v 2 = ( u + v ) 2 9
Thus, u + v 2 ( u + v ) 2. Taking the square root: Distance Formula u + v u + v In R, let a, b R. Find the formula for d(a, b): In R 2, let v = [a, b ], and v 2 = [a 2, b 2 ]. d(a, b) = a b = (a b) 2 d( v, v 2 ) = v v 2 = (a a 2 ) 2 (b 2 b 2 ) 2 In R n, let v, w be vectors. Define the distance from v to w as follows: Example d( v, w) = v w = n (v w ) 2 i= 2 u = 0 v = 2 2 d( u, v) = ( 2 0) 2 + ( 2) 2 + ( + 2) 2 = 2 + ( ) 2 + 2 = 4 = 2 0
Angle Between Vectors If we have two vectors v, u and the angle θ between them, we can use the law of cosines to find θ: u v 2 = u 2 + v 2 2 u v cos θ = ( u v) ( u v) = u u 2 u v + v v = u 2 2 u v + v 2 u 2 2 u v + v 2 = u 2 + v 2 2 u v cos θ 2 u v = 2 u v cos θ u v u v = cos θ Let u, v R n, the angle θ between v and u is: Example Find the angle between: cos θ = 2 u = 2 v = u v u v u v = 2() + () + ( 2)() = 2 + 2 = u = 2 2 + 2 + ( 2) 2 = 9 = 3 v = 2 + 2 + 2 = 3 cos θ = 3 3 θ.377rad 78.9
Orthogonal Vectors We say two vectors u and v are orthogonal if there s a right angle between them. cos( π 2 ) = u v u v u v 0 = u v u v = 0 Two vectors are orthogonal if their dot product is zero. Example u =,, 2 v = 3,, 2 The Pythagorean Theorem u v = (3) + () + ( 2)(2) = 3 + 4 = 0 For vectors u, v R n, suppose u and v are orthogonal. Then: Vector Projection u + v 2 = u 2 + v 2 ( u + v) ( u + v) = u u + 2 u v + v v The projection of v onto u described by p is: = u 2 + 2 u v + v 2 = u 2 + 0 + v 2 = u 2 + v 2 v p = proj u v u 2
u v u v proj u v = ( ) u = ( u u u ) u 2 Practice Exercise 48 Given: u = [ ] 2 v = 3 [ ] k + k Find all values of k such that u and v are orthogonal. u v = 0 2(k + ) + 3(k ) = 0 2k + 2 + 3k 3 = 0 5k = 0 k = 5 Practice Exercise 49 Given: k 2 u = v = k 2 3 Find all values k such that u and v are orthogonal. u v = 0 k 2 k + 2( 3) = 0 k 2 k 6 = 0 (k 3)(k + 2) = 0 k = 3 or k = 2 3
Practice Exercise 50 Describe all vectors v = [ ] x that are orthogonal to u = y u v = 0 3x + y = 0 y = 3x y is a line going through the origin with a slope of -3. [ ] 3. Practice Exercise 52 Under what conditions are the following true for vectors u, v R 2 or R 3. u + v = u + v Since: u + v 2 = ( u + v ) 2 ( u + v) ( u + v) = u 2 + 2 u v + v 2 u u + 2 u v + v v = u 2 + 2 u v + v 2 u v = u v cos θ = u v u v θ must be equal to zero since u v = u v. Therefore, this can only be true for u = c v, c > 0. Practice Exercise 55 Verify the stated property of distances: d( u, v) = u v d( u, v) = u v = ( )( v u) = v u = d( v, u) 4
Practice Exercise 56 d( u, w) d( u, v) + d( v, w) d( u, w) = u w = ( u v) + ( v w) u w + v + w = d( u, v) + d( v, w) Practice Exercise 58 Prove: u c v = c( u v) u (c v) = (c v) u = c( v u) = c( u v) You can find all my notes at http://omgimanerd.tech/notes. If you have any questions, comments, or concerns, please contact me at alvin@omgimanerd.tech 5