General (3A) /9/2
Copyright (c) 22 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". Please send corrections (or suggestions) to youngwlim@hotmail.com. This document was produced by using OpenOffice and Octave. /9/2
V: non-empty set of objects defined operations: addition scalar multiplication u + v k u if the following axioms are satisfied for all object u, v, w and all scalar k, m V: vector space objects in V: vectors. if u and v are objects in V, then u + v is in V 2. u + v = v + u 3. u + (v + w) = (u + v) + w 4. + u = u + = u (zero vector) 5. u + ( u) = ( u) + (u) = 6. if k is any scalar and u is objects in V, then ku is in V 7. k(u + v) = ku + kv 8. (k + m)u = ku + mu 9. k(mu) = (km)u. (u) = u 3 /9/2
Test for a. Identify the set V of objects 2. Identify the addition and scalar multiplication on V 3. Verify u + v is in V and ku is in V closure under addition and scalar multiplication 4. Confirm other axioms.. if u and v are objects in V, then u + v is in V 2. u + v = v + u 3. u + (v + w) = (u + v) + w 4. + u = u + = u (zero vector) 5. u + ( u) = ( u) + (u) = 6. if k is any scalar and u is objects in V, then ku is in V 7. k(u + v) = ku + kv 8. (k + m)u = ku + mu 9. k(mu) = (km)u. (u) = u 4 /9/2
Subspace a subset W of a vector space V If the subset W is itself a vector space the subset W is a subspace of V. if u and v are objects in W, then u + v is in W 2. u + v = v + u 3. u + (v + w) = (u + v) + w 4. + u = u + = u (zero vector) 5. u + ( u) = ( u) + (u) = 6. if k is any scalar and u is objects in W, then ku is in W 7. k(u + v) = ku + kv 8. (k + m)u = ku + mu 9. k(mu) = (km)u. (u) = u 5 /9/2
Subspace Example () In vector space R 2 any one vector (linearly indep.) spans R line through any two non-collinear vectors (linearly indep.) spans R 2 plane any three or more vectors (linearly dep.) spans R 2 plane Subspaces of R 2 {} v 3 = k v + k 2 v 2 R R 2 R 2 v 2 v 2 v 3 v v v v 6 /9/2
Subspace Example (2) In vector space R 2 any one vector (linearly indep.) spans line through R u v u v u+v u u 2u v v 2 v u = (,) v = (,2) u+v = (,3) vector space 7 /9/2
Subspace Example (3) In vector space R 3 any one vector (linearly indep.) spans R line through any two non-collinear vectors (linearly indep.) spans R 2 plane through any three vectors (linearly indep.) spans R 3 3-dim space non-collinear, non-coplanar any four or more vectors (linearly dep.) spans R 3 3-dim space Subspaces of R 2 {} R R 2 R 3 line through plane through 3-dim space 8 /9/2
Row & Column Spaces a a 2 a n ROW Space subspace of R n A = a 2 a 2 2 a 2n = span{r, r 2,, r m } a m a m2 a mn COLUMN Space subspace of = span{c, c 2,, c n } R m c c 2 c n c i R m r = a a 2 a n a a 2 a n r 2 = r m = a 2 a m a 22 a m2 a 2n a mn a 2 a m a 2 2 a m2 a 2 n a mn m r i R n n 9 /9/2
Row Space a a 2 a n ROW Space subspace of R n A = a 2 a 2 2 a 2n = span{r, r 2,, r m } a m a m2 a mn r i R n k r + k 2 r 2 + + k m r m r = a a 2 a n = k a a 2 a n r 2 = a 2 a 22 a 2n + k 2 a 2 a 22 a 2n r m = a m a m2 a mn + k m a m a m2 a mn n /9/2
Column Spaces a a 2 a n COLUMN Space subspace of R m A = a 2 a 2 2 a 2n = span{c, c 2,, c n } a m a m2 a mn c c 2 c n c i R m k c + k 2 c 2 + + k n c n a a 2 a n a a 2 a n m a 2 a 22 a 2 n a 2 a 22 = k + k 2 + k n a 2n a m a m2 a mn a m a m2 a mn /9/2
Null Space n m a a 2 a m a 2 a 22 a m2 a n a 2 n a mn x x n n = m NULL Space subspace of solution space R n a x + a 2 + a n x n a a 2 a n = a 2 x + a m x + a 2 2 + a m2 + a 2 n x n a mn x n a 2 a 2 2 = x + + x n a m a m2 a 2 n a mn A x = x c + c 2 + + x n c n = A x = x c + c 2 + + x n c n = b A x = A x = b 2 /9/2
Null Space n a a 2 a n x m a 2 a 22 a 2 n n = m a m a m2 a mn x n NULL Space solution space subspace of A x = R n Invertible A x = A = only trivial solution {} Non-invertible A A zero row(s) in a RREF free variables parameters s, t, u, one one a line through the origin two two a plane through the origin three three a 3-dim space through the origin R R 2 R 3 3 /9/2
Solution Space of Ax=b () 3 - -5 4 2-4 2 x + + x 3 = = Solve for a leading variable x + 3 x 3 = 4 x 3 = 2 x = 3 x 3 = 2 + 4 x 3 x 5 + x 3 = 4 x = 4 + 5 x 3 Treat a free variable as a parameter x 3 x = t = 3t = s x 3 = t x = 4 + 5 s t = 2 + 4 t = s x 3 = t x 3 = t 4 /9/2
Solution Space of Ax=b (2) x = 3t x = 4 + 5 s t = 2 + 4 t = s free variable x 3 = t free variable x 3 = t free variable [ x x 3] 2 x = [ 2 ] [ 3 + t 4 ] [ x x 3] 2 x = [ 4 ] [ 5 ] [ + s + t ] x 3 x 3 x 3 x 3 x x x x infinitely many solutions infinitely many solutions 5 /9/2
Solution Space of Ax=b (3) 3 - -5 4 2-4 2 x = 3t x = 4 + 5 s t = 2 + 4 t = s x 3 = t x 3 = t General Solution of A x = b [ x x 3] 2 x = [ 2 Particular Solution of A x = b ] [ 3 + t 4 ] General Solution of A x = [ x x 3] 2 x = [ 4 Particular Solution of A x = b ] [ 5 ] [ + s + t A x = ] General Solution of 6 /9/2
Linear System & Inner Product () Linear Equations Corresponding Homogeneous Equation a x + a 2 + + a n x n = a = (a, a 2,, a n ) x = (x,,, x n ) normal vector a x = b a x = each solution vector x of a homogeneous equation orthogonal to the coefficient vector a Homogeneous Linear System a x + a 2 + + a n x n = a 2 x + a 22 + + a 2 n x n = a m x + a m2 + + a mn x n = r x = r 2 x = r m x = 7 /9/2
Linear System & Inner Product (2) Homogeneous Linear System a x + a 2 + + a n x n = a 2 x + a 22 + + a 2 n x n = a m x + a m2 + + a mn x n = r x = r 2 x = r m x = each solution vector x of a homogeneous equation orthogonal to the row vector r i of the coefficient matrix Homogeneous Linear System A x = A : m n solution set consists of all vectors in R n that are orthogonal to every row vector of A 8 /9/2
Linear System & Inner Product (3) Non-Homogeneous Linear System Homogeneous Linear System A x = b A x = A : m n a particular solution A x = b solution set consists of all vectors in R n that are orthogonal to every row vector of + A a particular solution x A x = b R 3 z R 3 z x = (x, y, z) n = (a,b,c) y x = (x, y, z ) n = (a,b,c) y x = (x, y, z) x x 9 /9/2
Linear System & Inner Product (4) 3 - -5 4 2-4 2 2 r x = r 2 x = r x = 3 3 a line through the origin 2 R R 2 a plane through the origin x = 3t x = 4 + 5 s t = 2 + 4 t = s x 3 = t x 3 = t 2 /9/2
Consistent Linear System Ax=b a a 2 a n x a x + a 2 + a n x n a 2 a 22 a 2 n = a 2 x + a 2 2 + a 2 n x n a m a m2 a mn x n a m x + a m2 + a mn x n A x = b consistent a a 2 a n x c + c 2 + + x n c n = b expressed in linear combination of column vectors a 2 a 22 = x + + x n a m a m2 a 2 n a mn b is in the column space of A A x = x c + c 2 + + x n c n = b 2 /9/2
Dimension In a finite-dimensional vector space R n R all bases the same number of vectors n many bases but the same number of basis vectors basis {u, u 2 } R 2 basis {v, v 2 } R 2 basis {w, w 2 } R 2 u 2 (x u, y u ) v 2 (x v, y v ) w 2 (x w, y w ) v w u The dimension of a finite-dimensional vector space V dim(v) the number of vectors in a basis 22 /9/2
Dimension of a Basis () In vector space R 2 any one vector (linearly indep.) spans R 2 line through basis any two non-collinear vectors (linearly indep.) spans R 2 plane any three or more vectors (linearly indep.) spans R 2 plane In vector space R 3 any one vector (linearly indep.) spans R 3 line through any two non-collinear vectors (linearly indep.) spans R 3 plane through basis any three vectors non-collinear, non-coplanar (linearly indep.) spans R 3 3-dim space any four or more vectors (linearly indep.) spans R 3 3-dim space 23 /9/2
Dimension of a Basis (2) In vector space R n any n- vectors (linearly indep.)? spans R n line through basis n vectors of a basis (linearly indep.) spans R n plane any n+ vectors (linearly indep.)? spans R n plane a finite-dimensional vector space V a basis {v, v 2,, v n } a set of more than n vectors (linearly indep.) a set of less than n vectors spans V S = {v, v 2,, v n } non-empty finite set of vectors in V S is a basis S linearly independent S spans V 24 /9/2
Basis Test S = {v, v 2,, v n } S is a basis non-empty finite set of vectors in S linearly independent S spans V V V an n-dimensional vector space S = {v, v 2,, v n } a set of n vectors in V S linearly independent S is a basis S spans V S is a basis 25 /9/2
Plus / Minus Theorem S a nonempty set of vectors in a vector space V S v : linear independent a vector in V but outside of span(s) S {v } : linear independent v, u i S linear combination v = k u + k 2 u 2 + + k n u n span(s) = span(s {v}) v S {v } S {v } S {v } v v span(s) span(s) span(s) 26 /9/2
Finding a Basis S a nonempty set of vectors in a vector space V S v : linear independent a vector in V but outside of span(s) S {v } : linear independent if S is a linearly independent set that is not already a basis for V, then S can be enlarged to a basis for V by inserting appropriate vectors into S v, u i S linear combination v = k u + k 2 u 2 + + k n u n span(s) = span(s {v}) if S spans V but is not a basis for V, then S can be reduced to a basis for V by removing appropriate vectors from S 27 /9/2
Vectors in a S a nonempty set of vectors in a vector space V if S is a linearly independent set that is not already a basis for V, then S can be enlarged to a basis for V by inserting appropriate vectors into S Every linearly independent set in a subspace is either a basis for that subspace or can be extended to a basis for it if S spans V but is not a basis for V, then S can be reduced to a basis for V by removing appropriate vectors from S Every spanning set for a subspace is either a basis for that subspace or has a basis as a subset 28 /9/2
Dimension of a Subspace W a subspace of a finite-dimensional vector space W is finite-dimensional dim(w) dim(v) V W = V dim(w) = dim(v) 29 /9/2
Rank and Nullity a a 2 a n ROW Space subspace of R n A = a 2 a 2 2 a 2n = span{r, r 2,, r m } a m a m2 a mn COLUMN Space subspace of = span{c, c 2,, c n } R m NULL Space subspace of solution space A x = Invertible A x = A = only trivial solution R n Non-invertible A zero row(s) in a RREF free variables parameters s, t, u, dim(row space of A) = dim(column space of A) = rank(a) dim(null space of A) = nullity(a) 3 /9/2
Solution Space of Ax= 3-5 2-4 the same case x = 3t = 2 + 4 t x = 4 + 5 s t = s x 3 = t x 3 = t General Solution of A x = [ x x 3] 2 x = [ ] [ 3 + t 4 ] [ x x 3] 2 x = [ ] [ 5 ] [ + s + t ] dim(row space of A) dim(col space of A) rank(a) = 2 dim(null space of A) nullity(a) = 3 rank(a) = nullity(a) = 2 /9/2
Elementary Row Operation () ROW Space subspace of R n NULL Space subspace of R n = span{r, r 2,, r m } solution space A x = COLUMN Space subspace of R m free variables parameters s, t, u, = span{c, c 2,, c n } Elementary row operations do not change the null space of a matrix Elementary row operations do not change the row space of a matrix Elementary row operations do change the col space of a matrix Elementary row operations do not change the linear dependence and linear independence relationship among column vectors 32 /9/2
Elementary Row Operation (2) Elementary row operations do not change the null space of a matrix Elementary row operations do not change the row space of a matrix Elementary row operations do not change the linear dependence and linear independence relationship among column vectors Elementary row operations do change the col space of a matrix A R c c + + + + 33 /9/2
Elementary Row Operation (3) Elementary row operations do not change the null space of a matrix do not change the row space of a matrix do not change the linear dependence and linear independence relationship among column vectors do change the col space of a matrix A row space of A = row space of R null space of A = null space of R col space of A col space of R R a given set of column vectors linearly independent linearly independent the corresponding set of column vectors basis basis 34 /9/2
Bases of Row & Column Spaces () basis of row space of A = basis of row space of R basis of col space of A basis of col space of R the corresponding set of column vectors a given set of column vectors dim(row space of A) = dim(column space of A) = rank(a) 35 /9/2
Bases of Row & Column Spaces (2) basis of col space of A basis of col space of R the basis consisting of columns of A basis of col space of A basis of col space of R the basis consisting of rows of A 36 /9/2
Rank and Nullity n n (< m) m (< n) m # of leading variables # of leading variables n (=m) rank(a) < min(m, n) m (=n) # of zero rows = # of free var's n = rank(a) + nullity(a) m = (# of leading variables) + (# of zero rows) n = (# of leading variables) + (# of free variables) 37 /9/2
Overdetermined System n (< m) m = n column vectors can span at most R n b is in R m R m R n # of leading variables At least one vector b in R m does not lie in column space For such b in R m Ab = b inconsistent A x = b 38 /9/2
References [] http://en.wikipedia.org/ [2] Anton, et al., Elementary Linear Algebra, th ed, Wiley, 2 [3] Anton, et al., Contemporary Linear Algebra, /9/2