Liear Algebra Vectors ad Matrices Fudametal Operatios with Vectors Vector: a directed lie segmets that has both magitude ad directio =,,,..., =,,,..., = where 1, 2,, are the otatio: [ ] 1 2 3 1 2 3 compoets of the ector. if A(x 1, x 2, x ) is the iitial poit of the ector ad B(y 1, y 2,, y ) is the termial poit the the compoets ca be foud usig AB= y1 x1, y2 x2,..., y x, graphically, a ector is draw as a arrow such that the legth of the arro is the magitude of the ector ad the arrow poits i the directio of the ector A ector represets a sigle etity of magitude ad directio, oly oe ector is uique. The zero ector (0 ): the uique ector that has o magitude ad o directio, eery 0= 0,0,0,...,0. compoet is zero: [ ] The magitude of a ector : gie some ector with itial poit A(x 1, x 2, x ) ad termial poit B(y 1, y 2,, y ), the magitude of deoted is the distace from A to B: AB = = y x + y x + + y x = + + + ( ) ( ) ( ) 2 2 2 2 2 2 1 1 2 2... 1 2... The uit ector: ay ector u is a uit ector if its magitude is 1. The stadard uit ectors: ay ector e is a stadard uit ector for R is eery compoet of e is a zero except the th compoet, which is a 1. 2-D 3-D -D i = [1,0] i = [1,0,0] e 1 = [1, 0,0,0..,0] j = [0,1] j = [0,1,0] e 2 = [0,1,0,0..,0] k = [0,0,1] e 3 = [0,0,1,0..,0] e = [0, 0,0,0..,1] Operatio of Vectors: Let, w, ad z be ectors i R ad λ be a scalar (costat). The we defie: the sum of ad w as the ector + w= [ 1, 2,..., ] + [ w1, w2,..., w] = [ 1 + w1, 2+ w2,..., + w] λ= λ,,..., = λ, λ,..., λ the scalar product of λ ad is the ector [ ] [ ] the differece of ad w is the ector w= [ 1, 2,..., ] [ w1, w2,..., w] = [ 1 w1, 2 w2,..., w] =,,..., =,,...,. 1 2 3 1 2 3 the opposite of is the ector [ ] [ ] 1 2 1 2
2 w w + w - w ½ Note ad λ are always parallel (but may be i opposite directios if λ is egatie). From this we see a method to determie if two ectors are parallel: two ectors ad w are parallel if they are scalar multiples of oe aother. The Stadard Uit Vectors: Eery ector ca be writte as a liear combiatio of the stadard uit ectors. Let be a ector i R such that. Let e 1, e 2,..., e be the stadard uit ectors. The we see: = [ 1, 2,..., ] = [ 1,0,0,...,0] + [ 0, 2,0,...,0] + [ 0,0,..., ] = = 1,0,0,...,0 + 0,1,0,...,0 +... + 0,0,...,1 = e + e +... + e [ ] [ ] [ ] 1 2 1 1 2 2 Suppose we wat to fid a uit ector u i the same directio as a gie ector. How do we do it? This ca be doe by usig the followig formula u=. This works by the followig 1 Sice = 1 1, u = = = = 1 so sice the magitude of u is 1 ad is i the directio of the u is i the directio of. Ex: Gie = [ 1, 2,3] w = [ 4,1,1] z = [ 2, 3, 4] 1. Fid w ad z 2. 4 3. 3w 2+ 5z 4. The uit ector is the same directio as w ad aother i the same directio as.
Properties of Vector Operatios: Let = [ 1, 2,..., ], w= [ w1, w2,..., w], ad z = [ z1, z2,..., z] be ectors i R. Also let λ ad k be scalars ad 0 be the zero ector i R. The + w= w+ commutatie property for additio ( + w) + z = + ( w+ z) associatie property for additio + 0= 0+ = additie idetity property, which is zero + ( ) = ( ) + = 0 additie ierse property which is λ( + w) = λ+ λw ad ( λ+ k) = λ+ k distributie laws for scalars ( λk) = λ( k) = k( λ) associatie property for scalar multiplicatio 1 = 1 = multiplicatie idetity property for scalar multiplicatio, which is 1 ( ) ( ) Theorem: Let R ad λ R. I f λ = 0, the either λ = 0 or = 0 Liear Combiatio of Vectors: Let 1, 2,.. k be ectors i R the the ector is a liear combiatio of ectros 1, 2,, k if ad oly if there exists scalars λ 1, λ 2,, λ 3 such that = λ 1 1, λ 2 2,, λ k k. The Dot Product: Let ad w be ectors i R. The the dot product of ad w is the scalar deoted, w w=,,..., w, w,..., w = w + w +... + w gie by: [ ] [ ] 1 2 1 2 1 1 2 2 the dot product produces a scalar, ot a ector the dot product is also kow as the ier product there are may useful properties that come about from the dot product Properties of the Dot Product: Let, w, ad z be ectors i R ad λ R be a scalar. The the followig are true: w = w (commutatie law for dot product) 2 = 0 = 0 iff = 0 λ( w ) = ( λ ) w = ( λw ) ( w+ z) = ( w) + ( z) + w z= z + w z ( ) ( ) ( ) b. w c. z b Ex: Let = [ 3, 1, ] w= [ 5,2] z = [ 2, 1,3,5] b = [ 3, 2,1,4] a. w d. b e. z z, fid the followig if possible. f. z Iequalities Iolig the Dot Product Cauchy-Schwartz Iequality: Let ad w be ectors i R. The w w The Triagle Iequality: Let ad w be ector i R. The + w < + w (both proofs ca be foud i the textbook)
The Agle betwee Two Vectors: Let ad w be ectors i R such that that hae a commo iitial poit. Let β be the agle betwee the ectors, w β w w the cosβ = or β = arccos w w, also w= wcosβ two ectors ad w are orthogoal (perpedicular) if ad oly if w = 0 0 β π whe β is 0 or π (cosβ = ±1), ad w are parallel Theorem: Let ad w be ectors i R. The ad w are parallel if ad oly if w=± w Ex: Fid the agle betwee 3, 1,2 ad 1, 1, 2 ( β = π ) Projectio Vectors: Let ad w be ectors i R that hae the same iitial poit. Let l be a lie cotaiig w. acute obtuse w w l l The projectio of oto w (proj w ) is the ector with same itial poit as ad w ad foud by droppig perpedicular from the termial poit of oto the lie l cotaiig w to create to termial poit of the projectio. Therefore the projectio of oto w is a scalar multiple of w ad ca be foud by: w proj w = w 2 w We ca also decompose the ector usig the projectio ector ad the perpedicular 2 - proj w proj w Notice that proj w is parallel to w ad - proj w is orthogoal to w Ex: Gie = [3, - 2, 1] ad w = [4, 1, -5] fid proj w ad proj w, the fid the ector orthogoal to w
w is the scalar kow as the scalar compoet of i the same directio of w. 2 w Fudametal Operatio of Matrices Matrices: a matrix is a rectagular array of real umbers arraged i m rows ad colums. We say the size or dimesio of the matrix is mx. I geeral if matrix A is mx, the: a11 a12 a13... a1 a21 a22 a23... a 2.... A=........... ai 1 ai2 ai3... ai am 1 am2 am3... a m We use capital letters to deote matrices. We ca thik of a mx matrix A as a collectio of m row ectors of colum ectors. We typically use a ij to represet the real umber i the i th row ad j th colum. M m represets to set of all mx matrices The diagoal etries are all a 11, a 22, a 33 Special Matrices: A square matrix B is a matrix with the same umber of rows ad colums. Therefore the size of B is either mxm or x. A diagoal matrix D is a square matrix such that all the etries ot o the mai diagoal are zeros. We use D to represet the set of all x diagoal matrices. A idetity matrix I is a diagoal matrix such that all the mai diagoal etries are oes. We use I to represet the x idetity matrix. A upper triagular matrix is a square matrix such that all etries below the mai diagoal etries are zeros. U represets all x triagular matrices. A lower triagular matrix is a square matrix such that all etries aboe the mai diagoal are zeros. L represets all x lower triagular matrices. A zero matrix is ay matrix such that all etire are zeros, deoted 0 m. Additio of Matrices Let A ad B be mx matrices The sum of A ad B is the mx matrix whose ij th etry is equal to a ij + b ij. 3 1 4 6 6 1 7 9 Ex: + = 2 5 3 2 8 3 6 4
Let A be a mx matrix ad let λ be a real umber the the mx matrix λa, the scalar multiplicatio of A by λ, is the matrix whose ij th etry is λa ij. 3 2 Ex: C= 1 7 fid 5C 6 9 Properties of Additio ad Scalar Multiplicatio of Matrices Let A, B, ad C be mx matrices ad λ ad k be scalars. The we hae: A +B = B + A commutatie property (A + B) + C = A + (B + C) associatie property A+ 0 m = 0 m + A = A existece of a additie idetity A + (-1)A = -A + A = 0 m existece of a additie ierse k(a +B) = ka + kb distributie laws for scalar multiplicatio ad additio (k + λ)a = ka + λa distributie laws for scalar multiplicatio ad additio (kλ)a = (ka)λ = (λa)k associatiity of scalar multiplicatio 1A = A existece of idetity elemet for scalar multiplicatio The Traspose of a Matirx ad its Properties: Let A be ay mx matrix the the traspose matrix of A is the mx matrix A T such that the ij th etry of A T is the ji th etry from A. (the rows of A are the colums of A T ad ice ersa) Ex: Fid A T 3 1 7 2 4 if A = 2 5 6 9 3 Properties of the Traspose Let A ad B be mx matrices ad k be a scalar the: (A T ) T = A (A + B) T = A T + B T (ka) T = ka T Symmetric ad Skew-Symmetric Matrices A matrix A is a symmetric matrix iff A = A T A matrix B is a skew-symmetric matrix iff B = -B T oly square matrices ca be symmetric or skew symmetric 2 3 1 Ex: A= 3 4 6 is symmetric 1 6 7 Thereom: Eery square matrix A ca be decomposed uiquely as a sum of a symmetric matrix S ad a skew symmetric V. It ca be show that S= ½(A +A T ) ad V = ½ (A A T ).
Matrix Multiplicatio If A is a mx matrix ad B is a xp matrix, the their matrix product AB is the mxp matrix whose ij th etry is the dot product of the i th row of A ad the j th colum of B. The dimesio of AB is the (umber of rows of A) x (umber of colums of B) AB exists oly is the umber colums of A equals the umber of rows of B. AB may exist but BA may ot a11 a12... a 11 12... 1 1 11 11 12 21... b b b p a b + a b + + a1b 1 a21 a22... a b 2 21 b22... b 2 p a21b12 + a22b22 +... + a 2b 2 AB= =........... a 1 2... b 1 b2... bp am 1b1 p am2b2 p... a m am am + + + mbp 3 5 Ex: Let A 1 6 4 2 5 6 = ad B= 3 7 1 0 fid AB if possible 4 2 2 1 Ex: For D 0 5 2 1 = E= 3 0 4 3 Fid DE, ED, FG, GF, ad HD if possible. F = [ 6 3 1] 2 G= 4 5 H 3 0 = 2 1 I the special case where AB = BA we say A ad B commute If A is mx ad I represets ad idetity matrix with the appropriate umber of rows ad colums such that AI exists or I m A exists the the AI = A = I m A. I is called the multiplicatie idetity. Properties of Matrix Products Supoose that A,B, ad C are matirces for which the followig sums ad product are defied. Let λ be a scalar the: (AB)C = A(BC) associatie property A(B+C) = AB +BC distributie property (A+B)C = AC +BC distributie property λ(ab) = (λa)b = A(λB) scalar multiple Let A be a x matrix the the o-egatie powers of A are gie by A 0 = I A ad for k 2 A k = A (k 1) A If A is a square matrix ad s ad t are o-egatie itegers, the A s+t = A s A t ad (A s ) t = A st = (A t ) s If A is mx ad B is xp the the traspose of the matrix product is (AB) T = B T A T
Systems of Liear Equatios Solig Liear Systems usig Gaussia Elimiatio A system of m liear equatios i ariables is a collectio of m equatios each cotaiig a liear combiatio of the same ariables summig to a scalar: a11x1 + a12 x2+... + a1 x = b1 a21x1 + a22x2+... + a2x = b2... am 1x1 + am2x2+... + amx = bm A particular solutio (s 1, s 2,,s ) to a system of m equatios is ariables is a ordered -tuple of alues satisfyig eery equatio whe substituted for x 1, x 2, x. A complete solutio to a system of m equatios i ariables is the set of all ordered - tuples that satisfy eery equatio. The coefficiets of all the m ariables i all the ariables i the m equatios ca be collected i a matrix called a coefficiet matrix. a11 a12... a1 x1 b1 a21 a22... a 2 x A 2 b = we ca also let X 2 = ad B=...... am1 am2... am x bm The aboe otatio ca be writte as AX=B a11 a12... a1 x1 b1 a21 a22... a 2 x 2 b 2 =...... am1 am2... am x bm The augmeted matrix is the matrix A B a11 a12... a1 b1 a21 a22... a2 b2 A B =..... am 1 am2... am bm The Number of Solutios to a System Ay gie set of equatios i s system of liear equatios will hae oe of the followig possibilities for the total umber of solutios: oe solutio o solutio ifiitely may solutios (if a systems has more tha oe solutio it is guarateed to hae ifiitely may solutios)
Ay system that has a solutio is called cosistet ad if there is o solutio its icosistet. Cosider a system of two liear equatios: oe solutio o solutios ifiitely may solutios Gaussia Elimiatio Gaussia Elimiatio is a procedure that we use to sole a system. The procedure ioles multiplyig ad etire equatio by o-zero costats ad addig the equatios to aother to elimiate a ariable (elimiatio by additio). We will use Gaussia elimiatio o the augmeted matrix A B to sole the systems. There are three operatios we ca perform o ay gie row of a matrix so that the system retais to same solutio: 1. multiply a row by a o-zero scalar 2. addig a scalar multiple of oe row to aother row (while subtractig is a form of additio ad ca also be doe betwee rows is ery helpful to right operatios oly usig additio) 3. iterchage the positios of ay two rows It is helpful to otate what you are doig to a gie row usig the preious operatios. If we were plaig o performig a operatio of row 2 by multiplyig row 1 by 3 ad addig it to row 2 we would write that as: R 2 = 3r 2 + r 2. If we wat to iterchage row 1 ad row 2 we would write: R 1 R 2. We begi by selectig a specific row of the matrix called the home row ad a specific etry i that row called the piot. We will coert the piot to a 1 ad use the home row ad the piot to zero out the etries aboe ad below the piot. This process the repeats util a solutio is obtaied. Ex: Sole the system 5x 2y+ 2z= 14 3x+ y z= 8 2x+ 2y z= 3 The methods we are usig are called row reductio. WE are turig the augmeted matrix ito a upper/lower triagular matrix ad the back substitute. If we tur the augmeted matrix ito a diagoal matrix we do ot eed to back substitute.
Sometimes we hae a row or colum of all zeros. I these cases o piot ca be chose ad we moe o to the ext colum. If a row cosists of all zeros we will moe that row to the bottom of the matrix. 1 0 56 0 1 58 the third row haig all zeros to the right of the lie amouts to 0 0 0? 0x+ 0y+ 0 z=? If? = 0 there there are two possibilities: 1. ifiitely may solutios 2. oe solutio, the triial solutios (0,0,.0) If? 0 the there is o soluo to the system. If a colum to the right of the augmetatio bar has all zeros the there are ifiitely may solutios. Ex: Sole the system 3x+ y+ 7z+ 2w= 13 2x 4y+ 14z w= 10 5x+ 11y 7z+ 8w= 59 2x+ 5y 4z 3w= 39 Gauss-Jorda Row Reductios ad Reduced Row Reductio Form A matrix i row-echelo form has the followig properties: Ay rows cosistig etirely of zeros occur at the bottom of the matrix. The etry i row 1, colum 1 is a 1 ad zeros appear below it. The first o-zero etry of each row after the first row is a 1 with zeros below it. The disadatage to this method is we hae to back substitute to sole the system A matrix i reduded row echelo form has the followig properties: Ay rows cosistig etirely of zeros occur at the bottom of the matrix. The etry i row 1, colum 1 is a 1 ad zeros appear below it. The first o-zero etry of each row after the first row is a 1 with zeros below ad aboe it. This method does ot eed back substitutio. Solig a system usig row echelo form is called Gaussia elimiatio ad solig a system usig reduced row echelo form is called Gauss-Jorda elimiatio. Ex: Is the followig i reduced row echelo form? Why or why ot? 1 0 0 0 0 0 0 0 1
Number of solutios to a System of Liear Equatios Let AX = B be a system of liear equatios. Ad let C be the reduced row echelo form augmeted matrix. If a colum to the right of the augmetatio bar has all zeros the there are ifiitely may solutios. If a row has all zeros to the right of the augmetatio bar ad a zero to the right there are ifiitely may solutios If a row has all zeros to the right of the augmetatio bar ad a o-zero etry to the right there is o solutio. If oe of the preious apply there is a uique solutio. A system of liear equatios haig the form AX = 0 is called a homogeeous system. A homogeous system will hae the followig properties: If the reduced row echelo form augmeted matrix for a homogeeous system i ariables has fewer the ozero piot etries, the the system has oly the triial solutio (0,0,,0). Eery homogeeous system has at least the triial solutio. Therefore if the system also has a otriial solutio the it has ifiitely may solutios. I a homogeeous system if there are more ariables the equatios the there are ifiitely may solutios. Ex: Sole 2x1+ 4x2 x3+ 5x4+ 2x5 = 0 3x1 + 3x2 x3+ 3x4 = 0 5x1 6x2 + 2x3 + 6x4 x5 = 0 Equialet Systems, Rak, ad Row Space Equialet Systems ad Row Equialece of Matrices: Two systems of m liear equatios i ariables are equialet iff they hae the same solutio set. Two matrices are row equialet iff oe matrix is obtaied from the other by a fiite umber of row operatios. Each row operatio we perform is reersible. To go backwards is called a reerse or ierse row operatio. If a matrix D is a row equialet to a matrix C, the the matrix C is a row equialet to the matrix D. Let AX = B be a system of liear equatios. If C D system CX = D is equialet to the system AX = B is row equialet to A B the the
Rak of a Matrix: Eery matrix is row equialet to a uique matrix is reduced row echelo form. SO o matter how we reduce a matrix we will always get the same matrix i reduced row echelo form. The rak of a matrix A is the umber of ozero rows i the reduced row echelo from of A, deoted rak(a). Ex: Determie the rak of the followig: 1 2 3 1 3 5 7 8 A= 3 6 9 B= 2 5 3 6 1 7 14 21 8 20 12 24 4 Homogeeous Systems ad Rak: Let AX = 0 be a homogeeous system i ariables: If rak(a) < the system has a otriial solutio ( ifiitely may solutios) If rak(a) = the system has oly the triial solutio Corollary: Let AX = 0 be a homogeeous system o m liear equatios i ariables. If m < the the system has a otriial solutio. Liear Combiatios of Vectors: Recall: a liear combiatio of ectors is a sum of scalar multiples of the ectors x = 3, 1,4,5, y = 2,7, 1,6, ad z = 1,0,0, 3 fid 3x 2y + 5z Ex: Gie [ ] [ ] [ ] Questio: Gie a ector, ca we determie if the ector is a liear combiatio of some other ectors? Aswer: To determie whether is a liear combiatio of x, y, ad z, we must fid C 1, C 2, C 3 such that = C1x + C2 y + C3z. We ca thik of this as a system of liear x1c1 + y1c2 + z1c3 = 1 equatios, x2c1+ y2c2+ z2c3 = 2 ad use a augmeted matrix. x3c1 + y3c2+ z3c3 = 3 If there is a solutio the we ca write as a liear combiatio of x, y, ad z. IF o solutio the we ca t. Ex: Determie whether a = [ 2,2,3] is a liear combiatio of x= 6, 2,3, y= 0, 5, 1, z = 2,1,2 [ ] [ ] [ ] Row Space of a Matrix: Let A be a mx matrix. The subset of R cosistig of all ectors that are liear combiatios of the rows of A is called the row space of A.
Recall that if A is a mx matrix, the each of the rows of A is a ector with etries ( or i other words a ector i R ). We uderstad that the row space is a set of ectors that are all possible liear combiatios of the ectors makig up the rows of the matrix. The zero ector, 0 is guarateed to be i the row space of eery matrix. Each row of a matrix is also guarateed to be i the row space of the matrix. Lemma: Suppose is a liear combiatio of the ectors q 1, q 2,..., q ad suppose that 4 each of the ectors q 1, q 2,..., q is itself a liear combiatio of the ectors 4 r, r,..., r. 1 2 j The is a liear combiatio of r 1, r 2,..., r. j Suppose that A ad B are row equialet matrices, the the row space of A is equal to the row space of B. 3 1 1 Ex: Determie whether = [ 4,0, 3] is i the row space of A= 2 1 5 4 3 3 If is i the row space of A the there exists real umbers C 1, C 2, ad C 3 such that C 3,1,1 + C 2, 1,5 + C 4, 3,3 = [ ] [ ] [ ] 1 2 3 To check whether a ector is i the row space of matrix A we set up the augmeted T matrix: A the row reduce to see if there is exactly oe solutio. Ierse Matrices Let A be a square x matrix, if the ierse of A (deoted A -1 )exists the AA -1 =A -1 A=I ot all square matrices hae ierses (most do!) A x matrix A is sigular iff the ierse of A does ot exist. If the ierse does exists we say the matrix is o-sigular. Suppose the x matrices B ad C are ierses of the x matrix A, the B = C. If a ierse exists the that matrix is uique. Let A be a x o-sigular matrix. The we defie egatie powers of A usig the ierse A -1 of A. I other words for k 2 we hae A -k = (A -1 ) k. If A is a x o-sigular matrix ad s ad t are ay itegers, the we hae A s A t =A s+t ad (A s ) t =(A t ) s =A st Let A ad B be x o-sigular matrices, the: A -1 is o-sigular ad (A -1 ) -1 = A A k is o-sigular ad (A k ) -1 = (A -1 ) k for ay iteger k AB is o-sigular ad (AB) -1 = B -1 A -1 (otice order of matrices) A T is o-sigular ad (A T ) -1 = (A -1 ) T
Ierses of 2x2 matrices: Let δ = ad bc (the determiat) the it is true that: a b d b ad bc ab+ ab c d c a = cd dc cd da = + ad bc 0 δ 0 1 0 = = δ 0 ad bc 0 δ 0 1 If δ 0 we ca diide a 2x2 matrix by δ ad easily fid its ierse so that if a b 1 1 d b A= c d the A = δ c a If δ = 0 the A is sigular. 2 4 Ex: Gie A = 3 1 fid A -1 if possible. 3 4 Ex: Gie B= 6 8 fid B -1 if possible. Ierses of Larger Matrices If A is a x matrix to fid A -1 if it exists we eed to: Set up the augmeted matrix A I Reduce A to its row reduced echelo form If the LHS of the augmeted matrix caot be made ito I, the A is sigular. If the LHS of the augmeted matrix becomes I, the the RHS is A -1. 1 A I I A there is a way to fid the ierse for larger matrices usig determiats, we will discuss this later. 1 5 2 Ex: Fid the ierse if it exists for A= 3 1 2 5 7 6 Let A be a x matrix, the A is o-sigular iff rak(a) = 0 1 2 3 Ex: Fid the ierse if it exists for B= 0 1 4 5 6 0
Solig a System usig the Ierse of a Coefficiet Matrix Suppose AX= B is a system with the same umber of rows as ariables. We kow AX = B has a uique solutio iff A is o-sigular, the: AX = B A -1 (AX) = A -1 B (A -1 A)X = A -1 B I X = A -1 B X = A -1 B 5x+ 3y+ 6z= 4 Ex: Sole the system 3x y 7z= 11 2x + y + 2z = 2