Chapter 5 Linear Equations: Basic Theory and Practice

Transcription

1 Chapte 5 inea Equations: Basic Theoy and actice In this chapte and the next, we ae inteested in the linea algebaic equation AX = b, (5-1) whee A is an m n matix, X is an n 1 vecto to be solved fo, and b is an m 1 vecto. Relative to the mateial usually found in most engineeing mathematics books, ou pesentation is faily compehensive. In addition, we discuss functionality in atab that can be applied to this poblem. Row-Echelon Fom (Hemite omal Fom) With espect to given bases on vecto spaces U and V, suppose m n matix A epesents the linea tansfomation T : U V. As discussed in Chapte 3, a change of basis on space V can be effected by using an m m, nonsingula matix to elate the "oiginal" and "new" bases, and this poduces a new matix A = -1 A that epesents T with espect to the "oiginal" basis on U and the "new" basis on V. The logical question to ask hee is: How simple can we make A by changing only the basis on V? An answe to this question is given by the following theoem. Theoem 5.1 Given any m n matix A of ank ρ, thee exists a nonsingula, m m matix such that A -1 A has the fom descibed by the thee attibutes given below. 1) Thee is at least one non-zeo element in each of the fist ρ ows of A, and the elements in all emaining ows (afte ρ) ae zeo. 2) The fist non-zeo element appeaing in ow i (i ρ) is a 1 appeaing in column κ i, whee κ 1 < κ 2 <... < κ ρ. 3) In column κ i, 1 i ρ, the only non-zeo element is the 1 in ow i. The matix A is uniquely detemined by A. CH5.DC age 5-1

2 atix A descibed by 1) though 3) is said to be in ow-echelon fom (also known as Hemite omal Fom). It is impotant to ealize that pefoming the poduct -1 A is tantamount to pefoming ow opeations on A, ou next topic. Elementay Row peations and Elementay atices We define thee types of elementay ow opeations on an m n matix A. Type I: ultiply a ow by a non-zeo scala. Type II: Add a multiple of one ow to anothe ow. Type III: Intechange two ows. The poduct A = -1 A, in ow echelon fom, can be obtained by pefoming (usually multiple) elementay ow opeations on A. Each of the elementay ow opeations can be accomplished by multiplying m n matix A on the left by an elementay matix obtained by pefoming the elementay ow opeation on an m m identity matix. Fo example, the second ow of A can be multiplied by the scala c if matix A is multiplied on the left by the m m elementay matix c (5-2) And, if to the matix A, we add to the fist ow the constant k times the 3d ow we get the same esult as if we wee to multiply A on the left by the elementay matix 1 0 k (5-3) CH5.DC age 5-2

3 These elementay matices ae non-singula. Also, thei inveses ae elementay matices. As stated ealie, given an m n matix A thee exists a nonsingula m m matix such that A = -1 A is in ow-echelon fom. A can be geneated by applying elementay ow opeations to A. Apply these same ow opeation to the identity matix to obtain -1. Example A = , [A I ] = ultiply the 1st ow by 1/4; add to 2nd ow -5 times 1st ow; add to 3d ow -2 time 1st ow to obtain 1 3 / 4 1/ 2 1/ 4 1/ / 4 1/ 2 1/ 4 5 / / / 2 1/ ultiply 2nd ow by 4; add to 1st ow -3/4 times 2nd ow; add to 3d ow 7/2 times 2nd ow to obtain ultiply 3d ow by 1/5; add to 2nd ow -2 times 3d ow; add to 1st ow 1 times 3d ow to obtain CH5.DC age 5-3

4 / 5 2 / 5 1/ 5 1/ / 5 11/ 5 8 / 5 2 / / 5 18 / 5 14 / 5 1/ 5 We ae done! The ow-echelon fom and the tansfomation matix ae / 5 A = / / 5-1, = 2 / 5 1/ 5 1/ 5 11/ 5 8 / 5 2 / 5 18 / 5 14 / 5 1/ 5. ets check ou wok: -1 A = A? 2 / 5 1/ 5 1/ 5 11/ 5 8 / 5 2 / 5 18 / 5 14 / 5 1/ = / / / 5 YES! If matix A is nonsingula, the ow echelon fom is the identity matix, and -1 is the invese of A. This method of finding A -1 is one of the easiest available fo hand computation, and it is the ecommended technique. atab s RREF Function atab can educe a matix to ow echelon fom. The syntax is R = RREF(A), (5-4) whee A is a use-supplied m n matix. atab etuns the ow echelon fom in the matix R. This function can do moe than just the ow echelon fom of a matix. Consult the atab documentation fo infomation on the capabilities of (and full syntax fo) the RREF function. CH5.DC age 5-4

5 inea Algebaic Equations et T:U V be a linea tansfomation. et b V be any given vecto. The poblem of finding all X U such that T(X ) = b is a poblem of geat pactical impotance. f couse, once bases have been established fo U and V, we can ecast the linea algebaic poblem in tems of matices and coodinate vectos. Suppose m n matix A epesents tansfomation T with espect to some given set of bases. Then, the linea algebaic poblem consists of finding all coodinate vectos X such that AX = b, whee b is a coodinate vecto epesenting (with espect to a basis) some vecto in V. The meaning of X and b has to be taken fom context. If we wite T(X ) = b, we mean that X U and b V ae actual vectos. If we wite AX = b, we mean that X and b ae only coodinates that epesent eal vectos with espect to the undelying bases (and, with espect to the bases, matix A epesents tansfomation T). In tems of the tansfomation T, if b is not in the ange of T, then a solution to T(X ) = b does not exist, and the equation is said to be inconsistent. If b is in the ange of T, then at least one solution exists, and the equation is said to be consistent. Suppose X 1 and X 2 ae solutions of T(X ) = b. Then T(X 1 - X 2) = 0, so the diffeence X 1 - X 2 of two solutions is in the kenel of T (we wite X 1 - X 2 K(T)). Convesely, let Z K(T), and suppose that X 0 is any paticula solution of T(X ) = b. Then we have T(X 0 + Z ) = T(X 0) + T(Z ) = b + 0 = b, so X 0 + Z is also a solution. Since K(T) is a subspace of U, all solutions of T(X ) = b fom a tanslation of the subspace K(T). That is, the solution set fo equation T(X ) = b is a linea vaiety. f couse, these ideas have countepats when dealing with matices and coodinate vectos. Given a coodinate vecto X, the poduct AX is nothing moe than a linea combination of the columns of matix A. Hence, fo a solution of AX = b to exist, it is necessay and sufficient CH5.DC age 5-5

6 fo b to be epesented as a linea combination of the columns of A; that is b Span(A 1, A 2,..., A n), whee A 1, A 2,..., A n ae the n columns of the m n matix A (we wite A = A 1 A n ). ikewise, let X 1 and X 2 satisfy AX = b ; then A(X 1 - X 2) = 0, so that X 1 - X 2 K(A). Convesely, let X 0 be any paticula solution of AX = b ; fo all Z K(A) (i.e., AZ = 0 ), we have X 0 + Z a solution of AX = b. Hence, the solution set is a tanslation of K(A). Example A = , b = 1 3 bseve that X 0 = [2 1] T is a paticula solution of AX = b, and K(A) = α[1 1] T, whee α R. Hence, the set of all solutions of AX = b is = + α α 1+ α. ke(a) x 2 -axis solution set X 0 x 1 -axis et A be an m n matix. The system AX = b is said to be ovedetemined if m > n. ften, ovedetemined algebaic systems occu when we fomulate too many constaints when modeling a physical system. Sometimes, due to measuement eos, this may lead to an algebaic system that has no solution. The system is said to be undedetemined if m < n. When modeling a physical system, an undedetemined equation may esult if we have incomplete knowledge about the inteelationship between system vaiables. Solution Existence: A ecessay and Sufficient Condition A necessay and sufficient condition fo AX = b to have a solution can be stated in tems of the anks of matices. CH5.DC age 5-6

7 Theoem 5.2 Given A = a11 a12 a1n a21 a22 a2n b1 b b = 2,, (5-5) am1 am2 amn bm define the augmented system A b = a11 a12 a1n b1 a21 a22 a2n b2 am1 am2 amn bm,, (5-6) The system AX = b has a solution if and only if Rank A b = Rank ( A ). (5-7) When a solution(s) exist, the collection of solutions can be expessed in tems of n -, whee = Rank(A), independent paametes. oof: As discussed peviously, a solution exists if and only if b can be expessed as a linea combination of the columns of A. Hence, using b as an additional column cannot incease ank, and (5-7) must hold. ow, nullity(a) = dim(k(a)) = n -, so thee ae linealy independent Z 1, Z 2,..., Z n- such that AZ k= 0, 1 k n-. The complete solution of AX = b can be expessed as n X = X0 + α k Z k, (5-8) k= 1 CH5.DC age 5-7

8 whee α k, 1 k n-, ae independent paametes that can be assigned abitay values, and X 0 is any paticula solution. The complete solution can be found by using elementay opeations to educe [A b ] to ow echelon fom. Example A = b = 4, A b, = The ow echelon fom is A b = Hence, an equivalent set of equations is x1 + x4 = 1 x2 3x4 = 2 x3 + 2x4 = 3 U R V S W T x1 = 1 x4 x2 = 2 + 3x4, x3 = 3 2x4 wee x 4 can be thought of as an independent paamete that can be assigned an abitay value. This solution set can be expessed as CH5.DC age 5-8

9 x1 1 1 x2 2 3 x3 3 2 = + α, α R. x4 0 1 Hee, the vecto [ ] T is a paticula solution and [ ] T is in the one-dimensional kenel of A. The method illustated by the last example leads to an inconsistent esult when the system of equations has no solution. Example A = , 1 b = 2 1 atab gives ank(a) = 2 and ank[a b ] =3, which implies that the system is inconsistent. The augmented matix is A b = , atab's ef function yields ef{ A b } = The last ow confims ou initial analysis: the system is inconsistent since 0 1. CH5.DC age 5-9

10 As we have agued, the system AX = b has a solution if and only if b R(A) = {span of columns of A}. But, the whole coodinate space (the set of all coodinate vectos used to epesent vectos in V) fo V can be expessed as R(A) R(A). Hence, the system AX = b has a solution if and only if b is othogonal to all vectos in R(A) = {span of columns of A} (Equivalently, we can state that AX = b has a solution if and only if b is othogonal to any basis that spans R(A) ). A basis that spans R(A) can be obtained by consideing the adjoint of A. But fist, we must conside the concept of a linea functional. inea Functional et U denote a vecto space ove a scala field F. A mapping φ : U F (5-9) is temed a linea functional if φ( αx 1 + βx 2) = αφ(x 1) + βφ(x2 ) (5-10) fo all X 1, X 2 U and evey α, β F. That is, a linea functional is a linea mapping fom U into the scala field F. Example Fo abitay but fixed X U, we can define the functional φ X : U F as φ X ( X1) = X1, X, X1 U. (5-11) Clealy, as defined by (5-11), φ X is linea: CH5.DC age 5-10

11 φ ( αx1 + βx2) = αx1 + βx2, X = α X1, X + β X2, X X = αφx( X1) + βφx( X2) (5-12) fo all X 1, X 2 in U and α, β in F. This last example shows how elementay linea functionals can be. Howeve, accoding to Theoem 5.3, all linea functionals mapping U F must be epesentable in the fom given by (5-11). Theoem 5.3 et φ : U F be a linea functional on n-dimensional vecto space U. Fo evey such φ, thee exists a unique X 0 U (the vecto X 0 depends on the paticula φ) fo which φ( X) = X, X0 (5-13) fo all X U. That is, each linea functional can be epesented as an inne poduct with a functional-dependent vecto X 0. oof: Fist we show the existence of an X 0 with the stated popety. et α 1, α 2,..., α n be an othonomal basis of space U. Define X0 = φ( α1) α1 + φ( α2) α2 + + φ( αn ) αn (5-14) ( φ denotes the complex conjugate of φ). Then fo each i, 1 i n, we have αi, X0 = αi, φ( α1) α1 + φ( α2) α2 + + φ( αn ) αn = φ( αi). (5-15) CH5.DC age 5-11

12 That is, φ(x ) agees with X X, 0 on a basis of space U. ow, any vecto can be epesented in tems of basis α 1, α 2,..., α n. Hence, the equality φ( X) = X, X0 must hold on the whole space U, and we have poved the existence of an X 0 with the desied popety. ext, we show that, given φ, the vecto X 0 is unique. We do this by aiving at a contadiction afte assuming that thee ae two vectos with the stated popety. Suppose that X 0 and X 1 exist such that φ( X) = X, X0 = X, X1 (5-16) fo all X U. This would equie that X, X0 X1 = 0 fo all X U, and this means that X 0 and X 1 must be the same vecto. Hence, uniqueness of X 0 has been established, and this completes the poof of Theoem 5.3 Adjoint peato This subsection discusses the adjoint opeato, an opeato that plays a majo ole in linea analysis, both in finite and infinite-dimensional spaces. Theoem 5.4 et T : U V be a linea opeato. Then thee exists a unique linea opeato T* : V U such that T( X), Y = X, T ( Y) (5-17) CH5.DC age 5-12

13 fo evey X U and Y V. peato T* is called the adjoint of T. oof: et Y V be abitay but fixed; we geneate a T* that maps Y U T T* V back to U such that (5-17) holds. Fo all X U, we define φ( X) T( X ), Y, (5-18) a linea functional that maps U into F. Howeve, by Theoem 5.3 thee exists a unique vecto X Y U with the popety that φ( X) T( X ), Y = X, X Y, (5-19) fo all X U. Define T* : V U by T*(Y ) = X Y. Then we have T( X), Y = X, T ( Y) as claimed. ext, we show that T* is linea. Fo any Y 1 V and Y 2 V, and scalas α, β in F, we have X, T ( αy1 + βy2 ) = T( X), αy1 + βy2 = α T( X), Y1 + β T( X), Y2 = α X, T ( Y1 ) + β X, T ( Y2 ) F HG emembe:, is conjugate linea in its second enty! I KJ (5-20) = X, T ( αy1 ) + T ( βy2 ) CH5.DC age 5-13

14 fo all X U. Hence, we have T ( αy1 + βy2 ) = αt ( Y1 ) + βt ( Y2 ), (5-21) and we conclude that T* is linea. atix Repesentation of Adjoint With espect to given bases on spaces U and V, an m n matix A can be found that epesents the linea tansfomation T : U V. ikewise, with espect to these same bases, a matix epesentation fo the adjoint tansfomation T* : V U can be found. The question begs to be asked: is thee a simple elationship between the matices that epesent T and T*? As shown by the following theoem, the answe is a qualified yes. Theoem 5.5 et T : U V be a linea opeato, and let T* : V U be its adjoint. With espect to an othonomal basis α 1,..., α n fo U and an othonomal basis β 1,..., β m fo V, let m n matix A epesent T. Then the conjugate tanspose A* is the matix epesenting T* with espect to the bases α 1,..., α n fo U and β 1,..., β m fo V (intechange the ows and columns, and take the complex conjugate of the elements, of A to obtain A*). ote that A* is an n m matix. oof: With espect to the given othonomal bases, let n m matix B epesent T*. We show that B = A*. otationally, let {a ij } = A and {b ij } = B. Then by Theoem 3.2, we have aij = T( j i bij = α ), β T ( β j), α i. (5-22) Howeve, bij = j i = T ( β ), α αi, T ( βj) = T( αi), β j = aji, (5-23) CH5.DC age 5-14

15 whee ove bas denote complex conjugate. ote fom (5-23) that B = A* as claimed. Waning: Depending on what book you look in, thee ae two "adjoints", and the fist "adjoint" has absolutely nothing to do with the second "adjoint"! Fist, thee is an "adjoint" just as we have defined it in this chapte. Second, in the liteatue, the tanspose of the cofacto matix is called the "adjoint" (we do not discuss this "adjoint" hee). Bewae! Do not confuse one usage with the othe! A Useful Dichotomy of Vecto Spaces U and V Both the domain U and co-domain V can be patitioned in tems of the ange and kenel of T and T*. The next thee theoems establish this patition; the esults ae given in tems of the ange and kenel of matices A and A * used to epesent T and T *, espectively. As always R(A) and K(A) denote the ange and kenel, espectively, of matix A. Theoem 5.6 The subspace K(A * ) is othogonal to the subspace R(A). oof: et Y K(A*). Then A*Y = 0. Also, AX, Y = X, A Y = X, 0 (5-24) = 0 fo all X U. But R(A) = [AX : X U] so AX R(A). By (5-24), we conclude that K(A*) is othogonal to R(A) as claimed. Theoem 5.7 R(A) is the othogonal complement of K(A*). oof: As shown by Theoem 5.6, K(A*) is othogonal to R(A) so K(A*) R(A). et denote CH5.DC age 5-15

16 Vecto Space U Vecto Space V R(A*) = K(A) R(A) = K(A*) A : U V A* : V U K(A) = R(A*) K(A*) = R(A) A A * the ank of matix A. Then = ank[a] = ank[a*] since the numbe of linea independent columns is equal to the numbe of independent ows. Howeve, ank[a*] + nullity[a*] = m. So nullity[a*] = m -. So dim{k(a*)} = dim{r(a) }; this coupled with K(A*) R(A) leads to the conclusion that K(A*) = R(A), and K(A*) is the othogonal complement of R(A) as claimed. Theoem 5.8 R(A*) is the othogonal complement of K(A). oof: Apply Theoem 5.7 to A* and ealize that A** = A. 0 U can be split into R(A*) and K(A), pieces that ae othogonal to each othe. V can be split into R(A) and K(A*), pieces that ae othogonal to each othe. Figue 5.1: R(A*) and K(A) ae othogonal complements. R(A) and K(A*) ae othogonal complements. Theoems 5.7 and 5.8 state useful patitions of spaces U and V. R(A*) is the othogonal CH5.DC age 5-16

17 complement of K(A), and U = R(A*) K(A). Also, R(A) is the othogonal complement of K(A*), and V = R(A) K(A*). These vey impotant patitions ae illustated by Figue 5-1. Example A = , A = A : U = R 3 V = R 4 while A* : V = R 4 U = R 3. Rank[A] = 2, ullity[a] = 1, Rank[A] + ullity[a] = dim[ U ] = 3 Rank[A*]=2, ullity[a*] = 2, Rank[A*] + ullity[a*] = dim[v] = 4 R(A) = span{[ ] T, [ ] T }, K(A) = span{[1 1-1] T } R(A*) = span{[0 1 1] T, [1 0 1] T }, K(A*) = span{[ ] T, [ ] T } R(A) Κ(A*) and R(A) Κ(A*) = V = R 4 R(A*) K(A) and R(A*) Κ(A) = U = R 3. Theoem 5.2 povides necessay and sufficient conditions fo the existence of a solution(s) to the linea algebaic equation AX = b. By using Theoems 5.7 and 5.8, an altenative necessay and sufficient condition can be given fo the existence question. It is called Fedholm Altenative, and it plays a vey impotant ole in the theoy of finite dimensional space (like we study in this couse). oe geneal vesions of the Fedholm Altenative apply in infinite dimensional spaces whee they ae used to establish the existence of solution(s) to linea functional equations. Theoem 5.9 (Fedholm Altenative) The linea algebaic equation AX = b has a solution if and only if b K(A*). oof: Fom Theoem 5.2, we know that AX = b has a solution if and only if b R(A). This theoem follows fom the ealization that R(A) = K(A*). CH5.DC age 5-17