Lecture Note on Linear Algebra 1 Systems of Linear Equations Wei-Shi Zheng, 2012 1 Why Learning Linear Algebra( 5 ê)? Solving linear equation system is the heart of linear algebra Linear algebra is widely used in computer science and electronic engineering, eg artificial intelligence (<óœu), pattern recognition(ª O), computer vision(ožåàú), data mining(êâ ), machine learning(åìæs) It is fundamental of many other subjects/courses and research approaches 2 What Do You Learn from This Note Basic concept about linear equation( 5 ), system of linear equations( 5 ), matrix(ý) and its solution()) 3 What Is System of Linear Equations? Let us begin with an introduction of solving systems of linear equations We first show some examples of linear equations: x = 0, 2x = 1, 3x = 4(y 9), x 1 11x 2 = 1 3 (x 3 + e) In general, linear equation is defined as follows: Definition 1 (linear equation( 5 )) A linear equation with variables x 1, x 2,, x n is an equation that can be written in the form a 1 x 1 + + a n x n = b The first version of this lecture note was edited by Dr Jialun Huang 1
where b and the coefficients (Xê) a 1, a 2,, a n are real ( ê) or complex (Jê) numbers known in advance Definition 2 (system of linear equations( 5 )) A system of linear equations with variables (Cþ) x 1, x 2,, x n is a collection of finite linear equations with variables x 1, x 2,, x n Examples: 2x 1 3x 2 = 5 x 1 +2x 2 = 8 2x 1 +4x 2 = 0, x1 +4x 2 x 3 = 1 2 5 1 +5x 2 x 3 = 0 The general form of a system of m linear equations with n variables is: a 11 x 1 +a 12 x 2 + +a 1n x n = b 1 a 21 x 1 +a 22 x 2 + +a 2n x n = b 2 (1) a m1 x 1 +a m2 x 2 + +a mn x n = b m 4 Solution of System of Linear Equations Definition 3 (solution())) A list (s 1, s 2,, s n ) of numbers is called a solution of (1) iff (ie if and only if) all the equations in (1) are satisfied by substituting s 1, s 2,, s n for x 1, x 2,, x n The set of all solutions of (1) is called the solution set ()8) of (1) Two systems of linear equations are said to be equivalent (d) if they have the same solution set Does a system of linear equations always have a solution? Let us investigate some examples as follows: Examples: x1 2x 2 = 1 x 1 +3x 2 = 3, x1 2x 2 = 1 x 1 +2x 2 = 3, x1 2x 2 = 1 x 1 +2x 2 = 1 2
From the above examples, we see that not all systems of linear equations have a solution Generally speaking, a system of linear equations has either: 1 no solutions (Ã)), ie the solution set is empty (empty set), or 2 exactly one solution( )), ie the solution set contains only one element (singleton set), or 3 infinitely many solutions(ã õ)), ie the solution set contains infinitely many elements (infinite set) Definition 4 (consistence (ƒn)) A system of linear equations is said to be consistent if its solution set is non empty (ie either one solution or infinitely many solutions), otherwise it is inconsistent 5 Matrix Definition 5 (matrix (Ý)) A table of numbers with m rows (1) and n columns () as above is called an m n matrix we normally use a capital letter such as A, B, X etc to denote a matrix We define the m (n + 1) matrix a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a m1 a m2 a mn b m to be the corresponding augmented matrix (O2Ý) of system (1), where we call the corresponding m rows and n columns: a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn as the coefficient matrix (XêÝ) of system (1) Accordingly, we generate a one to one correspondence ( éa) between systems and matrices 3
Example: 4x 1 +5x 2 +9x 3 = 9 1 2 1 0 0 2 8 8 4 5 9 9 6 Solving a Linear System We are now introducing a procedure for solving systems of linear equations Basic strategy (ÄüÑ) The basic strategy is to replace one system with an equivalent system (ie one with the same solution set) that is easier to solve For example, if a linear system consists of three variables x 1, x 2 and x 3, then use the x 1 term in the first equation of a system to eliminate the x 1 terms in the other equations Then use the x x term in the second equation of a system to eliminate the x 1 terms in the other equations, and so on, until you finally obtain a very simple equivalent system of equations A system can be transformed into another equivalent system such that some variable is eliminated in some equation by applying an elementary operation(ðc ) After applying a series of elementary operations to the original system, the original system is transformed into an easy to solve system The following is a special example: 3x 1 +x 2 +x 3 = 5 2x 2 + x 3 = 1 Furthermore, by applying elementary operations, the above system can be transformed into the trivial system x 1 = 1 x 2 = 1 x 3 = 1 Operations Formally, three types of elementary operations on systems of linear equations are necessary for us to realize the above transformation Let S bs a linear system consists of m equations and n variables Let e i denote the i th equation of S The three types of elementary operations are defined 4
as follows: 1 Interchange(é C ): Perform exchange between the i th equation and the j th equation of S (e i e j ); 2 Scaling( C ): Multiply the i th equation by a nonzero number λ (e i := λe i ); 3 Replacement(\C ): Add the result of multiplying the j th equation by a number λ to the i th equation (e i := e i + λe j ) To solve the linear system S, we shall perform a sequence of elementary operations, resulting in a set of equivalent linear systems op 1 op 2 op S = S 0 S1 l Sl systematically, such that S l is an easy to solve or even a trivial system Example: 4x 1 +5x 2 +9x 3 = 9 x 2 = 16 e 3 :=e 3 +4e 1 e 2 :=e 2 +8e 3 e 1 :=e 1 e 3 x 1 = 29 x 2 = 16 3x 2 +13x 3 = 9 2x 2 = 32 x 1 2x 2 = 3 x 2 = 16 e 2 := 1 2 e 2 e 1 :=e 1 +2e 2 e 3 :=e 3 + 3 2 e 1 Theorem 6 Elementary operations are reversible(œ_) That is, if op S 1 S 2 where S 1, S 2 are systems and op is an elementary operation, then there is another elementary operation op 1 op such that S 1 2 S 1 Proof Exercise Connection to Row Operations In terms of matrix representation for a linear system of equations, the corresponding to elementary operations on 5
linear systems can be directly performed as elementary row operations (1 C ) on matrices as follows: 1 Interchange: Exchange the i th row and the j th row of a matrix (r i r j ); 2 Scaling: Multiply the i th row of a matrix by a nonzero number λ (r i := λr i ); 3 Replacement: Add the result of multiplying the j th row by a number λ to the i th row of a matrix (r i := r i + λr j ) Definition 7 (Row Equivalence (1d)) If matrix A can be transformed into matrix B by applying a series of elementary row operations on A then we say A is row equivalent to B and denote this equivalence by A B Obviously, A B if and only if their corresponding systems are equivalent From now on, we always represent a system of linear equations by its corresponding augmented matrix We shall use the more convenient matrix language in the remaining part of this course Reference David C Lay Linear Algebra and Its Applications (3rd edition) Pages 1 13 6