Section 1.7. Linear Independence

Section 1.7 Linear Independence

Motiation Sometimes the span of a set of ectors is smaller than you expect from the number of ectors. Span{, w} w Span{u,, w} w u This means you don t need so many ectors to express the same set of ectors. Notice in each case that one ector in the set is already in the span of the others so it doesn t make the span bigger. Today we will formalize this idea in the concept of linear (in)dependence.

Definition A set of ectors { 1, 2,..., p} in R n is linearly independent if the ector equation x 1 1 + x 2 2 + + x p p = 0 has only the triial solution x 1 = x 2 = = x p = 0. The opposite: The set { 1, 2,..., p} is linearly dependent if there exist numbers x 1, x 2,..., x p, not all equal to zero, such that x 1 1 + x 2 2 + + x p p = 0. This is called a linear dependence relation. Like span, linear (in)dependence is another one of those big ocabulary words that you absolutely need to learn. Much of the rest of the course will be built on these concepts, and you need to know exactly what they mean in order to be able to answer questions on quizzes and exams (and sole real-world problems later on).

Definition A set of ectors { 1, 2,..., p} in R n is linearly independent if the ector equation x 1 1 + x 2 2 + + x p p = 0 has only the triial solution x 1 = x 2 = = x p = 0. The set { 1, 2,..., p} is linearly dependent otherwise. The notion of linear (in)dependence applies to a collection of ectors, not to a single ector, or to one ector in the presence of some others.

Checking Linear Independence Question: 1 1 3 Is 1, 1, 1 linearly independent? 1 2 4 Equialently, does the (homogeneous) the ector equation 1 1 3 0 x 1 + y 1 + z 1 = 0 1 2 4 0 hae a nontriial solution? How do we sole this kind of ector equation? 1 1 3 1 1 1 row reduce 1 0 2 0 1 1 1 2 4 0 0 0 So x = 2z and y = z. So the ectors are linearly dependent, and an equation of linear dependence is (taking z = 1) 1 1 3 0 2 1 1 + 1 = 0. 1 2 4 0

Checking Linear Independence Question: 1 1 3 Is 1, 1, 1 linearly independent? 0 2 4 Equialently, does the (homogeneous) the ector equation 1 1 3 0 x 1 + y 1 + z 1 = 0 0 2 4 0 hae a nontriial solution? 1 1 3 1 1 1 row reduce 1 0 0 0 1 0 0 2 4 0 0 1 x 0 The triial solution y = 0 is the unique solution. So the ectors are z 0 linearly independent.

and Matrix Columns By definition, { 1, 2,..., p} is linearly independent if and only if the ector equation x 1 1 + x 2 2 + + x p p = 0 has only the triial solution. This holds if and only if the matrix equation Ax = 0 has only the triial solution, where A is the matrix with columns 1, 2,..., p: A = 1 2 p. This is true if and only if the matrix A has a piot in each column. Important The ectors 1, 2,..., p are linearly independent if and only if the matrix with columns 1, 2,..., p has a piot in each column. Soling the matrix equation Ax = 0 will either erify that the columns 1, 2,..., p of A are linearly independent, or will produce a linear dependence relation.

Linear Dependence Criterion If one of the ectors { 1, 2,..., p} is a linear combination of the other ones: 3 = 2 1 1 2 + 64 2 Then the ectors are linearly dependent: 2 1 1 2 3 + 64 = 0. 2 Conersely, if the ectors are linearly dependent 2 1 1 2 + 64 = 0, 2 then one ector is a linear combination of the other ones: 2 = 4 1 + 12 4. Theorem A set of ectors { 1, 2,..., p} is linearly dependent if and only if one of the ectors is in the span of the other ones.

Pictures in R 2 In this picture One ector {}: Linearly independent if 0. Span{}

Pictures in R 2 Span{w} w In this picture One ector {}: Linearly independent if 0. Two ectors {, w}: Linearly independent: neither is in the span of the other. Span{}

Pictures in R 2 Span{w} Span{} w u Span{, w} In this picture One ector {}: Linearly independent if 0. Two ectors {, w}: Linearly independent: neither is in the span of the other. Three ectors {, w, u}: Linearly dependent: u is in Span{, w}. Also is in Span{u, w} and w is in Span{u, }.

Pictures in R 2 w Two collinear ectors {, w}: Linearly dependent: w is in Span{} (and ice-ersa). Two ectors are linearly dependent if and only if they are collinear. Span{}

Pictures in R 2 Two collinear ectors {, w}: Linearly dependent: w is in Span{} (and ice-ersa). Two ectors are linearly dependent if and only if they are collinear. Span{} w u Three ectors {, w, u}: Linearly dependent: w is in Span{} (and ice-ersa). If a set of ectors is linearly dependent, then so is any larger set of ectors!

Pictures in R 3 Span{, w} In this picture Span{w} u w Two ectors {, w}: Linearly independent: neither is in the span of the other. Three ectors {, w, u}: Linearly independent: no one is in the span of the other two. Span{}

Pictures in R 3 In this picture Span{w} w Two ectors {, w}: Linearly independent: neither is in the span of the other. Three ectors {, w, x}: Linearly dependent: x is in Span{, w}. Span{}

Which subsets are linearly dependent? Think about Are there four ectors u,, w, x in R 3 which are linearly dependent, but such that u is not a linear combination of, w, x? If so, draw a picture; if not, gie an argument. Yes: actually the pictures on the preious slides proide such an example. Linear dependence of { 1,..., p} means some i is a linear combination of the others, not any.

Linear Dependence Stronger criterion Suppose a set of ectors { 1, 2,..., p} is linearly dependent. Take the largest j such that j is in the span of the others. For example, j = 3 and Is j is in the span of 1, 2,..., j 1? 3 = 2 1 1 2 + 64 2 Rearrange: (2 1 12 2 3 ) 4 = 1 6 so 4 is also in the span of 1, 2, 3, but 3 was supposed to be the last one that was in the span of the others. Better Theorem A set of ectors { 1, 2,..., p} is linearly dependent if and only if there is some j such that j is in Span{ 1, 2,..., j 1 }.

Increasing span criterion If the ector j is not in Span{ 1, 2,..., j 1 }, it means Span{ 1, 2,..., j } is bigger than Span{ 1, 2,..., j 1 }. If true for all j A set of ectors is linearly independent if and only if, eery time you add another ector to the set, the span gets bigger. Theorem A set of ectors { 1, 2,..., p} is linearly independent if and only if, for eery j, the span of 1, 2,..., j is strictly larger than the span of 1, 2,..., j 1.

Increasing span criterion: pictures Theorem A set of ectors { 1, 2,..., p} is linearly independent if and only if, for eery j, the span of 1, 2,..., j is strictly larger than the span of 1, 2,..., j 1. Span{, w} One ector {}: Linearly independent: span got bigger (than {0}). w Two ectors {, w}: Linearly independent: span got bigger. Span{}

Increasing span criterion: pictures Theorem A set of ectors { 1, 2,..., p} is linearly independent if and only if, for eery j, the span of 1, 2,..., j is strictly larger than the span of 1, 2,..., j 1. Span{, w, u} Span{, w} One ector {}: Linearly independent: span got bigger (than {0}). Span{} u w Two ectors {, w}: Linearly independent: span got bigger. Three ectors {, w, u}: Linearly independent: span got bigger.

Increasing span criterion: pictures Theorem A set of ectors { 1, 2,..., p} is linearly independent if and only if, for eery j, the span of 1, 2,..., j is strictly larger than the span of 1, 2,..., j 1. Span{, w, x} One ector {}: Linearly independent: span got bigger (than {0}). Span{} x w Two ectors {, w}: Linearly independent: span got bigger. Three ectors {, w, x}: Linearly dependent: span didn t get bigger.

Extra: Linear Independence Two more facts Fact 1: Say 1, 2,..., n are in R m. If n > m then { 1, 2,..., n} is linearly dependent: the matrix A = 1 2 n. cannot hae a piot in each column (it is too wide). This says you can t hae 4 linearly independent ectors in R 3, for instance. A wide matrix can t hae linearly independent columns. Fact 2: If one of 1, 2,..., n is zero, then { 1, 2,..., n} is linearly dependent. For instance, if 1 = 0, then is a linear dependence relation. 1 1 + 0 2 + 0 3 + + 0 n = 0 A set containing the zero ector is linearly dependent.