Linear Algebra- Review And Beyond. Lecture 3

Transcription

1 Linear Algebra- Review An Beyon Lecture 3 This lecture gives a wie range of materials relate to matrix. Matrix is the core of linear algebra, an it s useful in many other fiels. 1 Matrix Matrix is the concrete representation for a linear map between finite imensional vector spaces. Suppose we have two vector spaces V, W, an im V = m, im W = n, we can fin basis {α 1, α 2,, α m } of V an {β 1, β 2,, β n } of W. There is a linear map l : V W. For all v V, we have v = m j=1 x jα j, then l(v) = l( m x j α j ) = j=1 m x j l(α j ) So we just nee to consier l(α j ). l(α j ) is a vector in W, thus l(α j ) = n i=1 a ijβ i, an m m n n m l(v) = x j l(α j ) = x j a ij β i = ( a ij x j )β i More specifically, An j=1 j=1 i=1 j=1 v = {α 1, α 2,, α m } x 1 x 2. x m i=1 l(v) = {l(α 1 ), l(α 2 ),, l(α m )} l(α j ) = {β 1, β 2,, β n } l(v) = {β 1, β 2,, β n } a 1j a 2j. a nj j=1 x 1 x 2. x m a 11 a 12 a 1n a 21 a 22 a 2n... a n1 a n2 a nn 1 x 1 x 2. x m

2 Thus, for a linear mapping l : V W, the matrix uner basis {α j } an {β i } is A = (a ij ) where a ij = (l(α j )) βi The rule of aition an multiplication is omitte here. But you must be aware of the fact that the efinition of aition an multiplication is compatible with the meaning of linear mapping. Suppose U, V, W are all vector spaces with certain basis. Verify the following: If l : U V is a linear mapping with matrix A, then λa enotes the linear mapping λl. If l 1 : U V, l 2 : U V are linear mappings with matrices A 1, A 2, then A 2 + A 2 enotes the linear mapping l 1 + l 2. If l 1 : U V, l 2 : V W are linear mappings with matrices A 1, A 2, then A 2 A 1 enotes the linear mapping l 2 l 1. Remark The multiplication is associative because linear mapping is associative. proof is much clearer than the irect calculus. This way of Note that the multiplication of matrices is non-commutative. conition AB = BA is an interesting problem. However, uner what Verify that F m n is a vector space, an fin a basis. A R n n. Prove that if AB = BA for all B R n n, then there is some λ R such that A = λi. Definitions A is symmetric if A = A T. A is anti-symmetric if A = A T. A is Hermitian if A = ĀT. A is anti-hermitian if A = ĀT. A is nilpotent if A k = 0 for some k. A is iempotent if A 2 = A. A is involutory if A 2 = I. 2

3 A, B are square matrices of orer n, A = 2B I, then A is involutory if an only if B is iempotent. See F n n as a vector space Let S be the set of all symmetric matrices, an T be the set of all anti-symmetric matrices. Verify that S, T are subspaces, an fin im S, im T. Prove F n n = S T. Prove the following matrix is nilpotent: Suppose A F m n an rank(a) = 1, then α F m an β F n such that A = αβ T. This is useful ealing with matrices with rank 1. Definition(Invertible) BA = I n. A F m n is invertible if B F n m such that AB = I m an Fact A F m n is invertible if an only if m = n an et A 0. Definition(Ajoint Matrix) the ajoint matrix A = (a ij ) n n, an A ij is the algebraic cofactor of a ij, then A = A 11 A 21 A n1 A 12 A 22 A n2... A 1n A 2n A nn Fact AA = et A I. So if et A 0, A 1 = 1 et A A. Reprove Cramer theorem using x = A 1 β. I A I + A If A R n, an A k = 0 for some k. Fin the inverse of I + A + 1 2! A (k 1)! Ak 1 3

4 Definition(Elementary Transformation) Change two rows(columns) Multiply some row(column) with a non-zero number in F A λ times some row(column) to another row(column) Fin the corresponing matrices of elementary transformations. These matrices are calle elementary matrices. Remark The multiplication of matrix is non-commutative, thus you shoul be careful when oing elementary transformation by elementary matrix. Keep in min that left-multiplication is row transformation an right-multiplication is column transformation. (Schur s Formula) Suppose A F r r, B F r (n r), C F (n r) r an D F (n r) (n r), A is invertible. Prove: ( ) ( ) ( ) I r 0 A B A B CA 1 = I n r C D 0 D CA 1 B ( ) ( ) ( ) A B Ir A 1 B A 0 = C D 0 I n r C D CA 1 B ( ) ( ) ( ) ( ) I r 0 A B Ir A 1 B A 0 CA 1 = I n r C D 0 I n r 0 D CA 1 B A F n m, B F m n. Prove that λ m et(λi n AB) = λ n et(λi m BA). Suppose A F n n an rank(a) = 1. From the exercise above we know α, β F n such that A = αβ T. Prove that et(i A) = 1 β T α. Theorem(Equivalence Canonical Form) For any A F m n, we can fin P F m m, Q F n n such that ( ) Ir 0 P AQ = 0 0 where r = rank(a) an P is the prouct of elementary row transformation matrices, Q is the prouct of elementary column transformation matrices. Thus P, Q are invertible. Remark This theorem is extremely important. From now, when we want to stuy the property of matrix, we stuy the property of canonical form first. Canonical form makes problems easy to hanle. We also have other canonical form, but equivalence canonical form is the most funamental one. 4

5 Prove that equivalence is inee an equivalence relation. Theorem(Rank Inequality) ( ) A 0 rank = rank(a) + rank(b) 0 B ( ) A 0 rank rank(a) + rank(b) C B rank(ab) min{rank(a), rank(b)} If D is a submatrix of D, then rank(d ) rank(d) rank(a + B) rank(a) + rank(b) (Frobenius) rank(ab) + rank(bc) rank(b) rank(abc) If P, Q are invertible, then rank(p AQ) = rank(a) A is a matrix of orer n, prove: rank(a n ) rank(a n+1 ) rank(a n 1 ) rank(a n ) If rank(a m+1 ) = rank(a m ), then rank(a m ) = rank(a m+k ) for any integer k rank(a n ) = rank(a n+k ) for any integer k If A k = 0, then k n 2 Matrix analysis We have alreay known that F m n is a vector space. In lecture 1, we talke the structures on vector space, an norm is an important one. First we consier the norm of vector. Definition(Vector Norm) A function : R n R is a vector norm if x R n, x 0 an x = 0 if an only if x = 0 x R n, λ R, λx = λ x x, y R n, x + y x + y 5

6 Example For p 1, x p = ( n i=1 x i p ) 1 p is a vector norm. Specially, x 1 = n i=1 x i x 2 = ( n i=1 x i 2 ) 1 2 x = max 1 i n { x i } Remark The proof of triangle inequality is complicate. When p = 2, you can prove it by Cauchy-Schwarz inequality, which is the special case of Holer inequality. An one more wor, Holer inequality is a useful tool in real analysis an probability. Theorem(Holer Inequality) For p, q > 1, 1 p + 1 q = 1, xt y x p y q. We have efine many norms, but the following result tells us that all of them are equivalent. Theorem(Equivalent Norm) Suppose we have two norms α, β on R n, then there exists c 1, c 2 R such that for all x R n c 1 x α x β c 2 x α Suppose x k = (x 1 k,, xn k ) Rn an x = (x 1,, x n ) R n. Prove that lim x k x = 0 lim x i k x i = 0, i n n Although we have given the general efinition of norm, the matrix norm is ifferent, because we nee to consier the multiplication between matrices. To ensure the matrix norm has nice properties, we nee some more requirements. Definition(Matrix Norm) A function : R n n R is a matrix norm if A R n n, A 0 an A = 0 if an only if A = 0 A R n n, λ R, λa = λ A A, B R n n, A + B A + B A, B R n n, AB A B Remark Note that any matrix norm on R n n is also a vector norm on R n2. Thus matrix norm has all properties of vector norm. 6

7 Definition(Compatible) We have a matrix norm M on R n n an a vector norm v on R n, we say they are compatible if Example Define Ax v A M x v, A R n n, x R n A M = max Ax Ax v v = max x v=1 x x v This is a compatible matrix norm. This is calle the operator norm, an it will appear in functional analysis again. Actually, we often efine norm of linear operator between norme vector space in this way. Example In the last example, we can take v = 1, 2 an. In fact v = 1, then the inuce matrix norm is A 1 = max 1 j n n i=1 a ij v =, then the inuce matrix norm is A = max 1 i n n j=1 a ij A, B R n an A is invertible. If A B < 1 A 1 then B is also invertible. Definition(Convergence) say A n converges to A. A n, A R n, is a matrix norm, if lim n A n A = 0, we We claim that R n n with a matrix norm is a Banach space. Thus, any Cauchy sequence is convergent. A R n n, is a matrix norm, then A k A k. If A < 1, then I A is invertible, an (I A) 1 1. (Compare with 1 A the nilpotent case) For matrix A R n n, we efine e A = A k k=0. k! Prove that this is well-efine. Fin the inverse of e A. (Compare with the nilpotent case) Prove that if AB = BA, then e A+B = e A e B (Use convolution) Fin a counterexample such that e A+B e A e B 7

8 3 Calculus for vector an matrix value function In mathematical analysis, you have learne many things about vector value function. Suppose x(t) = (x 1 (t), x 2 (t),, x n (t)) is a vector value function of t in R n, we can iscuss continuity an erivative using the norm we learne above. Definition(Continuity) x(t) is continuous at t 0 if lim t t0 x(t) x(t 0 ) = 0. Definition(Derivative) If lim h 0 x(t 0+h) x(t 0 ) h at t 0, an the erivative at t 0 is x (t 0 ). x (t 0 ) = 0, we say x(t) is ifferentiable The case of matrix value function is all the same because we can see matrix as a vector. But note that when we put ifferentiation an linear function together, something nice appears. l is a linear function on R n, x(t) is a ifferentiable vector value function in R n, then t l(x(t)) = l( t x(t)) A(t) is a matrix value function, then t tr(a(t)) = tr( t A(t)) Theorem Suppose A(t) is a ifferentiable an invertible matrix value function, then A 1 (t) is ifferentiable, an t A 1 = A 1 ( t A)A 1 Remark Note that chain rule is not true for matrix value function because matrix multiplication is not commutative. An when we ifferentiate the prouct of matrix/vector, the orer can t be change. Prove by inuction that t Ak = A A k 1 + AA A k A k 1 A Theorem Suppose p is a polynomial, A(t) is a matrix value function, If A(t 0 ) is commutative with A (t 0 ), then the chain rule hols at t 0 : t tr(p(a)) = tr(p (A)A ) t p(a) t 0 = p (A)A t0 8

9 Now we come to consier multiple linear function L(a 1,, a n ). If x 1 (t),, x n (t) are ifferentiable vector value function, then L(x 1 (t),, x n (t)) is also ifferentiable. An it s not har to prove: t L(x 1(t),, x n (t)) = L(x 1(t),, x n (t)) + + L(x 1 (t),, x n(t)) As you may have guesse, an application of this formula is eterminant. Determinant is a multiple linear function, so t et(x 1(t),, x n (t)) = et(x 1(t),, x n (t)) + + et(x 1 (t),, x n(t)) Use the ifferentiation formula of eterminant, prove: X(t) is a ifferentiable matrix value function an X(0) = I, then t et X(t) t=0= tr(x (0)) X(t) is a ifferentiable matrix value function an X(t) is always invertible, then t ln et Y (t) = tr(y 1 (t)y (t)) 4 Applications in ifferential geometry I will introuce some basic ieas of ifferential geometry here. Definition(Curve) A mapping r : (a, b) R 3, t (x(t), y(t), z(t)) is a smooth curve if it s infinitely ifferentiable an r/t 0. We can take parameter s such that T (s) = r is the unit tangent vector. Because (T, T ) = 1, s ifferentiate, we know (T, T ) = 0, thus T T. Let N = 1 T, we immeiately know N T T an N = 1. Let κ = T = x 2 + y 2 + z 2, T = κn. An let B = T N, consier the relations between (T, N, B ) an (T, N, B). Differentiate (N, T ) = 0, we get (N, T ) = (N, T ) = κ. Let τ = (N, B), from (N, N ) = 0 an (T, N, B) is orthogonal, we know N = κt + τb. An B = T N + T N = T ( κt + τb) = τn Thus, using the language of matrix, we get T N = κ s B κ τ τ T N B 9

10 Suppose {e 1 (t), e 2 (t),, e n (t)} is a stanar orthogonal basis of R n, an e 1 (t) e 1 (t) e 2 (t) e 2 (t) t. e n (t) = A. e n (t) Prove that A + A T = 0. Definition(Smooth surface) A mapping r : D = {(u, v)} R 3, (u, v) (x(u, v), y(u, v), z(u, v)) is a smooth surface, if it s infinitely ifferentiable an r u r v 0 everywhere. Fix u = a, we get a curve r(a, v), an the tangent vector is r v (a, v). Similarly, r u (u, b) is a tangent vector. Consier a curve on u, say r(u(t), v(t)), then the tangent vector is r u +r t v v, t it s a linear combination of r u, r v. Thus, r u, r v span the tangent surface of, we enote this by T p. By the meaning of ifferentiation, a small arc r = r u u + r v v. To calculate the length, We let then (r, r) = (r u u + r v v, r u u + r v v) = (r u, r u )u 2 + 2(r u, r v )uv + (r v, r v )v 2 E = (r u, r u ) = ( x u )2 + ( y u )2 + ( z u )2 F = (r u, r v ) = x x u v + y y u v + z z u v G = (r v, r v ) = ( x v )2 + ( y v )2 + ( z v )2 ( E F r 2 = (u, v) F G This is calle the first funamental form of surface. ) ( u v ) Calculate the first funamental form Cyliner: r(u, v) = (x(u), y(u), v) Plane:r(u, v) = (u, v, 0) 10

11 5 Matrix group Suppose V is a vector space on fiel F, we use M n (F ) to enote all square matrices of orer n on F, an use GL n (F ) to enote all invertible matrices of orer n on F. M n (F ) is an abelian group uner matrix aition, but it s not a group uner matrix multiplication because not every matrix have an inverse. However, GL n (F ) is a group uner matrix multiplication, we call it general linear group. Example For n = 1, GL n (F ) = F. For a prime p, F p is a fiel. Fin GL n (F p ). We often stuy the subgroup of GL n (F ), which is calle classical group. Definition(Special Linear Group) SL n (F ) = {A GL n (F ) et A = 1}. ( a b Example SL 2 (Z/nZ) = { c ) a, b, c, Z/nZ, a bc = 1} Given the stanar inner prouct on R n, say, < x, y >= x T y, then A GL n (R) is an orthogonal matrix if A contains the inner prouct, i.e., < Ax, Ay >= x T A T Ay = x T y. Thus A T A = AA T = I. Definition(Orthogonal Group) O n (R) = {A GL n (R) A T A = AA T = I}. We have other inner prouct other than the stanar inner prouct, more specifically, bilinear form. An we know that for a nonegenerately bilinear form Q, there exists a basis such that Q(x, y) = x T ( Ip I q ) y In this case, the matrices satisfying Q(x, y) = Q(Ax, Ay) form a group. Definition(Generalize Orthogonal Group) O p,q = {A GL n (R) A T ( Ip I q ) A = ( Ip I q ) } An SO n (R) = O n (R) SL n (R) is calle the special orthogonal group, SO p,q (R) = O p,q (R) SL n (R) is calle the generalize special orthogonal group. 11

12 Example ( cos θ sin θ SO 2 (R) = { sin θ cos θ ) θ R} Prove that ϕ : R R, t e ta is a homomorphism for a fixe a. An all homomorphism from R to R must be of this form. Prove that ϕ : R GL n (R), t e ta is a homomorphism for a fixe A R n n. An all homomorphism from R to GL n (R) must be of this form. Let S n enote the symmetric group of egree n, an let F be an arbitrary fiel. To each permutation σ S n, efine M(σ) = n i=1 E σ(i),i. Show that M is a homomorphism from S n to GL n (K). 6 Numerical linear algebra 6.1 Perturbation You can regar this part as an introuction to computational mathematic. We focus on solving linear system Ax = b on a computer. Actually we can t get accurate solution because of the truncation error. Consier the linear system Ax = b an a perturbation on A an b we get (A + δa)(x + δx) = b + δb. Plugging in Ax = b, we get (A + δa)δx = δb δax Note that A+δA = A(I +A 1 δa). The perturbation is small enough such that A 1 δa < 1. Accoring to the result in exercise, we know I + A 1 δa is invertible, an Thus, (I + A 1 δa) A 1 δa δx = (A + δa) 1 (δb δax) = (I + A 1 δa) 1 A 1 (δb δax) δx (I +A 1 δa) 1 A 1 δb δax Diviing by x an note that b A x, 1 1 A 1 δa A 1 ( δb + δa x ) δx x A 1 A 1 A 1 δa ( δb + δa b A ) The number A A 1, name conition number gives an upper boun of the perturbation of solution, an we use κ(a) to enote it. The smaller κ(a) is, the more accurate the numerical solution is. 12

13 Solving a linear system is perfect theoretically, for example, Cramer s theorem, x = A 1 b. But we have mentione that these methos are awful in practice. So we nee to fin numerical methos. Generally speaking, there are two types of numerical metho:irectly an iteratively. 6.2 Direct metho The main iea of irect metho is ecomposition. First, consier the conition that A is upper triangle or lower triangle, then the linear system is easy to solve. For general coefficient matrix A, if we can ecompose A into LU where L is lower triangle, U is upper triangle, then the problem is easy. Suppose A R n n, we have n 2 elements in total. Both L an U have n 2 +n elements, so if we want to solve LU = A, we have n 2 equations an n 2 + n unknown 2 number. It has solution in general! In fact, we can assume the number of L in the iagonal positions are all 1. 1 l l n1 l n2 1 u 11 u 1,n 1 u 1n..... u n 1,n 1 u n 1,n u nn = a 11 a 12 a 1n a 21 a 22 a 2n... a n1 a n2 a nn We can get the first row of U, then the first column of L, then the secon row of U, the secon column of L an repeat this until we are one. On the other han, we can get the first column of U then the first row of L instea. However, this process is a little bit stupi. Recall the Gauss cancelation, we can o this by elementary row transformation. Let L k = I l k e T k, where l k = (0,, 0, l k+1,, l n ) T. Then for x = (x 1,, x n ) T R n, L k x = (x 1,, x k, x k+1 l k+1,k x k,, x n l n,k x k ) T Take l i,k = x i x k, i = k+1,, n, we have L k x = (x 1,, x k, 0,, 0) T. Thus, we can transform A into an upper triangle matrix U with prouct of such L k. An obviously the prouct of such L k, enote by L is lower triangle. So we get an LA = U. Note that Ax = b LAx = Lb Ux = Lb, we only nee to solve Ux = Lb. 6.3 Iterative metho For Ax = b, we ecompose A into three parts:a = D L U, where D is the iagonal elements of A, an L is lower triangle, U is upper triangle. Then we have (D L U)x = b. That is x = D 1 (L + U)x + D 1 b. For an initial value x 0 R n, let x k+1 = D 1 (L + U)x k + D 1 b we get a sequence of vectors. This is calle Jacobi iteration. 13

14 Another iea is we can use new terms of x k+1 to o iteration. That is x k+1 = D 1 Lx k+1 + D 1 Ux k + D 1 b. By calculation we know This is calle Gauss-Seiel iteration. x k+1 = (D L) 1 Ux k + (D L) 1 b Note that both iteration have the form x k+1 = Mx k + g. Suppose the accurate solution is x, we must have x = Mx = g. Let e k = x k x, we get e k+1 = Me k. If e k 0, then the iteration converges to the accurate solution. Note that e k = M k e 0, so if M < 1, the iteration must be efficient! 14