Introduction to Linear Algebra. Tyrone L. Vincent


 Joan Wood
 1 years ago
 Views:
Transcription
1 Introduction to Linear Algebra Tyrone L. Vincent Engineering Division, Colorado School of Mines, Golden, CO address: URL:
2
3 Contents Chapter. Revew of Vectors and Matricies. Useful Notation. Vectors and Matricies. Basic operations. Useful Properties of the Basic Operations 6 Chapter. Vector Spaces 9. Vector Space De nition 9. Linear Independence and Basis. Change of basis. Norms, Dot Products, Orthonormal Basis 5. QR Decomposition 6 Chapter. Projection Theorem Chapter. Matrices and Linear Mappings. Solutions to Systems of Linear Equations Chapter 5. Square Matrices, Eigenvalues and Eigenvectors 5. Matrix Exponential 6. Other Matrix Functions 6 Appendix A. Appendix A 9 iii
4
5 CHAPTER Revew of Vectors and Matricies. Useful Notation.. Common Abbrivations. In this course, as in most branches of mathematics, we will often utilize sets of mathematical objects. For example, there is the set of natural numbers, which begins ; ; ; : This set is often denoted N, so that is a member of N but is not. To specify that an object is a member of a set, we use the notation for "is a member of". For example N: Some of the sets we will use are R C R n R mn real numbers complex numbers n dimensional vectors of real numbers m n dimensional real matrices For these common sets, particular notation will be used to identfy members, namely lower case for a scalar or vector, and upper case for a matrix. The following table also includes some common operations x hx; yi A A T A det(a), jaj vector or scalar inner product between vectors x and y matrix transpose of A inverse of A determinant of A To specify a set, we can also use a bracket notation. For example, to specify E as the set of all positive even numbers, we can say either E = f; ; 6; 8; g when the pattern is clear, or use a : symbol, which means "such that": E = fx N : mod(x; ) = g : This can be read "The set of natural numbers x, such that x is divisible evenly by ". When talking about sets, we will often want to say when a property holds for every member of the set, or for at least one. In this case, the symbol 8; meaning "for all" and 9; meaning "there exists" are useful. For example, suppose I is the set numbers consisting of the IQs for people in this class. Then 8x I x >
6 . REVEW OF VECTORS AND MATRICIES means that all students in this class have IQ greater than while 9x I : x > means that at least one student in the class has IQ greater than. We will also be concerned with functions. Given a set X and a set Y a function f from X to Y maps an element from X to and element of Y and is denoted f : X! Y: The set X is called the domain, and f(x) is assumed to be de ned for every x X: The range, or image of f is the set of y for which f(x) = y for some x : Range(f) = fy Y : 9x X such that y = f(x)g : If Range(f) = Y; then f is called "onto". If there is only one x X y = f(x); then f is called "one to one".. Vectors and Matricies such that You are probably already familiar with vectors and matrices from previous courses in mathmatics or physics. We will nd matrices and vectors very useful when representing dynamic systems mathematically, however, we will need to be able to manipulate and understand these objects at a fairly deep level. Some texts use bold face for vectors and matrices, but ours does not, and I will not use that convention here, or during class. I will however, use lower case letters for vectors and upper case letters for matrices. A vector of n tuple real (or sometimes complex) numbers is represented as: x = 6 So that x is a vector, and x i are each scalars. We will use the notation x i R n to show that x is a length p vector of real numbers (or x i C n if the elements of x i are complex.) Sometimes we will want to index vectors as well, which can sometimes be confusing: Is x i the vector x i or the ith element of the vector x? To make the di erence clear, we will reserve the notation [x] i to indicate the ith element of x: As an example, consider the following illustration of addition and scalar multiplication for vectors: x + x = 6 [x ] + [x ] [x ] + [x ]. [x ] n + [x ] n x x. x n x = 6 A matrix is an m n array of scalars: a a a n a a a n A = a m a m a mn [x ] [x ]. [x ] n We use the notation A R mn indicate that A is a m n matrix. Addition and scalar multiplication are de ned the same way as for vectors. 7 5
7 . BASIC OPERATIONS. Basic operations You should already be faimilar with most of the basic operations on vectors and matricies listed in this section... Transpose. Given a matrix A R mn ; the transpose A T R nm is found by ipping all terms along the diagonal. That is, if a a a n a a a n A = a m a m a mn then a a a n A T a a a n = a m a m a nm Note that if the matrix is not square (m 6= n); then the shape of the matrix changes. We can also use transpose on a vector x R n by considering it to be an n by matrix. In this case, x T is the by n matrix: x T = [x] [x] [x] n.. Inner (dot) product. In three dimensional space, we are familiar with vectors as indicating direction. The inner product is an operation that allows us to tell if two vectors are pointing in a similar direction. We will use the notation hx; yi for inner product between x and y: In other courses, you may have seen this called the dot product with notation x y: The notation used here is more common in signal processing and control systems. The inner product of x; y R n is de ned to be the sum of product of the elements nx hx; yi = [x] i [y] i i= = x T y Recall that if x and y are vectors, the angle between them can be found using the formula hx; yi cos = p hx; xi hy; yi Note that the inner product satis es the following rules (inherited from transpose) hx + y; zi = hx; zi + hy; zi hy; zi = hy; zi.. Matrixvector multiplication. Suppose we have an m n matrix A a a a n a a a n A = a m a m a mn
8 . REVEW OF VECTORS AND MATRICIES and a length n vector x : Note that the number of columns of A are the same as the length of x : Multiplication of A and x is de ned as follows: where x = Ax (.) [x ] = a [x ] + a [x ] + + a n [x ] n (.a) [x ] = a [x ] + a [x ] + + a n [x ] n (.b). (.c) [x ] m = a m [x ] + a m [x ] + + a mn [x ] n (.d) Note that the result x is a length m vector (the number of rows of A). The notation (.) is a compact representation of the system of linear algebraic equations (.). Note that A de nes a mapping from R n to R m : Thus, we can write A : R n! R m : This mapping is linear. We can also consider a matrix to be a group of vectors. For example, if we group the vectors x ; x ; ; x n into a matrix M = x x x p and de ne the vector a = 6 Then all linear combinations of x ; x ; ; x p are given by. p 7 5 y = Ma = x + x + + p x p.. MatrixMatrix multiplication. If matrix A : R n! R m ; and matrix B : R m! R p ; we can nd mapping C: R n! R p which is the composition of A and B C = BA [c] ij = X k [b] ik [a] kj That is, the i; j element of C is the dot product of the ith row of B with the jth column of A: The dimension of C is pn: This can also be though of as B mapping a column of A at a time: That is, the rst column of C; [c] is B[a] ; B times the rst column of A: Clearly, two matricies can be multiplied only if they have compatible dimensions. Unlike scalars, the order of multiplication is important. If A and B are square matricies, AB 6= BA in general. The identity matrix I = is a square matrix with ones along the diagonal. If size is important, we will denote it via a subscript, so that I m is the m m identity matrix. The identity matrix is
9 . BASIC OPERATIONS 5 the multiplicative identity for matrix multiplication, in that AI = A and IA = A (where I has compatible dimensions with A):.5. Block Matricies. Matricies can also be de ned in blocks, using other matricies. For example, suppose A R mn B R mp C R qn and D R qp : Then we can "block ll" a (m + q) by (n + p) matrix X as A B X = C D Often we will want to specify some blocks as zero. We will denote a block of zeros as simply : The dimension can be worked out from the other matricies. For example, if A X = C D The zero block must have the name number of rows as A and the same number of columns as D: Matrix multiplication of block matricies uses the same rules as regular matricies, except as applied to the blocks. Thus A B A B A A = + B C A B + B D C D C D C A + D C C B + D D.6. Determinants. If A is a scalar matrix, that is, A R ; then the determinant of A is just equal to itself. For higher dimensions, the determinant is de ned recursively. If A R nn then nx det(a) = [a] ij c ij i= where c ij is the ijth cofactor, and is the determinant of a R n n matrix (this is what makes the de nition recursive) possibly times . In particular: c ij = ( ) i+j det(m ij ) where M ij is the n n submatrix created by deleting the ith row and jth column from A If A R ; then det(a) = a a a a (Check that this matches the de nition).7. Inverse. Given a square matrix A R nn ; the inverse of A is the unique matrix (when it exists) such that AA = I The inverse can be calculated as A = det(a) C T where [C] ij = c ij ; the ijth cofactor of A: C T is also called the adjugate of A: The inverse exists whenever det(a) 6= :
10 6. REVEW OF VECTORS AND MATRICIES Transpose. Useful Properties of the Basic Operations A T T = A (A + B) T = A T + B T (AB) T = B T A T Determinants det(ab) = det(ba) = det(a) det(b) det(a T ) = det(a) det(i) = Determinants for Block Matricies Very useful: the determinant of block triangular matricies is the product of the determininant of the diangonal blocks. In particular, if A and D are square A B A det = det = det(a) det(d) D C D if A and D are square and D exists, then A B det = det(a BD C) det(d) C D Inverse AA = A A = I (A T ) = (A ) T (AB) = B A Inverses for Block Matricies If A and D are square and invertible, A B A A = BD D D If A and D are square and D and = A BD C invertible, A B = BD D D D C D C BD + D.. Exercises. () Show that det(a ) = det(a) () Let N = Q = P = Im A O I n Im B I n Im A B I n (a) Explain why det(n) = det(q) = (b) Compute NP and QP (c) Show that det(np ) = det(i m + AB) and det(qp ) = det(i n + BA)
11 . USEFUL PROPERTIES OF THE BASIC OPERATIONS 7 (d) Show that det(i m + AB) = det(i n + BA) () Using the results of problem, explain why (I m + AB) exists if and only if (I n + BA) exists. Show by verifying the properties of the inverse that (I m + AB) = I m A(I n + BA) B: (That is, multiply the right hand side by I m + AB and show that you get the identity) () Verify the block inversion equations.
12
13 CHAPTER Vector Spaces. Vector Space De nition Definition. A vector space (F,X ) consists of a set of elements X, called vectors, and a eld F (such as the real numbers) which satisfy the following conditions: () To every pair of vectors x and x in X ; there corresponds a vector x = x + x in X : () Addition is commutative: x + x = x + x () Addition is associative: (x + x ) + x = x + (x + x ) () X contains a vector, denoted, such that + x = x for every x in X (5) To every x in X there is a vector x in X such that x + x = (6) To every in F, and every x in X ; there corresponds a vector x in X (7) Scalar multiplication is associative: For any ; in F and any x in X, (x) = ()x (8) Scalar multiplication is distributive with respect to vector addition: (x + x ) = x + x (9) Scalar multiplication is distributed with respect to scalar addition: ( + )x = x + x () For any x in X ; x = x: You can verify that R n (or C n ) is a vector space. It is interesting to see that other mathematical objects also qualify to be vector spaces. For example: Example. X =R n [s]; the set of all polynomials with real coe cients with degree less than n; F = R; with addition and multiplication de ned in the usual way: if x = a s n + a s n + + a n x = b s n + b s n + + b n ; then x + x = (a + b )s n + (a + b )s n + + (a n + b n ) kx = ka s n + ka s n + + ka n We can show that this is a vector space by verifying that it satis es the conditions: () Given any x = a s n +a s n + +a n and x = b s n +b s n + + b n ; we see that x + x = (a + b )s n + (a + b )s n + + (a n + b n ) is indeed a polynomial of degree less than n, so x + x is in X () obvious from de nition of addition () obvious from de nition of addition () Select x = as the zero vector 9
14 . VECTOR SPACES (5) Given x = a s n +a s n + +a n ; select x = a s n a s n a n (6) Given x = a s n + a s n + + a n ; we see that ax = aa s n + aa s n + + aa n is a polynomial of degree less than n; so that ax is in X (7) obvious from de nition of scalar multiplication (8) obvious from de nition of addition and scalar multiplication (9) obvious from de nition of addition and scalar multiplication () select x = as the unit vector.. Exercises. () Show that X =C; the set of all continuous functions is a vector space withf = R; with addition and multiplication de ned as x = f(t); x = g(t); x + x = f(t) + g(t); ax = af(t): This can be shown to be a vector space in the same way as above. () Show that X =C n ; the set of all n tuples of complex numbers is a vector space with F = C; the eld of complex numbers () Show that X =R n ; F = C is not a vector space. () Show that X =fx : x + _x + = g is a vector space. with F = R.. Linear Independence.. Linear Independence and Basis Definition. A linear combination of the vectors x ; x ; ; x p is a sum of the form x + x + + p x p : A linear combination can also be written in matrixvector form, x x x p p A vector x is said to be linearly dependent upon a set S of vectors if x can be expressed as a linear combination of vectors from S: A vector x is said to be linearly independent of S if it is not linearly dependent on S: A set of vectors is said to be a linearly independent set if each vector in the set is linearly independent of the remainder of the set. This de nition immediately leads to the following tests: Theorem. A set of vectors S = fx ; x ; : : : ; x p g are linearly dependent if there exists i with at least one 6= such that x + x + + p x p = Theorem. A set of vectors S = fx ; x ; : : : ; x p g is linearly independent if and only if implies i = i = ; ; ; p: x + x + + p x p =
15 . LINEAR INDEPENDENCE AND BASIS Example. Consider the set of vectors x = 5 ; x = 5 ; x = 5 6 This set is linearly dependent, for if we select = have x + x + x = Example. Consider the set of vectors x = ; x = ; = and = ; we This set is linearly dependent, for if we select = and = ; then x + x = Note that the zero vector is linearly dependent on all other vectors. The maximal number of linearly independent vectors in a vectors space is an important characteristic of that vector space. Definition. The maximal number of linearly independent vectors in a vector space is called the dimension of the vector space Example. Show that the dimension of the vector space (R ; R) is Note that the vectors and are linearly independent. Thus the dimension of (R ; R) is greater than or equal to : Given three vectors x = a b ; x = c d ; x = e f, Then we have x + x + x = if = ; and and are solutions to the system of equations a + c = e b + d = f which always has at least one solution. Thus no set of three vectors are linearly independent, and the dimension of (R ; R) is less than ; implying that the dimension of (R ; R) is... Basis. Definition. A set of linearly independent vectors from a vector space (F; X ) is a basis for X if every vector in X can be expressed as a unique linear combination of these vectors. It is a fact that in an ndimensional vector space, any set of n linearly independent vectors quali es as a basis. We have seen that there are many di erent mathematical objects which qualify at vector spaces. However, all ndimensional vectors spaces (X ; R) have a one to one correspondence with the vector space (R n ; R) once a basis has been chosen. Suppose e ; e ; ; e n is a basis for X : Then for all x in X x = e e e n where = n and i are scalars. Thus the vector x can be identi ed with the unique vector in R n :Consider the vector space (R [s]; R)
16 . VECTOR SPACES where R [s] is the set of all real polynomials of degree less than. This vector space has dimension, with one basis as e = ; e = s; e = s : The vector x = s + s can be written as x = e e e 5 So that the representation with respect to this basis is : However, if we choose the basis e = ; e = s ; e = s s (verify that this set of vectors is independent), x = s + s = + 5(s ) + (s s) = e e e 5 5 so that the representation of x with respect to this basis is 5 :.. Standard basis. For R n ; the standard basis are the unit vectors that point in the direction of each axis i = ; i = ; i n = Exercises. () Find the dimension of the vector space given by all (real) linear combinations of x = 5 ; x = 5 ; x = 5 ; 5 That is, X = fx : x = x + x + x ; i Rg This is called the vector space spanned by fx ; x ; x g : () Show that the space of all solutions to the di erential equation x + _x + x = t is a dimensional vector space. (Verify the properties of a vector space). Change of basis Since the vectors are made up of polynomials which are mathematical objects quite di erent from n tuples of numbers, the ideas of separation between vectors and their representations with respect to basis is fairly clear. This becomes more complicated when consider the native vector space (R n ; R): When n = ; it is natural to visualize these vectors in the plane, as shown in Figure. In order to represent the vector x; we need to choose a basis. The most natural basis for (R ; R) is the array i = i =
17 . CHANGE OF BASIS Figure. A twodimensional real vector space In this basis, we have the following representation for x : x = = i i Note that the vector and its representation look identical. However, if we choose a di erent basis, say e = e = then x = = e e so the representation of x in this basis is We have seen that a vector x can have di erent representations for di erent basis. A natural desired operation would be to transform between one basis and another. Suppose a vector x has representations with respect to e e e n as and with respect to e e e n as ; so that x = e e e n = e e e n (.) what is the relationship between and? The answer is most easily found by nding the relationship between the bases themselves. Each basis vector has a representation in the other basis. That is, there exists p i such that e i = e e e n pi
18 . VECTOR SPACES If we group the vectors e i into a matrix, we can write e e e n = e e e n p p p n p p p n = e e e p p p n n p n p n p nn = e e e n P (.) where we see that the matrix P takes the vectors p i as its columns. Substituting (.) into (.), we get e e e n P = e e e n Since the representation of a vector with respect to its basis is unique, we must have = P : Thus, in order to transform from basis ( e e e n ) to basis ( e e e n ), we must form the matrix P; where P = 6 ith column: the representation of basis vector i (e i ) with respect to basis ( e e e n ) 7 5 It turns out that P will always be an invertible matrix, so that = P must have ith column: the representation of P = Q = 6 basis vector i (e i ) 7 with respect to basis ( 5 e e e n ) ; and we. Norms, Dot Products, Orthonormal Basis.. Vector Norms. A vector norm, denoted kxk ; is a real valued function of x which is measure of its length. You are probably already familar with a common norm de ned by the Euclidean length of a vector, but in fact, there are many possibilities. A valid norm satis es the following properties: x: () (Always positive unless x = ) kxk for every x and kxk = imples x = () (homogeneity) kxk = jj kxk for scalar : () (Triangle inequality) kx + x k kx k + kx k The most common vector norms are the following:... norm. The norm is the sum of the absolute value of the elements of kxk := nx j[x] i j i=
19 . NORMS, DOT PRODUCTS, ORTHONORMAL BASIS norm. The norm corresponds to Euclidean distance, and is the square root of the sum of squares of the elements of x: v ux kxk := t n ([x] i ) Note that the sum of squares of the elements can also be written as x T x: Thus i= kxk = p x T x... norm. The norm is simply the largest component of x kxk = max [x] i i.. Dot products, orthogonality and projection. As discussed earlier, the dot product between two vectors is given by nx hx; yi = [x] i [y] i Note that i= = x T y kxk = p hx; xi If two vectors have a dot product of zero, then they are said to be orthogonal. A set of vectors fx i g which are pairwise orthogonal and unit norm are said to be orthonormal and will satisfy hx i ; x j i = x T i x j = i = j i 6= j The projection of one vector (say x) on another (say y) is given by z = hx; yi kyk y The vector z points in the same direction as y; but the length is chosen so that the di erence between z and x is orthogonal to y: * + hx; yi hz x; yi = kyk y x; y since hy; yi = kyk hx; yi = hy; yi kyk = hx; yi.. Orthonormal Basis  GramSchmidt Proceedure. An orthonormal basis is a vector space basis which is also orthonormal. Operations are often much easier when vectors are de ned using an orthonormal basis. The gramschimidt proceedure can be used to transform a general basis into an orthonormal basis. It does so by building up the orthonormal basis one vector at a time.
20 6. VECTOR SPACES Suppose we had a basis of two vectors fe ; e g: We can make an orthonormal basis as follows: Set the rst basis vector to point in the same direction as e ; but with unit length: q = e ke k : We need to pick a section vector which is orthogonal to q ; but spans the same space as fe ; e g: This can be done by subtracting the part of e which points in the same direction as q : Let Then u = e hq ; e i q hq ; u i = hq ; e hq ; e i q i = hq ; e i hq ; hq ; e i q i = hq ; e i hq ; e i hq ; q i = hq ; e i hq ; e i = since hq ; q i = : Thus u is orthogonal to q : We can get an orthonormal set by letting q = e ke : Yet, k " # hq ;e i q q = e e ke k ku k ku k which is clearly an invertible change of basis. The general proceedure is as follows. Let fe ; ; e n g be a basis. Let u = e q = u ku k u = e hq ; e i q q = u kuk.. nx u n = e n hq k ; e n i q k q n = un ku nk k= The orthonormal basis given by fq ; ; q n g spans the same space as fe ; ; e n g 5. QR Decomposition The gramschmidt proceedure can be viewed as a matrix decomposition. Let E = e e e n be a matrix with columns made up of n independent vectors e i : Then the relationship between the orthonormal vectors q obtained via the gramschmit proceedure and the original vectors can be written as ku k hq ; e i hq ; e n i e e e n = q q q q ku k hq ; e i hqn ; e n i5 ku n k or E = QR
21 5. QR DECOMPOSITION 7 where Q is a matrix with orthonormal columns, and R is an upper diagonal matrix. Since Q has orthonormal columns, you can verify that QQ T = I; implying that Q = Q T : A matrix whose transpose is also its inverse is called and orthonormal matrix, and satis es QQ T = Q T Q = I (so its rows are also orthonormal.) It turns out that the gramschimdt proceedure as described in the last section is not very well conditioned, numerically, meaning that small errors will accumulate as the algorithm progresses. However, much more numerically stable algorithms are available using Householder or Givens transformations. We will examine the former, but both are covered in detail in textbooks on numerical linear algebra, such as Golub G. H and C. F Van Loan, Matrix Computations, John Hopkins Press, 989. Consider the following problem: we have a vector x; and we would like to nd an orthonormal matrix P such that P x = = i where is an arbitrary number. It turns out that a matrix of the form P = I vv T will do the job, where v is restricted to be unit length (kvk = ): First, lets check that P is indeed orthonormal for any v : P P T = I vv T I vv T T = I vv T I vv T = I vv T + (vv T vv T ) = I vv T + (v kvk v T ) = I vv T + vv T = I Where we have used the fact that kvk = : Now, lets see if we can indeed pick an appropriate v: P x = I vv T x = x v(v T x)
22 8. VECTOR SPACES Let s pick v = P x = x = x = x+kxki kx+kxki : Then k kx + kxk i k (x + kxk i ) ((x + kxk i ) T x) kx + kxk i k (x + kxk i ) (kxk + kxk x T i ) kx + kxk i k x kxk + kxk x T i i kx + kxk i k kxk + kxk x T i x = kx + kxk i k kxk + kxk x T i + kxk ki k x kxk + kxk x T i i kxk + kxk x T i x note that ki k = ; thus P x = kx + kxk i k kxk + kxk x T i x kxk + kxk x T i x kxk + kxk x T i i = kxk + kxk x T i kx + kxk i k i so that the desired transformation occurs with = (kxk +kxk x T i ) : kx+kxki k You can verify in a similar manner that another possible choice for v is x kxki kx kxki : k In practice, one would choose the v for which kx kxk i k is largest, to avoid dividing by a small number. Now, and QR decomposition can be accomplished as follows. () Given E R nn E = e e e n pick v = ekeki ke : Apply P ke ki k = I v v T to get P E = where is an arbitrary number, is a vector zero of length n are arbitrary vectors of length n : () Pick v = e ke ki and apply P ke ke ki k = I v v T to get P P E = 5 e e n e e n and e i Note that because of the way we chose P ; the rst column of P E remains the same, and we zero out the correct parts of the second column. () Continue in this manner, until with P = P n P n P ; we get P E = R; where R is an upper diagonal matrix. Then with Q = P T ; E = QR:
23 5. QR DECOMPOSITION 9 The keys to numerical stability are that at each step, the modi cation of E involes an orthonormal matrix, and speci cation of this orthonormal matrix is well conditioned when kvk is away from zero.
24
25 CHAPTER Projection Theorem The close connection between inner product and the norm comes into play in vector minimization problems that involve the norm. Suppose we have a matrix A; and a vector y: We would like to nd the vector x which gets mapped through A to a vector which is as close as possible to y: That is, we have the folowing problem: min kax yk x (.) Useful facts: () When A is a matrix, there is always a solution to this minimization problem. () When A is an arbitrary linear operator, there is always a solution to this minimization problem if the image, (or range space) of A is closed. () The solution can be found using dot products. Let s try to understand the minimization problem. Theorem. (Projection Theorem) x is a minimizer of (.) if and only if hy Ax; Axi = for all x: X: Proof. (if) Suppose x satis es hy kax yk = ka(x + x x) yk = kax y + A(x x)k Ax; Axi = : Let x be another vector in = kax yk + hax y; A(x x)i + ka(x x)k Now, x x is a vector in X; so that hax y; A(x x)i = ; and kax yk = kax yk + ka(x x)k since ka(x x)k ; kax yk kax yk : Thus x is a minimizer (only if) Now, suppose ^x does not satisfy hy A^x; Axi = for some x X; e.g. hy A^x; Ax d i = c: Then that is is a minimizer ka(^x + x d ) yk = ka^x yk + ha^x y; Ax d i + kx d k =
26
27 CHAPTER Matrices and Linear Mappings A matrix is an m n array of scalars that represents a linear map from one vector space to another: If we group the vectors x ; x ; ; x n into a matrix M = x x x p and de ne the vector a = 6. 5 p Then all linear combinations of x ; x ; ; x p are given by The space is called the range space of M 7 y = Ma = x + x + + p x p S = fy : y = Ma a R p g Definition 5. The range space (or just range) of M is the subset of R n to which vectors are mapped by M, that is R(M) = fy : y = Mx; x R m g In matrix notation, we can say that the column vectors of M are linearly independent if and only if Ma = implies a = : If a matrix does not consist of linearly independent column vectors, then there exists a set of a such that Ma = : This set is a subspace of R p ; and is called a null space. Definition 6. The null space of M is the subset of R m which is mapped to the zero vector, that is N(M) = fx : = Mx; x R m g The dimension of the range space is called the rank of a matrix. Theorem. The rank of M is given by the maximal number of linearily independent columns (or rows)... MATLAB. MATLAB has commands to nd the range space, null space and rank of a matrix. Consider the matrix Orth, null and rank... Exercises.
28 . MATRICES AND LINEAR MAPPINGS. Solutions to Systems of Linear Equations Given vector spaces X and Y and linear operator A : X! Y: Given equation Ax = y (.) () Determine if there is a solution for a particular y () Determine if there is a solution for every y () Determine if the solution is unique () If the solution exists, nd it.... Existence and uniqueness of a solution. y in range space of A: null space of A:... Finding the solution Case : A invertible. De nition of the inverse of A...5. Finding the solution Case : A not invertible.
29 CHAPTER 5 Square Matrices, Eigenvalues and Eigenvectors If A is a square matrix, i.e. A R nn ; then A maps vectors back to the same space. These matrices can be characterized by their eigenvalues and eigenvectors Definition 7. Given matrix A R nn ; (or C nn ) if there exists scalar and vector x 6= such that Ax = x then is an eigenvalue of A; and x is an eigenvector of A: To nd eigenvalues and eigenvectors, we can use the concept of rank. If is an eigenvalue of A; then there exists x such that or Ax x = (A I)x = From above, an x 6= only exists if the rank of (A I) is less than n: Recall that the rank of a square matrix is less than n if and only if the determinant of the matrix is zero. This implies that the eigenvalues of A satisfy Then Example 5. Let det(a I) = A = A I = det(a I) = ( )( ) = = ( )( + ) Thus the eigenvalues are and : For = ; we nd the corresponding eigenvector via (A + I) x = x = x which has solution x = : Note that there are many possible solutions, each can be obtained through a scale factor. 5
30 6 5. SQUARE MATRICES, EIGENVALUES AND EIGENVECTORS For = ; we nd the corresponding eigenvector via which has solution x = : (A I) x = x = x. Matrix Exponential Given A R nn ; we de ne the matrix exponential as follows: e A : = I + A + A! + A! + : = X k= Using the de nition, you can verify the following properties: A k k! e = I; where is a matrix of zeros. e A = e A d dt eat = Ae At = e At A For the last property, note that and similarly for e At A: d X (At) k dt k= k! = X k= = X `= = A X = Ae At ka k t k k! (` + )A`+ t` (` + )! `= A`t` `!. Other Matrix Functions Note that the de nition of the matrix exponential corresponded with the usual scalar de nition, that is e x = + x + x! + e A = I + A + A! + In general, a matrix function is de ned using its series expansion. In particular, we de ne the matrix natural log ln(a) = (A I) (A I) + (A I) (A I) + It turns out that for a given matrix, the value of the matrix function can be found using a nite expansion as well. The key result is called the CaleyHamilton Theorem, which will be stated without proof:
31 . OTHER MATRIX FUNCTIONS 7 Theorem 5. (CaleyHamilton) Given an ndimensional square matrix A; let a() = det(a I) = n + n + + n = be the characteristic equation. Then a(a) = ; that is A n + A n + + n I = ( A satis es its own characteristic equation) In particular, this indicates that A n = A n n I A n+ = A n A n n A = A n n I A n n A = A n n I and in general, A k for k > n can be written as a linear combination of the terms I; A; A ; A n : Thus for a given A; and a given matrix function f(); there is an n dimensional polynomial representation of f(a); that is f(a) = A n + A n + + n I Although we will not prove this in detail, since both A and i have the property that a( i ) = a(a) = ; and because a() is what is used to simplify a(a); we can use the function values at the eigenvalues to more easily evaluate the matrix function. Given n dimensional matrix A and function f(); to evaluate f(a) when the eigenvalues of A are simple (not repeated), perform the following steps: Find the eigenvalues of A; that is, ; ; n Solve the following equations for through n f( ) = n + n + + n f( ) = n + n + + n. f( n ) = n n + n n + + n Find f(a) = A n + A n + + n I When the eigenvalues are repeated, there will not be enough equations to solve for i ; so the additional conditions are added d d f( i) = d d n + n + + n. d m d m f( i ) = dm d m n + n + + n where m is the index of the repeated eigenvalue. #! Example 6. Find ln det " p " p p p : The characteristic equation is #! = p! + = p +
32 8 5. SQUARE MATRICES, EIGENVALUES AND EIGENVECTORS and = p q = p j = ej 6 : Now ln e j 6 = j + k k = ; ; ; 6 ln e j 6 = j + k k = ; ; ; 6 Note that there are multiple solutions. This is because the function ln is not one to one. The solution of interest depends on the context. For now, let s pick the solutions with k = : Then j 6 = e j 6 + or This has solution and j = 6 e j 6 + " p # j 6 j = + j p 6 j = = ln(a) = p A 6 I = = " p # + j p j 6 j j 6 p p 6 p p 6 6 p 6 For an example with repeated roots, see Example B. in the text. Note that, because ln(e x ) = x; ln(e A ) = A; and the matrix log is the inverse function for the matrix exponential. 5
33 APPENDIX A Appendix A 9
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector
More information4 Linear Algebra Review
Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the text 1 Vectors and Matrices For the context of data analysis, the
More informationLinear Algebra Primer
Introduction Linear Algebra Primer Daniel S. Stutts, Ph.D. Original Edition: 2/99 Current Edition: 4//4 This primer was written to provide a brief overview of the main concepts and methods in elementary
More informationMatrices and Linear Algebra
Contents Quantitative methods for Economics and Business University of Ferrara Academic year 20172018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationMath Linear Algebra Final Exam Review Sheet
Math 151 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a componentwise operation. Two vectors v and w may be added together as long as they contain the same number n of
More information4.3  Linear Combinations and Independence of Vectors
 Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be
More informationMatrix Algebra: Summary
May, 27 Appendix E Matrix Algebra: Summary ontents E. Vectors and Matrtices.......................... 2 E.. Notation.................................. 2 E..2 Special Types of Vectors.........................
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationIntroduction to Matrices
214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the
More informationELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices
ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationa11 a A = : a 21 a 22
Matrices The study of linear systems is facilitated by introducing matrices. Matrix theory provides a convenient language and notation to express many of the ideas concisely, and complicated formulas are
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationMath Bootcamp An pdimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationElementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationLinear Algebra March 16, 2019
Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More informationLinear Algebra Primer
Linear Algebra Primer D.S. Stutts November 8, 995 Introduction This primer was written to provide a brief overview of the main concepts and methods in elementary linear algebra. It was not intended to
More informationBasic Elements of Linear Algebra
A Basic Review of Linear Algebra Nick West nickwest@stanfordedu September 16, 2010 Part I Basic Elements of Linear Algebra Although the subject of linear algebra is much broader than just vectors and matrices,
More informationMath113: Linear Algebra. Beifang Chen
Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data TwoDimensional Plots Programming in
More informationIntroduction. Vectors and Matrices. Vectors [1] Vectors [2]
Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector  one dimensional array Matrix 
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationMatrices. Chapter Definitions and Notations
Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an mbyn array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More information3 a 21 a a 2N. 3 a 21 a a 2M
APPENDIX: MATHEMATICS REVIEW G 12.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a twodimensional array of numbers 2 A = 6 4 a 11 a 12... a 1N a 21 a 22... a 2N. 7..... 5 a M1 a M2...
More informationEIGENVALUES AND EIGENVECTORS 3
EIGENVALUES AND EIGENVECTORS 3 1. Motivation 1.1. Diagonal matrices. Perhaps the simplest type of linear transformations are those whose matrix is diagonal (in some basis). Consider for example the matrices
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationJim Lambers MAT 610 Summer Session Lecture 1 Notes
Jim Lambers MAT 60 Summer Session 20090 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra
More informationReview of linear algebra
Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationLinear Algebra. James Je Heon Kim
Linear lgebra James Je Heon Kim (jjk9columbia.edu) If you are unfamiliar with linear or matrix algebra, you will nd that it is very di erent from basic algebra or calculus. For the duration of this session,
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More information3 Matrix Algebra. 3.1 Operations on matrices
3 Matrix Algebra A matrix is a rectangular array of numbers; it is of size m n if it has m rows and n columns. A 1 n matrix is a row vector; an m 1 matrix is a column vector. For example: 1 5 3 5 3 5 8
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationMATH 235. Final ANSWERS May 5, 2015
MATH 235 Final ANSWERS May 5, 25. ( points) Fix positive integers m, n and consider the vector space V of all m n matrices with entries in the real numbers R. (a) Find the dimension of V and prove your
More informationMobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti
Mobile Robotics 1 A Compact Course on Linear Algebra Giorgio Grisetti SA1 Vectors Arrays of numbers They represent a point in a n dimensional space 2 Vectors: Scalar Product ScalarVector Product Changes
More information[ Here 21 is the dot product of (3, 1, 2, 5) with (2, 3, 1, 2), and 31 is the dot product of
. Matrices A matrix is any rectangular array of numbers. For example 3 5 6 4 8 3 3 is 3 4 matrix, i.e. a rectangular array of numbers with three rows four columns. We usually use capital letters for matrices,
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationMatrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices
Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated
More informationLinear Algebra Final Exam Review
Linear Algebra Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.
More informationLinear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions
Linear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions AnnaKarin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 Main problem of linear algebra 2: Given
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationIntroduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661  Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationLinear Algebra Practice Final
. Let (a) First, Linear Algebra Practice Final Summer 3 3 A = 5 3 3 rref([a ) = 5 so if we let x 5 = t, then x 4 = t, x 3 =, x = t, and x = t, so that t t x = t = t t whence ker A = span(,,,, ) and a basis
More informationLecture Summaries for Linear Algebra M51A
These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture
More informationA Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
More informationFinal Review Sheet. B = (1, 1 + 3x, 1 + x 2 ) then 2 + 3x + 6x 2
Final Review Sheet The final will cover Sections Chapters 1,2,3 and 4, as well as sections 5.15.4, 6.16.2 and 7.17.3 from chapters 5,6 and 7. This is essentially all material covered this term. Watch
More informationMATH Topics in Applied Mathematics Lecture 12: Evaluation of determinants. Cross product.
MATH 311504 Topics in Applied Mathematics Lecture 12: Evaluation of determinants. Cross product. Determinant is a scalar assigned to each square matrix. Notation. The determinant of a matrix A = (a ij
More informationElementary Linear Algebra
Matrices J MUSCAT Elementary Linear Algebra Matrices Definition Dr J Muscat 2002 A matrix is a rectangular array of numbers, arranged in rows and columns a a 2 a 3 a n a 2 a 22 a 23 a 2n A = a m a mn We
More informationOnline Exercises for Linear Algebra XM511
This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2
More informationMTH 2032 SemesterII
MTH 202 SemesterII 201011 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationSUMMARY OF MATH 1600
SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More informationA Brief Outline of Math 355
A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting
More informationSection 6.4. The Gram Schmidt Process
Section 6.4 The Gram Schmidt Process Motivation The procedures in 6 start with an orthogonal basis {u, u,..., u m}. Find the Bcoordinates of a vector x using dot products: x = m i= x u i u i u i u i Find
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342  Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationa 11 a 12 a 11 a 12 a 13 a 21 a 22 a 23 . a 31 a 32 a 33 a 12 a 21 a 23 a 31 a = = = = 12
24 8 Matrices Determinant of 2 2 matrix Given a 2 2 matrix [ ] a a A = 2 a 2 a 22 the real number a a 22 a 2 a 2 is determinant and denoted by det(a) = a a 2 a 2 a 22 Example 8 Find determinant of 2 2
More information2. Every linear system with the same number of equations as unknowns has a unique solution.
1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations
More informationProblem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show
MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology  Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationECON0702: Mathematical Methods in Economics
ECON0702: Mathematical Methods in Economics Yulei Luo SEF of HKU January 12, 2009 Luo, Y. (SEF of HKU) MME January 12, 2009 1 / 35 Course Outline Economics: The study of the choices people (consumers,
More informationMath 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam
Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system
More information[Disclaimer: This is not a complete list of everything you need to know, just some of the topics that gave people difficulty.]
Math 43 Review Notes [Disclaimer: This is not a complete list of everything you need to know, just some of the topics that gave people difficulty Dot Product If v (v, v, v 3 and w (w, w, w 3, then the
More informationMatrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices
Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationn n matrices The system of m linear equations in n variables x 1, x 2,..., x n can be written as a matrix equation by Ax = b, or in full
n n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in n variables x 1, x 2,..., x n a 11 x 1 + a 12 x
More information22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices
m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix
More information4 Linear Algebra Review
4 Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the course 41 Vectors and Matrices For the context of data analysis,
More informationLinear Algebra and Matrices
Linear Algebra and Matrices 4 Overview In this chapter we studying true matrix operations, not element operations as was done in earlier chapters. Working with MAT LAB functions should now be fairly routine.
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 WeiTa Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationChapter Two Elements of Linear Algebra
Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to
More informationSolution of Linear Equations
Solution of Linear Equations (Com S 477/577 Notes) YanBin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass
More informationMath 554 Qualifying Exam. You may use any theorems from the textbook. Any other claims must be proved in details.
Math 554 Qualifying Exam January, 2019 You may use any theorems from the textbook. Any other claims must be proved in details. 1. Let F be a field and m and n be positive integers. Prove the following.
More informationNumerical Linear Algebra
Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an nvector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix
More informationMAT 610: Numerical Linear Algebra. James V. Lambers
MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic
More informationMATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS
MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS There will be eight problems on the final. The following are sample problems. Problem 1. Let F be the vector space of all real valued functions on
More information5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.
Linear Algebra  Test File  Spring Test # For problems  consider the following system of equations. x + y  z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More informationReduction to the associated homogeneous system via a particular solution
June PURDUE UNIVERSITY Study Guide for the Credit Exam in (MA 5) Linear Algebra This study guide describes briefly the course materials to be covered in MA 5. In order to be qualified for the credit, one
More informationLINEAR ALGEBRA 1, 2012I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS
LINEAR ALGEBRA, I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,
More informationMath 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 78pm in 117 MacMillan What will be covered The exam will cover material from the lectures
More informationA primer on matrices
A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous
More information