Linear Algebra Review
|
|
- Clement Nichols
- 5 years ago
- Views:
Transcription
1 Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite rapidly. 1.1 Vector Spaces The standard object in linear algebra is a vector space. Definition 1.1. A vector space V over a field F (the scalars) is a set of vectors with two operations: vector addition vector + vector = vector, which makes V into an Abelian group, and scalar multiplication with properties scalar vector = vector, α (βv) = (αβ) v, α(v + w) = αv + αw, (α + β)v = αv + v. A subspace is a subset of V which is closed under addition and scalar multiplication. In this course, we will only consider R (real numbers) and C (complex numbers) as scalars. I will state the definitions for the complex case, if it makes a difference, but all of the examples and homework problems will be real (except possibly in the section on eigenvalues). There are two standard examples of finite-dimensional vector spaces. 1
2 2 CHAPTER 1. LINEAR ALGEBRA REVIEW The Geometric View: Arrows A vector is an arrow, given by a direction and a length. You can move it around to any place you want. Examples are forces in physics, or velocities The Analytic View: Columns of Numbers The vector space is V = R n or C n. v 1 v 1 + w 1 αv 1 v 2 v =., v + w = v 2 + w 2.., αv = αv 2.. v n v n + w n αv n Note: Vectors are columns of numbers, not rows. In this course, we basically only use the analytic view, but the geometric view is occasionally useful to get an intuitive understanding of what some theorem or algorithm means Linear Independence and Bases Let {v i } be a collection of vectors, and {α i } some scalars. A linear combination of the vectors is α i v i. i The set of all linear combinations of {v i } is called the span. The span is always a subspace. The {v i } are called linearly dependent if there is a set of coefficients {α i }, not all zero, for which α i v i = 0. i Otherwise, they are linearly independent. A basis of V is a collection {v i } so that every w V can be written uniquely as a linear combination.
3 1.2. LINEAR MAPS 3 Theorem 1.2. (a) Every vector space has a basis (usually infinitely many of them) (b) Every basis has the same number of elements. This is called the dimension of the space. Take an arbitrary n-dimensional vector space V over F. Pick a basis {e 1, e 2,..., e n }. Then we can equate α 1 w = α i e i w = α 2 α n V F n. Thus, every finite-dimensional vector space over F is isomorphic to F n. We will frequently use the notation e 1 = 0, e 2 = 0, for the standard basis vectors. Sideline: As a generalization of F n, the space of infinite sequences {α 1, α 2, α 3,...} is also a vector space, with dimension infinity. Even more general, the space of functions on an interval [a, b] is a vector space. In finite dimensions, we have subscripts 1,..., n. In the case of sequences, we have subscripts 1, 2,.... In the case of functions, you can think of the x in f(x) as a subscript. Linear algebra for infinite-dimensional spaces is called functional analysis, and is its own topic. 1.2 Linear Maps A homomorphism L between vector spaces needs to preserve the two vector space operations. This means L(v + w) = L(v) + L(w), L(αv) = αl(v),
4 4 CHAPTER 1. LINEAR ALGEBRA REVIEW or equivalently L(αv + βw) = αl(v) + βl(w). Such a mapping is called a linear map. Assume L is a linear map from V to W. Definition 1.3. R(L) = range of L = {Lv : v V } W, N(L) = nullspace of L = kernel of L = {v V : Lv = 0} V. Both of these are subspaces. The dimension of R(L) is called the rank of L. Theorem 1.4. dim(r(l)) + dim(n(l)) = dim(v ). Note: From the definitions, a linear map L from V to W is one-to-one if and only if N(L) = {0}. It is onto if and only if R(L) = W. By the theorem and a dimension count, a mapping L from V into itself is one-to-one if and only if it is onto. 1.3 Matrices So far, L is just a mapping between vector spaces. It maps vectors to other vectors. Now we want to introduce coordinates. We pick a basis {e i }, i = 1,..., n in V, and another basis {f j }, j = 1,..., m in W. If v = i v ie i is an arbitrary vector in V (expressed in terms of the basis {e i }), then by linearity, w = Lv = v i Le i. i We can express Le i in the basis of W : Le i = j l ji f j. We collect all these numbers in a matrix L. Then L v = w l 11 l 12 l 1n l 21 l 22 l 2n.. l m1 l m2 l mn v 1. v n = w 1. w m. So, the columns of the matrix represent the images of the basis vectors.
5 1.3. MATRICES 5 Note the numbering in L: the first subscript always refers to the row, the second to the column. Entry l 35 is in row 3, column 5. The map L goes from F n to F m, and is of size m n (which is the opposite order). There are two ways to think about matrix times vector multiplication. The first one is the dot product interpretation: The jth entry in w is the dot product between the jth row of L, and v. v 1 v 2 w j = (l j1, l j2,..., l jn ). = l j1v 1 + l j2 v l jn v n. v n The second one is the linear combination interpretation: w is the linear combination of the columns of L, with coefficients from v. This is the way we derived it above. When you program this on a computer, (matrix times vector) corresponds to a double loop. The loop can be executed in either order, corresponding to the two interpretations. Depending on the computer architecture, one way may be faster than the other. In Matlab-like code: % dot product % linear combination w = 0; w = 0; for i = 1:n for j = 1:n for j = 1:n for i = 1:n w(i) = w(i) + L(i,j)*v(j) w(i) = w(i) + L(i,j)*v(j) end end end end Remark: Technically, I should distinguish between the mapping L and the matrix L, but I won t. Just keep in mind that the matrix depends on the choice of basis, the mapping does not Matrix Multiplication Suppose you have a linear map L from V (dimension n) to W (dimension m), and M from W to X (dimensions p). We can then consider the combined map N = M L (backwards again).
6 6 CHAPTER 1. LINEAR ALGEBRA REVIEW You can verify that for the matrices, N = M L (p n) = (p m) (m n) n ij = k m ik l kj In words: the entry n ij in the product is the dot product of row i in M with column j in L. Note: The middle dimension has to match, and gets canceled. The size of the result comes from the outer numbers. Likewise in the sum, the middle index gets canceled. On a computer, (matrix times matrix) corresponds to a triple loop, which can be executed in 6 orders, corresponding to 3 different viewpoints. (Each one shows up twice, depending on the order in which the product matrix gets filled in). The first two are the dot product and linear combination interpretations from above. The third one is the sum of rank one matrices interpretation. If w is a row vector (size 1 n), and v is a column vector (size n 1), then w v is a scalar (1 1 matrix), and vw is a matrix of rank 1: v 1 w 1 v 1 w 2 v 1 w n v 2 w 1 v 2 w 2 v 2 w n. v n w 1 v n w 2 v n w n You can think of matrix multiplication ML as the sum of the rank one matrices produced from the products between the i th column of M and the i th row of L. Matrix multiplication is not commutative in general: LM M L. Unless the matrices are square, the two products are not even both defined, or not the same size. However, matrix multiplication is associative and distributive: (LM)N = L(MN) L(M + N) = LM + LN (L + M)N = LN + MN
7 1.3. MATRICES 7 One more observation about matrix multiplication: If N = ML, then then first column of N depends only on the first column of L, not any of the other numbers in L. The second column of N depends only on the second column in L, and so on. Likewise, the first row in N depends only on the first row in M, and so on. This means that if you want to solve a system of matrix equations AX = B, you can treat this as a sequence of matrix-vector problems: where b i, x i are the columns of B, X Basis Change Ax 1 = b 1 Ax m = b m The mapping L is independent of the choice of bases in V, W, but the matrix L depends on the bases. How does the matrix change when you change the bases? We will just consider this for the case where V = W. Suppose you have the standard basis {e i }, and a new basis {f i }. Let F be the matrix with columns f i. Let x be an arbitrary vector. In the original basis, it is expressed with coefficients v i, in the new basis with coefficients w i. x = i v i e i = w i f i = F w. (Use the interpretation of the matrix-vector product F w as a linear combination of the columns of F ). So, v = F w or w = F 1 v. To get from the original representation v in basis {e i } to the new representation w in basis {f i } you have to multiply by F 1. Now assume we have a linear map from V to V. In basis {e i } it is represented by a square matrix L. In the original basis, consider Convert to the new basis y = Lx. F 1 y = ( F 1 LF ) ( F 1 x ). Thus, the mapping L is represented in the new basis as ( F 1 LF ). The matrices L and F 1 LF are called conjugates of each other, or similar matrices. Similar matrices can be considered as the same mapping represented in two different bases. Properties of a matrix that are geometric, such as the determinant or eigenvalues, are preserved under conjugation. Other properties that are analytic are not preserved, such as the special shapes of matrices listed below.
8 8 CHAPTER 1. LINEAR ALGEBRA REVIEW Special Matrices Let F m n be the set of matrices of size m n. This becomes a vector space over F of dimension mn, with entry-by-entry addition and scalar multiplication. The unit element for addition is the zero matrix: O = In F n n, the identity matrix I = satisfies I L = L I = L. For a given L, the inverse matrix L 1 satisfies L L 1 = L 1 L = I. The inverse matrix may or may not exist. Example: Take ( ) a b L = C 2 2. c d The inverse is L 1 = 1 ad bc ( d b c a which can be verified by multiplying. The inverse exists if and only if ad bc 0. The term ad bc in the example is the determinant of the matrix. There are formulas for the determinant of a larger matrix, but we won t need them. Here is what the determinant means. The unit basis vectors e i form a square (in dimension 2) or cube (in dimension 3) of area or volume 1. After the mapping, the images form a parallelogram or parellelepiped. The absolute value of the determinant is the area (or volume) of that. The sign of the determinant has to do with orientation. This is the reason the Jacobian determinant shows up in multidimensional change of variable in integrals. If the determinant is zero, that means that the square or cube gets flattened into something lower-dimensional, and there is no inverse. ),
9 1.4. EIGENVALUES 9 Theorem 1.5. Properties of the determinant: det(i) = 1 det(ab) = det(a) det(b) det(a 1 ) = 1/ det(a) det(triangular matrix) = product of diagonal terms Properties of the inverse: 1.4 Eigenvalues A 1 exists if and only if det(a) 0 A 1 is unique A A 1 = A 1 A = I (AB) 1 = B 1 A 1 Assume A is a square matrix, of size n n. A nonzero vector v is called an eigenvector to eigenvalue λ, if Av = λv (A λi)v = 0. It is obvious that if v is an eigenvector, so is any multiple of v. More generally, linear combinations of eigenvectors to the same eigenvalue are again eigenvectors, so it makes more sense to think of the eigenspace E(λ) = N(A λi) = {v : (A λi)v = 0}. The dimension of E(λ) is called the geometric multiplicity of λ, written γ(λ). From the definition, a nonzero v exists for a given λ if and only if det(a λi) = 0. It turns out that this determinant is a polynomial of degree n in λ. This is called the characteristic polynomial χ(λ). By the fundamental theorem of algebra, χ(λ) has precisely n roots, possibly complex, possibly multiple. The multiplicity of λ as a root of χ(λ) is called the algebraic multiplicity of λ, written α(λ). Example: The n n identity matrix has χ(λ) = (1 λ) n, and every vector is an eigenvector to eigenvalue 1. Algebraic and geometric multiplicity are both n. Example: The matrix ( ) 1 1 A = 0 1 has χ(λ) = (1 λ) 2, so λ = 1 has algebraic multiplicity 2. The only eigenvector is (1, 0) T, so the geometric multiplicity is 1.
10 10 CHAPTER 1. LINEAR ALGEBRA REVIEW We always have 1 γ(λ) α(λ). If γ(λ) = α(λ) for all λ, then A is called diagonalizable. In this case, there is a basis of eigenvectors in which A becomes a diagonal matrix: V 1 AV = Λ A = V ΛV 1. Remark: In theoretical linear algebra courses, you spend a lot of time investigating what happens to matrices which are not diagonalizable. There is something called the Jordan Normal Form that leads to lots of interesting mathematics. For numerical purposes, this is completely irrelevant. We simply assume that every matrix is diagonalizable, which is actually pretty close to the truth: for a matrix with randomly chosen entries, the probability that it is not diagonalizable is 0. Some more properties of eigenvalues: The product of all eigenvalues (with appropriate algebraic multiplicity) is the determinant. If A is triangular, the eigenvalues are the numbers on the diagonal. If H is Hermitian, the eigenvalues are real, and H is diagonalizable. If Q is unitary, the eigenvalues are on the complex unit circle: λ i = 1, and Q is diagonalizable. 1.5 The Inner Product In addition to addition and scalar multiplication, many vector spaces also have an inner product: vector times vector = scalar. The standard inner product on C n is v, w = v i w i, i where the bar denotes complex conjugation. For R n, just ignore the bar. Any inner product produces a norm by v = v, v = v i 2. A norm in general is a measure for the length or size of a vector. This one is called the Euclidean norm, and corresponds to the geometrical length of the vector. If v, w are real, then v, w = v w cos θ, where θ is the angle between v and w. In particular, v and w are orthogonal if v, w = 0. i
11 1.6. SPECIAL MATRIX TYPES 11 For complex vectors you cannot talk about angles, but the definition of orthogonality is the same. Once you have an inner product, you can define the adjoint A of a linear map A by Av, w = v, A w. For matrices, it turns out that A = A T ( A transpose ) if A is real, or A = A H ( A Hermitian transpose ) if A is complex. If A has size m n, then A has size n m, and (A T ) ij = A ji, (A H ) ij = A ji. I frequently use a notation like v = (1, 2, 3, 4) T. That means that v is a column vector, but typesetting it like that would take up too much space on the page, so I write it as the transpose of a row vector. Theorem 1.6. The adjoint has the following properties: (AB) = B A (A ) 1 = (A 1 ) det(a ) = det(a) 1.6 Special Matrix Types Matrices With Certain Patterns of Zeros Some matrices have certain patterns of zeros that make them easy to work with. These patterns do not usually stay the same under a basis transformation. In fact, many algorithms are based on finding a basis that makes a general matrix take such a shape. The notation just means any entry that does not have to be zero (but it could be). Diagonal
12 12 CHAPTER 1. LINEAR ALGEBRA REVIEW Tridiagonal More generally, a banded matrix has several bands of numbers near the diagonal. Triangular (upper or lower) Hessenberg (upper or lower). This is triangular, with one more band along the diagonal Hessenberg matrices are used for eigenvalue problems. It turns out that you cannot transform a general matrix into triangular form with identical eigenvalues in a finite number of steps, but you can get it into Hessenberg form. Then, you start an iterative method to wipe out the extra band. Note that a symmetric Hessenberg matrix is tridiagonal. All of these matrix types can also be defined for non-square matrices. You just start at the top left and follow the diagonal, until you run into an edge. The following matrices all count as upper triangular:
13 1.7. BLOCK MATRICES Hermitian and Unitary Matrices Here are two other special kinds of matrices that come up a lot in numerical analysis. A matrix which is equal to its adjoint is called self-adjoint in general, but that word is usually only used in the infinite-dimensional setting. For real matrices, it is called symmetric, for complex matrices it is called Hermitian. For solving Ax = b for a symmetric or Hermitian matrix, we only need half the storage space, and half the number of calculations. For eigenvalue problems of real symmetric matrices, we know that the eigenvalues and eigenvectors are real, so we don t need complex arithmetic. Also, the Hessenberg form becomes tridiagonal, which speeds up the algorithm significantly. A square matrix whose columns are mutually orthogonal is called orthogonal in the real case, and unitary in the complex case. Orthogonal matrices are very popular in computation for two reasons: U 1 = U, so we have the inverse handy if we need it These matrices are extremely numerically stable (very little round-off error). Since all eigenvalues of a unitary matrix are on the unit circle, the determinant also has absolute value 1. In the real case, the determinant is either 1 or ( 1). Determinant 1 means the matrix represents a rotation of the coordinate system. Determinant ( 1) means it is a rotation pluys a reflection. There are two special examples of orthogonal matrices that are worth mentioning. A permutation matrix has one 1 in each row or column, the rest are 0. This corresponds to a permutation of the basis vectors If A is any matrix, then AP is A with its columns in different order. P A is A with its rows in different order. The general 2 2 orthogonal matrix is ( ) ( ) cos θ sin θ cos θ sin θ (det = 1), or (det = 1). sin θ cos θ sin θ cos θ This is a rotation by angle θ (and maybe a reflection). 1.7 Block Matrices You can partition matrices into blocks:
14 14 CHAPTER 1. LINEAR ALGEBRA REVIEW ( ) A B C =. D E F As long as the dimensions all fit together, you can add/multiply block matrices just like regular matrices: ( ) A B C G ( ) H AG + BH + CI =. D E F DG + EH + F I I This is used extensively in parallel computing, for splitting the calculations among multiple processors.
Linear Algebra Review
Chapter 1 Linear Algebra Review It is assumed that you have had a beginning course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc I will review some of these terms here,
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationMath Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More information235 Final exam review questions
5 Final exam review questions Paul Hacking December 4, 0 () Let A be an n n matrix and T : R n R n, T (x) = Ax the linear transformation with matrix A. What does it mean to say that a vector v R n is an
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More informationFinal Review Sheet. B = (1, 1 + 3x, 1 + x 2 ) then 2 + 3x + 6x 2
Final Review Sheet The final will cover Sections Chapters 1,2,3 and 4, as well as sections 5.1-5.4, 6.1-6.2 and 7.1-7.3 from chapters 5,6 and 7. This is essentially all material covered this term. Watch
More informationMATH 583A REVIEW SESSION #1
MATH 583A REVIEW SESSION #1 BOJAN DURICKOVIC 1. Vector Spaces Very quick review of the basic linear algebra concepts (see any linear algebra textbook): (finite dimensional) vector space (or linear space),
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More information2. Every linear system with the same number of equations as unknowns has a unique solution.
1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationDefinitions for Quizzes
Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does
More information6 Inner Product Spaces
Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space
More informationMatrices and Linear Algebra
Contents Quantitative methods for Economics and Business University of Ferrara Academic year 2017-2018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationA = 3 B = A 1 1 matrix is the same as a number or scalar, 3 = [3].
Appendix : A Very Brief Linear ALgebra Review Introduction Linear Algebra, also known as matrix theory, is an important element of all branches of mathematics Very often in this course we study the shapes
More informationMATH 423 Linear Algebra II Lecture 20: Geometry of linear transformations. Eigenvalues and eigenvectors. Characteristic polynomial.
MATH 423 Linear Algebra II Lecture 20: Geometry of linear transformations. Eigenvalues and eigenvectors. Characteristic polynomial. Geometric properties of determinants 2 2 determinants and plane geometry
More informationA VERY BRIEF LINEAR ALGEBRA REVIEW for MAP 5485 Introduction to Mathematical Biophysics Fall 2010
A VERY BRIEF LINEAR ALGEBRA REVIEW for MAP 5485 Introduction to Mathematical Biophysics Fall 00 Introduction Linear Algebra, also known as matrix theory, is an important element of all branches of mathematics
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More information1. In this problem, if the statement is always true, circle T; otherwise, circle F.
Math 1553, Extra Practice for Midterm 3 (sections 45-65) Solutions 1 In this problem, if the statement is always true, circle T; otherwise, circle F a) T F If A is a square matrix and the homogeneous equation
More informationCS 143 Linear Algebra Review
CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationA Review of Linear Algebra
A Review of Linear Algebra Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab: Implementations
More informationBASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x
BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,
More informationUniversity of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm
University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm Name: The proctor will let you read the following conditions
More informationMATH 310, REVIEW SHEET 2
MATH 310, REVIEW SHEET 2 These notes are a very short summary of the key topics in the book (and follow the book pretty closely). You should be familiar with everything on here, but it s not comprehensive,
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW JC Stuff you should know for the exam. 1. Basics on vector spaces (1) F n is the set of all n-tuples (a 1,... a n ) with a i F. It forms a VS with the operations of + and scalar multiplication
More informationRemark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.
Sec 6 Eigenvalues and Eigenvectors Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called an eigenvalue of A if there is a nontrivial
More informationEigenvectors and Hermitian Operators
7 71 Eigenvalues and Eigenvectors Basic Definitions Let L be a linear operator on some given vector space V A scalar λ and a nonzero vector v are referred to, respectively, as an eigenvalue and corresponding
More informationTopic 15 Notes Jeremy Orloff
Topic 5 Notes Jeremy Orloff 5 Transpose, Inverse, Determinant 5. Goals. Know the definition and be able to compute the inverse of any square matrix using row operations. 2. Know the properties of inverses.
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More informationMATH 221, Spring Homework 10 Solutions
MATH 22, Spring 28 - Homework Solutions Due Tuesday, May Section 52 Page 279, Problem 2: 4 λ A λi = and the characteristic polynomial is det(a λi) = ( 4 λ)( λ) ( )(6) = λ 6 λ 2 +λ+2 The solutions to the
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More information(b) If a multiple of one row of A is added to another row to produce B then det(b) =det(a).
.(5pts) Let B = 5 5. Compute det(b). (a) (b) (c) 6 (d) (e) 6.(5pts) Determine which statement is not always true for n n matrices A and B. (a) If two rows of A are interchanged to produce B, then det(b)
More informationOnline Exercises for Linear Algebra XM511
This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2
More informationand let s calculate the image of some vectors under the transformation T.
Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =
More informationLecture Summaries for Linear Algebra M51A
These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture
More informationMATH 23a, FALL 2002 THEORETICAL LINEAR ALGEBRA AND MULTIVARIABLE CALCULUS Solutions to Final Exam (in-class portion) January 22, 2003
MATH 23a, FALL 2002 THEORETICAL LINEAR ALGEBRA AND MULTIVARIABLE CALCULUS Solutions to Final Exam (in-class portion) January 22, 2003 1. True or False (28 points, 2 each) T or F If V is a vector space
More informationSymmetric and anti symmetric matrices
Symmetric and anti symmetric matrices In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally, matrix A is symmetric if. A = A Because equal matrices have equal
More informationc c c c c c c c c c a 3x3 matrix C= has a determinant determined by
Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.
More informationLinear Algebra. Min Yan
Linear Algebra Min Yan January 2, 2018 2 Contents 1 Vector Space 7 1.1 Definition................................. 7 1.1.1 Axioms of Vector Space..................... 7 1.1.2 Consequence of Axiom......................
More information33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM
33AH, WINTER 2018: STUDY GUIDE FOR FINAL EXAM (UPDATED MARCH 17, 2018) The final exam will be cumulative, with a bit more weight on more recent material. This outline covers the what we ve done since the
More informationft-uiowa-math2550 Assignment NOTRequiredJustHWformatOfQuizReviewForExam3part2 due 12/31/2014 at 07:10pm CST
me me ft-uiowa-math2550 Assignment NOTRequiredJustHWformatOfQuizReviewForExam3part2 due 12/31/2014 at 07:10pm CST 1. (1 pt) local/library/ui/eigentf.pg A is n n an matrices.. There are an infinite number
More informationMIT Final Exam Solutions, Spring 2017
MIT 8.6 Final Exam Solutions, Spring 7 Problem : For some real matrix A, the following vectors form a basis for its column space and null space: C(A) = span,, N(A) = span,,. (a) What is the size m n of
More informationMATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018
Homework #1 Assigned: August 20, 2018 Review the following subjects involving systems of equations and matrices from Calculus II. Linear systems of equations Converting systems to matrix form Pivot entry
More informationDIAGONALIZATION. In order to see the implications of this definition, let us consider the following example Example 1. Consider the matrix
DIAGONALIZATION Definition We say that a matrix A of size n n is diagonalizable if there is a basis of R n consisting of eigenvectors of A ie if there are n linearly independent vectors v v n such that
More informationProblem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show
MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,
More informationMATH 235. Final ANSWERS May 5, 2015
MATH 235 Final ANSWERS May 5, 25. ( points) Fix positive integers m, n and consider the vector space V of all m n matrices with entries in the real numbers R. (a) Find the dimension of V and prove your
More informationMATH 304 Linear Algebra Lecture 34: Review for Test 2.
MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1
More informationA Brief Outline of Math 355
A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationLinear Algebra. Carleton DeTar February 27, 2017
Linear Algebra Carleton DeTar detar@physics.utah.edu February 27, 2017 This document provides some background for various course topics in linear algebra: solving linear systems, determinants, and finding
More informationALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA
ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationG1110 & 852G1 Numerical Linear Algebra
The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the
More informationNumerical Linear Algebra
Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix
More informationSolving a system by back-substitution, checking consistency of a system (no rows of the form
MATH 520 LEARNING OBJECTIVES SPRING 2017 BROWN UNIVERSITY SAMUEL S. WATSON Week 1 (23 Jan through 27 Jan) Definition of a system of linear equations, definition of a solution of a linear system, elementary
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationScientific Computing: Dense Linear Systems
Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationReview of linear algebra
Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for
More informationMATRICES ARE SIMILAR TO TRIANGULAR MATRICES
MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,
More informationEigenvalues and Eigenvectors
Sec. 6.1 Eigenvalues and Eigenvectors Linear transformations L : V V that go from a vector space to itself are often called linear operators. Many linear operators can be understood geometrically by identifying
More informationCalculating determinants for larger matrices
Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationOHSx XM511 Linear Algebra: Solutions to Online True/False Exercises
This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)
More informationChapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form
Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationRemark 1 By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.
Sec 5 Eigenvectors and Eigenvalues In this chapter, vector means column vector Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationSolutions to Final Exam
Solutions to Final Exam. Let A be a 3 5 matrix. Let b be a nonzero 5-vector. Assume that the nullity of A is. (a) What is the rank of A? 3 (b) Are the rows of A linearly independent? (c) Are the columns
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationLinear Algebra- Final Exam Review
Linear Algebra- Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.
More informationLinear Algebra Methods for Data Mining
Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 1. Basic Linear Algebra Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki Example
More informationLINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS
LINEAR ALGEBRA, -I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,
More informationELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices
ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex
More informationEigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems
More informationSYLLABUS. 1 Linear maps and matrices
Dr. K. Bellová Mathematics 2 (10-PHY-BIPMA2) SYLLABUS 1 Linear maps and matrices Operations with linear maps. Prop 1.1.1: 1) sum, scalar multiple, composition of linear maps are linear maps; 2) L(U, V
More informationExtra Problems for Math 2050 Linear Algebra I
Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More information[Disclaimer: This is not a complete list of everything you need to know, just some of the topics that gave people difficulty.]
Math 43 Review Notes [Disclaimer: This is not a complete list of everything you need to know, just some of the topics that gave people difficulty Dot Product If v (v, v, v 3 and w (w, w, w 3, then the
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationNOTES FOR LINEAR ALGEBRA 133
NOTES FOR LINEAR ALGEBRA 33 William J Anderson McGill University These are not official notes for Math 33 identical to the notes projected in class They are intended for Anderson s section 4, and are 2
More informationGRE Subject test preparation Spring 2016 Topic: Abstract Algebra, Linear Algebra, Number Theory.
GRE Subject test preparation Spring 2016 Topic: Abstract Algebra, Linear Algebra, Number Theory. Linear Algebra Standard matrix manipulation to compute the kernel, intersection of subspaces, column spaces,
More informationName: Final Exam MATH 3320
Name: Final Exam MATH 3320 Directions: Make sure to show all necessary work to receive full credit. If you need extra space please use the back of the sheet with appropriate labeling. (1) State the following
More informationQuestion: Given an n x n matrix A, how do we find its eigenvalues? Idea: Suppose c is an eigenvalue of A, then what is the determinant of A-cI?
Section 5. The Characteristic Polynomial Question: Given an n x n matrix A, how do we find its eigenvalues? Idea: Suppose c is an eigenvalue of A, then what is the determinant of A-cI? Property The eigenvalues
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More information