Solving Linear Systems of Equations
|
|
- Anna Hoover
- 5 years ago
- Views:
Transcription
1 November 6, 2013
2 Introduction The type of problems that we have to solve are: Solve the system: A x = B, where a 11 a 1N a 12 a 2N A =.. a 1N a NN x = x 1 x 2. x N B = b 1 b 2. b N To find A 1 (inverse of a matrix) The determinant of a matrix A The eigenvalues and eigenvectors of a matrix A
3 Gauss Method Let assume that we have to solve the linear system A x = B with det (A) 0. a 11 x 1 + a 12 x 2 + a 13 x a 1N x N = b 1 a 21 x 1 + a 22 x 2 + a 23 x a 2N x N = b 2 a N1 x 1 + a N2 x 2 + a N3 x a NN x N = b N. (1) We will try to transform it into an upper-triangular linear system. The steps are the following:
4 Gauss Method STEP 1 Multiply the 1st equation with a 21 /a 11 and substract it from the 2nd. Similarly, multiply the 1st with a 31 /a 11 and substract it from the 3rd etc a 11 x 1 + a 12 x 2 + a 13 x a 1N x N = b a (1) 22 x 2 + a (1) 23 x a (1) 2N x N = b (1) 2 where, for example, we set: 0 + a (1) N2 x 2 + a (1) N3 x a (1) NN x N = b (1) N.. (2). a (1) 22 = a 12a 21 a 11 a 22
5 Gauss Method STEP 2 The 1st row and the 1st column remain unchanged and we multiply the 2nd equation with a (1) 32 /a(1) 22 and substract it from the 3rd and so on. Thus after n 1 operations we get a 11 x 1 + a 12 x 2 + a 13 x a 1N x N = b a (1) 22 x 2 + a (1) 23 x a (1) 2N x N = b (1) a (2) 33 x a (2) 3N x N = b (2) 3 Where for example we have set: a (2) N3 x a (2) NN x N = b (2) N.. (3). a (2) 33 = a(1) 32 a(1) 33 a (1) 22 a (1) 33
6 Gauss Method After N 1 steps we get the following upper-triangular system: a 11 x 1 + a 12 x 2 + a 13 x a 1N x N = b 1 a (1) 22 x 2 + a (1) 23 x a (1) 2N x N = b (1) 2 a (2) 33 x a (2) 3N x N = b (2) 3. (4) a (N 1) NN x N = b (N 1) N For the Nth equation of the above system (5)we get : x N = b(n 1) N a (N 1) NN for a (N 1) NN 0 (5) while the rest of the values can be calculated via the relation: x i = b (i 1) i N k=i+1 a (i 1) ii a (i 1) ik x k for a (i 1) ii 0 (6) The no of arithmetic operations needed is: (4N 3 + 9N 2 7N)/6.
7 Determinant If a matrix is transformed into an upper-triangular or lower-triangular or diagonal form then det A = a 11 a 22 a 33 a NN = N a ii (7) i=1
8 Pivoting DEFINITION: The number a ii in the position (i, i) that is used to eliminate x i in rows i + 1, i + 2,...,N is called the ith pivotal element and the ith row is called the pivotal row. Pivoting to avoid a (i) ii = 0 : If a (i) ii = 0, row i cannot be used to eliminate, the elements in column i below the diagonal. It is neccesary to find a row j, where a (i) ji 0 and j > i and then interchange row i and j so that a nonzero pivot element is obtained. Pivoting to reduce error : If there is more than one nonzero element in column i that lies on or below the diagonal, there is a choice to determine which rows to interchange. Because, the computer uses fixed-precision arithmetic, it is possible that a small error is introduced each time an arithmetic operation is performed. One should check the magnitude of all the elements in column i that lie on below the diagonal, and locate a row j in which the element has the largest absolute value, that is a ji = max{ a ii, a i+1 i,..., a N 1 i, a N i }, and then switch row i with row j if j > i. Usually, the larger pivot element will result in a smaller error being propagated.
9 Pivoting : Example Let s assume that after a number of operations the last two equations of an N N system are 0x N 1 + x N = 1 and 2x N 1 + x N = 3 The obvious solution is x N 1 = x N = 1. But due to accumulation of numerical errors the system will be: ɛx N 1 + x N = 1 and 2x N 1 + x N = 3 ɛx N 1 + x N = 1 and (1 2/ɛ) x N = 3 2/ɛ x N = 3 2/ɛ 1 2/ɛ 1 correct, but x N 1 = 1 x N! ɛ I.e. x N 1 is the ratio of two small numbers with questionable accuracy. Then this erroneous term will be used for the estimation of x N 2,..., x 1 with disastrous results. With pivoting, the system will be written as: 2x N 1 + x N = 3 and ɛx N 1 + x N = 1 2x N 1 + x N = 3 and (1 ɛ/2) x N = 1 3ɛ/2 then x N 1.0 and x N 1 = 3 x N
10 Gauss - Jordan Method It is a variation of Gauss s method. Here instead of transforming the original system into an upper-triangular one we transform it into a diagonal one and then the solution is obvious. The method follows the steps of Gauss s method with parallel elimination of the elements over the diagonal. Thus after the 1st step of Gaussian elimination we multiply the 2nd equation with a 12 /a (1) 22 and substract it from the 1st. Thus the system becomes: a 11 x a (2) 13 x a (2) 1N x N = b (2) a (1) 22 x 2 + a (1) 23 x a (1) 2N x N = b (1) a (2) 33 x a (2) 3N x N = b (2) a (2) N3 x a (2) NN x N = b (2) N. (8) and in the next step we eliminate the terms with coefficients a (2) 13 anda(1) 23.
11 Gauss - Jordan Method Thus after N steps we get the system a 11 x 1 = b (N 1) 1 a (1) 22 x 2 = b (N 1) 2. (9) a (N 1) NN x N = b (N 1) N with obvious solution x i = b(n 1) i a (i 1) ii. (10)
12 The Jacobi Method (iterative) It is a generalization of the method x = g (x) presented in the previous section. Let s assume the following system of N linear equations with N unknowns: f 1 (x 1, x 2,..., x N ) = 0 f 2 (x 1, x 2,..., x N ) = 0 f n (x 1, x 2,..., x N ) = 0 this can be easily written in the form: or in a close form: x 1 = g 1 (x 2, x 3,..., x N ) x 2 = g 2 (x 1, x 3,..., x N )... (11)... (12) x N = g N (x 1, x 2,..., x N 1 ) x i = b i a ii 1 a ii N j=1, j i a ij x j (13)
13 The Jacobi Method (iterative) Then by giving N initial values x (0) 1, x (0) 2,..., x (0) N recurrence relation x (k+1) i which will converge to the solution of the system if:, we create the = g i (x (k) 1,..., x (k) N ). (14) a ii > N j=1, j i a ij (15) independent on the choice of the initial values x (0) 1, x (0) 2,..., x (0) N. The recurrence relation can be written in a matrix form as: x (k+1) = D 1 B D 1 Cx (k) where A = D + C i.e. the matrix D includes only the diagonal elements of A and the matrix C all the rest.
14 Jacobi Method : Example 4x y + z = 7 4x 8y + z = 21 2x + y + 5z = 15 we can create the recurrence relations: wih solutions x = 2, y = 4, z = 3 x (k+1) = 7 + y (k) z (k), y (k+1) = x (k) + z (k) 4 8 z (k+1) = x (k) y (k) 5 then assuming (1, 2, 2) we get the following sequence of solutions (1, 2, 2) (1.75, 3.375, 3) (1.844, 3.875, 3.025) (1.963, 3.925, 2.963) (1.991, 3.977, 3.0) (1.994, 3.995, 3.001) I.e. with 5 iterations we reached the solution with 3 digits accuracy.
15 Gauss - Seidel Method Variation of Jacobi s method. Let s assume an trial solution x (0) i then from the 1st equation we estimate x (1) 1 and in the 2nd equation we use the following set of trial values (x (1) 1, x (0) 2, x (0) 3 Then we use the trial values (x (1) 1, x (1) 2, x (0) 3 so on. The recurrence relations will be: x (k+1) 1 = 1 N b 1 a 1j x (k) j a 11 x (k+1) 2 = 1 a 22 x (k+1) i = 1 a ii j=2 b 2 a 21 x (k+1) 1 i 1 b i j=1 a ij x (k+1) j,..., x (0) N,..., x (0) N (1) ) to estimate x 2. (1) ) to estimate x 3 and N a 2j x (k) j=3 N j=i+1 j a ij x (k) (16) j
16 Gauss - Seidel Method The method will converge if: a ii > N j=1, j i a ij i = 1, 2,..., N (17) This procedure in a matrix form will be written as: where x (k+1) = D 1 ( B Lx (k+1) Ux (k)) (18) A = L + lower D diagonal + U upper. The matrix L has the elements of below the diagonal A, the matrix D only the diagonal elements of A and finally the matrix U the elements of matrix A over the diagonal.
17 Gauss - Seidel : Example The recurrence relations for the previous example are the same i.e. x (k+1) = 7 + y (k) z (k) 4 y (k+1) = x (k) + z (k) 8 z (k+1) = x (k) y (k) 5 But with Gauss - Seidel method we get the following sequence of approximate solutions: (1, 2, 2) (1.75, 3.75, 2, 95) (1.95, 3.97, 2.99) (1.996, 3.996, 2.999) i.e. here we need only 3 iterations to find the solution while with Jacobi s method we need 5 iterations.
18 Inverse of a matrix ( ) a b If we are seeking the inverse of the matrix A = this will be ( ) c d x y another matrix B = with the following property: z w ( ) ( ) ( ) a b x y 1 0 A B = = = I c d z w 0 1 In order to find the elements (x, y, z, w) of the inverse matrix we must solve two linear systems: ( ) ( ) ( ) ( ) ( ) ( ) a b x 1 a b y 0 = and = c d z 0 c d w 1 This means that the inverse of a matrix cannot help in the solution of a linear system since one needs to solve the system to find it. Notice that in order to solve both of the above systems we need to diagonalize (with the Gauss - Jordan procedure) the same matrix A. The only difference between the two systems is the vector on the right hand side which corresponds to different column of the unit matrix.
19 Inverse of a matrix We then construct the following scheme: a 11 a a 1N a 21 a a 2N a N1 a N2... a NN A I (19) and whatever transformation we do in the left side (matrix A) exactly the same we do on the right hand side (matrix I) ã 11 ã ã 1N ã 21 ã ã 2N (20) ã N1 ã N2... ã NN Thanks to the Gauss-Jordan method on the left hand side we reduce to the unit matrix matrix I and on the right hand side on the inverse ã ij = A 1.
20 Eigenvalues and Eigenvectors If A is a N N matrix then the scalars, λ for which there exists a non-zero vector x such that A x = λ x (21) will be called eigenvalues of the matrix A and the vectors x eigenvectors. In the following example: = A u 1 λ 1 u 1 the vector u 1 = (2, 1, 2) is the e-vector and λ 1 = 1 the corresponding e-value of A. Equation (21) is equivalent to: det (A λi) = 0 that is det 1 λ λ λ = λ3 5λ 2 + 3λ + 9 = 0 (22) and the e-values will be the roots of the characteristic polynomial (here λ i = 1, 3 and 3).
21 Eigenvalues and Eigenvectors : The power method For a matrix A there will always be a dominant e-value λ 1 (this means that λ 1 is absolutely larger than any other e-value) λ 1 > λ 2 λ 3... λ N (23) and also any vector x can be written as a linear combination of the N e-vectors { u (1), u (2),..., u (N)} i.e. then A u (i) = λ i u (i) (1 i N) (24) x = a 1 u (1) + a 2 u (2) + + a N u (N) (25) If we multiply both sides of (25) A, we get: x (1) A x = a 1 λ 1 u (1) + a 2 λ 2 u (2) + + a N λ N u (N) (26) If we multiply k times eqn (26) with the matrix A we get: x (k) A k x = a 1 λ k 1u (1) + a 2 λ k 2u (2) + + a N λ k Nu (N) ( ( ) k ( ) ) k = λ k 1 a 1 u (1) λ2 + a 2 u (2) λn + + a N u (N) (27) λ 1 λ 1
22 Since λ 1 is the absolute larger e-value (eqn 23), (λ j /λ 1 ) k 0 as k. Thus x (k) = A k x λ k 1 a 1 u (1) (28) then the ratio r (k) x (k+1) x (k) = Ak+1 x A k x λk+1 1 a 1 u (1) λ k 1 a λ 1 u (1) 1 (29) leads to the e-value λ 1 and the vector defined in (28) is the corresponding e-vector.
23 Eigenvalues and Eigenvectors : Example The matrix A = has e-values: λ 1 = , λ 2 = 2 & λ 3 = and corresponding e-vectors : u (1) = (0.3694, 1, ), u (2) = (0, 1, 0) & u (3) = (0.7735, 1, ). Then if we assume a trial vector x = (1, 2, 1), and multiply it with the 5th and 6th power of A A 5 = A 6 = we get : x (5) = A 5 x = (232, 628, 560) & x (6) = A 6 x = (792, 2144, 1912). This leads to λ 1 x (6) 792 = x (5) and the e-vector u (1) x (6) (0.3694, 1, ).
24 Eigenvalues and Eigenvectors : Aitken Acceleration The elements of the vector r (k) in eqn (29) are approximations to the exact e-value λ 1 and the error will be ɛ k = r k λ 1 where r is any element of r (k). Obviously for r (k+1) we get ɛ k+1 = r k+1 λ 1. Since the convergence of the method is linear:, i.e. ɛ k+1 = A ɛ k (30) we can use Aitken s method to accelerate the convergence. That is, we will use the formula derived in the previous section for 3 approximate values: r k, r k+1 and r k+2 to find a better approximation to the e-value i.e. λ 1 r k r k+2 r 2 k+1 r k+2 2 r k+1 + r k. (31) Application: From the example of the previous slide r 5 = 3.414, r 4 = and r 3 = Then Aitken s method gives λ
25 Eigenvalues and Eigenvectors : Inverse Power Method Theorem: If λ is an e-value of a matrix A, then λ 1 is an e-value of the A 1. Thus if λ 1 > λ 2 λ N 1 > λ N > 0, are the e-values of A then λ 1 > λ 1 λ 1 > 0 (32) N N 1 will be the e-values of A 1. Thus λ 1 N is the largest e-value of A 1. To find the e-value instead of calculating the inverse of the matrix A to get the expression ( A 1) (k+1) x we can solve by using Gaussian elimination the system A x (k+1) = x (k), where x (k) is defined as x (k) = A k x. That is for a initial x (0), we get x (1) = A 1 x (0) i.e. Ax (1) = x (0). Thus the solution of the linear system defines the value of x (1). With this procedure we get the vector x (k+1) via x (k+1) = A 1 x (k) A x (k+1) = x (k) (33) 1
26 Eigenvalues and Eigenvectors : Shifting Power Method Theorem: If the N values λ i with (i = 1,..., N) are the e-values of a N N matrix A, then for any complex number µ the matrix A µ I (where I is the unit matrix) will have as e-values the complex numbers (λ i µ) for (i = 1,... N). Thus if in the interval (λ N, λ 1 ) we consider a point µ such that for an e-value λ k to hold 0 < λ k µ < ɛ while for all the rest we get λ i µ > ɛ; then λ k µ will be the minimum e-value of A µ I. This means that we can estimate it by using the inverse power method Remember that you need to calculate x (k+1) by solving the system: ( A µ I ) x (k+1) = x (k) Thus if we calculate the r k, the actual e-value λ i will be: λ i = 1 r k + µ. (34)
Eigenvalues and Eigenvectors
5 Eigenvalues and Eigenvectors 5.2 THE CHARACTERISTIC EQUATION DETERMINANATS nn Let A be an matrix, let U be any echelon form obtained from A by row replacements and row interchanges (without scaling),
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationEigenvalues and Eigenvectors
5 Eigenvalues and Eigenvectors 5.2 THE CHARACTERISTIC EQUATION DETERMINANATS n n Let A be an matrix, let U be any echelon form obtained from A by row replacements and row interchanges (without scaling),
More informationLU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationTMA4125 Matematikk 4N Spring 2017
Norwegian University of Science and Technology Institutt for matematiske fag TMA15 Matematikk N Spring 17 Solutions to exercise set 1 1 We begin by writing the system as the augmented matrix.139.38.3 6.
More information10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method
54 CHAPTER 10 NUMERICAL METHODS 10. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More informationCPE 310: Numerical Analysis for Engineers
CPE 310: Numerical Analysis for Engineers Chapter 2: Solving Sets of Equations Ahmed Tamrawi Copyright notice: care has been taken to use only those web images deemed by the instructor to be in the public
More informationMATH 3511 Lecture 1. Solving Linear Systems 1
MATH 3511 Lecture 1 Solving Linear Systems 1 Dmitriy Leykekhman Spring 2012 Goals Review of basic linear algebra Solution of simple linear systems Gaussian elimination D Leykekhman - MATH 3511 Introduction
More informationMatrix notation. A nm : n m : size of the matrix. m : no of columns, n: no of rows. Row matrix n=1 [b 1, b 2, b 3,. b m ] Column matrix m=1
Matrix notation A nm : n m : size of the matrix m : no of columns, n: no of rows Row matrix n=1 [b 1, b 2, b 3,. b m ] Column matrix m=1 n = m square matrix Symmetric matrix Upper triangular matrix: matrix
More informationGauss-Seidel method. Dr. Motilal Panigrahi. Dr. Motilal Panigrahi, Nirma University
Gauss-Seidel method Dr. Motilal Panigrahi Solving system of linear equations We discussed Gaussian elimination with partial pivoting Gaussian elimination was an exact method or closed method Now we will
More informationMath 313 Chapter 1 Review
Math 313 Chapter 1 Review Howard Anton, 9th Edition May 2010 Do NOT write on me! Contents 1 1.1 Introduction to Systems of Linear Equations 2 2 1.2 Gaussian Elimination 3 3 1.3 Matrices and Matrix Operations
More informationDepartment of Aerospace Engineering AE602 Mathematics for Aerospace Engineers Assignment No. 4
Department of Aerospace Engineering AE6 Mathematics for Aerospace Engineers Assignment No.. Decide whether or not the following vectors are linearly independent, by solving c v + c v + c 3 v 3 + c v :
More information1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
More informationDepartment of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015
Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two
More informationMATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018
Homework #1 Assigned: August 20, 2018 Review the following subjects involving systems of equations and matrices from Calculus II. Linear systems of equations Converting systems to matrix form Pivot entry
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationGaussian Elimination without/with Pivoting and Cholesky Decomposition
Gaussian Elimination without/with Pivoting and Cholesky Decomposition Gaussian Elimination WITHOUT pivoting Notation: For a matrix A R n n we define for k {,,n} the leading principal submatrix a a k A
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationLinear Equations and Matrix
1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
More informationDirect Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
More informationMAC Module 3 Determinants. Learning Objectives. Upon completing this module, you should be able to:
MAC 2 Module Determinants Learning Objectives Upon completing this module, you should be able to:. Determine the minor, cofactor, and adjoint of a matrix. 2. Evaluate the determinant of a matrix by cofactor
More informationDirect Methods for Solving Linear Systems. Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le
Direct Methods for Solving Linear Systems Simon Fraser University Surrey Campus MACM 316 Spring 2005 Instructor: Ha Le 1 Overview General Linear Systems Gaussian Elimination Triangular Systems The LU Factorization
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationChapter 3. Linear and Nonlinear Systems
59 An expert is someone who knows some of the worst mistakes that can be made in his subject, and how to avoid them Werner Heisenberg (1901-1976) Chapter 3 Linear and Nonlinear Systems In this chapter
More informationSystem of Linear Equations
Chapter 7 - S&B Gaussian and Gauss-Jordan Elimination We will study systems of linear equations by describing techniques for solving such systems. The preferred solution technique- Gaussian elimination-
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationLemma 8: Suppose the N by N matrix A has the following block upper triangular form:
17 4 Determinants and the Inverse of a Square Matrix In this section, we are going to use our knowledge of determinants and their properties to derive an explicit formula for the inverse of a square matrix
More informationChapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
More informationThe Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
More informationLinear Algebraic Equations
Linear Algebraic Equations Linear Equations: a + a + a + a +... + a = c 11 1 12 2 13 3 14 4 1n n 1 a + a + a + a +... + a = c 21 2 2 23 3 24 4 2n n 2 a + a + a + a +... + a = c 31 1 32 2 33 3 34 4 3n n
More informationChapter 4. Solving Systems of Equations. Chapter 4
Solving Systems of Equations 3 Scenarios for Solutions There are three general situations we may find ourselves in when attempting to solve systems of equations: 1 The system could have one unique solution.
More informationIntroduction to PDEs and Numerical Methods Lecture 7. Solving linear systems
Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Dr. Noemi Friedman, 09.2.205. Reminder: Instationary heat
More informationLINEAR SYSTEMS (11) Intensive Computation
LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY
More informationIterative Methods. Splitting Methods
Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition
More informationJACOBI S ITERATION METHOD
ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes
More informationANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3
ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ISSUED 24 FEBRUARY 2018 1 Gaussian elimination Let A be an (m n)-matrix Consider the following row operations on A (1) Swap the positions any
More informationMATRICES. a m,1 a m,n A =
MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of
More informationGAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511)
GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511) D. ARAPURA Gaussian elimination is the go to method for all basic linear classes including this one. We go summarize the main ideas. 1.
More informationElementary Linear Algebra
Matrices J MUSCAT Elementary Linear Algebra Matrices Definition Dr J Muscat 2002 A matrix is a rectangular array of numbers, arranged in rows and columns a a 2 a 3 a n a 2 a 22 a 23 a 2n A = a m a mn We
More information. =. a i1 x 1 + a i2 x 2 + a in x n = b i. a 11 a 12 a 1n a 21 a 22 a 1n. i1 a i2 a in
Vectors and Matrices Continued Remember that our goal is to write a system of algebraic equations as a matrix equation. Suppose we have the n linear algebraic equations a x + a 2 x 2 + a n x n = b a 2
More informationLinear Algebra: Lecture Notes. Dr Rachel Quinlan School of Mathematics, Statistics and Applied Mathematics NUI Galway
Linear Algebra: Lecture Notes Dr Rachel Quinlan School of Mathematics, Statistics and Applied Mathematics NUI Galway November 6, 23 Contents Systems of Linear Equations 2 Introduction 2 2 Elementary Row
More informationNumerical Methods for Chemical Engineers
Numerical Methods for Chemical Engineers Chapter 3: System of Linear Algebraic Equation Morteza Esfandyari Email: Esfandyari.morteza@yahoo.com Mesfandyari.mihanblog.com Page 4-1 System of Linear Algebraic
More informationDefinition (T -invariant subspace) Example. Example
Eigenvalues, Eigenvectors, Similarity, and Diagonalization We now turn our attention to linear transformations of the form T : V V. To better understand the effect of T on the vector space V, we begin
More informationNUMERICAL METHODS C. Carl Gustav Jacob Jacobi 10.1 GAUSSIAN ELIMINATION WITH PARTIAL PIVOTING
. Gaussian Elimination with Partial Pivoting. Iterative Methods for Solving Linear Systems.3 Power Method for Approximating Eigenvalues.4 Applications of Numerical Methods Carl Gustav Jacob Jacobi 8 4
More informationDeterminants by Cofactor Expansion (III)
Determinants by Cofactor Expansion (III) Comment: (Reminder) If A is an n n matrix, then the determinant of A can be computed as a cofactor expansion along the jth column det(a) = a1j C1j + a2j C2j +...
More informationCOURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems.
COURSE 9 4 Numerical methods for solving linear systems Practical solving of many problems eventually leads to solving linear systems Classification of the methods: - direct methods - with low number of
More informationCHAPTER 6. Direct Methods for Solving Linear Systems
CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to
More informationGaussian Elimination and Back Substitution
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 4 Notes These notes correspond to Sections 31 and 32 in the text Gaussian Elimination and Back Substitution The basic idea behind methods for solving
More informationA Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
More informationMatrices and systems of linear equations
Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.
More informationSystems of Linear Equations
Systems of Linear Equations Last time, we found that solving equations such as Poisson s equation or Laplace s equation on a grid is equivalent to solving a system of linear equations. There are many other
More informationLinear Algebra. Solving Linear Systems. Copyright 2005, W.R. Winfrey
Copyright 2005, W.R. Winfrey Topics Preliminaries Echelon Form of a Matrix Elementary Matrices; Finding A -1 Equivalent Matrices LU-Factorization Topics Preliminaries Echelon Form of a Matrix Elementary
More informationChapter 12: Iterative Methods
ES 40: Scientific and Engineering Computation. Uchechukwu Ofoegbu Temple University Chapter : Iterative Methods ES 40: Scientific and Engineering Computation. Gauss-Seidel Method The Gauss-Seidel method
More informationMidterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015
Midterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015 The test lasts 1 hour and 15 minutes. No documents are allowed. The use of a calculator, cell phone or other equivalent electronic
More informationMath Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationSimultaneous Linear Equations
Simultaneous Linear Equations PHYSICAL PROBLEMS Truss Problem Pressure vessel problem a a b c b Polynomial Regression We are to fit the data to the polynomial regression model Simultaneous Linear Equations
More informationStudy Guide for Linear Algebra Exam 2
Study Guide for Linear Algebra Exam 2 Term Vector Space Definition A Vector Space is a nonempty set V of objects, on which are defined two operations, called addition and multiplication by scalars (real
More informationAMS 209, Fall 2015 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems
AMS 209, Fall 205 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems. Overview We are interested in solving a well-defined linear system given
More informationChapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form
Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1
More informationMAC Module 2 Systems of Linear Equations and Matrices II. Learning Objectives. Upon completing this module, you should be able to :
MAC 0 Module Systems of Linear Equations and Matrices II Learning Objectives Upon completing this module, you should be able to :. Find the inverse of a square matrix.. Determine whether a matrix is invertible..
More informationNumerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
More informationMATH 310, REVIEW SHEET 2
MATH 310, REVIEW SHEET 2 These notes are a very short summary of the key topics in the book (and follow the book pretty closely). You should be familiar with everything on here, but it s not comprehensive,
More informationHere is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J
Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.
More informationLinear Algebra. James Je Heon Kim
Linear lgebra James Je Heon Kim (jjk9columbia.edu) If you are unfamiliar with linear or matrix algebra, you will nd that it is very di erent from basic algebra or calculus. For the duration of this session,
More informationProcess Model Formulation and Solution, 3E4
Process Model Formulation and Solution, 3E4 Section B: Linear Algebraic Equations Instructor: Kevin Dunn dunnkg@mcmasterca Department of Chemical Engineering Course notes: Dr Benoît Chachuat 06 October
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS We want to solve the linear system a, x + + a,n x n = b a n, x + + a n,n x n = b n This will be done by the method used in beginning algebra, by successively eliminating unknowns
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationMAC1105-College Algebra. Chapter 5-Systems of Equations & Matrices
MAC05-College Algebra Chapter 5-Systems of Equations & Matrices 5. Systems of Equations in Two Variables Solving Systems of Two Linear Equations/ Two-Variable Linear Equations A system of equations is
More informationApplied Linear Algebra
Applied Linear Algebra Gábor P. Nagy and Viktor Vígh University of Szeged Bolyai Institute Winter 2014 1 / 262 Table of contents I 1 Introduction, review Complex numbers Vectors and matrices Determinants
More informationMIDTERM 1 - SOLUTIONS
MIDTERM - SOLUTIONS MATH 254 - SUMMER 2002 - KUNIYUKI CHAPTERS, 2, GRADED OUT OF 75 POINTS 2 50 POINTS TOTAL ) Use either Gaussian elimination with back-substitution or Gauss-Jordan elimination to solve
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More information5.7 Cramer's Rule 1. Using Determinants to Solve Systems Assumes the system of two equations in two unknowns
5.7 Cramer's Rule 1. Using Determinants to Solve Systems Assumes the system of two equations in two unknowns (1) possesses the solution and provided that.. The numerators and denominators are recognized
More informationCS412: Lecture #17. Mridul Aanjaneya. March 19, 2015
CS: Lecture #7 Mridul Aanjaneya March 9, 5 Solving linear systems of equations Consider a lower triangular matrix L: l l l L = l 3 l 3 l 33 l n l nn A procedure similar to that for upper triangular systems
More information2.1 Gaussian Elimination
2. Gaussian Elimination A common problem encountered in numerical models is the one in which there are n equations and n unknowns. The following is a description of the Gaussian elimination method for
More informationII. Determinant Functions
Supplemental Materials for EE203001 Students II Determinant Functions Chung-Chin Lu Department of Electrical Engineering National Tsing Hua University May 22, 2003 1 Three Axioms for a Determinant Function
More informationMTH50 Spring 07 HW Assignment 7 {From [FIS0]}: Sec 44 #4a h 6; Sec 5 #ad ac 4ae 4 7 The due date for this assignment is 04/05/7 Sec 44 #4a h Evaluate the erminant of the following matrices by any legitimate
More informationLinear System of Equations
Linear System of Equations Linear systems are perhaps the most widely applied numerical procedures when real-world situation are to be simulated. Example: computing the forces in a TRUSS. F F 5. 77F F.
More informationMatrices and Linear Algebra
Contents Quantitative methods for Economics and Business University of Ferrara Academic year 2017-2018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2
More informationLinear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
More informationCHAPTER 5. Basic Iterative Methods
Basic Iterative Methods CHAPTER 5 Solve Ax = f where A is large and sparse (and nonsingular. Let A be split as A = M N in which M is nonsingular, and solving systems of the form Mz = r is much easier than
More information(b) If a multiple of one row of A is added to another row to produce B then det(b) =det(a).
.(5pts) Let B = 5 5. Compute det(b). (a) (b) (c) 6 (d) (e) 6.(5pts) Determine which statement is not always true for n n matrices A and B. (a) If two rows of A are interchanged to produce B, then det(b)
More informationDEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationMAT 610: Numerical Linear Algebra. James V. Lambers
MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationBoundary Value Problems - Solving 3-D Finite-Difference problems Jacob White
Introduction to Simulation - Lecture 2 Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Reminder about
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 2: Direct Methods PD Dr.
More informationMAC Module 1 Systems of Linear Equations and Matrices I
MAC 2103 Module 1 Systems of Linear Equations and Matrices I 1 Learning Objectives Upon completing this module, you should be able to: 1. Represent a system of linear equations as an augmented matrix.
More information1 Introduction. 2 Determining what the J i blocks look like. December 6, 2006
Jordan Canonical Forms December 6, 2006 1 Introduction We know that not every n n matrix A can be diagonalized However, it turns out that we can always put matrices A into something called Jordan Canonical
More informationTHE EIGENVALUE PROBLEM
THE EIGENVALUE PROBLEM Let A be an n n square matrix. If there is a number λ and a column vector v 0 for which Av = λv then we say λ is an eigenvalue of A and v is an associated eigenvector. Note that
More informationMath 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationSection 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
More information1 Number Systems and Errors 1
Contents 1 Number Systems and Errors 1 1.1 Introduction................................ 1 1.2 Number Representation and Base of Numbers............. 1 1.2.1 Normalized Floating-point Representation...........
More informationLinear Algebra. Carleton DeTar February 27, 2017
Linear Algebra Carleton DeTar detar@physics.utah.edu February 27, 2017 This document provides some background for various course topics in linear algebra: solving linear systems, determinants, and finding
More information