Notes on vectors and matrices

Size: px
Start display at page:

Download "Notes on vectors and matrices"

Transcription

1 Notes on vectors and matrices EE103 Winter Quarter L Vandenberghe 1 Terminology and notation Matrices, vectors, and scalars A matrix is a rectangular array of numbers (also called scalars), written between brackets, as in A An important attribute of a matrix is its size or dimensions, ie, thenumbersofrows and columns The matrix A above, for example, has 3 rows and 4 columns, so its size is 3 4 (Size is always given as rows columns) The entries or coefficients or elements of a matrix are the values in the array The i, j entry is the value in the ith row and jth column, denoted by double subscripts: the i, j entry ofamatrixc is denoted C ij (which is a number) The positive integers i and j are called the (row and column, respectively) indices For our example above, A 13 23, A 32 1 The row index of the bottom left entry (which has value 41)is 3; its column index is 1 A matrix with only one column, ie, with size n 1, is called a column vector or just a vector Sometimes the size is specified by calling it an n-vector The entries of a vector are denoted with just one subscript (since the other is 1), as in a 3 The entries are sometimes called the components of the vector As an example, v is a 4-vector (or 4 1 matrix); its third component is v 3 33 We sometimes write a vector by listing its components between parentheses and separated by commas, as in v (1, 2, 33, 03) It is important to keep in mind that this denotes the same column vector as defined above BasedontheMatrix Primer by S Boyd (Stanford University) 1

2 A matrix with only one row, ie, with size 1 n, is sometimes called a row vector As an example, w is a row vector (or 1 3 matrix); its third component is w 3 0 Sometimes a 1 1 matrix is considered to be the same as a scalar, ie, anumber Two matrices are equal if they are the same size and all the corresponding entries (which are numbers)are equal The set of all m n matrices is denoted R m n, and the set of all n-vectors is denoted R n In other words, A R m n means the same as A is a matrix with size m n, and x R n means x is an n-vector Notational conventions Some authors try to use notation that helps the reader distinguish between matrices, vectors, and scalars (numbers) For example, Greek letters (α, β, )might be used for numbers, lower-case letters (a, x, f, )for vectors, and capital letters (A, F, )for matrices Other notational conventions include matrices given in bold font (G), or vectors written with arrows above them ( a) Unfortunately, there are about as many notational conventions as authors, so you should be prepared to figure out what things are (ie, scalars, vectors, matrices)despite the author s notational scheme (if any exists) Zero and identity matrices The zero matrix (of size m n)is the matrix with all entries equal to zero Sometimes the zero matrix is written as 0 m n, where the subscript denotes the size But often, a zero matrix is denoted just 0, the same symbol used to denote the number 0 In this case you ll have to figure out the size of the zero matrix from the context (More on this later) Note that zero matrices of different sizes are different matrices, even though we use the same symbol to denote them (ie, 0) In programming we call this overloading: wesaythe symbol 0 is overloaded because it can mean different things depending on its context (ie, the equation it appears in) When a zero matrix is a (row or column)vector, we call it a zero (row or column)vector An identity matrix is another common matrix It is always square, ie, has the same number of rows as columns Its diagonal entries, ie, those with equal row and column index, are all equal to one, and its off-diagonal entries (those with unequal row and column indices) are zero Identity matrices are denoted by the letter I Sometimes a subscript denotes the size, as in I 4 or I 2 2 But more often the size must be determined from context (just like zero matrices) Formally, the identity matrix of size n is defined by { 1 i j I ij 0 i j 2

3 Perhaps more illuminating are the examples , which are the 2 2and4 4 identity matrices (Remember that both are denoted with the same symbol I) The importance of the identity matrix will become clear later 2 Matrix operations Matrices can be combined in various operations to form other matrices Matrix transpose If A is an m n matrix, its transpose, denoted A T (or sometimes A ),isthen m matrix given by ( A T ) ij A ji In words, the rows and columns of A are transposed in A T For example, T If we transpose a matrix twice, we get back the original matrix: ( A T ) T A Matrix addition Two matrices of the same size can be added together, to form another matrix (of the same size), by adding the corresponding entries (which are numbers) Matrix addition is denoted by the symbol + (Thus the symbol + is overloaded to mean scalar addition when scalars appear on its left and right hand side, and matrix addition when matrices appear on its left and right hand sides)for example, Note that (row or column)vectors of the same size can be added, but you cannot add together a row vector and a column vector (except if they are both scalars!) Matrix subtraction is similar As an example, I

4 Note that this gives an example where we have to figure out what size the identity matrix is Since you can only add (or subtract)matrices of the same size, we conclude that I must refer to a 2 2 identity matrix Matrix addition is commutative, ie, ifa and B are matrices of the same size, then A + B B + A It s also associative, ie, (A + B)+C A +(B + C), so we write both as A + B + C We always have A +00+A A, ie, adding the zero matrix to a matrix has no effect Scalar multiplication Another operation is scalar multiplication: multiplying a matrix by a scalar (ie, number), which is done by multiplying every entry of the matrix by the scalar Scalar multiplication is usually denoted by juxtaposition, with the scalar on the left, as in ( 2) Scalar multiplication obeys several laws you can figure out for yourself, eg, if A is any matrix and α, β are any scalars, then (α + β)a αa + βa It s useful to identify the symbols appearing in this formula The + symbol on the left is addition of scalars, while the + symbol on the right denotes matrix addition Another simple property is (αβ)a (α)(βa), where α and β are scalars and A is a matrix On the left hand side we see scalar-scalar multiplication (αβ)and scalar-matrix multiplication; on the right we see two cases of scalar-matrix multiplication Note that 0 A 0 (where the lefthand zero is the scalar zero, and the righthand zero is a matrix zero of the same size as A) Matrix multiplication It s also possible to multiply two matrices using matrix multiplication You can multiply two matrices A and B provided their dimensions are compatible, which means the number of columns of A equals the number of rows of B Suppose A and B are compatible, ie, A has size m p and B has size p n The product matrix C AB, which has size m n, is defined by p C ij A ik B kj A i1 B 1j + + A ip B pj, i 1,,m, j 1,,n k1 This rule looks complicated, but there are several ways to remember it To find the i, j entry of the product C AB, you need to know the ith row of A and the jth column of B The summation above can be interpreted as moving left to right along the i row of A while moving top to bottom down the jth column of B As you go, you keep a running sum of the product of entries: one from A and one from B 4

5 Now we can explain why the identity is called the identity: if A is any m n matrix, then AI A and IA A, ie, when you multiply a matrix by an identity matrix, it has no effect (The identity matrices in the formulas AI A and IA A have different sizes what are they?) One very important fact about matrix multiplication is that it is (in general) not commutative: wedon t (in general)have AB BA In fact, BA may not even make sense, or, if it makes sense, may be a different size than AB (so that equality in AB BA is meaningless) For example, if A is 2 3andB is 3 4, then AB makes sense (the dimensions are compatible)but BA doesn t even make sense (much less equal AB) Even when AB and BA both make sense and are the same size, ie, whena and B are square, we don t (in general)have AB BA As a simple example, consider: , Matrix multiplication is associative, ie, (AB)C A(BC)(provided the products make sense) Therefore we write the product simply as ABC Matrix multiplication is also associative with scalar multiplication, ie, α(ab) (αa)b, where α is a scalar and A and B are matrices (that can be multiplied) Matrix multiplication distributes across matrix addition: A(B + C) AB + AC and (A + B)C AC + BC Matrix-vector product A very important and common case of matrix multiplication is y Ax, wherea is an m n matrix, x is an n-vector, and y is an m-vector We can think of matrix vector multiplication (with an m n matrix)as a function that transforms n-vectors into m-vectors The formula is y i A i1 x A in x n, i 1,,m Inner product Another important special case of matrix multiplication occurs when v is a row vector and w is a column vector (of the same size) Then vw makes sense, and has size 1 1, ie, isa scalar: vw v 1 w v n w n This occurs often in the form x T y where x and y are both n-vectors In this case the product (which is a number)is called the inner product or dot product of the vectors x and y Other notation for the inner product is x, y or x y The inner product of a vector with itself is the square of its (Euclidean)norm x : Matrix powers x x x x 2 n x T x When a matrix A is square, then it makes sense to multiply A by itself, ie, to form A A We refer to this matrix as A 2 Similarly, k copies of A multiplied together is denoted A k 5

6 (Non-integer powers, such as A 1/2 (the matrix squareroot), are pretty tricky they might not make sense, or be ambiguous, unless certain conditions on A hold This is an advanced topic in linear algebra) By convention we set A 0 I Useful identities We ve already mentioned a handful of matrix identities, that you could figure out yourself, eg, A +0A Here we list a few others, that are not hard to derive, and quite useful transpose of product: (AB) T B T A T transpose of sum: (A + B) T A T + B T products of powers: A k A l A k+l (for k, l 1 in general, and for all k, l if A is invertible) Block matrices and submatrices In some applications it s useful to form matrices whose entries are themselves matrices, as in F I A B C,, 0 G where A, B, C, F,andGare matrices (as are 0 and I) Such matrices are called block matrices; the entries A, B, etc are called blocks and are sometimes named by indices Thus, F is called the 1, 1 block of the second matrix Of course the block matrices must have the right dimensions to be able to fit together: matrices in the same (block)row must have the same number of rows (ie, the same height ); matrices in the same (block)column must have the same number of columns (ie, thesame width ) Thus in the examples above, A, B and C musthavethesamenumberofrows(eg, they could be 2 3, 2 2, and 2 1) The second example is more interesting Suppose that F is m n Then the identity matrix in the 2, 2 position must have size m m (since it must have the same number of rows as F ) We also see that G must have m columns, say, dimensions p m That fixes the dimensions of the 0 matrix in the 2, 1block itmustbe p m You can also divide a larger matrix (or vector)into blocks In this context the blocks are sometimes called submatrices of the big matrix As a specific example, suppose that C, D Then we have D C

7 It s often useful to write an m n matrix as a 1 n block matrix of m-vectors (which are just its columns), or as an m 1 block matrix of n-row-vectors (which are its rows) Block matrices can be added and multiplied as if the entries were numbers, provided the corresponding entries have the right sizes (ie, conform )and you re careful about the order of multiplication Thus we have A B C D X Y AX + BY CX + DY provided the products AX, BY, CX,andDY makes sense 3 Cost of basic matrix operations Complexity analysis via flop count The cost of a numerical linear algebra algorithm is often expressed by giving the total number of floating-point operations or flops required to carry it out, as a function of various problem dimensions We define a flop as one addition, subtraction, multiplication, or division of two floating-point numbers (Some authors define a flop as one multiplication followed by one addition, so their flop counts are smaller by a factor up to two)to evaluate the complexity of an algorithm, we count the total number of flops, express it as a function (usually a polynomial)of the dimensions of the matrices and vectors involved, and simplify the expression by ignoring all terms except the leading (ie, highest order or dominant) terms As an example, suppose that a particular algorithm requires a total of m 3 +3m 2 n + mn +4mn 2 +5m +22 flops, where m and n are problem dimensions We would normally simplify this flop count to m 3 +3m 2 n +4mn 2 flops, since these are the leading terms in the problem dimensions m and n If in addition we assumed that m n, we would further simplify the flop count to 4mn 2 Flop counts were originally popularized when floating-point operations were relatively slow, so counting the number gave a good estimate of the total computation time This is no longer the case: Issues such as cache boundaries and locality of reference can dramatically affect the computation time of a numerical algorithm However, flop counts can still give us a good rough estimate of the computation time of a numerical algorithm, and how the time grows with increasing problem size Since a flop count no longer accurately predicts the computation time of an algorithm, we usually pay most attention to its order or orders, ie, its largest exponents, and ignore differences in flop counts smaller than a factor of two or so For example, an algorithm with flop count 5n 2 is considered comparable to one with aflopcount4n 2, but faster than an algorithm with flop count (1/3)n 3 7

8 Vector operations To compute the inner product x T y of two vectors x, y R n we form the products x i y i, and then add them, which requires n multiplies and n 1 additions, or 2n 1 flops As mentioned above, we keep only leading term, and say that the inner product requires 2n flops, or even more approximately, order n flops A scalar-vector multiplication αx, where α R and x R n costs n flops The addition x + y of two vectors x, y R n also costs n flops If the vectors x and y are sparse, ie, have only a few nonzero terms, these basic operations can be carried out faster (assuming the vectors are stored using an appropriate data structure) For example, if x is a sparse vector with N nonzero entries, then the inner product x T y can be computed in 2N flops Matrix-vector multiplication A matrix-vector multiplication y Ax where A R m n costs 2mn flops: wehaveto calculate m components of y, each of which is the product of a row of A with x, ie, an inner product of two vectors in R n Matrix-vector products can often be accelerated by taking advantage of structure in A For example, if A is diagonal, then Ax can be computed in n flops, instead of 2n 2 flops for multiplication by a general n n matrix More generally, if A is sparse, with only N nonzero elements (out of mn),then2n flops are needed to form Ax, since we can skip multiplications and additions with zero As a less obvious example, suppose the matrix A is represented (stored)in the factored form A UV,whereU R m p, V R p n,andp m, p n Then we can compute Ax by first computing V x(which costs 2pn flops), and then computing U(V x)(which costs 2mp flops),sothetotalis2p(m + n)flops Since p min{m, n}, this is small compared to 2mn Matrix-matrix multiplication The matrix-matrix product C AB, wherea R m n and B R n p,costs2mnp flops We have mp elements in C to calculate, each of which is an inner product of two vectors of length n Again, we can often make substantial savings by taking advantage of structure in A and B For example, if A and B are sparse, we can accelerate the multiplication by skipping additions and multiplications with zero If m p and we know that C is symmetric, then we can calculate the matrix product in m 2 n flops, since we only have to compute the (1/2)m(m + 1)in the lower triangular part To form the product of several matrices, we can carry out the matrix-matrix multiplications in different ways, which have different flop counts in general The simplest example is computing the product D ABC, wherea R m n, B R n p,andc R p q Here we can compute D in two ways, using matrix-matrix multiplies One method is to first form the product AB (2mnp flops), and then form D (AB)C (2mpq flops), so the total is 2mp(n + q)flops Alternatively, we can first form the product BC (2npq flops), and then form D A(BC)(2mnq flops), with a total of 2nq(m + p)flops The first method is better 8

9 when 2mp(n + q) < 2nq(m + p), ie, when 1 n + 1 q < 1 m + 1 p This assumes that no structure of the matrices is exploited in carrying out matrix-matrix products For products of more than three matrices, there are many ways to parse the product into matrix-matrix multiplications Although it is not hard to develop an algorithm that determines the best parsing (ie, the one with the fewest required flops)given the matrix dimensions, in most applications the best parsing is clear 9

A primer on matrices

A primer on matrices A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous

More information

A primer on matrices

A primer on matrices A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous

More information

Phys 201. Matrices and Determinants

Phys 201. Matrices and Determinants Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1

More information

Matrices BUSINESS MATHEMATICS

Matrices BUSINESS MATHEMATICS Matrices BUSINESS MATHEMATICS 1 CONTENTS Matrices Special matrices Operations with matrices Matrix multipication More operations with matrices Matrix transposition Symmetric matrices Old exam question

More information

L. Vandenberghe EE133A (Spring 2017) 3. Matrices. notation and terminology. matrix operations. linear and affine functions.

L. Vandenberghe EE133A (Spring 2017) 3. Matrices. notation and terminology. matrix operations. linear and affine functions. L Vandenberghe EE133A (Spring 2017) 3 Matrices notation and terminology matrix operations linear and affine functions complexity 3-1 Matrix a rectangular array of numbers, for example A = 0 1 23 01 13

More information

Elementary Row Operations on Matrices

Elementary Row Operations on Matrices King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix

More information

[ Here 21 is the dot product of (3, 1, 2, 5) with (2, 3, 1, 2), and 31 is the dot product of

[ Here 21 is the dot product of (3, 1, 2, 5) with (2, 3, 1, 2), and 31 is the dot product of . Matrices A matrix is any rectangular array of numbers. For example 3 5 6 4 8 3 3 is 3 4 matrix, i.e. a rectangular array of numbers with three rows four columns. We usually use capital letters for matrices,

More information

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

More information

1 Matrices and matrix algebra

1 Matrices and matrix algebra 1 Matrices and matrix algebra 1.1 Examples of matrices A matrix is a rectangular array of numbers and/or variables. For instance 4 2 0 3 1 A = 5 1.2 0.7 x 3 π 3 4 6 27 is a matrix with 3 rows and 5 columns

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

Linear Algebra and Matrix Inversion

Linear Algebra and Matrix Inversion Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much

More information

Matrices. Chapter Definitions and Notations

Matrices. Chapter Definitions and Notations Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which

More information

Mathematics for Graphics and Vision

Mathematics for Graphics and Vision Mathematics for Graphics and Vision Steven Mills March 3, 06 Contents Introduction 5 Scalars 6. Visualising Scalars........................ 6. Operations on Scalars...................... 6.3 A Note on

More information

Basic Concepts in Linear Algebra

Basic Concepts in Linear Algebra Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear

More information

Two matrices of the same size are added by adding their corresponding entries =.

Two matrices of the same size are added by adding their corresponding entries =. 2 Matrix algebra 2.1 Addition and scalar multiplication Two matrices of the same size are added by adding their corresponding entries. For instance, 1 2 3 2 5 6 3 7 9 +. 4 0 9 4 1 3 0 1 6 Addition of two

More information

Review of Basic Concepts in Linear Algebra

Review of Basic Concepts in Linear Algebra Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra

More information

Math 123, Week 2: Matrix Operations, Inverses

Math 123, Week 2: Matrix Operations, Inverses Math 23, Week 2: Matrix Operations, Inverses Section : Matrices We have introduced ourselves to the grid-like coefficient matrix when performing Gaussian elimination We now formally define general matrices

More information

ICS 6N Computational Linear Algebra Matrix Algebra

ICS 6N Computational Linear Algebra Matrix Algebra ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix

More information

Linear Algebra V = T = ( 4 3 ).

Linear Algebra V = T = ( 4 3 ). Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional

More information

Lecture 3 Linear Algebra Background

Lecture 3 Linear Algebra Background Lecture 3 Linear Algebra Background Dan Sheldon September 17, 2012 Motivation Preview of next class: y (1) w 0 + w 1 x (1) 1 + w 2 x (1) 2 +... + w d x (1) d y (2) w 0 + w 1 x (2) 1 + w 2 x (2) 2 +...

More information

Matrices. 1 a a2 1 b b 2 1 c c π

Matrices. 1 a a2 1 b b 2 1 c c π Matrices 2-3-207 A matrix is a rectangular array of numbers: 2 π 4 37 42 0 3 a a2 b b 2 c c 2 Actually, the entries can be more general than numbers, but you can think of the entries as numbers to start

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

Matrices: 2.1 Operations with Matrices

Matrices: 2.1 Operations with Matrices Goals In this chapter and section we study matrix operations: Define matrix addition Define multiplication of matrix by a scalar, to be called scalar multiplication. Define multiplication of two matrices,

More information

M. Matrices and Linear Algebra

M. Matrices and Linear Algebra M. Matrices and Linear Algebra. Matrix algebra. In section D we calculated the determinants of square arrays of numbers. Such arrays are important in mathematics and its applications; they are called matrices.

More information

A FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic

A FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Matrix Arithmetic Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)

More information

Chapter 2: Matrices and Linear Systems

Chapter 2: Matrices and Linear Systems Chapter 2: Matrices and Linear Systems Paul Pearson Outline Matrices Linear systems Row operations Inverses Determinants Matrices Definition An m n matrix A = (a ij ) is a rectangular array of real numbers

More information

Matrix Basic Concepts

Matrix Basic Concepts Matrix Basic Concepts Topics: What is a matrix? Matrix terminology Elements or entries Diagonal entries Address/location of entries Rows and columns Size of a matrix A column matrix; vectors Special types

More information

Math 360 Linear Algebra Fall Class Notes. a a a a a a. a a a

Math 360 Linear Algebra Fall Class Notes. a a a a a a. a a a Math 360 Linear Algebra Fall 2008 9-10-08 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are

More information

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

Matrices and Linear Algebra

Matrices and Linear Algebra Contents Quantitative methods for Economics and Business University of Ferrara Academic year 2017-2018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2

More information

CLASS 12 ALGEBRA OF MATRICES

CLASS 12 ALGEBRA OF MATRICES CLASS 12 ALGEBRA OF MATRICES Deepak Sir 9811291604 SHRI SAI MASTERS TUITION CENTER CLASS 12 A matrix is an ordered rectangular array of numbers or functions. The numbers or functions are called the elements

More information

Introduction. Vectors and Matrices. Vectors [1] Vectors [2]

Introduction. Vectors and Matrices. Vectors [1] Vectors [2] Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector - one dimensional array Matrix -

More information

Matrix operations Linear Algebra with Computer Science Application

Matrix operations Linear Algebra with Computer Science Application Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the

More information

CS 246 Review of Linear Algebra 01/17/19

CS 246 Review of Linear Algebra 01/17/19 1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

More information

Linear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.

Linear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations. POLI 7 - Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes - Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems

More information

Notation, Matrices, and Matrix Mathematics

Notation, Matrices, and Matrix Mathematics Geographic Information Analysis, Second Edition. David O Sullivan and David J. Unwin. 010 John Wiley & Sons, Inc. Published 010 by John Wiley & Sons, Inc. Appendix A Notation, Matrices, and Matrix Mathematics

More information

DM559 Linear and Integer Programming. Lecture 3 Matrix Operations. Marco Chiarandini

DM559 Linear and Integer Programming. Lecture 3 Matrix Operations. Marco Chiarandini DM559 Linear and Integer Programming Lecture 3 Matrix Operations Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline and 1 2 3 and 4 2 Outline and 1 2

More information

MATH 320, WEEK 7: Matrices, Matrix Operations

MATH 320, WEEK 7: Matrices, Matrix Operations MATH 320, WEEK 7: Matrices, Matrix Operations 1 Matrices We have introduced ourselves to the notion of the grid-like coefficient matrix as a short-hand coefficient place-keeper for performing Gaussian

More information

Prepared by: M. S. KumarSwamy, TGT(Maths) Page

Prepared by: M. S. KumarSwamy, TGT(Maths) Page Prepared by: M. S. KumarSwamy, TGT(Maths) Page - 50 - CHAPTER 3: MATRICES QUICK REVISION (Important Concepts & Formulae) MARKS WEIGHTAGE 03 marks Matrix A matrix is an ordered rectangular array of numbers

More information

Getting Started with Communications Engineering. Rows first, columns second. Remember that. R then C. 1

Getting Started with Communications Engineering. Rows first, columns second. Remember that. R then C. 1 1 Rows first, columns second. Remember that. R then C. 1 A matrix is a set of real or complex numbers arranged in a rectangular array. They can be any size and shape (provided they are rectangular). A

More information

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections ) c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side

More information

Finite Mathematics Chapter 2. where a, b, c, d, h, and k are real numbers and neither a and b nor c and d are both zero.

Finite Mathematics Chapter 2. where a, b, c, d, h, and k are real numbers and neither a and b nor c and d are both zero. Finite Mathematics Chapter 2 Section 2.1 Systems of Linear Equations: An Introduction Systems of Equations Recall that a system of two linear equations in two variables may be written in the general form

More information

Systems of Linear Equations and Matrices

Systems of Linear Equations and Matrices Chapter 1 Systems of Linear Equations and Matrices System of linear algebraic equations and their solution constitute one of the major topics studied in the course known as linear algebra. In the first

More information

Chapter 2. Matrix Arithmetic. Chapter 2

Chapter 2. Matrix Arithmetic. Chapter 2 Matrix Arithmetic Matrix Addition and Subtraction Addition and subtraction act element-wise on matrices. In order for the addition/subtraction (A B) to be possible, the two matrices A and B must have the

More information

Matrix Arithmetic. j=1

Matrix Arithmetic. j=1 An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column

More information

A Review of Matrix Analysis

A Review of Matrix Analysis Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value

More information

Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Numerical Analysis Lecture Notes Peter J Olver 3 Review of Matrix Algebra Vectors and matrices are essential for modern analysis of systems of equations algebrai, differential, functional, etc In this

More information

Chapter 1 Matrices and Systems of Equations

Chapter 1 Matrices and Systems of Equations Chapter 1 Matrices and Systems of Equations System of Linear Equations 1. A linear equation in n unknowns is an equation of the form n i=1 a i x i = b where a 1,..., a n, b R and x 1,..., x n are variables.

More information

GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511)

GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511) GAUSSIAN ELIMINATION AND LU DECOMPOSITION (SUPPLEMENT FOR MA511) D. ARAPURA Gaussian elimination is the go to method for all basic linear classes including this one. We go summarize the main ideas. 1.

More information

Calculus II - Basic Matrix Operations

Calculus II - Basic Matrix Operations Calculus II - Basic Matrix Operations Ryan C Daileda Terminology A matrix is a rectangular array of numbers, for example 7,, 7 7 9, or / / /4 / / /4 / / /4 / /6 The numbers in any matrix are called its

More information

1 Matrices and Systems of Linear Equations. a 1n a 2n

1 Matrices and Systems of Linear Equations. a 1n a 2n March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real

More information

Jim Lambers MAT 610 Summer Session Lecture 1 Notes

Jim Lambers MAT 610 Summer Session Lecture 1 Notes Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra

More information

Matrix-Vector Operations

Matrix-Vector Operations Week3 Matrix-Vector Operations 31 Opening Remarks 311 Timmy Two Space View at edx Homework 3111 Click on the below link to open a browser window with the Timmy Two Space exercise This exercise was suggested

More information

Inverses. Stephen Boyd. EE103 Stanford University. October 28, 2017

Inverses. Stephen Boyd. EE103 Stanford University. October 28, 2017 Inverses Stephen Boyd EE103 Stanford University October 28, 2017 Outline Left and right inverses Inverse Solving linear equations Examples Pseudo-inverse Left and right inverses 2 Left inverses a number

More information

Announcements Wednesday, October 10

Announcements Wednesday, October 10 Announcements Wednesday, October 10 The second midterm is on Friday, October 19 That is one week from this Friday The exam covers 35, 36, 37, 39, 41, 42, 43, 44 (through today s material) WeBWorK 42, 43

More information

Matrices. Math 240 Calculus III. Wednesday, July 10, Summer 2013, Session II. Matrices. Math 240. Definitions and Notation.

Matrices. Math 240 Calculus III. Wednesday, July 10, Summer 2013, Session II. Matrices. Math 240. Definitions and Notation. function Matrices Calculus III Summer 2013, Session II Wednesday, July 10, 2013 Agenda function 1. 2. function function Definition An m n matrix is a rectangular array of numbers arranged in m horizontal

More information

Announcements Monday, October 02

Announcements Monday, October 02 Announcements Monday, October 02 Please fill out the mid-semester survey under Quizzes on Canvas WeBWorK 18, 19 are due Wednesday at 11:59pm The quiz on Friday covers 17, 18, and 19 My office is Skiles

More information

Matrix Algebra 2.1 MATRIX OPERATIONS Pearson Education, Inc.

Matrix Algebra 2.1 MATRIX OPERATIONS Pearson Education, Inc. 2 Matrix Algebra 2.1 MATRIX OPERATIONS MATRIX OPERATIONS m n If A is an matrixthat is, a matrix with m rows and n columnsthen the scalar entry in the ith row and jth column of A is denoted by a ij and

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

Introduction to Matrix Algebra

Introduction to Matrix Algebra Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

More information

n n matrices The system of m linear equations in n variables x 1, x 2,..., x n can be written as a matrix equation by Ax = b, or in full

n n matrices The system of m linear equations in n variables x 1, x 2,..., x n can be written as a matrix equation by Ax = b, or in full n n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in n variables x 1, x 2,..., x n a 11 x 1 + a 12 x

More information

v = ( 2)

v = ( 2) Chapter : Introduction to Vectors.. Vectors and linear combinations Let s begin by saying what vectors are: They are lists of numbers. If there are numbers in the list, there is a natural correspondence

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2 MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

Linear Equations in Linear Algebra

Linear Equations in Linear Algebra 1 Linear Equations in Linear Algebra 1.7 LINEAR INDEPENDENCE LINEAR INDEPENDENCE Definition: An indexed set of vectors {v 1,, v p } in n is said to be linearly independent if the vector equation x x x

More information

Chapter 2. Ma 322 Fall Ma 322. Sept 23-27

Chapter 2. Ma 322 Fall Ma 322. Sept 23-27 Chapter 2 Ma 322 Fall 2013 Ma 322 Sept 23-27 Summary ˆ Matrices and their Operations. ˆ Special matrices: Zero, Square, Identity. ˆ Elementary Matrices, Permutation Matrices. ˆ Voodoo Principle. What is

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

Matrix-Matrix Multiplication

Matrix-Matrix Multiplication Chapter Matrix-Matrix Multiplication In this chapter, we discuss matrix-matrix multiplication We start by motivating its definition Next, we discuss why its implementation inherently allows high performance

More information

Matrix Algebra: Definitions and Basic Operations

Matrix Algebra: Definitions and Basic Operations Section 4 Matrix Algebra: Definitions and Basic Operations Definitions Analyzing economic models often involve working with large sets of linear equations. Matrix algebra provides a set of tools for dealing

More information

Matrix Algebra. Matrix Algebra. Chapter 8 - S&B

Matrix Algebra. Matrix Algebra. Chapter 8 - S&B Chapter 8 - S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number

More information

Linear Algebra. Linear Equations and Matrices. Copyright 2005, W.R. Winfrey

Linear Algebra. Linear Equations and Matrices. Copyright 2005, W.R. Winfrey Copyright 2005, W.R. Winfrey Topics Preliminaries Systems of Linear Equations Matrices Algebraic Properties of Matrix Operations Special Types of Matrices and Partitioned Matrices Matrix Transformations

More information

Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds

Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds These notes are meant to provide a brief introduction to the topics from Linear Algebra that will be useful in Math3315/CSE3365, Introduction

More information

Matrix & Linear Algebra

Matrix & Linear Algebra Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A

More information

Basic Linear Algebra in MATLAB

Basic Linear Algebra in MATLAB Basic Linear Algebra in MATLAB 9.29 Optional Lecture 2 In the last optional lecture we learned the the basic type in MATLAB is a matrix of double precision floating point numbers. You learned a number

More information

Order of Operations. Real numbers

Order of Operations. Real numbers Order of Operations When simplifying algebraic expressions we use the following order: 1. Perform operations within a parenthesis. 2. Evaluate exponents. 3. Multiply and divide from left to right. 4. Add

More information

Matrices. Chapter What is a Matrix? We review the basic matrix operations. An array of numbers a a 1n A = a m1...

Matrices. Chapter What is a Matrix? We review the basic matrix operations. An array of numbers a a 1n A = a m1... Chapter Matrices We review the basic matrix operations What is a Matrix? An array of numbers a a n A = a m a mn with m rows and n columns is a m n matrix Element a ij in located in position (i, j The elements

More information

Kevin James. MTHSC 3110 Section 2.1 Matrix Operations

Kevin James. MTHSC 3110 Section 2.1 Matrix Operations MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j

More information

Appendix C Vector and matrix algebra

Appendix C Vector and matrix algebra Appendix C Vector and matrix algebra Concepts Scalars Vectors, rows and columns, matrices Adding and subtracting vectors and matrices Multiplying them by scalars Products of vectors and matrices, scalar

More information

Multiplying matrices by diagonal matrices is faster than usual matrix multiplication.

Multiplying matrices by diagonal matrices is faster than usual matrix multiplication. 7-6 Multiplying matrices by diagonal matrices is faster than usual matrix multiplication. The following equations generalize to matrices of any size. Multiplying a matrix from the left by a diagonal matrix

More information

Matrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices

Matrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in

More information

x + 2y + 3z = 8 x + 3y = 7 x + 2z = 3

x + 2y + 3z = 8 x + 3y = 7 x + 2z = 3 Chapter 2: Solving Linear Equations 23 Elimination Using Matrices As we saw in the presentation, we can use elimination to make a system of linear equations into an upper triangular system that is easy

More information

An Introduction To Linear Algebra. Kuttler

An Introduction To Linear Algebra. Kuttler An Introduction To Linear Algebra Kuttler April, 7 Contents Introduction 7 F n 9 Outcomes 9 Algebra in F n Systems Of Equations Outcomes Systems Of Equations, Geometric Interpretations Systems Of Equations,

More information

What is the Matrix? Linear control of finite-dimensional spaces. November 28, 2010

What is the Matrix? Linear control of finite-dimensional spaces. November 28, 2010 What is the Matrix? Linear control of finite-dimensional spaces. November 28, 2010 Scott Strong sstrong@mines.edu Colorado School of Mines What is the Matrix? p. 1/20 Overview/Keywords/References Advanced

More information

Systems of Linear Equations and Matrices

Systems of Linear Equations and Matrices Chapter 1 Systems of Linear Equations and Matrices System of linear algebraic equations and their solution constitute one of the major topics studied in the course known as linear algebra. In the first

More information

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 2

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 2 EECS 6A Designing Information Devices and Systems I Spring 9 Lecture Notes Note Vectors and Matrices In the previous note, we introduced vectors and matrices as a way of writing systems of linear equations

More information

Matrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices

Matrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated

More information

Math 304 (Spring 2010) - Lecture 2

Math 304 (Spring 2010) - Lecture 2 Math 304 (Spring 010) - Lecture Emre Mengi Department of Mathematics Koç University emengi@ku.edu.tr Lecture - Floating Point Operation Count p.1/10 Efficiency of an algorithm is determined by the total

More information

. a m1 a mn. a 1 a 2 a = a n

. a m1 a mn. a 1 a 2 a = a n Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by

More information

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

More information

Review from Bootcamp: Linear Algebra

Review from Bootcamp: Linear Algebra Review from Bootcamp: Linear Algebra D. Alex Hughes October 27, 2014 1 Properties of Estimators 2 Linear Algebra Addition and Subtraction Transpose Multiplication Cross Product Trace 3 Special Matrices

More information

Matrices and Vectors

Matrices and Vectors Matrices and Vectors James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 11, 2013 Outline 1 Matrices and Vectors 2 Vector Details 3 Matrix

More information

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

More information

Lecture 3: Matrix and Matrix Operations

Lecture 3: Matrix and Matrix Operations Lecture 3: Matrix and Matrix Operations Representation, row vector, column vector, element of a matrix. Examples of matrix representations Tables and spreadsheets Scalar-Matrix operation: Scaling a matrix

More information

Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.

Section 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A

More information

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33 Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct

More information

Introduction to Matrices

Introduction to Matrices POLS 704 Introduction to Matrices Introduction to Matrices. The Cast of Characters A matrix is a rectangular array (i.e., a table) of numbers. For example, 2 3 X 4 5 6 (4 3) 7 8 9 0 0 0 Thismatrix,with4rowsand3columns,isoforder

More information

Mathematics 13: Lecture 10

Mathematics 13: Lecture 10 Mathematics 13: Lecture 10 Matrices Dan Sloughter Furman University January 25, 2008 Dan Sloughter (Furman University) Mathematics 13: Lecture 10 January 25, 2008 1 / 19 Matrices Recall: A matrix is a

More information

MATRICES AND MATRIX OPERATIONS

MATRICES AND MATRIX OPERATIONS SIZE OF THE MATRIX is defined by number of rows and columns in the matrix. For the matrix that have m rows and n columns we say the size of the matrix is m x n. If matrix have the same number of rows (n)

More information