Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional column vector: W V Subscripts are used to denote single elements of the vector For example W V 4 A row vector is a list of numbers stored horizontally The dimension of a row vector is the number of values in the vector U is a 4-dimensional row vector and T is a -dimensional row-vector: 4 5 U 4 T 4 Elements are accessed by subscript: U T 4 The term vector is used to refer to either a row vector or a column vector in situations where it doesn t matter whether the values are stored in a column or in a row
Two vectors with a common dimension can be added or subtracted: 4 + Any vector can be multiplied by a scalar coefficient: 6 Combining addition and scalar multiplication gives a linear combination: 4 Geometrically a vector V can be viewed as a line segment in d-dimensional space with its tail at the origin and its head at the point V 4 V - U - - -4-4 - - - 4 V is the vector and U is the vector --
The scalar product also called the dot product or inner product is formed from two vectors V and W having the same dimension The scalar product is a single number a scalar and is notated V W or V W The definition of the scalar product is V W i V i W i For example if V and W If U 4 4 and T V W + + 4 U T 4 + + + 4 9 If two d-dimensional vectors U and V have inner product U V then U and V are orthogonal or perpendicular Viewing -dimensional vectors as points in the plane V x y U x y then V U when V and U are perpendicular in the geometric sense the angle between them is π/ radians 4 V - - U - -4-4 - - - 4 V is the vector and U is the vector -
A linear combination of d-dimensional vectors V V V m is an expression of the form c V + c V + + c m V m where c c m are scalars called coefficients For example if V 4 V and c c c then c V + c V + c V is V A set of d-dimensional vectors V V m are linearly dependent if there exist scalars c c m not all of which are zero such that the linear combination c V + + c m V m is zero For example V V V 6 are linearly dependent since if c c and c then c V + c V + c V A set of d-dimensional vectors V V m are linearly independent if they are not linearly dependent That is whenever c V + + c m V m then c c m For example suppose V V 5 V 6 and c V + c V + c V The linear combination can be written c + 5c 6c c c c + c c + c + c From the second line c and from the final line c c From the third line c c so we conclude c c hence V V V are linearly independent 4
It is a fact that any set of m > d d-dimensional vectors must be linearly dependent For example there can never be a set of 4 linearly independent vectors having dimension Matrices A m n matrix A is a m n array of numbers where M ij refers to the value in the i th row and j th column For example the following is a matrix A 4 where A A etc The following is a matrix B where B B etc Matrices of the same shape can be added and subtracted: 4 Any matrix can be multiplied by a scalar: + 4 4 4 4 4 We can form linear combinations of matrices: 9 6 4 6 7 5
A n-dimensional column vector is also a n matrix A n-dimensional row vector is also a n matrix The transpose of an m n matrix A written A is an n m matrix where A ij A ji For example 4 A matrix A is symmetric if A A For example 4 is symmetric while S 4 5 T 4 6 5 is not If A is a m n matrix and V is a n-dimensional column vector we can form the matrix vector product W AV where W i is the dot product of V with row i of A For example 4 + + 4 7 6
The nullspace of M written NullM is the set of all vectors V such that MV For example if then 4 V is in NullM The zero vector is always in NullM and if the columns of M are linearly independent the zero vector is the only vector in NullM But if the columns of M are linearly dependent there will be infinitely many nonzero vectors in NullM A square m m matrix with nullspace containing only the zero vector is nonsingular otherwise it is singular Equivalently a square matrix is nonsingular if and only if its columns are linearly independent If A is a m n matrix and B is a n r matrix we can form the matrix matrix product C AB where C is a m r matrix whose elements are defined as: C ij is the dot product of row i of A with column j of B For example 4 where for example + + 4 7 5 7
Rectangular matrices can only be multiplied if the number of columns in the first matrix is equal to the number of rows in the second matrix ie AB can be formed only if A is m n and B is n r For square matrices the products AB and BA can both be formed However it is important to note that they are different: 4 5 7 4 5 7 For any matrix m n matrix A the products A A and AA can always be formed The first product is n n and the second product is m m The product A A is the column-wise inner product matrix since A A ij product of the i th column of A with the j th column of A is the inner The product AA is the row-wise inner product matrix since AA ij is the inner product of the i th row of A with the j th row of A 8
For example if A 4 then A A 5 6 6 7 and AA Note that both A A and AA are symmetric 6 6 6 7 The identity matrix is a special square n n matrix I where I jj and I ij if i j For example the 4 4 identity matrix is I The identity matrix acts like for matrix multiplication: AI A and IA A if A is m n the first I is the m m identity matrix and the second I is the n n identity matrix 9
If A is a square n n matrix with linearly independent columns then an n n matrix A can be constructed such that AA A A I where I is the n n identity For example A 7 A / / 7/ / For a matrix the general form of the inverse is a b A c d A ad bc d b c a where if ad bc A is singular and has no inverse For d > the formula for the inverse of a d d matrix is very complicated but inverses can be easily calculated on a computer