Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review

Outline 1 Matrices and Vectors Basic Matrix Operations Addition and subtraction Scalar multiplication Matrix multiplication The Identity Matrix Matrix inverse Systems of linear equations Representing summation using matrix notation Portfolio math with matrix algebra Bivariate normal distribution Derivatives of simple matrix functions Eric Zivot (Copyright 2015) Matrix Algebra Review 2 / 54

Matrices and vectors Matrix A = (n m) a 11 a 12 a 1m a 21 a 22 a 2m...... a n1 a n2 a nm Vector Square matrix : n = m x = (n 1) x 1 x 2 n = # of rows, m = # of columns. x n Eric Zivot (Copyright 2015) Matrix Algebra Review 3 / 54

Remarks R is a matrix oriented programming language Excel can handle matrices and vectors in formulas and some functions Excel has special functions for working with matrices. There are called array functions. Must use: <ctrl>-<shift>-<enter> to evaluate array function Eric Zivot (Copyright 2015) Matrix Algebra Review 4 / 54

Transpose of a Matrix Interchange rows and columns of a matrix: A = transpose of A (m n) (n m) Example, A = x = 1 2 3 4 5 6 1 2 3, x =, A = 1 2 3 1 4 2 5 3 6 Eric Zivot (Copyright 2015) Matrix Algebra Review 5 / 54

Symmetric Matrix A square matrix A is symmetric if: A = A Example, 1 2 A = 2 1, A = 1 2 2 1 Remark: Covariance and correlation matrices are symmetric. Eric Zivot (Copyright 2015) Matrix Algebra Review 7 / 54

Matrix Multiplication (not element-by-element) A = (3 2) a 11 a 12 a 21 a 22 a 31 a 32, B = (2 3) b11 b 12 b 13 b 21 b 22 b 23 Note: A and B are comformable matrices: # of columns in A = # of rows in B: A B (3 2) (2 3) = a 11 b 11 + a 12 b 21 a 11 b 12 + a 12 b 22 a 11 b 13 + a 12 b 23 a 21 b 11 + a 22 b 21 a 21 b 12 + a 22 b 22 a 21 b 13 + a 22 b 23 a 31 b 11 + a 32 b 21 a 31 b 12 + a 32 b 22 a 31 b 13 + a 32 b 23 Remark: In general, A B B A Eric Zivot (Copyright 2015) Matrix Algebra Review 11 / 54

Example A = A B = R operator A%*%B Excel function 1 2 3 4, B = 5 6 7 8 5 + 14 6 + 16 15 + 28 18 + 32 MMULT(matrix1, matrix2) <ctrl>-<shift>-<enter> = 19 22 43 50 Eric Zivot (Copyright 2015) Matrix Algebra Review 12 / 54

Identity Matrix The n dimensional identity matrix has all diagonal elements equal to 1, and all off diagonal elements equal to 0. Example, 1 0 I 2 = 0 1 Remark: The identity matrix plays the roll of 1 in matrix algebra, 1 0 a11 a 12 I 2 A = 0 1 a 21 a 22 = = A a11 + 0 a 12 + 0 0 + a 21 0 + a 22 = a11 a 12 a 21 a 22 Eric Zivot (Copyright 2015) Matrix Algebra Review 13 / 54

Identity Matrix cont. Similarly, a11 a 12 A I 2 = a 21 a 22 a11 a 12 = a 21 a 22 1 0 0 1 = A R function diag(n) creates n dimensional identity matrix. Eric Zivot (Copyright 2015) Matrix Algebra Review 14 / 54

Matrix Inverse Let A (n n) = square matrix. A 1 = inverse of A satisfies: A 1 A=I n AA 1 =I n Remark: A 1 is similar to the inverse of a number: a = 2, a 1 = 1 2 a a 1 = 2 1 2 = 1 a 1 a = 1 2 2 = 1 Eric Zivot (Copyright 2015) Matrix Algebra Review 15 / 54

Representing Systems of Linear Equations Using Matrix Algebra Consider the system of two linear equations x + y = 1 2x y = 1 The equations represent two straight lines which intersect at the point: x = 2 3, y = 1 3 Eric Zivot (Copyright 2015) Matrix Algebra Review 17 / 54

Representing Systems of Linear Equations Using Matrix Algebra cont. Matrix algebra representation: 1 1 x 1 = 2 1 y 1 or, A z = b where, A = 1 1 2 1, z = x y and b = 1 1. Eric Zivot (Copyright 2015) Matrix Algebra Review 18 / 54

Representing Systems of Linear Equations Using Matrix Algebra cont. We can solve for z by multiplying both sides by A 1 : A 1 A z = A 1 b = I z = A 1 b = z = A 1 b or, x y = 1 1 2 1 1 1 1 Remark: As long as we can determine the elements in A 1, we can solve for the values of x and y in the vector z. Since the system of linear equations has a solution as long as the two lines intersect, we can determine the elements in A 1 provided the two lines are not parallel. Eric Zivot (Copyright 2015) Matrix Algebra Review 19 / 54

Matrix Inverse (2 2 case) There are general numerical algorithms for finding the elements of A 1 and programs like Excel and R have these algorithms available. However, if A is a (2 2) matrix then there is a simple formula for A 1. Let, a11 a 12 A =. a 21 a 22 Then, where, A 1 = 1 a22 a 12 det(a) a 21 a 11 det(a) = a 11 a 22 a 21 a 12 0. Eric Zivot (Copyright 2015) Matrix Algebra Review 20 / 54

Matrix Inverse (2 2 case) cont. Let s apply the above rule to find the inverse of A in our example: A 1 1 1 1 1 1 = = 3 3. 1 2 2 1 2 3 1 3 Notice that, A 1 A = 1 3 2 3 1 3 1 3 1 1 2 1 = 1 0 0 1. Eric Zivot (Copyright 2015) Matrix Algebra Review 21 / 54

Representing Systems of Linear Equations Using Matrix Algebra cont. In general, if we have n linear equations in n unknown variables we may write the system of equations as: a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2. =. a n1 x 1 + a n2 x 2 + + a nn x n = b n which we may then express in matrix form as: a 11 a 12 a 1n b 1 a 21 a 22 a 2n x 1 x 2... = b 2. a n1 a n2 a nn x n Eric Zivot (Copyright 2015) Matrix Algebra Review 23 / 54 b n

Representing Systems of Linear Equations Using Matrix Algebra cont. or, A (n n) x = (n 1) b. (n 1) The solution to the system of equations is given by: x = A 1 b where A 1 A = I and I is the (n n) identity matrix. If the number of equations is greater than two, then we generally use numerical algorithms to find the elements in A 1. Eric Zivot (Copyright 2015) Matrix Algebra Review 24 / 54

Representing Summation Using Matrix Notation cont. Sum of Squares: n x 2 i = x 2 1 + x 2 2 + + x 2 n i=1 ( ) x x = x 1 x 2 x n x 1 x 2. x n n = x 2 1 + x 2 2 + + x 2 n = x 2 i i=1 Eric Zivot (Copyright 2015) Matrix Algebra Review 29 / 54

Representing Summation Using Matrix Notation cont. Sums of cross products: n x i y i = x 1 y 1 + x 2 y 2 + + x n y n i=1 ( ) x y = x 1 x 2 x n y 1 y 2. y n n = x 1 y 1 + x 2 y 2 + + x n y n = x i y i i=1 = y x Eric Zivot (Copyright 2015) Matrix Algebra Review 30 / 54

Computational tools R function t(x)%*%y, t(y)%*%x crossprod(x,y) Excel function MMULT(TRANSPOSE(x),y) MMULT(TRANSPOSE(y),x) <ctrl>-<shift>-<enter> Eric Zivot (Copyright 2015) Matrix Algebra Review 31 / 54

Portfolio Math with Matrix Algebra Three Risky Asset Example: Let R i denote the return on asset i = A, B, C and assume that R A, R B and R C are jointly normally distributed with means, variances and covariances: µ i = ER i, σ 2 i = var(r i ), cov(r i, R j ) = σ ij Portfolio x : x i = share of wealth in asset i x A + x B + x C = 1 Portfolio return: R p,x = x A R A + x B R B + x C R C. Eric Zivot (Copyright 2015) Matrix Algebra Review 33 / 54

Portfolio Math with Matrix Algebra cont. Portfolio expected return: µ p,x = ER p,x = x A µ A + x B µ B + x C µ C Portfolio variance: σ 2 p,x = var(r p,x ) = x 2 Aσ 2 A + x 2 Bσ 2 B + x 2 Cσ 2 C + 2x A x B σ AB + 2x A x C σ AC + 2x B x C σ BC Portfolio distribution: R p,x N(µ p,x, σ 2 p,x) Eric Zivot (Copyright 2015) Matrix Algebra Review 34 / 54

Portfolio Math with Matrix Algebra cont. Matrix algebra representation: R A µ A R = R B, µ = µ B R C µ C x = x A x B x C, Σ = Portfolio weights sum to 1: x 1 = x A x B x C ) = x A + x B + x C = 1, 1 = 1 1 1 σa 2 σ AB σ AC σ AB σb 2 σ BC σ AC σ BC σc 2 1 1 1 Eric Zivot (Copyright 2015) Matrix Algebra Review 35 / 54

Digression on Covariance Matrix Using matrix algebra, the variance-covariance matrix of the N 1 return vector R is defined as: var(r) N N = cov(r) = E(R µ)(r µ) = Σ Because R has N elements, Σ is the N N matrix: σ 2 1 σ 12 σ 1n σ 12 σ 2 2 σ 2n Σ =...... σ 1n σ 2n σn 2 Eric Zivot (Copyright 2015) Matrix Algebra Review 36 / 54

Digression on Covariance Matrix cont. For the case N = 2, we have: ( ) E(R µ) (R µ ) R1 µ 1 = E (R 1 µ 1, R 2 µ 2 ) 2 1 1 2 R 2 µ 2 = E = = ( (R1 µ 1 ) 2 (R 1 µ 1 )(R 2 µ 2 ) (R 2 µ 2 )(R 1 µ 1 ) (R 2 µ 2 ) 2 ( E(R1 µ 1 ) 2 E(R 1 µ 1 )(R 2 µ 2 ) E(R 2 µ 2 )(R 1 µ 1 ) E(R 2 µ 2 ) 2 ( var(r1 ) cov(r 1, R 2 ) cov(r 2, R 1 ) var(r 2 ) ) = ( σ 2 1 σ 12 σ 12 σ 2 2 ) ) ) = Σ. Eric Zivot (Copyright 2015) Matrix Algebra Review 37 / 54

Portfolio Math with Matrix Algebra cont. Portfolio return: R p,x = x R = ( x A x B x C ) = x A R A + x B R B + x C R C = R x Portfolio expected return: µ p,x = x µ = ( x A x B x X ) = x A µ A + x B µ B + x C µ C = µ x R A R B R C Eric Zivot (Copyright 2015) Matrix Algebra Review 38 / 54 µ A µ B µ C

Portfolio Math with Matrix Algebra cont. Portfolio variance: σ 2 p,x = var(x R) = E ( x R x µ ) 2 = E ( x (R µ) ) 2 = Ex (R µ)x (R µ) = Ex (R µ)(r µ) x = = x E(R µ)(r µ) x = x Σx = ( x A x B x C ) σa 2 σ AB σ AC σ AB σb 2 σ BC σ AC σ BC σc 2 x A x B x C = x 2 Aσ 2 A + x 2 Bσ 2 B + x 2 Cσ 2 C + 2x A x B σ AB + 2x A x C σ AC + 2x B x C σ BC Eric Zivot (Copyright 2015) Matrix Algebra Review 40 / 54

Computational tools Excel formulas MMULT(TRANSPOSE(xvec),MMULT(sigma,xvec)) MMULT(MMULT(TRANSPOSE(xvec),sigma),xvec) <ctrl>-<shift>-<enter> Note: x Σx = (x Σ) x = x (Σx) R formulas t(x)%*%sigma%*%x Portfolio distribution R p,x N(µ p,x, σp,x) 2 Eric Zivot (Copyright 2015) Matrix Algebra Review 41 / 54

Covariance Between 2 Portfolio Returns 2 portfolios: x = x A x B x C, y = x 1 = 1, y 1 = 1 Portfolio returns: R p,x = x R R p,y = y R Covariance: cov(r p,x, R p,y ) = x Σy = y Σx y A y B y C Eric Zivot (Copyright 2015) Matrix Algebra Review 42 / 54

Covariance Between 2 Portfolio Returns cont. Derivation: cov(r p,x, R p,y ) = cov(x R, y R) = E(x R x µ)(y R y µ) = Ex (R µ)y (R µ) = Ex (R µ)(r µ) y = x E(R µ)(r µ) y = x Σy Eric Zivot (Copyright 2015) Matrix Algebra Review 43 / 54

Computational tools Excel formula MMULT(TRANSPOSE(xvec),MMULT(sigma,yvec)) MMULT(TRANSPOSE(yvec),MMULT(sigma,xvec)) <ctrl>-<shift>-<enter> R formula t(x)%*%sigma%*%y Eric Zivot (Copyright 2015) Matrix Algebra Review 44 / 54

Bivariate Normal Distribution Let X and Y be distributed bivariate normal. The joint pdf is given by: 1 f(x, y) = 2πσ X σ Y 1 ρ 2 XY { (x ) 1 2 ( ) µx y 2 µy exp 2(1 ρ 2 XY ) + 2ρ XY (x µ X )(y µ σ X σ Y σ X σ Y where EX = µ X, EY = µ Y, sd(x) = σ X, sd(y ) = σ Y, and ρ XY = cor(x, Y ). Eric Zivot (Copyright 2015) Matrix Algebra Review 46 / 54

Bivariate Normal Distribution Let X and Y be distributed bivariate normal. The joint pdf is given by: f(x, y) = 1 2πσ X σ Y 1 ρ 2 XY { ( ) 1 exp x µx 2 ( ) } 2(1 ρ 2 XY ) σ X + y µy 2 σ Y 2ρ XY (x µ X )(y µ Y ) σ X σ Y where EX = µ X, EY = µ Y, sd(x) = σ X, sd(y ) = σ Y, and ρ XY = cor(x, Y ). Eric Zivot (Copyright 2015) Matrix Algebra Review 47 / 54

Bivariate Normal Distribution cont. Define X = ( X Y ), x = ( x y ), µ = ( µx µ Y ) ( σ 2, Σ = X σ XY σ XY σy 2 ) Then the bivariate normal distribution can be compactly expressed as: where, f(x) = 1 2π det(σ) 1/2 e 1 2 (x µ) Σ 1 (x µ) det(σ) = σ 2 Xσ 2 Y σ 2 XY = σ 2 Xσ 2 Y σ 2 Xσ 2 Y ρ 2 XY = σ 2 Xσ 2 Y (1 ρ 2 XY ). We use the shorthand notation: X N(µ, Σ) Eric Zivot (Copyright 2015) Matrix Algebra Review 48 / 54

Derivatives of Simple Matrix Functions Result: Let A be an n n symmetric matrix, and let x and y be an n 1 vectors. Then, x x y = x Ax= x x Ax = x 1 x y. x n x y x 1 (Ax). x n (Ax) x 1 x Ax. x n x Ax = y, = A, = 2Ax. Eric Zivot (Copyright 2015) Matrix Algebra Review 50 / 54

Example We will demonstrate these results with simple examples. Let, ( ) ( ) ( ) a b x1 y1 A =, x =, y = b c For the first result we have, x y = x 1 y 1 + x 2 y 2. x 2 y 2 Then, x x y = ( x 1 x y x 2 x y ) = ( x 1 (x 1 y 1 + x 2 y 2 ) x 2 (x 1 y 1 + x 2 y 2 ) ) = ( y1 y 2 ) = y. Eric Zivot (Copyright 2015) Matrix Algebra Review 51 / 54

Example cont. Next, consider the second result. Note that, ( ) ( ) ( ) a b x1 ax1 + bx 2 Ax = = b c x 2 bx 1 + cx 2 and, (Ax) = (ax 1 + bx 2, bx 1 + cx 2 ) Then, ( x Ax= x 1 (ax 1 + bx 2, bx 1 + cx 2 ) x 2 (ax 1 + bx 2, bx 1 + cx 2 ) ) ( a b = b c ) = A Eric Zivot (Copyright 2015) Matrix Algebra Review 52 / 54

Example cont. Finally, consider the third result. We have, ( ) ( ) ( ) x a b x1 Ax = x 1 x 2 = ax 2 1 + 2bx 1 x 2 + cx 2 b c 2. x 2 Then, ( ( x x Ax= x 1 ax 2 1 + 2bx 1 x 2 + cx 2 ) ) ( 2 2ax1 + 2bx ( x 2 ax 2 1 + 2bx 1 x 2 + cx 2 ) 2 = 2 2bx 1 + 2cx 2 ( ) ( ) a b x1 = 2 = 2Ax. b c x 2 ) Eric Zivot (Copyright 2015) Matrix Algebra Review 53 / 54