Matrix inversion and linear equations

Transcription

1 Learning objectives. Matri inversion and linear equations Know Cramer s rule Understand how linear equations can be represented in matri form Know how to solve linear equations using matrices and Cramer s rule. reminder. The concept of inverse only applies to square nn matrices. To calculate A - First check that det. A Then calculate the matri of minors, M Then from this calculate the matri of co-factors C Then A - (/det. A)C T.

2 Inverses for (square) matrices Idea: In standard multiplication every number has an inverse (ecept maybe zero unless you count infinity) The inverse of is /; the inverse of 7 is /7, the inverse of -. -/. Also a number times its inverse equals : (/) and the inverse times the number equals : (/) and the inverse of the inverse is the original number /(/)

3 Inverses for (square) matrices The rules for the inverse of a matri are similar (but not identical) If A is an nn matri then the inverse of A, written A -, is an nn matri such that:. AA - I. A - A I Notes:. This means that A is the inverse of A -. But.... A - may not always eist

4 An Inverse matri eample A. Suppose and B / / Then AB / / ()(/ ) + ()(/ ) ()(/ ) + ( )(/ ) ()() + ()( ) ()() + ( )( ) So given AA - I it must be that in this case BA -

5 In general need to introduce some more terminology before can invert a matri. Only square matrices can be inverted. Not all square matrices can be inverted however. A matri that can be inverted is said to be nonsingular (so squareness is a necessary but not sufficient condition to invert). The sufficient condition is that the columns (or rows since it is square) be linearly dependent - think of this as being separate equations so the equations must be independent (n equations and n unknowns) if a solution is to be found

6 Eg 4 d A d 5 d + 4 d 5 + d So that the st row of A is twice that of the nd row and there is linear dependence One equation is redundant (no etra information) and the system reduces to a single equation with unknowns So no unique solution for and eists

7 Rank of a matri The idea of vector rank can be easily etended to a matri The rank of a matri is the maimum number of linearly independent rows or columns If the matri is square the maimum number of independent rows must be the same as the maimum number of independent columns If the matri is not square then the rank is equal to the smaller of the maimum number of rows or columns, ρ<min(rows, cols) If a matri of order n is also of rank n, the matri is said to be of full rank Important: Only full rank matrices can be inverted Matri ranks are closely linked to the concept of determinants

8 Can you see how this relates to equation ()? Let A be an nn matri then the determinant of A is a unique number (scalar), defined as: () Determinants j n + j j det( A) A ( ) a j det( A ) j Notes: In each term there are three components:. (-) +j. a j. Det(A j ) 4. What does this mean? Start with a matri A a a a a A a a a a which gives a single number (scalar) as the answer as do all determinants

9 Determinants Eg 4 A 8 5 A (*5) (4*8) 8 What is the determinant of B 5 6 C 8 4 So matrices that are not full rank have linear dependent rows/columns - have zero determinants (will come back to this) and are singular

10 Determinants. The determinant of a matri is defined iteratively. An nn is calculated as the sum of terms involving the determinants of n(n-)(n-) (ie n!) matrices. Each (n-)(n-) matri determinant is the sum of terms involving n- determinants of (n-)(n-) matrices and so on. Since we know how to calculate the determinant of a matri we can always use this definition to find the determinant of an nn matri 4. In practice we shall not go above matrices (unless using a computer program) but we need to know the general formula for an inverse

11 General properties of determinants A 4 B 4 ) If B A, then det. B det. A ) If B is constructed from A by swapping two rows, then det. B -det. A ) If B is constructed from A by swapping two columns, then det. B -det. A 4) If B is constructed from A by multiplying one row (or column) by a constant, c, then det. B c det. A 5) If B is constructed from A by adding a multiple of one row to another, then det. B det. B

12 Determinants of triangular matrices A B Are eamples of respectively an upper triangular and a lower triangular matri (zeros below or above the main diagonal) The determinant of either an upper or lower triangular matri is equal to the product of the elements on the main diagonal Eg det.a (4-) - () + () 4 *4*6

13 Determinants For a matri, using j n det( A) A ( ) a det( A ) j + j j j a a a A a a a a a a a a a a a a A a a + a a a a a a a Question: What is the determinant of A

14 Method : Laplace epansion of an n n matri. Can generalise this rule for the determinant of any n by n matri j n j n + j A ( ) aj Mj aj Cij j j As part of this method, you need to know the following:. Minor M. Co-factor C (which are also essential to invert a matri)

15 nn Matri inversion - minors There is a minor M ij for each element a ij in the square matri.. To find it construct a new matri by deleting the row i and deleting the column j.. Then find the determinant of what s left. E.g. M A a a n a a n nn ; so M a a n a a n nn 4. Eample M A Delete row and column i. e. A 4 5 6; so M

16 N n matri inversion: Co-factor and adjoint matrices The cofactor is C ij is a minor with a pre-assigned algebraic sign given to it.for each element a ij, work out the minor.then multiply it by (-) i+j.in simpler language: if i+j is even then C ij M ij 4.If i+j is odd, then C ij -M ij 5.The co-factor matri is just C c cn c c n nn 6.The adjoint matri is C i.e. the transpose of C.

17 Co-factor, adjoint matrices and the inverse matri. The inverse matri, A - is just A adj. A C A A So i) find the determinant if it is non-zero, the matri is non-singular so its inverse eists ii) Find the cofactors of all the elements of A and arrange them in the cofactor matri iii) Transpose this matri to get the adjoint matri iv) Divide the adjoint matri by the determinant to get the inverse

18 Eample ( matri) A. If find A - Use the formula A adj. A C A A A a a ( aa aa) a a First find the determinant which is non-zero so can continue ( ) A aa aa ()() ()() Now find matri of cofactors, which in the case is a set of X determinants C C C C C

19 Eample ( matri) Now transpose the matri of cofactors to get the adjoint matri adj. A C ' Now using the formula above A adj. A C A A ( ) which is non-zero so can continue C C C C A aa aa ()() ()() then / / A ( ) NB. Always check that the answer is right by looking if AA - I AA + + ( ) ( ) + +

20 While Eample : matri ; M so A ; M ; ; C C ()( ) ()(6) ()( ) n n A a c a c a c

21 Eample continued ; A so A

22 Quiz. In each case find the matri of minors. Find the determinant. Find the inverse and check it. A B

23 Linear Equations Recall that linear equations are those where the variables enter in a linear form: e.g. 4 + is linear (variables appear in additive form) 4 + is not linear because of the term Linear equations can be written a more concise form Eg. 4 + can be written in terms of vector products 4 ( )

24 Matri form for linear equations If there are several simultaneous linear equations, they can be stacked in matri form e.g In general, a system of m linear equations in n unknowns can be represented as A b where A is an mn matri, is a n matri (or column vector) and b is an m matri Conversely A b can be interpreted as a system of m linear equations in n unknowns

25 Solving linear equation systems In general we want to find the solution to the equation system Ab There are three possible cases: ) There is no solution ) There is eactly one solution ) There is more than one solution (typically an infinite number) It is not always obvious which case applies Eample. No solution 4 This says that 4 + and + simultaneously Thus 4, which is nonsense Hence there is no solution

26 Solving linear equation systems Eample. One solution This says that 4 + and +. Thus - and 5 Eample. Many solutions This says that 4 + and 8 + So, all we can say is that 4- If, then 4; if then etc

27 Solving when there are n equations and n unknowns- Cramer s Rule This (n & n) is a special case (in general we may have more equations than unknowns or vice versa) If n unknowns then m n so in the equation A b, A is a square matri. Suppose det. A so that A - eists. If A b then A - A A - b or A - b and we have our solution Eample: So, A b A

28 Solving when there are n equations and n unknowns. No solution. Suppose b then if there is no solution A - does not eist i.e. det. A. Eample: 4 Since A (*) (*) solution A - does not eist and can t solve Ab for

29 Solving when there are n equations and n unknowns. Many solutions. Suppose b then if there are multiple solutions A - does not eist i.e. det. A. Eample: 4 8 So any 4- is a solution. In this case. Det. A -. Compare ) and ). When there are no solutions det. A and when there are multiple solutions, det. A. So when det. A all we know is that there isn t one solution. We don t know if we re in the no solution or the multiple solutions case. To find out, we need to check to see if the equations are consistent. In case. the equations are inconsistent hence there are no solutions. In case, the equations are consistent hence there are many solutions.

30 Quiz II.. Use matri inversion to find the solution to the following set of equations.. Why in this case is it quicker not to use matri inversion? 4

31 Part II Cramer s rule. Introduction: Cramer s rule Often when faced with Ab we are not interested in a complete solution for We may only wish to find or 4 Cramer s rule is a short cut for finding a particular i It s particularly useful when A is or bigger It is not sensible to use it if you need to find several i finding A - is generally quicker

32 Suppose you have the system of equations, A b Define the matri A i as the result of replacing in the i th column of A with b Eample. Replaces the n th column with the column vector b Eample. Replaces the st column with the column vector b Cramers rule n n nn n n b b a a a a n n n b a b a A 4 4 A so

33 Cramer s rule Suppose you have the system of equations A b, then, if det. A, () i Ai A So to find the value of the j th variable, replace the j th column of A by the vector of constants b - Intuitively the solution ( ) A b adj. A * b A effectively does () Eample. (Recall that the solution to this system was -, 5) 4 det. A det A ; det A So /- - and -5/

34 Quiz Suppose, 4 Use Cramer s rule to find and

35 Cramer s rule in macroeconomics Many macroeconomic models involve a system of linear equations Cramer s rule can be used to solve for one particular variable Eample. Consider a simple macro model of the economy Y C + I + G C a + by < b < I I G G Write this system in matri form then use Cramer s rule to find consumption, C Step : identify the endogenous variables and the eogenous variables The endogenous variables correspond to the vectors in the previous eample The eogenous variables are equivalent to the parameters of the system and correspond to the b vector Eample: here C and Y are endogenous. I and G are the eogenous variables

36 Cramer s rule in macroeconomics Step : Simplify the system of equations if possible then write down the system in such a way that all the endogenous variables are on one side of the equation and all the eogenous variables are on the other side Eample: simplify the equations Y C + I + G C a + by. Rewrite: Y - C I + G C by a Step. Put into matri form Matri form: b Y C I + G a

37 Cramer s rule in macroeconomics Step 4. then use Cramer s rule So, to find C we replace the second column of the matri with the column vector of parameters. Quiz II. Find Y using the same procedure. + a G I C Y b b G I b a b a b G I C ) (

38 Some guidance on solving mn equation systems Often the set of unknowns and equations to solve them are not equal The general problem involves m equations and n unknowns. Many systems of equations involve fewer equations than variables, m<n Some involve more equations than variables, n < m. In either case you cannot use matri inversion to characterise the solution (if it eists). Eample. When m n we seek to do two things:.find out if any solution eists..if at least one solution eists, identify its features.

39 Some guidance on solving mn equation systems. The rank of a matri provides a guide to the number of solutions Reminder: The rank of a matri is the largest number of linearly independent rows or columns. Note that the column rank and the row rank will be the same. Note that the rank cannot be larger than the smaller of m and n. i.e. if A is an mn matri rank(a) min(m,n) Eample. A The rank of this matri is at most, but in fact rank(a). Note that for an nn matri (det A ) rank(a) < n. We can see from the properties of determinants. If rank(a) < n we can add and subtract rows to create a row of zeros. The determinant of this new matri is therefore, but by property 5 adding and subtracting rows does not change the determinant. So det(a).

40 Some guidance on solving mn equation systems Find the rank of the system. Note that the maimum possible rank is min(m,n). Let s suppose this is n i. If rank(a) n, then there may be a unique solution ii. If rank(a) < n then there cannot be a unique solution

41 Summary skills you should be able to do Use Cramer s rule to solve for a variable Write down a linear macroeconomic system in matri form Work out the rank of a matri Characterise the solution of Ab.

42 Markov Chains A common application of matrices in macroeconomics is based on the idea of modelling changes over time as a Markov process where the levels of any variables of interest are modelled as Eg the vector t M t t- t t Unemployed t Employed and the transition matri M t ee ue.95. eu uu.5.7 Gives the set of probabilities of moving between the (two) states during time t- and t Eg euno. flowing from E to U/No. in employment at time t- Note ee eu (column probabilities add to one)

43 Note that M Employed ee ue Employed t t t t t Unemployed t eu uu Unemployed t Employedt ee* Employedt + ue* Unemployedt Unemployed eu * Employed + uu * Unemployed t t t So that the stock now proportion staying in that state + proportion moving in out of other state If these transition matrices hold at every point in time then can calculate a steady state stock as t M n t-n (ie the constant transition matri raised to the power n)

44 Eg in period M employed unemployed employed unemployed employed.55 unemployed Note that as number of time periods increases the matri M n will converge to a particular set of values where the columns are identical M the steady state transition probabilities that ensure the stocks of employment and unemployment are stable over time Note that you can calculate the inflows and outflows from each state as eu*e and ue*u in a steady state these flows should be equal

45

46 Some guidance on solving mn equation systems. Eample: This has a rank of. We can take the second row away from the first row to create: Or - and +. This is as far as we can go in defining a solution. One variable is free to take on any value which then determines the value of the other two variables. For eample, given any we can calculate the other two variables. In general, if the equation system is consistent, then the number of free variables is n-rank(a). b A

47 Practical tips on finding solutions If the matri is square then the first issue is whether the determinant is zero. So if the matri is square you need to first find its determinant. Eample: So, A - eists. In fact, Thus 4 ) ()( ) )( ()( ) ()(. det c a c a c a A A b A

48 Practical tips on finding solutions What s the solution to this set of linear equations? Obviously this is related to the last set of equations - has been relabelled as and vice versa. So, Note that the new inverse is obtained from the old inverse by swapping columns not rows A

49 Practical tips on finding solutions Recall, general properties of determinants. ) If B A T, then det. B det. A. ) If B is constructed from A by swapping two rows, then det. B -det. A ) If B is constructed from A by swapping two columns, then det. B -det. A 4) If B is constructed from A by multiplying one row (or column) by a constant, c, then det. B c det. A 5) If B is constructed from A by adding a multiple of one row to another, then det. B det. B.

50 Practical tips on finding solutions Some implications of properties of determinants. ) You can use any row to calculate the determinant (often there are easy rows to use). A 4 Using row, det. A (-)+4(+)+6(-) 6 Using row, det. A (.-.6) 6 ) If one row (column) is a multiple of another row (column) then det. A. ) If one row (column) can be constructed by adding/subtracting multiples of other rows (columns) then det. A. 6

51 Quiz III.. Show that all the determinants equal 5 5 ; ; C B A