EC5555 Economics Masters Refresher Course in Mathematics September 2013

Transcription

1 EC5555 Economics Masters Reresher Course in Mathematics September 3 Lecture 5 Unconstraine Optimization an Quaratic Forms Francesco Feri

2 We consier the unconstraine optimization or the case o unctions with many variables: where is a vector ma () subject to S To ace this topic we nee some preliminariy notions: - Quaratic orms - Concavity an conveity o unctions o many variables

3 Deinition o quaratic orms A orm is a polynomial unction in which each component has the same eponent sum: - a linear orm is (,y,z) = 4 9y + z (each term has eponents that a to one (the irst egree ) - a quaratic orm is ( y z) = 4 + zy z +z (each term has eponents that a to two (the secon egree ) A polynomial equation in which each term is o the n egree (sum o the integer eponents = ) is a quaratic orm Deinition A quaratic orm in n variables is a unction Q,. n = b + b + +b ij i j + + b nn n = n n = b ij i j i= j= where b ij or i =, n an j =, n n are constants. 3

4 Eample The unction Q, = is a quaratic orm in two variables. We can write it using matrices Q(, ) = 3 5 Note: we can simpliy this unction Q, = + 5 That can be written as: Q(, ) = Where the matri is symmetric. 4

5 In general we can in act write any quaratic orm as Q() = A where - is the column vector o i 's an - A is a symmetric n n matri or which the (i, j)th element is a ij = (/)(b ij + b ji ) note that i j = j i or any i an j, so that b ij i j + b ji j i can be written as (b ij +b ji ) i j or (b ij + b ji ) i j + (b ij + b ji ) j i 5

6 Eample Q, = + a + b c Q, = 3 a + b c a + b c 5 3 6

7 Conitions or einiteness With quaratic orms there are ways o establishing whether their signs are positive or negative an this will help etermine whether the unction o interest is concave or conve Deinition Let Q() be a quaratic orm, an let A be the symmetric matri that represents it (i.e. Q() = 'A). Then the associate matri A (an the quaratic orm) is:. positive einite i 'A > or all. negative einite i 'A < or all 3. positive semieinite i 'A or all 4. negative semieinite i 'A or all 5. ineinite i it is neither positive nor negative semieinite (i.e. i 'A > or some an 'A < or some ). 7

8 Eamples ) a + c = a c is positive einite or a, c > because a + c > or a, c > an ) + + = is positive semieinite because we can write it as ( + ) that is non negative or all, It is not positive einite because or =, = its value is. 3) Prove that the orm + a is ineinite or any value o a 8

9 Positive or Negative einite matrices To obtain conitions or an n-variable quaratic orm to be positive or negative einite, we nee to eamine the eterminants o some o its submatrices. Deinition: The leaing principal matrices o a nn square matri are the matrices oun by eleting. The last n- rows an columns to give D. The last n- rows an columns to give D an the original matri D n Deinition: The leaing principal minors o a matri are the eterminants o these leaing principal matrices. 9

10 Eample: the leaing principal matrices are then D = A an (D = ) an the eterminants (principal minors) are 5 an Eample. Fin D D an D 3 3 A A

11 I a square matri is negative einite then the leaing principal minors have the ollowing signs D ; D ; D3... a positive einite matri requires leaing principal minors are all positive, i.e. D ; D ; D3... To check i a square matri is negative semi-einite we have to compute all principal minors (not only the leaing principal minors)

12 Positive or Negative semieinite matrices To obtain conitions or an n-variable quaratic orm to be positive or negative semieinite, we nee to eamine the eterminants o some o its submatrices. Deinition: The principal matrices o a nn square matri are the matrices oun by eleting. n- rows an columns in all possible combinations. n- rows an columns in all possible combinations an the original matri Deinition: The principal minors o a matri are the eterminants o the principal matrices.

13 Let A = a b b c The irst-orer principal minors o A are a an c, an the secon-orer principal minor is the eterminant o A, namely ac b. 3

14 Let A = This matri has 3 irst-orer principal minors, obtaine by eleting the last two rows an last two columns the irst an thir rows an the irst an thir columns the irst two rows an irst two columns which gives us simply the elements on the main iagonal o the matri: 3,, an. The matri also has 3 secon-orer principal minors, obtaine by eleting the last row an last colunm the secon row an secon column the irst row an irst column which gives us 4,, an. The matri has one thir-orer principal minor, namely its eterminant, 9. 4

15 Let A be an n n symmetric matri. Then: A is positive semieinite i an only i all the principal minors o A are nonnegative. A is negative semieinite i an only i all the kth orer principal minors o A are i k is o an i k is even. Eample 4 The two irst-orer principal minors an an, an the secon-orer principal minor is. Thus the matri is positive semieinite. 5

16 Proceures or checking the einiteness o a matri. Fin the leaing principal minors an check i the conitions or positive or negative einiteness are satisie. I they are, you are one.. the conitions are not satisie, check i they are strictly violate. I they are, then the matri is ineinite. 3. I the conitions are not strictly violate, in all its principal minors an check i the conitions or positive or negative semieiniteness are satisie. Note that i a matri is positive einite, it is certainly positive semieinite, an i it is negative einite, it is certainly negative semieinite 6

17 Three Variable Quaratic Forms Can always write a quaratic orm in 3 variables q = + y + 3 z + y + y + 3 yz + 3 z + 3 zy 33 z in matri orm A where = (, y, z) an A is a symmetric 3 by 3 matri T ' A y z y z here are now 3 leaing principal minors rom the iscriminants o A D ; D ; D

18 Once again can convert into an epression where the 3 variables appear only as square terms D D D ) ( z z y z y q 8

19 An can show that q < (>) i the terms outsie the brackets are all negative (positive) an these terms are respectively: D ; D D ; D 3 D I D <, D >, D 3 < the matri is sai to be negative einite i D >, D >, D 3 > the matri is sai to be positive einite 9

20 Characteristic root test A secon test to check einiteness Given an n n: matri D, we in a scalar r an an n vector such that: D = r r is the characteristic root o matri D (or eigenvalue) is the characteristic vector o matri D (or eigenvector) This equation is rewritten as: (D ri) = The conition that satisies this is i the matri (D ri) is singular; i.e., its eterminant is zero The iea is to solve or r an then

21 Eample D r r r ri D 6 r r r r ri D So the characteristic roots are r = 3 an r = For r = ) ( ri D Note that the rows o the matri are linearly epenent as epecte or a singular matri giving an ininite number o solutions =

22 To orce out a unique solution, we nee to normalise by imposing a restriction: an in general or n unknowns n i i - This is arbitrary but whichever rule is chosen, all subsequent values will be relate Then an () 5 / 5; / 5 Thus, the st characteristic vector (eigenvector) is 5 5 an or r =, the n characteristic vector (eigenvector) is Properties: ) normalisation implies that the prouct o characteristic vectors, i.e. = ) Each pair o characteristic vectors are orthogonal, i.e. = ' 5 5 '

23 Characteristic root test or the sign einiteness o a matri D. D is positive einite i an only i every characteristic root is positive, i.e. >. D is negative einite i an only i every characteristic root is negative, i.e. < 3. D is positive einite i an only i every characteristic root is nonnegative, i.e. 4. D is negative einite i an only i every characteristic root is nonpositive, i.e. 3

24 When the unction consists o more than variable the concavity conition is very similar we replace the variable by the vector, an remember to get the orer o multiplication right. Now We also nee the vector o irst partial erivatives o, an the matri o secon orer partial erivatives, H J is calle Jacobian o the unction H is calle the Hessian o the unction ' ( ' ) ' n n n ; n n n n H 4 Fining i a unction with more variables is concave

25 The concavity conition is now: + The Taylor approimation o is now + + H + Replacing in the irst equation we get H + Simpliying we get H Then matri H has to be a negative semi-einite matri 5

26 Let be a unction o many variables with continuous partial erivatives o irst an secon orer on the conve open set S an enote the Hessian o at the point by H(). Then is concave i an only i H() is negative semieinite or all S i H() is negative einite or all S then is strictly concave is conve i an only i H() is positive semieinite or all S i H() is positive einite or all S then is strictly conve. Note to say that is concave (conve) we nee to prove that H() is negative (positive) einite 6

27 Putting it all together So given a unction () To in out whether the unction is concave we nee to know i H i.e. whether H is negative semi-einite.fin the Hessian matri o secon orer erivatives, H.From H in the leaing principal matrices by eliminating:. The last n- rows an columns written as D. The last n- rows an columns written as D The original matri - D n

28 3. Compute the eterminants o these leaing principal matrices 4. i the eterminants have the ollowing pattern (with not all zero): D <, D >, D 3 <, then is concave; i the eterminants are all strictly positive then is conve 5. i some conition is violate by equality you nee to check the sign o all principal minors (conition or semieiniteness) 6. i these conitions o not hol you ve prove that the unction is not concave or conve

29 Fin whether the unction () = is concave We nee the Hessian matri o secon orer erivatives, H - The Jacobian is = - The Hessian is H 9

30 From H in the leaing principal matrices by eliminating:.the last n- rows an columns written as D = ().The last n- rows an columns written as D = H Compute the eterminants o these leaing principal matrices..det. D =.Det. H = 4 which is negative is concave i the leaing principal minors are is conve i the leaing principal minors are Leaing principal minors o not have one o this patterns so is not concave, not conve H ; ; D D 3 ; ; D D

31 UNCONSTRAINED OPTIMIZATION WITH MORE VARIABLE We generalize the results or a single variable to the case o many variables Consier the problem: ma () subject to S where is a vector Proposition Let be a ierentiable unction o n variables eine on the set S. I the point in the interior o S is a local or global maimizer or minimizer o then i () = or i =,, n. (Note, i means the i th partial erivative o ) Then the conition that all partial erivatives are equal to zero is a necessary conition or an interior optimum (an thereore or an optimum in an unconstraine optimization where each element o coul be any o the real numbers. 3

32 Conitions uner which a stationary point is a local optimum Let be a unction o n variables with continuous partial erivatives o irst an secon orer, eine on the set S. Suppose that * is a stationary point o in the interior o S (so that i '(*) = or all i). I H(*) is negative einite then * is a local maimizer. I * is a local maimizer then H(*) is negative semieinite. I H(*) is positive einite then * is a local minimizer. I * is a local minimizer then H(*) is positive semieinite. where H() enotes the Hessian o at. 3

33 Conitions uner which a stationary point is a global optimum Suppose that the unction has continuous partial erivatives in a conve set S an let be in the interior o S.. i is concave then is a global maimizer o in S i an only i it is a stationary point o. i is conve then is a global minimizer o in S i an only i it is a stationary point o. H(z) is negative semieinite or all z S [ is a global maimizer o in S i an only i is a stationary point o ] H(z) is positive semieinite or all z S [ is a global minimizer o in S i an only i is a stationary point o ], where H() enotes the Hessian o at. 33

34 Eample : Unconstraine Maimization with two variables For eample Utility = U(, y) or Output = F(K, L) Now try to in the values o an y which maimise a unction (, y) Three steps:. Set both st orer conitions equal to zero = an y = (the slope o the unction with respect to both variables simultaneously zero). Solve the equations simultaneously or an y must be However this is a necessary but not suicient conition (sale points, points o inlection) 3. Secon orer conitions H = y y yy <, yy < an yy y > 34

35 (,y) = 4 + y y. (i). = y = (ii). y = y =. Solve: rom (ii) we have = y insert into (i) to get = or 4 = or = so y = = 3. H = y y yy = 4 The irst orer leaing principal minor is = -4 < The secon orer leaing principal minor is yy y = (-4)(-) () = 4> Then the matri H is negative einite is (strictly) concave, so we have a maimum point where = an y =

36 Quiz Maimize. z = y + y. z = y + y 36

37 Eample Maimize () = The irst orer conitions are: Is this a maimum? it will be i unction is concave. H is, 4 4 H 37

38 From H in the leaing principal matrices by eliminating:. The last n- rows an columns written as D = (-). The last n- rows an columns written as D = H Compute the eterminants o these leaing principal matrices.. D =. H = 8 Then the matri H is negative einite is (strictly) concave the values o which satisy FOC ( an ) give a maimum. 38

39 Eample 3 Total revenue R = q + 8q Total Cost = q + q q +q Fin the values o q an q that maimise proit Proit = revenue cost = q + 8q - (q + q q +q ) The irst orer conitions are: q 4q q 8 q 4q q Solving or q an q gives q = an q =4 Is this a maimum? it will be i unction is concave 39

40 The Hessian is From H in the leaing principal matrices by eliminating:.the last n- rows an columns written as D = (-4).The last n- rows an columns written as D = H Compute the eterminants o these leaing principal matrices.. D = 4. H = ( 4) ( 4) = 5 So H is negative einite, then is (strictly) concave an the values or q an q maimise proits 4 4 q q q q q q H 4

41 Eample with three variables Maimize () = 3 The irst orer conitions are: The Hessian is: H 4

42 From H in the leaing principal matrices by eliminating:. The last n- rows an columns D = ( ). The last n- rows an columns D = 4 3. The last rows an columns D 3 = H. Compute the eterminants o these leaing principal matrices.. D =,. D = 8 3. H = 6 H is negative einite, then is (strictly) concave 4

43 Summing up two variable maimization. Dierentiate () an solve the irst orer conitions are:. Check concavity o to see i the conitions represent a maimum. a. We compute the Hessian b. We check i it is negative einite c. i.e. check i, or all an, an or 43

44 3. I these conitions hol, H is negative einite, is strictly concave an the stationary point is a maimum 4. I these conitions are violate by equality, i.e. are equal to zero, check the conitions or semi einiteness 5. I these conitions hol, H is negative semieinite, is concave an the stationary point is a maimum 6. I these conitions are violate, we nee urther investigation 44

45 Summing up 3 variable maimization. Dierentiate () an solve the the irst orer conitions are: 3. Check concavity o to see i the conitions represent a maimum. a. We compute the Hessian b. We check i it is negative einite or

46 3. I these conitions hol, H is negative einite, is strictly concave an the stationary point is a maimum 4. I these conitions are violate by equality, i.e. are equal to zero, check the conitions or semi einiteness, I these conitions hol, H is negative semieinite, is concave an the stationary point is a maimum 6. I these conitions are violate, we nee urther investigation 46