Basics of Calculus and Algebra

Transcription

1 Monika Department of Economics ISCTE-IUL September 2012

2 Basics of linear algebra Real valued Functions Differential Calculus Integral Calculus Optimization

3 Introduction I A matrix is a rectangular array of numbers, symbols, or expressions, arranged in rows and columns. The individual items in a matrix are called its elements or entries. Examples of matrices: A = ( ) B = [ ]

4 Introduction II A n m matrix (n and m are called dimensions) has the form a 11 a a 1n a 21 a a 2n... a m1 a m2... a mn Entries have the form a ij, where the first index i = 1, 2,..., m denotes the row, and the second index j = 1, 2,..., n denotes the column.

5 Introduction III Special cases: A n n matrix is called a square matrix of a dimension n. A matrix for which n m is called rectangular. Given a square matrix, we call a main diagonal, a diagonal formed of elements a ij for which i = j. These entries are called main elements. A matrix for which all expect for the main elements are equal to 0 is called a diagonal matrix. If all the main elements of a diagonal matrix are equal to 1, we call this matrix unit or identity matrix. If only all entries above (below) the main diagonal are zero, a matrix is called a lower triangular matrix (upper triangular matrix, respectively).

6 Basic Operations I Matrix addition: Matrix addition is the operation of adding two matrices by adding the corresponding entries together: C = A + B c ij = a ij + b ij It is only possible if the matrices are of the same dimension.

7 Basic Operations I Matrix addition: Matrix addition is the operation of adding two matrices by adding the corresponding entries together: C = A + B c ij = a ij + b ij It is only possible if the matrices are of the same dimension. Example Consider matrices A, B, C and D [ ] [ A =, B = [ ] [ ] C = D = ]

8 Basic Operations II [ 2 3 A + C = 2 1 [ 3 3 = 4 6 ] ] [ ] = [ ] [ B + D = = ] [ ] [ = ] [ ]

9 Basic Operations III Scalar multiplication: The scalar multiplication λa of a matrix A and a number λ is given by multiplying every entry of A by λ: B = λa b ij = λa ij λ R

10 Basic Operations III Scalar multiplication: The scalar multiplication λa of a matrix A and a number λ is given by multiplying every entry of A by λ: B = λa b ij = λa ij λ R Example Given a real number and a matrix: λ = 3 ; A = [ ] we have λa = 3 [ ] = [ ( 2) ] = [ ]

11 Basic Operations IV Multiplication: A product of two matrices is calculated by multiplying the rows of the first matrix by the columns of the second matrix. It is, therefore only possible if the number of columns of the first matrix is equal to the number of rows of the second matrix. Definition A multiplication of a matrix A of dimensions m n bya matrix B of dimensions p q is possible if n = p, and the product is a matrix C, of dimensions m p, elements of which are of the form: c ij = a i1 b 1j + a i2 b 2j a in b nj = n a ik b kj k=1 where { i = 1, 2,, m j = 1, 2,, p

12 Basic Operations V Example Find a product of A and B, where A (2 3) = [ ], B (3 2) = It is possible to find a product A 2 3 B 3 2 and the result will be a matrix C 2 2. It is not possible to find a product B A!

13 Basic Operations VI A B = [ ] [ ( 1) ( 1) ( 1) = ( 1) [ ] [ ] = = ]

14 Basic Operations VII Kronecker Product: If A is n m matrix and B is p q matrix, then the Kronecker product A B is the mp nq block matrix a 11 B a 1n B A B = a m1 B a mn B More explicitly, we have a 11b 11 a 11b 12 a 11b 1q a 1nb 11 a 1nb 12 a 1nb 1q a 11b 21 a 11b 22 a 11b 2q a 1nb 21 a 1nb 22 a 1nb 2q a 11b p1 a 11b p2 a 11b pq a 1nb p1 a 1nb p2 a 1nb pq A B = a m1b 11 a m1b 12 a m1b 1q a mnb 11 a mnb 12 a mnb 1q a m1b 21 a m1b 22 a m1b 2q a mnb 21 a mnb 22 a mnb 2q a m1b p1 a m1b p2 a m1b pq a mnb p1 a mnb p2 a mnb pq

15 Basic Operations VIII Example Find A B where and A = B [ ] [ ] [ ] [ ] 0 5 = =

16 Basic Operations IX Transpose: The transpose of a matrix A is another matrix A T (also denoted: A ) created by any one of the following equivalent actions: reflect A over its main diagonal (which runs top-left to bottom-right) write the rows of A as the columns of A T write the columns of A as the rows of A T If A is m n then A T is n m. [A T ] ij = [A] ji Example A = [ ] A T =

17 Basic Operations IX Transpose: The transpose of a matrix A is another matrix A T (also denoted: A ) created by any one of the following equivalent actions: reflect A over its main diagonal (which runs top-left to bottom-right) write the rows of A as the columns of A T write the columns of A as the rows of A T If A is m n then A T is n m. [A T ] ij = [A] ji Example A = [ ] A T = A symmetric matrix is a matrix which is equal to its transpose. If A = A T the matrix is called antisymmeric.

18 Basic Operations X Row operations: The following operations, known as elementary row operations are allowed between the rows of a matrix: A row within the matrix can be switched with another row. Each element in a row can be multiplied by a non-zero constant. A row can be replaced by the sum of that row and a multiple of another row. A rank of a matrix A - r(a), is the maximum number of linearly independent row vectors of A. It can be obtained by transforming a matrix into a upper triangular matrix (row echelon form) by row operations and calculating the number of non zero rows.

19 Determinants I Every square matrix A of a dimension n can be assigned a real number called a determinant. It is denoted det(a) or A = [a ij ] i,j=1,...,n det (A) = A A determinant of a matrix that contains one element, is equal to this element: A = [12] A = 12 = 12

20 Determinants II A determinant of a 2 2 matrix can be found using the following formula: [ ] a11 a A = 12 A = a 21 a 22 a 11 a 12 a 21 a 22 = a 11a 22 a 12 a 21 More precisely, it is a difference between the product of the main diagonal elements and the elements on the other diagonal: A = = ( 1) = 17

21 Determinants III Determinant of a 3 3 matrix can be found using the Sarrus rule. Consider a matrix: A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33

22 Determinants IV The Sarrus rule: A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 a 11 a 12 a 13 a 21 a 22 a 23 A = (a 11 a 22 a 33 + a 21 a 32 a 13 + a 31 a 12 a 23 ) (a 13 a 22 a 31 + a 23 a 32 a 11 + a 33 a 12 a 21 ) Actually, the same trick can be done with columns!

23 Determinants V Example Find the determinant of: A = A = ( ) ( ) = ( ) ( ) = = 1

24 Determinants VI A minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows or columns. The minor M i,j is defined to be the determinant of the (n1) (n1) matrix that results from A by removing the i-th row and the j-th column. Example: A = , D 12 = D 33 = = 1 1 (0) (2) = = 2 0 ( 1) ( 1) = 1

25 Determinants VI Determinant can be found using the Laplace formula. The determinant of A is given by: det(a) = n ( 1) i+j a i,j M i,j = j=1 n ( 1) i+j a i,j M i,j i=1

26 Determinants VII Example For the example 3-by-3 matrix A = [ ] det(a) = ( 1) det 2 1 [ ] ( 1) det ( 1) det [ ] = ( 2) (( 1) ( 1) 2 3) + 1 (( 2) ( 1) 2 ( 3)) = ( 2) ( 5) + 8 = 18

27 Inverse of a matrix I An inverse of a matrix A (here B) is matrix that satisfies If B exists it is denoted A 1. AB = BA = I A matrix is invertible or nonsingular if det(a) 0 or in other words it has a full rank its rank is equal to its dimension. There are several ways to find the inverse.

28 Inverse of a matrix II Using the adjugate matrix: Define the (i, j) cofactor of A as:  ij = ( 1) i+j M ij Define the cofactor matrix of A as a n n matrix  whose (i, j) is the (i, j) cofactor of A. The adjugate of A is the transpose of the cofactor matrix of A: adj(a) = ÂT

29 Inverse of a matrix II Using the adjugate matrix: Define the (i, j) cofactor of A as:  ij = ( 1) i+j M ij Define the cofactor matrix of A as a n n matrix  whose (i, j) is the (i, j) cofactor of A. The adjugate of A is the transpose of the cofactor matrix of A: adj(a) = ÂT The inverse can be found as: A 1 = 1 (ÂT ) A ij = 1 ) (Âji A = 1 A  11  21  n1  12  22  n  1n  2n  nn

30 Inverse of a matrix III Example Find the inverse of (if it is possible!) [ 2 3 A = 5 1 ] A 1 = ÂT A [ ] 1 5 Â = 3 2 ÂT = [ ]

31 Inverse of a matrix IV Then: A 1 = ÂT A = 1 [ [ ] 1/17 3/17 = 5/17 2/17 ]

32 Inverse of a matrix V Using Gauss Jordan elimination: This can be done by augmenting the square matrix with the identity matrix of the same dimensions and applying the following matrix operations: [AI] A 1 [AI] [IA 1 ] Example: A = Augment by an identity matrix: [AI] =

33 Inverse of a matrix VI Perform elementary row operations on the augmented matrix until it reaches: [IA ] = Finally we have: I = A 1 =

34 Proprieties of determinants If a matrix has a zero row its determinant is 0; Also if two or more rows are identical. A = A T, A 1 = A 1 Interchanging two rows or columns multiplies the determinant by 1. In general a permutation of the rows or columns multiplies the determinant by the sign of the permutation. det(ab) = det(a) det(b) det(ca) = c n det(a) for a n n matrix. If A is a triangular matrix, the determinant equals the product of the entries on the main diagonal: det(a) = a 1,1 a 2,2 a n,n = n i=1 a i,i

35 Eigenvalues I Let A be a square matrix of a dimension n. An eigenvalue of A is a scalar λ which satisfies det(a λi n ) = 0 A matrix can have between 1 and n eigenvalues. The equation det(a λi n ) = 0 is known as the characteristic equation and it is a polynomial of degree n of variable λ of the form: det (A λi n ) = λ n + c n 1 λ n 1 + c n 2 λ n c 1 λ + c 0,

36 Eigenvalues II Example Find the eigenvalues of A. A = [ We need to find the characteristic polynomial of A, given by ([ ] [ ]) 5 2 λ 0 det (A λi 2 ) = det λ [ ] 5 λ 2 = det 2 2 λ ]. = (λ + 5)(λ + 2) 4 = λ 2 + 7λ + 6

37 Eigenvalues III We solve the characteristic equation: det (A λi 2 ) = 0 λ 2 + 7λ + 6 = 0 (λ + 1)(λ + 6) = 0 The solutions are λ 1 = 1 and λ 2 = 6 which are the eigenvalues.

38 Basics of linear algebra Real valued Functions Differential Calculus Integral Calculus Optimization

39 Introduction A function of one real variable f : D R R is given by the correspondence y = f (x) where x independent variable y dependent variable A function of two real variables f : D R 2 R is given by the correspondence z = f (x, y) where x, y independent variables z dependent variable Example Cobb-Douglas Production Function: f : R 2 + R +, f (k, l) = k α l β, where k (capital), l (labour) are independent variables and production z = f (k, l) is the dependent variable.

40 Convex vs. concave functions A convex function: a function f : [a, b] R R is convex if the graph of the function lies below the line segment joining any two points of the graph that is if for x and y on [a, b] and t [0, 1], it holds that f(tx + (1 t)y) tf(x) + (1 t)f(y) Equivalently, a function is convex if its epigraph (the set of points on or above the graph of the function) is a convex set. A concave function: a function f : [a, b] R R is concave if it is not convex.

41 Exponential function Exponential function: f : R R + Properties: f (x) = e x e f (x) = a x, a > 0 e A e B = e A+B e A, e B = ea B, a x = e x ln a a x b x = (ab) x, e 0 = 1, e = 0, e + = +

42 Logarythmic function Logarythmic function: f : R + R Properties: f (x) = log a x, a > 0, a 1 a = e log natural ln ln A + ln B = ln (AB), ln A ln B = ln A ln B = ln B A, ln 1 = 0, ln e = 1 ln 0 + =, ln(+ ) = + ( ) A B

43 Power function Power function: f : R R f (x) = ax k, a, k R

44 Basics of linear algebra Real valued Functions Differential Calculus Integral Calculus Optimization

45 Derivatives I Differentiation is a method to compute the rate at which a dependent variable y changes with respect to the change in the independent variable x. This rate of change is called the derivative of y with respect to x. If x and y are real numbers, and if the graph of y is plotted against x, the derivative measures the slope of this graph at each point.

46 Derivatives I Differentiation is a method to compute the rate at which a dependent variable y changes with respect to the change in the independent variable x. This rate of change is called the derivative of y with respect to x. If x and y are real numbers, and if the graph of y is plotted against x, the derivative measures the slope of this graph at each point. If y = f(x) is differentiable at a, then f must also be continuous at a (necessary condition). The graph of a differentiable function must have a non vertical tangent line at each point in its domain no kinks, jumps etc.

47 Derivatives II Here: k is a real number and u, v are real valued functions of one variable. Rules of differentiation: (ku) = ku (u + v) = u + v ( ) x k ( ) = kx k 1 u k = ku k 1 u ( (uv) u ) = u v + uv u v uv = v v 2 (e x ) = e x (e u ) = u e u (a x ) = a x ln a, a > 0 (a u ) = u a u ln a, a > 0 (ln x) = 1 x (ln u) = u u

48 Derivatives III Some examples: 1 f (x) = 4x 3 = f (x) = 12x 2 = f (x) = 24x 2 f (x) = (x 1) 2 = f (x) = 2 (x 1) = f (x) = 2 3 f (x) = 2 x 2 = f (x) = 2 ( x 2) = f (x) = 4 x 3 4 f (x) = x 3 e x = f (x) = x 2 e x (3 + x) 5 f (x) = (ln x) 4 = f (x) = 4 ln3 x x

49 Partial Derivatives I A (first order) partial derivative of f(x, y) with respect to x is denoted f f(x, y) x is a derivative of f wrt x while treating y as constant. A (first order) partial derivative of f(x, y) with respect to y is denoted f f(x, y) y is a derivative of f wrt y while treating x as constant.

50 Partial Derivatives II The function f(x, y) is differentiable at the point (a, b) if the partial derivatives are continuous and finite at this point. In this case the first order differential of function f at point (a, b) is given by: df(a, b) = ( f x ) (a, b) dx + where dx and dy are infinitesimals. ( f y ) (a, b) dy

51 Partial Derivatives II The function f(x, y) is differentiable at the point (a, b) if the partial derivatives are continuous and finite at this point. In this case the first order differential of function f at point (a, b) is given by: df(a, b) = ( f x ) (a, b) dx + where dx and dy are infinitesimals. ( f y ) (a, b) dy When the function is differentiable at (a, b), we can use the following approximate formula to calculate value of the function in the neighborhood of the point: f(a + h, b + k) f(a, b) + dx ( ) f + dy x (a,b) ( ) f y (a,b)

52 Partial Derivatives III When we calculate the partial derivatives of the partial derivatives of the first order, we obtain four partial derivatives of the second order: ( f x ( f y ) x ) x = x = x ( ) f x ( ) f y = 2 f x 2 ; = 2 f y x ; ( f x ) ( f y = ( ) f = 2 f y y x x y ) ( ) f = 2 f y y 2 y = y

53 Partial Derivatives III When we calculate the partial derivatives of the partial derivatives of the first order, we obtain four partial derivatives of the second order: ( f x ( f y ) x ) x = x = x ( ) f x ( ) f y = 2 f x 2 ; = 2 f y x ; ( f x ) ( f y = ( ) f = 2 f y y x x y ) ( ) f = 2 f y y 2 y = y A second order differential at point (a, b) is defined as: ( d 2 2 ) ( f f(a, b) = x 2 (a, b) dx 2 2 ) f + 2 (a, b) dxdy y x ( 2 ) f + y 2 (a, b) dy 2

54 Partial Derivatives IV Example: The function f(x, y) = x 3 + 2y 2 has the following first and second order partial derivatives: f (x, y) = 3x2 x f (x, y) = 4y y 2 ( f f (x, y) = x2 x 2 ( f f (x, y) = y x x 2 ( f f (x, y) = y2 y ) x ) y ) y = ( 3x 2) x = 6x = ( 3x 2) y = 0 = (4y) y = 4

55 Partial Derivatives V Example cont d: The first and second differentials of f at point (1, 2) are given by: ( ) ( ) f f df(a, b) = (1, 2) dx + (1, 2) dy x y = ( 3x 2) (1, 2) dx + (4y) (1, 2) dy = 3dx + 8dy ( d 2 2 ) ( f f(a, b) = x 2 (a, b) dx 2 2 ) f + 2 (a, b) dxdy y x ( 2 ) f + y 2 (a, b) dy 2 = (6x) (1, 2) dx (0) (1, 2) dxdy + 4 (1, 2) dy 2 = 6dx 2 + 4dy 2

56 Basics of linear algebra Real valued Functions Differential Calculus Integral Calculus Optimization

57 Introduction I Throughout this part we refer to Riemann integration. An indefinite integral is a function F, whose derivative is the given function f: F = f(x)dx A definite integral b a f(x)dx is the area bounded by the graph of f and the vertical lines x = a and x = b. The two definitions are connected by the formula: b a f(x)dx = F (b) F (a)

58 Introduction II Notation f(x)dx represents integration dx indicates the variable of integration, thus that we are integrating over x For definite integration we might have f(x)dx when we integrate over a domain D or D b if the domain is an interval [a, b]. a f(x)dx

59 The Fundamental Theorem of Calculus Theorem Let f be a continuous real valued function defined on a closed interval [a, b]. Let F be the function defined for all x in [a, b] by F (x) = x a f(t)dt Then, F is continuous on [a, b], differentiable on (a, b) and for all x in (a, b) F (x) = f(x)

60 Some Formulas 1 b f(x)dx = for a > b a a b f(x)dx

61 Some Formulas 1 b f(x)dx = a a b for a > b 2 a f(x)dx = 0 a f(x)dx

62 Some Formulas 1 b f(x)dx = a a b for a > b 2 a f(x)dx = 0 a f(x)dx 3 b f(x)dx = c f(x)dx + b a a c f(x)dx

63 Computation of Integrals I Some integrals can be found directly with the help of formulas which can be found in tables. Examples: x a dx = xa+1 a C 1 dx = ln x + C x e x dx = e x + C cos xdx = sin x + C

64 Computation of Integrals II: Integration by Parts Recall the formula (f(x)g(x)) = f (x)g(x) + f(x)g (x)

65 Computation of Integrals II: Integration by Parts Recall the formula (f(x)g(x)) = f (x)g(x) + f(x)g (x) Integrate the whole expression: f(x)g(x) = f (x)g(x)dx + f(x)g (x)dx

66 Computation of Integrals II: Integration by Parts Recall the formula (f(x)g(x)) = f (x)g(x) + f(x)g (x) Integrate the whole expression: f(x)g(x) = f (x)g(x)dx + f(x)g (x)dx Rearrange: f(x)g (x)dx = f(x)g(x) f (x)g(x)dx

67 Computation of Integrals III: Integration by Parts Example Find xe x dx 1 According to our formula we set f(x) = x and g (x) = e x

68 Computation of Integrals III: Integration by Parts Example Find xe x dx 1 According to our formula we set f(x) = x and g (x) = e x 2 We have f (x) = 1 and g(x) = g (x) = e x

69 Computation of Integrals III: Integration by Parts Example Find xe x dx 1 According to our formula we set f(x) = x and g (x) = e x 2 We have f (x) = 1 and g(x) = g (x) = e x 3 Substitute back to the formula: xe x dx = xe x 1 e x dx = xe x e x + C

70 Computation of Integrals IV: Integration by Substituion Recall the formula [f(g(t))] = f (g(t)) g (t)

71 Computation of Integrals IV: Integration by Substituion Recall the formula [f(g(t))] = f (g(t)) g (t) Integrate both sides twice: f(g(t))dt = g(b) g(a) f(x)dx = f(g(t))g (t)dt The trick is to find a substitution which makes it easy.

72 Computation of Integrals V: Integration by Substituion Example Find x cos(x 2 + 1)dx 1 Make the substitution u = x so we obtain du = 2xdx.

73 Computation of Integrals V: Integration by Substituion Example Find x cos(x 2 + 1)dx 1 Make the substitution u = x so we obtain du = 2xdx. 2 x cos(x 2 + 1)dx = 1 2x cos(x 2 + 1)dx 2 = 1 cos udu = sin u + C = 1 2 sin(x2 + 1) + C

74 Basics of linear algebra Real valued Functions Differential Calculus Integral Calculus Optimization

75 Extrema I Definition: Let f : A R R and a Int(A) a is a local (global) minimum of a function f if in the neighborhood V of a it holds that f (x) f (a), x V ( x A) a is a local (global) maximum of a function f if in the neighborhood V of a it holds that f (x) f (a), x V ( x A) Minima and maxima are jointly denoted as extrema (singular: extremum).

76 Extrema II Definition: Let f(x) be a differentiable function defined on A R with values in R. A necessary condition (First order condition) for existence of an extremum at point a A is that f (a) = 0 Points with f = 0 are known as stationary points.

77 Extrema II Definition: Let f(x) be a differentiable function defined on A R with values in R. A necessary condition (First order condition) for existence of an extremum at point a A is that f (a) = 0 Points with f = 0 are known as stationary points. Definition: A sufficient condition (second order condition), which characterizes whether a stationary point is a maximum, minimum or a saddle point is given by: if f (a) > 0 a is a minimum if f (a) < 0 a is a maximum if f (a) = 0 a is a saddle point

78 Extrema III Definition: Let f : A R 2 R and (a, b) Int(A) (a, b) is a local (global) minimum of a function f if in the neighborhood V of (a, b) it holds that f (x, y) f (a, b), x, y V ( x, y A) (a, b) is a local (global) maximum of a function f if in the neighborhood V of (a, b) it holds that f (x, y) f (a, b), x, y V ( x, y A)

79 Extrema IV A necessary condition (FOC) for existence of an extremum at a point (a, b) A is that ( ) f (x, y) = 0 x ( ) f (x, y) = 0 y

80 Extrema V Sufficient conditions, which determine whether a stationary point is a local maximum or a minimum are given by the determinant of the Hessian matrix evaluated at the stationary point. The Hessian matrix is defined as: 2 f H (x, y) = x 2 2 f x y 2 f x y 2 f y 2 (x,y)

81 Extrema VI If det(h(x, y)) > 0 and f xx (a, b) > 0 then (a, b) is a local minimum of f. If det(h(x, y)) > 0 and f xx (a, b) < 0 then (a, b) is a local maximum of f. If det(h(x, y)) < 0 then (a, b) is a saddle point of f. If det(h(x, y)) = 0 the test is inconclusive.

82 Extrema VI If det(h(x, y)) > 0 and f xx (a, b) > 0 then (a, b) is a local minimum of f. If det(h(x, y)) > 0 and f xx (a, b) < 0 then (a, b) is a local maximum of f. If det(h(x, y)) < 0 then (a, b) is a saddle point of f. If det(h(x, y)) = 0 the test is inconclusive. One can extend these results straightforwardly to a case or more than two variables.

83 Example I Example Find the extrema of The necessary conditions: f (x, y) = x 3 + 4xy y 2. { f x (x, y) = 0 { 3x 2 + 4y = 0 { 3x 2 + 4y = 0 f y (x, y) = 0 4x 2y = 0 y = 2x { { { 3x (2x) = 0 3x 2 + 8x = 0 x = 0 or x = 8/3 y = 2x y = 2x y = 0 or y = 16/3 Thus, we have two stationary points: (x, y) = (0, 0) and (x, y) = (8/3, 16/3).

84 Example II Sufficient conditions: The Hessian matrix is given by: [ ] [ ] f x (x, y) f H (x, y) = 2 xy (x, y) 6x 4 = f xy (x, y) f y (x, y) Evaluated at the point (x, y) = (0, 0) we obtain [ ] [ ] [ 6x H (0, 0) = = = (0,0) ]

85 Example III where D 1 = 2 = 2 < 0 D 2 = = 16 < 0 so the point (0, 0) is a saddle point.

86 Example IV For the point (x, y) = (8/3, 16/3) we obtain [ ] 6x 4 H (8/3, 16/3) = = 4 2 (8/3,16/3) [ 48/ ] where D 1 = 48/3 = 48/3 < 0 D 2 = 48/ = 48/3 > 0 so the point (8/3, 16/3) is a local maximum.

87 Conditional Optimization I Definition: The problem of conditional optimization consists of a real valued function f : A R 2 R (objective function) variables of which are constrained by a condition g(x, y) = 0 Finding of conditional extrema of the basic problem is equivalent to finding of extrema of the following function: L (x, y; λ) = f (x, y) + λg(x, y) known as the Lagrange function (Lagrangian). An auxiliary variable λ is known as the Lagrange multiplier.

88 Conditional Optimization II After we define the Lagrangian we find the stationary points (necessary condition) to obtain a system of three equations: ( ) L (x, y; λ) = 0 x ( ) L (x, y; λ) = 0 ( y ) L (x, y; λ) = 0 λ

89 Conditional Optimization III Similar to the case of unconditional extrema, the characterization of the stationary points depends on the signs of the principal minors of a bordered Hessian. For the 3 3 case we have: 0 H 2 (a, b; λ g ) = x g y g x 2 L x 2 2 L x y where (a, b, λ ) are the stationary points. g y 2 L x y 2 L y 2 (a,b,λ ) If H 2 (a, b; λ ) > 0, the point (a, b) is a conditional maximum. If H 2 (a, b; λ ) < 0, the point (a, b) is a conditional minimum.

90 Example I Example Determine the conditional extrema of the function f (x, y) = x 2 + y 2 s.t. g (x, y) = x + y 2 = 0 Step 1:We find the Lagrange function: L (x, y, λ) = f (x, y) + λg (x, y) = x 2 + y 2 + λ (x + y 2) Step 2: First order conditions: ( ) L (x, y; λ) = 0 x ( ) L (x, y; λ) = 0 ( y ) L (x, y; λ) = 0 λ 2x + λ = 0 2y + λ = 0 x + y 2 = 0

91 Example II x = λ/2 y = λ/2 x = 2 y x = λ/2 y = λ/2 λ = 2 x = λ/2 y = λ/2 λ/2 = 2 + λ/2 x = 1 y = 1 λ = 2 Therefore, (1, 1, 2) is the only stationary point of the Lagrangian.

92 Example III Step 3: The second order condition: 0 g H 2 (1, 1, 2) = x g y = thus (1, 1) is a conditional minimum. g g x y 2 L 2 L x 2 x y 2 L 2 L x y y 2 = 4 < 0 (1,1, 2) (1,1, 2)