Optimization Methods: Optimization using Calculus - Equality constraints 1. Module 2 Lecture Notes 4

Transcription

1 Optimization Methods: Optimization using Calculus - Equality constraints Module Lecture Notes 4 Optimization of Functions of Multiple Variables subect to Equality Constraints Introduction In the previous lecture we learnt the optimization of functions of multiple variables studied for unconstrained optimization. This is done with the aid of the gradient vector and the Hessian matri. In this lecture we will learn the optimization of functions of multiple variables subected to equality constraints using the method of constrained variation and the method of Lagrange multipliers. Constrained optimization A function of multiple variables, f(), is to be optimized subect to one or more equality constraints of many variables. These equality constraints, g (), may or may not be linear. The problem statement is as follows: Maimize (or minimize) f(x), subect to g (X) 0,,,, m where X n with the condition that m n; or else if m > n then the problem becomes an over defined one and there will be no solution. Of the many available methods, the method of constrained variation and the method using Lagrange multipliers are discussed. Solution by method of Constrained Variation For the optimization problem defined above, let us consider a specific case with n and m before we proceed to find the necessary and sufficient conditions for a general problem using Lagrange multipliers. The problem statement is as follows: Minimize f(, ), subect to g(, ) 0 For f(, ) to have a minimum at a point X* [ *, *], a necessary condition is that the total derivative of f(, ) must be zero at [ *, *]. df d + d 0 () ML4

2 Optimization Methods: Optimization using Calculus - Equality constraints Since g( *, *) 0 at the minimum point, variations d and d about the point [ *, *] must be admissible variations, i.e. the point lies on the constraint: g( * + d, * + d ) 0 () assuming d and d are small the Taylor series epansion of this gives us or g( * + d, * + d ) g( *, *) + ( *, *) d + ( *, *) d 0 (3) dg d + d Assuming / 0 (4) can be rewritten as which is the condition that must be satisfied for all admissible variations. g/ 0 at [ *, *] (4) / d ( *, *) d (5) which indicates that once variation along (d ) is chosen arbitrarily, the variation along (d ) is decided automatically to satisfy the condition for the admissible variation. Substituting equation (5) in ()we have: df / d / ( *, *) 0 (6) The equation on the left hand side is called the constrained variation of f. Equation (5) has to be satisfied for all d, hence we have 0 ( *, *) (7) This gives us the necessary condition to have [ *, *] as an etreme point (maimum or minimum) Solution by method of Lagrange multipliers Continuing with the same specific case of the optimization problem with n and m we define a quantity λ, called the Lagrange multiplier as Using this in (6) λ / g/ (8) ( *, *) ML4

3 Optimization Methods: Optimization using Calculus - Equality constraints 3 + λ 0 ( *, *) (9) And (8) written as + λ 0 ( *, *) (0) Also, the constraint equation has to be satisfied at the etreme point g (, ) 0 () ( *, *) Hence equations (9) to () represent the necessary conditions for the point [ *, *] to be an etreme point. Note that λ could be epressed in terms of / as well and / has to be non-zero. Thus, these necessary conditions require that at least one of the partial derivatives of g(, ) be non-zero at an etreme point. The conditions given by equations (9) to () can also be generated by constructing a function L, known as the Lagrangian function, as L (,, λ) f(, ) + λg (, ) () Alternatively, treating L as a function of, and λ, the necessary conditions for its etremum are given by (,, λ) (, ) + λ (, ) 0 (,, λ) (, ) + λ (, ) 0 (3) (,, λ) g (, ) 0 λ The necessary and sufficient conditions for a general problem are discussed net. Necessary conditions for a general problem For a general problem with n variables and m equality constraints the problem is defined as shown earlier Maimize (or minimize) f(x), subect to g (X) 0,,,, m where X n ML4

4 Optimization Methods: Optimization using Calculus - Equality constraints 4 In this case the Lagrange function, L, will have one Lagrange multiplier λ for each constraint (X) as g L (,,..., λ, λ,..., λ ) f( X) + λ g( X) + λ g( X) λ g ( X) n, m m m (4) L is now a function of n + m unknowns,,,..., n, λ, λ,..., λ m, and the necessary conditions for the problem defined above are given by ( ) + ( X) 0, i,,..., n,,..., m m X λ i i i g ( X) 0,,,..., m λ (5) which represent n + m equations in terms of the n + m unknowns, i and λ. The solution to this set of equations gives us X * * and n * λ* λ * λ * λm * The vector X corresponds to the relative constrained minimum of f(x) (subect to the verification of sufficient conditions). Sufficient conditions for a general problem A sufficient condition for f(x) to have a relative minimum at X* is that each root of the polynomial in, defined by the following determinant equation be positive. (6) L L L L g g L g n m L L L g g g n m M O M M O M L L L L g g L g n n nn n n mn g g L g 0 L L 0 n g g g n g g L g 0 L L 0 m m mn M O M M O M M M 0 (7) ML4

5 Optimization Methods: Optimization using Calculus - Equality constraints 5 where Li ( X*, λ*), for i,,..., n,,..., m i p gpq ( X*), where p,,..., m and q,,..., n q (8) Similarly, a sufficient condition for f(x) to have a relative maimum at X* is that each root of the polynomial in, defined by equation (7) be negative. If equation (7), on solving yields roots, some of which are positive and others negative, then the point X* is neither a maimum nor a minimum. Eample Minimize f ( X) Subect to + 5 Solution g ( X ) L(,,..., λ, λ,..., λ ) f( X) + λ g ( X) + λ g ( X) λ g ( X) n, m m m with n and m L λ ( + 5) λ 0 > + (7 + λ ) 6 > 5 (7 + λ ) 6 or λ λ 0 > 3+ 5 (5 + λ) > 3( + ) + (5 + λ) ML4

6 Optimization Methods: Optimization using Calculus - Equality constraints 6 and, Hence X*, ; λ* [ 3 ] L L L g 6 ( X*,λ* ) g L 6 ( X*,λ* ) ( X*,λ* ) 0 ( X*,λ* ) g ( X*,λ* ) L L g L L g 0 g g 0 The determinant becomes ( 6 )[ ] ( 6)[ ] + [ ] 0 or > Since is negative, X*, λ * correspond to a maimum. ML4