ACM 113 Introduction to Optimization - Final Exam. June 6, 2001

Size: px

Start display at page:

Download "ACM 113 Introduction to Optimization - Final Exam. June 6, 2001"

Blake Powers
5 years ago
Views:

1 ACM 113 Introduction to Optimization - Final Exam Ling Li, ling@cscaltechedu June 6, The Baywatch Problem As shown in Figure 1, denote the position of the victim as a, b The lifeguard is at 0, d d > 0, and will enter the sea at point x, 0 The problem is a, b victim min x fx = We need to solve f x = get the minimizer x x + d v 1 + a x + b v x + x a v 1 x +d v a x +b = 0 to If represented in θ 1 and θ π < θ 1, θ < π, the problem is min θ fθ = d v 1 cos θ 1 + b v cos θ, subject to d tan θ 1 + b tan θ = a The Lagrangian is Solving Lθ, λ = 0, 0 θ 1 lifeguard 0, d d b + λd tan θ 1 + b tan θ a v 1 cos θ 1 v cos θ θ Lθ, λ = [ d sin θ1 v 1 cos θ 1 λd cos θ 1 b sin θ v cos θ λb cos θ θ x, 0 sea sand Figure 1: The path of the lifeguard gives sin θ 1 = λv 1 and sin θ = λv If a 0, then the constraint requests λ 0, so sin θ 1 sin θ = v 1 v, which means the lifeguard should mae a larger angle where his/her speed is faster Local Convergence of Trust Region Methods a The Cauchy point is ] = 0 p c = f λ f, 0 λ, 1

2 and p c minimizes m p subject to p along direction f Thus m 0 m p c = f T pc 1 pc T f p c = λ f 1 λ f T f f f λ f 1 λ f T f f f for any 0 λ 1 If f T f f 0, for λ = in 1, we have m 0 m p c f 1 f min Otherwise, f T f f > 0 From ref Homewor 1, f f we get from 1, f T f f f f, If m 0 m p c λ f 1 λ f f = 1 f f f f, then for λ = f, is f m 0 m p c 1 f f f = 1 f min otherwise for λ =, becomes So, overall, we have m 0 m p c f = 1 f + f λ f f f f 1 f = 1 f min m 0 m p c 1 f min, f f, f ; f f, f f In the CG-Newton trust region method, we initialize the inner CG by p 0 = 0 and d 0 = f If negative curvature is met during the first chec, p = p c is returned Otherwise, the CG iteration starts from p c, and ensures descent So the CG-Newton method yields at least much reduction as p c T f = f d 0T B d 0 During the first iteration of CG, α 0 = r0t r 0 and p 1 = p 0 +α 0 d 0 = f f f T f f f T f f minimizes m p along f without p bound Thus the following p 1 chec ensures p 1 = p c, or returns p = p c

3 b During the CG-Newton iterations, r = f p + f Since the region constraint is inactive, the stopping criterion requests r η f Thus p p N = f 1 f p + f = f 1 r η f 1 f η f 1 f p N = η κ p N, where κ is the condition number of f, which is bounded for sufficient large Since η 0, so p p N = o p N 3 LP Sensitivity Analysis The primal-dual optimality conditions are Ax = b, A T y + s = c, x 0, s 0, x T s = 0 The current basis is not affected iff these conditions are not affected by the perturbation a b b + b This doesn t change y and s If B 1 b + b = x B + B 1 b 0, the basis is not affected However, x B is changed by B 1 b and the objective is changed by c T B B 1 b = y T b If x B + B 1 b 0, the basis is affected Since y, s have not been affected, x, y, s is primal infeasible but still dual feasible We can use the dual simplex algorithm to restore the primal feasibility b c Ni c Ni + c Ni This doesn t affect x, y However, s Ni is changed by c Ni If s Ni + c Ni 0, the optimality conditions still hold, so the basis is not affected And there is no change to the objective, since x N = 0 Otherwise, the basis is affected x, y, s is primal feasible but dual infeasible The primal simplex algorithm can be used to restore the dual feasibility and optimality c N i N i + N i This doesn t affect x, y However, since s N = c N N T y, s Ni is changed by Ni T y If s N i Ni T y 0, ie, s N i Ni T y, the basic and the objective are not changed, since c, x remain the same If s Ni < Ni T y, x, y, s is primal feasible but dual infeasible We can use the primal simplex algorithm to restore the dual feasibility d x t added with c t and A t Regard x t as a non-basic variable and let x t = 0 y is not affected s N is appended by s t = c t A T t y If s t 0, the basis has not been affected and the objective doesn t change Otherwise s t < 0 Since x, y, s now is primal feasible but dual infeasible The primal simplex algorithm can be used to restore the dual feasibility and optimality 4 Augmented Lagrangian Methods For less writing, define fx = e x 1x x 3 x 4 x 5, cx = c 1 x, c x, c 3 x T, and c 1 x = x 1 + x + x 3 + x 4 + x 5 10, c x = x x 3 5x 4 x 5, c 3 x = x x

4 error of x error of λ error of x error of λ error 10 5 error iteration # iteration # a a 0 = 01, a +1 = 6a b a = 05 fixed Figure : Error for x and λ are defined as x x and λ λ, respectively a Thus the augmented Lagrangian is L a x, λ = fx λ T cx + a cx = fx λ 1 c 1 x λ c x λ 3 c 3 x + a [ c 1 x + c x + c 3x ] 3 And x L a x, λ = fx cxλ + a cxc = x x 3 x 4 x 5 fx λ 1 x 1 3λ 3 x 1 + a [ x 1 c 1 x + 3x 1 c 3x ] x 1 x 3 x 4 x 5 fx λ 1 x λ x 3 3λ 3 x + a [ x c 1 x + x 3 c x + 3x c 3x ] x 1 x x 4 x 5 fx λ 1 x 3 λ x + a [x 3 c 1 x + x c x] x 1 x x 3 x 5 fx λ 1 x 4 + 5λ x 5 + a [x 4 c 1 x 5x 5 c x] 4 x 1 x x 3 x 4 fx λ 1 x 5 + 5λ x 4 + a [x 5 c 1 x 5x 4 c x] b The program trunc newtonm is reused as the unconstrained sub-iteration which is Hessianfree Program aug lagranm controls the augmented Lagrangian method by calling trunc newtonm and updating λ as λ +1 = λ a cx +1, a little different from the normal way: λ +1 = λ a cx The objf 4m defines the augmented objective and derivative as 3 and 4 c Using finite difference, superlinear convergence [x,l,f] = aug lagran objf 4,[-;;;-1;-1],[0;0;0],[01,6],1,15,1e-8,5e-8; we get at the 6 th iteration, x = , λ =

5 The convergence of x and λ can be seen in Figure a Here we let a increase along the iterations and the converge rate is more than linear If a is fixed at 05 Figure b, the rate is linear The previous one uses only 6 iterations, while fixed a uses 8 While x x and λ λ, cx 0 The augmented part of L a x, λ tends to be close to 0 if a is fixed So fixed a results in linear convergence, while a would speed up the convergence However, larger a maes L a x, λ more difficult to be optimized due to larger condition number 5 Protein Design is N P -complete a The decision form is: For p positions where position i has n i amino acid side-chain alternatives, can we select the one side-chain at each position, st, E total = Ei r, j s L? i j,j<i b If a PRODES has a yes solution, we can calculate E total and verify whether E total L This can be done with pp 1 additions and at most pp 1 table looing-up to get Ei r, j s Thus this verification requires poly-time Note that we may also verify the validity of the given solution by checing that exact one side-chain is selected at each position, in Op time Thus PRODES NP c Mae transformation from SAT to PRODES as shown in the table SAT PRODES E total 0 clause i position i literal side-chain # of literals in clause i n i That is, convert each clause into a position For each literal in clause i, convert it as one side-chain at position i The pairwise interaction energy is defined as { 1, if ir and j Ei r, j s = s are a variable and its negative; 0, otherwise For example, Ex 1, x = 0, E x 1, x 1 = 1, Ex 1, x 1 = 0 Such transformation requires p p O n i time, and the calculation of E requires O n i n j time So totally we need i=1 i=1 j<i p On time, where n = n i is the number of literals in SAT problem, or, the size of the problem i=1 If SAT has a solution, then at least one literal is true in clause i Select any true literal as the selected side-chain Since x and x can t both be true in the solution, by the definition of E, E total = 0 If there is a solution to PRODES E total 0, then mae those selected literals to be true Such assignments are consistent, since E total 0 assures each pair in the selection is not a pair of a variable and its negative Then we now this is also a solution to SAT Variables that are not selected could be assigned with any value Thus, in Op time, a solution to SAT can be transformed to a solution to PRODES E total 0, and vice verse SAT is polynomial-transformable to PRODES So PRODES N P - complete 5

The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006

The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 1 Simplex solves LP by starting at a Basic Feasible Solution (BFS) and moving from BFS to BFS, always improving the objective function,