Stationarity results for generating set search for linearly constrained optimization

Size: px

Start display at page:

Download "Stationarity results for generating set search for linearly constrained optimization"

Melissa Jackson
5 years ago
Views:

1 Stationarity results for generating set search for linearly constrained optimization Virginia Torczon, College of William & Mary Collaborators: Tammy Kolda, Sandia National Laboratories Robert Michael Lewis, College of William & Mary

2 The problem: minimize f(x) subject to Ax b, where f : R n R, x R n, A is an m n matrix, and b R m. We use Ω to denote the feasible region: Ω = { x R n : Ax b }. We assume that f is continuously differentiable on Ω but that gradient information is not computationally available. We do not assume that the constraints are nondegenerate.

3 The stationarity measure: 1 χ(x) max x+w Ω, w 1 f(x)t w, where χ(x) is a continuous function with the property if and only if χ(x) = 0 for x Ω x is a KKT point of the linearly constrained problem. 1 A. R. Conn, N. Gould, A. Sartenaer, and P. L. Toint, Convergence propreties of an augmented Lagrangian algorithm for optimization with a combination of general equality and linear constraints, SIOPT, 1996.

4 First-order stationarity Main result: Once k is small enough the subsequence x k for k U {0, 1, 2,...} produced by a Generating Set Search (GSS) method for the linearly constrained optimization problems satisfies χ(x k ) = O( k ). Observe: a careful specification of a GSS method for linearly constrained optimization ensures, at a minimum, that lim inf k = 0, so that firstorder stationarity for this subsequence of iterates is immediate. But that is not the primary motivation for this investigation since global convergence to KKT points for the linearly constrained problems have already been established [see May, 1974; Yu/Li, 1981; Lewis/Torczon, 2000; Lucidi/Sciandrone/Tseng, 2002;....]

5 What are GSS methods for linearly constrained optimization? Look at one simple example applied to the problem: where minimize x R 2 f(x 1, x 2 ) f(x) = (3 2x 1 )x 1 2x (3 2x 2 )x 2 x , the modified Broyden tridiagonal function augmented with three linear constraints. Use a feasible iterates approach.

6 The initial set of search directions:

7 Identify feasible improvement:

8 Move Northeast and keep the set of search directions

9 Identify feasible improvement:

10 Move Northeast and change the set of search directions

11 No feasible improvement:

12 Contract and change the set of search directions

13 Identify feasible improvement:

14 Move Northeast and change the set of search directions

15 No feasible improvement:

16 Contract and change the set of search directions

17 Critical: When close to the boundary of the feasible region, the set of directions must conform to the geometry of the nearby constraints. Essential features: identifying the nearby constraints obtaining a set of search directions finding a step of an appropriate length accepting a step

18 Identifying the nearby constraints Find the outward-pointing normals within distance ε of the current iterate. ε a 2 x 2 a 2 a 3 ε ε Ω x 1 a 1 Ω Ω x 3 The conditions on ε depend on the convergence analysis in effect.

19 Obtaining a set of search directions: Part I Translate the outward-pointing normals within distance ε of the current iterate x to obtain the ε-normal cone N(x, ε) and its polar, the ε-tangent cone T (x, ε). N(x,ε 1 ) N(x,ε 2 ) x ε 1 x ε 2 ε 3 x Ω Ω Ω T(x,ε 1 ) T(x,ε 2 ) T(x,ε 3 )

20 Obtaining a set of search directions: Part II The set of search directions must contain generators for the ε-tangent cone T (x, ε). N(x,ε 1 ) N(x,ε 2 ) x x x Ω Ω Ω T(x,ε 1 ) T(x,ε 2 ) T(x,ε 3 ) Conditioning is important!

21 Conditioning: concern When T (x, ε) is a lineality space or half space, there is freedom in choosing the generators: N(x,ε) d (1) d (3) poor N(x,ε) d (1) N(x,ε) d (1) x ε x ε x ε Ω d (2) f(x) Ω d (2) d (3) good f(x) Ω d (2) f(x) d (3) ideal T(x,ε) T(x,ε) T(x,ε)

22 Conditioning: requirement that must be enforced There exists a constant κ min > 0, independent of k, such that for all k there exists a set of generators G for T (x k, ε k ) and, furthermore, κ(g) min max v R n d G v K 0 v T d v K d κ min.

23 Conditioning: gratis There exists a constant ν min > 0, independent of k, such that for all k there exists a set of generators A for N(x k, ε k ) such that κ(a) ν min. The for free is because each N(x, ε) is generated by at most m rows of the constraint matrix A.

24 Obtaining a set of search directions: Part III What precisely the set of search directions contains depends on the convergence analysis in effect. Choices: At a minimum, generators for the ε-tangent cones T (x k, ε) for all ε [0, ε ]. [Lewis/Torczon, 2000] Only generators for the ε-tangent cone T (x k, ε k ). [Lucidi/Sciandrone/Tseng, 2002] At a minimum, generators for the ε-tangent cone T (x k, ε k ), where ε k = min{ε max, β max k }. [Kolda/Torczon/Lewis, 2004] The sets used for the example contained generators for both T (x k, ε k ) and N(x k, ε k ).

25 The initial set of search directions:

26 Move Northeast and keep the set of search directions

27 Move Northeast and change the set of search directions

28 Contract and change the set of search directions

29 Move Northeast and change the set of search directions

30 Contract and change the set of search directions

31 Finding steps of an appropriate length First: bound the lengths of the direction vectors. Specifically, there must exist β min and β max, independent of k, such that for all k the following holds: β min d β max for all d G k.

32 Finding steps of an appropriate length Second: once the lengths of the direction vectors are bounded for all k, tie the lengths of the steps tried to a step-length control parameter k. Requirement: If x k + k d (i) k Ω, then the actual step taken along d (i) must be of length k, just as for unconstrained generating set search methods. Question: What to do when x k + k d (i) k Ω?

33 Finding feasible steps of an appropriate length x k x k x k Once again, the analysis allows multiple options.

34 Accepting a step If x k + k d k Ω and f(x k + k ) < f(x k ) ρ( k ) then k S, x k+1 = x k + k and k+1 = φ k k for a choice of φ k 1. Otherwise, k U, x k+1 = x k, and k+1 = θ k k, for some choice θ k (0, 1).

35 The forcing function: 1. The function ρ( ) is a nonnegative continuous function on [0, + ). 2. The function ρ( ) is o(t) as t 0; i.e., lim t 0 ρ(t)/t = The function ρ( ) is nondecreasing; i.e., ρ(t 1 ) ρ(t 2 ) if t 1 t 2.

36 The stationarity result: If the set F = {x Ω f(x) f(x 0 )} is bounded and the gradient of f is Lipschitz continuous with constant M on F, then there exists γ > 0 such that for all x F, f(x) < γ. Further, for all k U, if ε k = β max k, then χ(x k ) ( M κ min + γ ν min ) k β max + 1 κ min β min ρ( k ) k.

37 Conclusions: k can be used to assess progress toward a KKT point. The bound on χ(x k ) illuminates what algorithmic parameters can and should be monitored to assure the effectiveness of an implementation. Our analysis yields an estimate that makes it possible to use linearly constrained GSS methods with the augmeted Lagrangian approach of Conn/Gould/Sartenaer/Toint to handle problems with general constraints.

1. Introduction. We consider ways to implement direct search methods for solving linearly constrained nonlinear optimization problems of the form

1. Introduction. We consider ways to implement direct search methods for solving linearly constrained nonlinear optimization problems of the form IMPLEMENTING GENERATING SET SEARCH METHODS FOR LINEARLY CONSTRAINED MINIMIZATION ROBERT MICHAEL LEWIS, ANNE SHEPHERD, AND VIRGINIA TORCZON Abstract. We discuss an implementation of a derivative-free generating