Fakultät für Mathematik und Informatik

Transcription

1 Fakultät für Mathematik und Informatik Preprint S. Dempe, F. Mefo Kue Discrete bilevel and semidefinite programg problems ISSN

2 S. Dempe, F. Mefo Kue Discrete bilevel and semidefinite programg problems TU Bergakademie Freiberg Fakultät für Mathematik und Informatik Prüferstraße FREIBERG

3 ISSN Herausgeber: Herstellung: Dekan der Fakultät für Mathematik und Informatik Medienzentrum der TU Bergakademie Freiberg

4 1 Introduction 2 Abstract: In this paper, we study the bilevel programg problem with discrete polynomial lower level. We start by transforg the problem into a bilevel problem comprising a semidefinite program (SDP for short) at the lower level. Then, we are able to deduce some conditions of existence of solutions for the original problem. After that, we again change the bilevel problem with SDP in the lower level into a semi-infinite program. With the aid of the exchange technique, for simple bilevel programs, an algorithm for computing a global optimal solution is suggested, the convergence is shown, and a numerical example is given. Keywords: Bilevel Programg, Semidefinite Programg, Semi-infinite Programg, Solution Algorithm. Subclass: 90C11, 90C22, 90C30, 90C34. 1 Introduction Bilevel programg problems are optimization problems which are partly constrained by another optimization problem. Formulated as a hierarchical game, the first and second player are called leader and follower, respectively. In this article, we focus on the study of bilevel programg problems with discrete lower level and continuous upper level variables. Our interest for this type of problem is motivated by a number of important applications in bilevel programg problems with parametric graph problems in the lower level such as the imum spanning tree problem, the toll problem, the shortest path problem, the matching problem in a bipartite graph, or the imum knapsack problem. Just recently, several solution algorithms have been developed for solving bilevel polynomial optimization problems [9, 13]. The authors in [9] found a method to get global imizers using a sequence of SDP relaxations, and in [13], a similar idea is realized through Lasserretype semidefinite relaxations and the exchange technique. In this paper, we also consider polynomial optimization techniques in order to transform the original bilevel problem into bilevel problem with SDP in the lower level. In the next section, we first present some basic notations and background material to be used in the paper. We reformulate the original bilevel problem in Section 3 and using the theory of perturbation analysis of continuous optimization problems [2], we are able to derive conditions of existence of solutions. Section 4 describes the solution algorithm for simple bilevel programs (i.e. the feasible set of the lower level problem is independent of the parameter) through the exchange technique from semi-infinite programg. A numerical example is given as well. 2 Background material and basic definitions We begin by fixing notations, definitions, and preliaries. In this paper, we use N n, R n, R n m, S p, S p +, Sp, Sp ++ to denote the set of all natural vectors with n components, the set of all real vectors with n components, the set of all real matrices with n rows and m columns, the linear space of p p symmetric matrices, the cone of p p symmetric positive semidefinite matrices, the cone of p p symmetric negative semidefinite matrices, and the set of p p S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

5 2 Background material and basic definitions 3 symmetric positive definite matrices, respectively. The expression A S p + (resp. Sp ) is also represented by A 0 (resp. A 0). The real vector space S p is endowed with the Frobenius inner product defined by A, B S p : A, B S p := p p A ij B ij. i=1 Therefore, for A S p, A := A, A S p S p associated to the Frobenius norm. holds and we postulate S to be the unit sphere of We consider the parametric discrete optimization problem given by y f(x, y) s.t. g j (x, y) 0 j = 1,..., l y {0, 1} m, (2.1) where the functions f, g j : R n R m R, j = 1,..., l are real polynomials in y. We adopt the notation from [10], which for the sake of clarity, we reproduce here. Let 1, y 1, y 2,..., y m, y 2 1, y 1 y 2,..., y 1 y m, y 2 2, y 2 y 3,..., y 2 m,..., y r 1,..., y r m (2.2) be a basis for the vector space R m [y] r of real-valued polynomials on R m of degree at most r, and let s(r) := Cm+r m be its dimension [11]; Cm+r m is the number of ways of selecting m items from a set of m + r elements. Then, an r-degree polynomial p : R m R is written by p(y) = p α y α, y R m, where y α := y α 1 1 yα yαm m, with α := m α i. The vector α N m 0 is called multi-index and α is its order. For each α, p αy α is a monomial with coefficient p α R and degree α. As mentioned just above, {y α α r} is the base of the vector space R m [y] r. We set f(x, y) = f α (x)y α, g j (x, y) = gα(x)y j α, j = 1,..., l, and unless otherwise specified, we suppose in the rest of the paper that the leading coefficient of each polynomial g j (x, y) j = 1,..., l does not vanish. In this way, the respective degree cannot depend on x. Now, we define two important notions: the moment matrix and the localizing matrix. Let {z α α N m } an s(2r) sequence (i.e. a finite sequence with s(2r) = C m m+2r elements). The moment matrix denoted M r (z) is a matrix of dimension s(r), with rows and columns labeled by (2.2), and constructed as follows [11]: M r (z)(α, β) = z α+β, i=1 S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

6 3 Reformulation of the bilevel programg problem 4 where α, β are the orders of y α and y β, respectively. For example, for m = 2 and r = 2, we obtain 1 = z 00 z 10 z 01 z 20 z 11 z 02 z 10 z 20 z 11 z 30 z 21 z 12 M 2 (z) = z 01 z 11 z 02 z 21 z 12 z 03. z 20 z 30 z 21 z 40 z 31 z 22 z 11 z 21 z 12 z 31 z 22 z 13 z 02 z 12 z 03 z 22 z 13 z 04 Another way of constructing M r (z) using some block matrices can be found in [10]. Let θ : R m R be a given polynomial with coefficient vector {θ α } α (here, the coefficient vector can depend on x). The localizing matrix associated to the polynomial θ, denoted M r (θz), is defined by M r (θz)(i, j) = θ α z {β(i,j)+α}, 1 i, j s(r), α where β(i, j) denotes the index β of the entry (i, j) in the moment matrix M r (z). For example, for m = 1 = r [ ] 1 = z0 z M 1 (z) = 1 z 1 z 2 and for y θ(x, y) = 4y + (2 x), we obtain [ ] 4z1 + (2 x) 4z M 1 (θz) = 2 + (2 x)z 1. 4z 2 + (2 x)z 1 4z 3 + (2 x)z 2 Remark 2.1. The moment matrix defines a bilinear form, z on the space R m [y] r of realvalued polynomials of degree at most r, by p, q z := p, M r (z)q p, q R m [y] r, where p, q R s(r) denote the vectors coefficients of p and q. Then, if z is defined by z α = [0,1] y α dy, we have from [11] that m q, M r (z)q = q(y) 2 dy [0,1] m and q, M r (g(x)z)q = g(x)q(y) 2 dy [0,1] m for every polynomial q R m [y] r with coefficient vector q R s(r). 3 Reformulation of the bilevel programg problem We consider now the following bilevel program with discrete lower level problem x,y F (x, y) s.t. G(x) 0 y Ψ(x). (3.1) S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

7 3 Reformulation of the bilevel programg problem 5 Therein, Ψ(x) := Arg{f(x, y) g j (x, y) 0, j = 1,..., l y {0, 1} m }, y the functions F : R n R m R and G: R n R p are given. We denote by X the upper level feasible set defined by X := {x R n G(x) 0} and suppose G continuous. The mapping defined by ϕ(x) = y {f(x, y) g j (x, y) 0, j = 1,..., l y {0, 1} m } is called the optimal value function. In this section, we want to transform the bilevel programg problem into a bilevel problem with semidefinite programg in the lower level. For that aim, we start by stating a main result of Lasserre from [10] which provides an explicit equivalent positive semidefinite program of (2.1) in 2 m 1 variables. For fixed i N, we denote by M i (z) and M i (g j (x)z) the moment matrix and the localizing matrix, in which we have replaced z α, α = (α 1,..., α m ) with z β, β j = 1 whenever α j 1, respectively. Theorem 3.1. [10, Theorem 3.2, Remark 3.3] Depending on its parity, let w j := 2v j or w j := 2v j 1 be the degree of the polynomial g j (x, y), j = 1,..., l and let v := max,...,l v j. We consider the following problem Then, z f α(x)z α s.t. Mm (z) 0 M m+v vj (g j (x)z) 0, j = 1,..., l. (3.2) (3.2) is solvable and problems (3.2) and (2.1) have the same objective function value at any optimal solution as long as (2.1) is feasible. Every optimal solution ȳ = (ȳ 1,..., ȳ m ) of (2.1) corresponds to the optimal solution of (3.2). z = (ȳ 1,..., ȳ m,..., (ȳ 1 ) 2(m+v),..., (ȳ m ) 2(m+v) ) Every optimal solution z of (3.2) satisfies z α = t z γ j ȳj α with t z := rank M m ( z), t z γ j = 1, γ j 0 and ȳ j Ψ(x), j 1,..., t z. Remark 3.2. It is important to see that the feasible set of (3.2) is bounded because it is a subset of the set {z R s(r) M m (z) 0} which is in turn by construction bounded since the set {0, 1} m is bounded. In order to illustrate the previous theorem, let us have a look at the following example. Example 3.3. Consider the following one level discrete programg problem with parameter x 2y y s.t. g 1 (x, y) := 4y + (2 x) 0 (3.3) g 2 (x, y) := 4y + (2 + x) 0 y {0, 1}. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

8 3 Reformulation of the bilevel programg problem 6 From Theorem 3.1, the equivalent form of problem (3.3) is 2z 1 z 1 [ ] 1 z1 s.t. M1 (z) = 0 z 1 z 1 [ ] 4z1 + (2 x) (6 x)z M 1 (g 1 (x)z) = 1 0 (6 x)z 1 (6 x)z 1 [ ] 4z1 + (2 + x) (6 + x)z M 1 (g 2 (x)z) = 1 0. (6 + x)z 1 (6 + x)z 1 (3.4) On the other hand, remembering that a symmetric matrix A is positive semidefinite if and only if all its principal ors are nonnegative, problem (3.4) can again be rewritten as z 1 2z 1 s.t. z 1 0, z 1 z 2 1 0, 4z x 0, (6 + x)z 1 0, (x + 2)(x + 6)(z 1 z 2 1 ) 0, (x 2)(x 6)(z 1 z 2 1 ) 0, (6 x)z 1 0 4z x 0. (3.5) We can easily see that if z 1 z1 2 > 0, problem (3.5) does not have a solution. Furthermore, problem (3.5) has a solution if z 1 z1 2 = 0. Therefore, problem (3.5) is equivalent to z 1 {0,1} 2z 1 s.t. 4z x 0 4z x 0 (3.6) which is actually the same problem as (3.3). As postulated in Section 2, we assumed in this part, that for any x X the functions g j (x, ) j = 1,..., s are polynomial with nonvanishing leading coefficient and that the map F (x, ) is a polynomial and G is continuous. Next, we want to transform the bilevel programg problem into a bilevel problem with semidefinite programg in the lower level. For that aim, we use the notations introduced in Theorem 3.1. Theorem 3.4. Consider the problem x,z F 1 (x, z) s.t. G(x) 0 z Ψ 1 (x) (3.7) { with Ψ 1 (x) := Arg f α(x)z α M m (z) 0, Mm+v vj (g j (x)z) 0, j = 1,..., l} z and F 1 is the function obtained from F by replacing y α = y α yαm m by z β with β j = 1 if α j 1 and 0 otherwise. The following assertions hold: If ( x, ȳ) is a global optimal solution of problem (3.1), then the point ( x, z) with solves (3.7) globally. z = (ȳ 1,..., ȳ m,..., (ȳ 1 ) 2(m+v),..., (ȳ m ) 2(m+v) ) S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

9 3 Reformulation of the bilevel programg problem 7 If ( x, z) is a global optimal solution of problem (3.7), then z α = t z γ j ȳ α j, where t z = rank( M m ( z)), t z γ j = 1 and for all j = 1,..., t z, γ j 0 and ȳ j Ψ( x). Further, the point ( x, ȳ j0 ) with ȳ j0 Arg{F ( x, ȳ j ), j = 1,..., t z } is a global solution of (3.1). Proof. Let ( x, ȳ) be a global optimal solution of (3.1) and (x, z) a feasible point of (3.7). From Theorem 3.1, z α = t z γ j yj α, where t z = rank( M m (z)), t z γ j = 1 and for all j = 1,..., t z, y j Ψ(x), γ j 0. Therefore, (x, y j ) is feasible for (3.1) and F ( x, ȳ) F (x, y j ) j = 1,..., t z. This implies F ( x, ȳ) t z γ j F (x, y j ). Noticing that F ( x, ȳ) = F 1 ( x, z) and F (x, y j ) = F 1 (x, (y j ) 1,..., (y j ) m,..., ((y j ) 1 ) 2(m+v),..., ((y j ) m ) 2(m+v) ), we get from the definition of F 1 that it is linear w.r.t. its second variable and that F 1 ( x, z) = F ( x, ȳ) = t z t z = F 1 (x, γ j F (x, y j ) γ j F 1 (x, (y j ) 1,..., (y j ) m,..., ((y j ) 1 ) 2(m+v),..., ((y j ) m ) 2(m+v) ) t z γ j (y j ) 1,..., t z γ j ((y j ) m ) 2(m+v) )) = F 1 (x, z). Consequently, ( x, z) is a global optimal solution of problem (3.7). Now let us prove the second statement. For that, we take ( x, z) a global optimal solution of problem (3.7) and (x, y) a feasible point of (3.1). Then, Theorem 3.1 implies the existence of z = (y 1,..., y m,..., (y 1 ) 2(m+v),..., (y m ) 2(m+v) ) such that (x, z) is feasible for (3.7). Then F 1 ( x, z) F 1 (x, z) = F (x, y). Working along the proof of the first assertion and by definition of ȳ j0, we get F ( x, ȳ j0 ) t z γ j F ( x, ȳ j ) = F 1 ( x, z) F (x, y) and the result follows. Next, we will see the relation between (3.7) and (3.1) w.r.t. local solutions. Corollary 3.5. Let ( x, z) be a local solution of problem (3.7). If t = rank M m ( z) = 1, then ( x, ȳ) is a local solution of (3.1) with ȳ α = z α for all α. Proof. Suppose by contradiction that ( x, ȳ) is not a local solution of (3.1). Then, there exists a sequence {(x n, y n )} feasible for (3.1) which converges to ( x, ȳ) and F (x n, y n ) < F ( x, ȳ) for all n is satisfied. The feasibility of (x n, y n ) for (3.1) entails that the point given by z n = (y n 1,..., y n m,..., (y n 1 ) 2(m+v),..., (y n m) 2(m+v) ) is such that (x n, z n ) is feasible for (3.7). Therefore, F 1 (x n, z n ) = F (x n, y n ) < F ( x, ȳ) = F 1 ( x, z) since rank M m ( z) = 1. Clearly, (x n, z n ) ( x, z) and this contradicts the initial assumption. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

10 3 Reformulation of the bilevel programg problem 8 In Example 4.8, the necessity of the assumption on the rank of the moment matrix at the reference point is highlighted. On the hand, we can see that the assumption in Corollary 3.5 is satisfied for instance if the variable z takes the values 0 or 1. Problem (3.7) is a bilevel programg problem with SDP in the lower level. This problem has been studied in [7] in a more general setting where some optimality conditions using different approaches have been found. In the sequel, we are going to use the resultant problem (3.7) in order to get some conditions for the existence of solutions of problem (3.1). The following theorem combines some results from parametric SDP and parametric nonlinear programg. The proof is straightforward from the given references and by observing that Ψ 1 is locally compact (see Remark 3.2). Proposition 3.6. [2, Proposition 4.4], [5, Remark 3.2] Consider the bilevel programg problem (3.7). Let x be a given point in the parameter space X and Φ(x) := {z R s(r) M m (z) 0, Mm+v vj (g j (x)z) 0, j = 1,..., l} the lower feasible set of problem (3.7). Suppose that 1. the functions F and f are continuous on X R s(r), 2. the multifunction Φ( ) is closed, 3. there exist β R such that for every x in a neighborhood of x 0, the level set lev β f(, x) := z Φ(x) f α (x)z α β is nonempty and uniformly bounded, 4. for any neighborhood V of the set Ψ 1 (x 0 ) there exists a neighborhood V X of x 0 such that V Φ(x) for all x V X. Then an optimal solution of problem (3.1) exists if the set X is nonempty compact. It is worth mentioning that the first three conditions allow us to get lower semicontinuity of ϕ and the fourth condition gives the upper semicontinuity of ϕ. Furthermore, Robinson s CQ is sufficient for the fourth condition when for all j = 1,..., l, the mappings g j are continuous [2]. In the case where the feasible set does not depend on the parameter x, we know that an optimal solution of (3.1) exists if the feasible sets of the lower and upper level problem are nonempty and compact and the respective objective functions are continuous [6]. Similarly, under these assumptions we easily see that from Proposition 3.6, the solution set mapping is upper semicontinuous (indeed, conditions 2 and 4 are obviously satisfied) therefore an optimal solution of (3.7) exists. However, we have to say that the condition 4 in Proposition 3.6 is pretty strong. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

11 4 Solution algorithm 9 It should be noted that the semidefinite programg problem we get from the previous example does not fulfill Robinson s CQ. In fact, if we set ḡ 0 (x, z) := M 1 (z), ḡ j (x, z) := M 1 (g j (x)z), j = 1, 2, and ḡ(x, z) := (ḡ 0, ḡ 1, ḡ 2 )(x, z), we can show that for x = 2 and for all feasible points z 0 / int{ḡ(2, z) + z ḡ(2, z)r S+ 2 S+ 2 S+}. 2 Since int ( S+ 2 S+ 2 S+} 2 ), the last condition is equivalent to ḡ(2, z) + z ḡ(2, z)λ / int ( S+ 2 S+ 2 S+ 2 ) with λ R. In fact, we have [ ] [ ] [ ] 4z 4z 4 4 4z + 4λ 4z + 4λ ḡ 1 (2, z) + z ḡ 1 (2, z)λ = + λ = 4z 4z 4 4 4z + 4λ 4z + 4λ / S In Proposition 3.6, Robinson s CQ is used to prove the upper semicontinuity of the mapping Ψ which in turn is an essential assumption for existence of an optimal solution of the optimistic bilevel optimization problem. Therefore, an optimal solution of problem (3.7) does not exist in general. Another way to see that the mapping Ψ is not upper semicontinuous is to consider problem (3.5) and see that it is equal to { {0} if x [ 2, 2] Ψ(x) = {1} if x [ 6, 2[ ]2, 6]. Consequently, Ψ is not upper semicontinuous and an optimal solution of the associated bilevel does not necessarily exist (see [5, Remark 3.2]). In addition, the optimal value function given by { 0 if x [ 2, 2] ϕ(x) = 2 if x [ 6, 2[ ]2, 6]. is not continuous. However, the Robinson s CQ can be verified in some other settings as we can see from Remark 2.1. Indeed, this constraint qualification which here is equivalent to the Slater s CQ is satisfied for (3.2) whenever the only constraint in the lower level of (3.1) is given by y {0, 1} m. That leads to the fact that the results from [7] can be easily applied in this framework. 4 Solution algorithm In the previous section, we have transformed our bilevel problem with discrete lower level into a bilevel problem with SDP in the lower level. Now, we would like to think about a solution algorithm to solve the original bilevel problem. For that, we assume that the lower level constraint does not depend on the parameter x (this simplification can help us to have ϕ concave). Then, in order to make our problem as simple as possible, we suppose that the only constraint is given by y {0, 1} m (it should be noted that the whole theory behind can be generalized in the case we have some constraints like g(y) 0) and make the following assumption. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

12 4 Solution algorithm 10 Assumption 4.1. The mapping x f α (x) is concave for all α (i.e. each coefficient of the polynomial f(x, y) is a concave function in x) and continuously differentiable. We considered now the transformed problem of (3.1) given by: x,z F 1 (x, z) s.t. G(x) 0 z Ψ 1 (x) = Arg{ f α(x)z α M m (z) 0}. (4.1) We mention that the parameter is only in the objective function of the lower level problem. The optimal value reformulation of (4.1) leads to x,z F 1 (x, z) s.t. G(x) 0 { } f α(x)z α ϕ(x) := z f α(x)z α M m (z) 0 M m (z) 0. (4.2) Problem (4.2) is an optimization problem with two main difficulties to overcome: firstly, we have the optimal value function which is in general nondifferentiable and secondly, we have the constraint M m (z) 0. Note that the previous constraint distinguishes this class of optimization problems from other types of problems in constrained optimization. It is worth to mention that several algorithms have been developed in the literature in order to solve problem (4.2) without the constraint f α (x)z α ϕ(x) (4.3) using for example a generalization of the sequentially quadratic programg method see [3] and the references therein. These methods cannot be applied to (4.2) straightforwardly since the constraint (4.3) is the one which really defines the bilevel programg problem and is not so easy to handle (as said earlier). Here, our idea is to transform problem (4.2) equivalently into a semi-infinite program and then use the exchange technique (see [1,8]) in order to solve it. The next proposition from [12] shows how problem (4.2) can be reformulated as a semiinfinite program. We include here another proof different from the one that the reader can find in [12]. Proposition 4.2. We have where λ(z) := max Ω S S s(r) {z R s(r) M m (z) 0} = {z R s(r) λ(z) 0}, Ω, M m (z). Proof: Clearly, we see that for all z such that M m (z) 0, we have max Ω S S s(r) Ω, M m (z) 0 S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

13 4 Solution algorithm 11 i.e. λ(z) 0. To show the reverse inclusion, we take an arbitrarily z such that λ(z) 0 i.e. for all Ω S S s(r), Ω, M m (z) 0. We want to show M m (z) 0. For that, we will prove that for all positive semidefinite matrices A S s(r) +, we get A, M m (z) 0 (see [14]). In order to ensure that, we consider two cases for each A S s(r) +. If A = 1, then A S S s(r) and from the assumption, we have A, M m (z) 0 i.e. A, M m (z) 0. If A 1 with A O (the case A = O is trivially satisfied) then A A S Ss(r) The proof is completed. A A Ss(r) + follows. Hence, A is valid. This implies that A, M m (z) 0 i.e. A, M m (z) 0. Remark 4.3. As mention earlier, the suggested solution algorithm can be generalized when we have some constraints which do not depend of the parameter x. In fact, we also get with {z R s(r) M m+v vj (g j z) 0} = {z R s(r) λ 1 (z) 0} λ 1 (z) := max Ω S S s(r) Ω, M m+v vj (g j z). After that, we can define a new function given by { λ 2 (z, Ω, Ω ) = max Ω, M m (z), Ω, M } m+v vj (g j z). Therefore from Proposition 4.2, problem (4.2) is equivalent to x,z F 1 (x, z) s.t. G(x) 0 f α(x)z α ϕ(x) Ω, M m (z) 0 Ω S S s(r). (4.4) Problem (4.4) is a semi-infinite program on which we can apply the exchange technique. On the other hand, the concavity of the lower level optimal value function can be shown in the next proposition and we recall that for any concave function h: R n R, the superdifferential of h is given by h( x) := {x R n : h(x) h( x) + x (x x) x R n }. Proposition 4.4. The function ϕ is concave at any point x R n and we have { f α( x) z α z Ψ1 ( x)} ϕ( x) for any x X. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

14 4 Solution algorithm 12 Proof: The concavity of the optimal value function comes from the fact that the mapping x f α (x) is concave for all α. In fact, ϕ equals the imum of concave functions. let z Ψ 1 ( x) and suppose that f α( x) z α / ϕ( x). Since the function ϕ is concave, the set ϕ( x) reduces to the superdifferential of ϕ at x. Hence, from the definition of supergradient, there exists x R n such that ϕ(x) > ϕ( x) + f α( x) z α, x x = f α( x) z α + f α( x) z α, x x f α(x) z α ϕ(x), where the penultimate inequality comes from the fact that the mapping x f α(x)z α, for fixed z is concave and differentiable and the last inequality is true because z is feasible for the lower level problem. However, this inequality is a contradiction. Consequently, using the previous proposition and the definition of supergradient, we can relax problem (4.4) by x,z F 1 (x, z) s.t. G(x) 0 f α(x)z α f α(x 0 )z 0 α + ( f α(x 0 )z 0 α), (x x 0 ) Ω, M m (z) 0 Ω S S s(r) (4.5) for arbitrary x 0 X and z 0 Ψ 1 (x 0 ). It is clear that the feasible set of (4.5) is larger than the one of (4.4) or (4.2). Problem (4.5) is a semi-infinite program that we can solve using a classical method, namely the exchange one [1,8]. The idea of the algorithm is to solve at each iteration a relaxation of (4.2) and then check if the solution we get is feasible for the initial problem. If not, we can add some cuts (inequalities which are similar to the second constraint of problem (4.5) where x 0 and z 0 are replaced by other reference points) to the feasible set of the relaxation in order to approximate as accurate as possible the feasible set of (4.2) (similar idea can be found in [4], [5]). In the sequel, we give the different steps of operations which have to be performed in order to compute a solution of (4.2). We define S k and X k as finite subsets of S S s(r) Algorithm 4.5. and X at each iteration, respectively. Step 0: Set k = 0, choose a vector for the set S 0 and X 0, respectively. Step 1: For x k X k, solve the problem z f α(x k )z α s.t. Mm (z) 0. Let z k be one of its optimal solutions and go to Step 2. (4.6) Step 2: Solve the problem x,z F 1 (x, z) s.t. G(x) 0 Ω, M m (z) 0 Ω S k f α(x)z α l=1,...,k f α( x l ) z l α + f α( x l ) z l α, x x l. (Q k ) S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

15 4 Solution algorithm 13 Let (x k, z k ) be a global optimal solution of (Q k ) and go to Step 3. Step 3: Solve the problem max Ω s.t. Ω, M m (z k ) Ω S S s(r). (4.7) Let Ω k be an optimal solution. If Ω k, M m (z k ) 0, go to Step 4. If not, go to Step 5. Step 4: If (x k, z k ) solves the lower level problem, stop. Otherwise, go to Step 5. Step 5: Set S k+1 = S k {Ω k }, X k+1 = X k {x k }, k = k + 1, and go to Step 1. Remark 4.6. As we can see, problem (4.7) which has to be solved in the Step 3 is a nonlinear semidefinite programg one. One way to handle Step 3 is to directly check whether M m (z k ) is a semidefinite positive matrix since by Proposition 4.2, it is equivalent to verify that max Ω S S s(r) Ω, M m (z k ) 0. In other words, Step 3 actually verifies the feasibility of the point z k for the lower level problem. Now, we can prove the convergence result. Theorem 4.7. Assume that the set X is bounded and let (x k, z k ) be a sequence of points generated by Algorithm 4.5. If ( x, z) is one of the accumulation points of (x k, z k ), then ( x, z) solves problem (4.2) globally. Proof: Since the set {(x, z) X R s(r) M m (z) 0} is compact, we have the existence of accumulation points. We suppose without loss of generality that the sequence (x k, z k ) converges to ( x, z). Since ( x, z) is feasible for (Q k ) and has the best objective function value w.r.t. (4.5), we only need to show that it is feasible for (4.2). We assume that it is not true. Then, from its feasibility for (Q k ) we have f α ( x) z α > ϕ( x) or M m ( z) / S s(r) +. α If f α( x) z α > ϕ( x), then by the feasibility of (x k, z k ) for (Q k ), we have f α (x k )zα k f α (x k 1 ) z α (k 1) + = ϕ(x k 1 ) + f α (x k 1 ) z α (k 1), x k x k 1 f α (x k 1 ) z α (k 1), x k x k 1 with z (k 1) Ψ 1 (x k 1 ). Since ϕ is continuous, taking the limit gives f α ( x) z α ϕ( x) which contradicts the initial assumption. S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

16 4 Solution algorithm 14 If M m ( z) / S s(r) +, then by Proposition 4.2 there exits Ω S S s(r) such that Ω, M m ( z) > 0. We have then Ω, M m (z k ) > 0 for sufficiently large k. On the other hand, from the sequence {z k } we can construct a sequence {Ω k } S S s(r) converges to a certain Ω S S s(r) we have Taking the limit leads to (solution of (4.7)) which (which is a compact set). By the definition of Ω k Ω k, M m (z k ) Ω, M m (z k ) > 0. Ω, M m ( z) Ω, M m ( z) > 0. Moreover, we have from Step 2 that Ω k+1, M m (z k+1 ) 0 which gives Ω, M m ( z) 0 which is absurd. Therefore, in each case, we have a contradiction. That means ( x, z) is feasible for (4.2), hence it is a global optimal solution. As we observed in Step 2 of the Algorithm 4.5, we need to compute a global solution, but for problem (Q k ) being nonconvex it might be difficult to find a global optimal solution. That is why in the following theorem we are going to compute a local solution of (4.1). Theorem 4.8. Assume that the set X is bounded and let (x k, z k ) be a sequence of points generated by Algorithm 4.5 where in the Step 2, the relaxed problem (Q k ) is solved locally instead of globally. Assume that there is an ε > 0 and an iteration index k 0 such that k k 0 (x, z) feasible for (Q k ) with (x, z) (x k, z k ) ε we have F (x, z) F (x k, z k ). If ( x, z) is one of the accumulation points of (x k, z k ), then ( x, z) solves problem (4.2) locally. Proof: We suppose w.l.o.g that (x k, z k ) converges to ( x, z). The feasibility of ( x, z) and its existence follows from Theorem 4.7 and the compactness of the set {(x, z) X R s(r) M m (z) 0}, respectively. Let (x, z) a feasible point of (4.1) with (x, z) ( x, z) < ε 2. For k sufficiently large, (x, z) (x k, z k ) < ε 2. Then the assumption leads to F (x, z) F (x k, z k ). Taking the limit on the previous inequality gives F (x, z) F ( x, z) and the proof is completed. For illustration purposes, we consider the following bilevel programg problem. Example 4.9. We consider the problem x,y (x 1 + y) 2 x 2 y s.t. x 2 1, x 1, x 2 0 y Ψ(x) = Arg{(x 1 + x 2 )y y {0, 1}} y The transformed bilevel optimization problem is given by x,z (x 1 + z) 2 x 2 z s.t. x 2 1, x 1, x 2 0 z Ψ 1 (x) = Arg{(x 1 + x 2 )z M 1 (z) 0} z [ ] with M 1 z 1 (z) =. The algorithm works as follows z z (4.8) (4.9) S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

17 5 Conclusion 15 [ ] Step 0: We initialize the sets S 0 and X 0 with Ω 0 = respectively. Step 1: We solve the lower level problem The solution is z 0 {0} = Ψ 1 (x 0 ). z z s.t. M 1 (z) 0. Step 2: We solve the problem at the first iteration given by x,z (x 1 + z) 2 x 2 z s.t. x 1, x 1, x 2 0 (x 1 + x 2 )z z 0. and x 0 := (x 0 1, x0 2 ) = (0.5, 0.5), The local (resp. global) solution is (x 1 1, x1 2, z 1) = (0, 0, 0.5) (resp. (0, 1, 0)), x 1 := (x 1 1, x1 2 ). Step 3: The problem has max Ω s.t. Ω 1 = Ω, M 1 (z 1 ) Ω S S 2 [ 0.21 ] as a solution. We see that Ω 1, M 1 (z 1 ) 0 with [ ] M 1 (z 1 ) = In others words [ M 1 (z 1 ] ) is a positive semidefinite matrix. In the same vein, we can see 1 0 that the matrix is positive semidefinite. 0 0 Step 4: The points (0, 0, 0.5) and (0, 1, 0) solve the lower level problem. Then, (0, 0, 0.5) (resp. (0, 1, 0)) is a local solution of (4.9) (resp. (4.8)). From this example, we conclude that any local optimal solution of (3.7) is not necessarily a local optimal solution for the original bilevel problem (3.1) without additional assumptions. We easily see that the rank of the moment matrix at the point 0.5 is not equal to 1, see Section 3. 5 Conclusion We considered a bilevel programg problem with discrete lower level. In order to establish some conditions of existence, a surrogate problem has been constructed using tools of semidefinite programg. The relation w.r.t. global and local solutions between the two problems S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

18 References 16 has been investigated. We proposed an algorithm for solving the original problem using the optimal value reformulation and we have seen that the assumption to get equivalence on the local solution can not be weakened. It has to be mentioned that the results used to get the surrogate problem are mainly based on the fact that the leading coefficient of each polynomial g j (x, y) j = 1,..., l does not vanish. A future research is to be able to get such a substitute problem in the case the degree of one of these polynomials is not constant. References [1] J. W. Blankenship and J. E. Falk. Infinitely constrained optimization problems. Journal of Optimization Theory and Applications, 19(2): , [2] J. F. Bonnans and A. Shapiro. Perturbation Analysis of Optimization Problems. Springer, [3] R. Correa and H. C. Ramirez. A global algorithm for nonlinear semidefinite programg. SIAM Journal on Optimization, 15(1): , [4] S. Dempe and S. Franke. Solution algorithm for an optimistic linear stackelberg problem. Computers and Operations Research, 41: , [5] S. Dempe, V. V. Kalashnikov, G. A. Pérez-Valdés, and N. Kalashnykova. Bilevel programg problems. Energy Systems. Springer, Berlin, [6] S. Dempe, F. Mefo Kue, and P. Mehlitz. Optimality conditions for mixed discrete bilevel optimization problems. submitted for publication. [7] S. Dempe, F. Mefo Kue, and P. Mehlitz. Optimality conditions for special semidefinite bilevel optimization problems. submitted for publication. [8] R. Hettich and K. O. Kortanek. Semi-infinite programg: theory, methods, and applications. SIAM review, 35(3): , [9] V. Jeyakumar, J. B. Lasserre, G. Li, and T. S. Pham. Convergent semidefinite programg relaxations for global bilevel polynomial optimization problems. SIAM Journal on Optimization, 26(1): , [10] J. B. Lasserre. An explicit equivalent positive semidefinite program for nonlinear 0-1 programs. SIAM Journal on Optimization, 12(3): , [11] J. B. Lasserre. Moments, positive polynomials and their applications, volume 1. World Scientific, [12] B. S. Mordukhovich and T. T. A. Nghia. Nonsmooth cone-constrained optimization with applications to semi-infinite programg. Mathematics of Operations Research, 39(2): , S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems

19 References 17 [13] J. Nie, L. Wang, and J. Ye. Bilevel polynomial programs and semidefinite relaxation methods. arxiv preprint arxiv: , [14] L. Tunçel. Polyhedral and semidefinite programg methods in combinatorial optimization, volume 27. American Mathematical Soc., S. Dempe, F. Mefo Kue: Discrete bilevel and semidefinite programg problems