EXTENDING MODELING SYSTEMS: STRUCTURE AND SOLUTION

Transcription

1 EXTENDING MODELING SYSTEMS: STRUCTURE AND SOLUTION Michael C. Ferris Computer Sciences Department, University of Wisconsin, Madison, WI Steven P. Dirkse GAMS Corporation, 1217 Potomac Street, Washington, DC Jan Jagla GAMS Software GmbH, Eupener Str , Cologne, Germany Alexander Meeraus GAMS Corporation, 1217 Potomac Street, Washington, DC 2007 Abstract Keywords Some extensions of a modeling system are described that facilitate higher level structure identification within a model. These structures, which often involve complementarity relationships, can be exploited by modern large scale mathematical programg algorithms. A specific implementation of an extended mathematical programg tool is outlined that communicates structure in a computationally beneficial manner from the GAMS modeling system to an appropriate solver. Modeling, complementarity, nonlinear programg, variational inequalities Introduction Chemical engineering has been a user of optimization techniques and technology for several decades. Many chemical engineers have made significant contributions to the field of optimization, and in several cases have developed software that has had impact far beyond the chemical engineering discipline. Packages such as DICOPT (Duran and Grossman, 1986), BARON (Tawarmalani and Sahinidis, 2004), ALPHAECP (Westerlund et al., 1998) and IPOPT (Wächter and Biegler, 2006) have influenced the debate on tractability of hard, practical nonlinear optimization problems, setting the gold standard for the solution of nonconvex, global optimization problems. Accessing these solvers, and many of the other algorithms that have been developed over the past three decades has been made easier by the advent of modeling languages. A modeling language (Bisschop and Meeraus, 1982, Fourer et al., 1990) provides a natural, convenient way to represent mathematical programs. They typically have efficient procedures to handle vast amounts of data, take advantage of the numerous options for solvers and model types, and can quickly generate a large number of models. For this reason, modeling languages are heavily used in practical applications. Although, we will use GAMS (Brooke et al., 1988), the system we are intimately familiar with, most what will be said could as well be applied to other algebra based modeling systems like AIMMS, AMPL, MOSEL, MPL and OPL. We believe that further advancements in applications of optimization can be achieved via identification of specific problem structures within a model. In some cases, such structures can be automatically extracted from a model that is formulated in a modeling system, but in general it may be difficult to tease out particular structures from a large complex model. As a concrete example, we cite the domain of applied general equilibrium modeling within economics, where a model consists of a collection of interacting agents each of which maximizes some utility

2 function of some demanded goods, which in turn are generated from an optimization of production functions. Since current solvers require the (economic) modeler to formulate the problem as a complementarity problem or a system of nonlinear equations, the utility function is maximized (by hand) to derive demands which are normally rather complex nonlinear functions; similarly for the supplies that are derived from a cost imization involving the production functions. It is typically difficult to detere the structure of the underlying model from the resulting nonlinear complementarity system. This paper aims to alleviate this difficulty. The purpose of our work is to extend the classical nonlinear program from the traditional model: x f(x) s.t. g(x) 0, h(x) = 0, (1) where f, g and h are assumed sufficiently smooth, to a more general format that allows new constraint types to be specified precisely. Some extensions of this format have already been incorporated into modeling systems. There is support for integer, sos, and semicontinuous variables, and some limited support for logical constructs. GAMS (Ferris and Munson, 2000), AMPL (Ferris et al., 1999) and AIMMS have support for complementarity constraints, and there are some extensions that allow the formulation of second-order cone programs within GAMS. AMPL (Fourer et al., 1993) has facilities to model piecewise linear functions. Much of this development is tailored to specific structures. We aim to develop more general annotation schemes to allow extended mathematical programs to be written clearly and succinctly. In the following sections we outline some new features of GAMS that we refer to as extended mathematical programs (EMP), that incorporate many of the extensions menitoned above but also allows a variety of other structures to be described at the modeling level. We believe such extensions may have benefits on several levels. Firstly, we think this will make the modelers task easier, in that the model can be described more naturally and perhaps at a higher level. (Of course, there are several examples of this already in the literature including the use specialized languages such as MPSGE (Rutherford, 1999) to facilitate general equilibrium models, and specialized (graphical) interfaces to allow queueing system or process system design.) Secondly, the automatic generation of the model can be done more reliably based on techniques such as automatic differentiation and problem reformulation - duality constructs, or specific ways to write certain constraints. Thirdly, if an algorithm is given additional structure, it may be able to exploit that in an effective computational manner; knowing the problem is a cone program, or the problem involves the optimality conditions of a nonlinear program can be treated in a variety of different ways, some of which may be distinctly superior to others in certain settings. Indeed, the availability of such structure to a solver may well foster the generation of new features to existing solvers or drive the development of new classes of algorithms. Extended Mathematical Programs Complementarity Problems The necessary and sufficient optimality conditions for the linear program c T x x s.t. Ax b, x 0 are that x and some λ satisfy 0 c A T λ x 0 0 Ax b λ 0. (2) (3) Here, the sign signifies (for example) that in addition to the constraints 0 Ax b and λ 0, each of the products (Ax b) i λ i is constrained to be zero. An equivalent viewpoint is that either (Ax b) i = 0 or λ i = 0. Within GAMS, these constraints can be modeled simply as positive variables lambda, x; model complp / defd.x, defp.lambda /; where defp and defd are the equations that define primal and dual feasibility (Ax b, c A T λ) respectively. Other linear programs with specialized constraint structure are just as easy to specify. For example c T x x s.t. Ax = b, x [l, u] has similarly expressed optimality conditions: 0 (c A T λ) j if x j = l j 0 = (c A T λ) j if l j < x j < u j 0 (c A T λ) j if x j = u j 0 = Ax b λ free. (4) Note that the first three complementarity relationships in (4) can be written more succinctly as (c A T λ) x [l, u]. This is translated into GAMS syntax as follows:

3 variables lambda, x; x.lo(i) = l(i); x.up(i) = u(i); model complp / defd.x, defp.lambda /; Such a problem is an instance of a linear mixed complementarity problem, for which we use the acronym MCP. Note that the bounds on the variables x detere the nature of the relationship on c A T λ at the solution. (It is possible to introduce explicit multipliers on the constraints x l and x u, and to rewrite the optimality conditions in terms of x, λ, and these multipliers. The notation enables us to write these relationships much more succinctly.) Complementarity problems do not have to arise as the optimality conditions of a linear program; the optimality conditions of the nonlinear programs (1) are the following MCP: 0 = f(x) + λ T g(x) + µ T h(x) x free 0 g(x) λ 0 0 = h(x) µ free. (5) Many examples are no longer simply the optimality conditions of an optimization problem. A specific example arises in chemical phase equilibrium. In this setting, different conditions are satisfied at an equilibrium depending on whether we are in vapor, liquid or two-phase state. Letting α represent the fraction in vapor, the problem is to find f(α) α [0, 1] where f(α) = z i (x i K i x i ), x i = K i i α + 1 α, for given data K i and z i. Ferris and Pang (1997) catalogue a number of other applications both in engineering and economics that can be written in a similar format. It should be noted that robust large scale solvers exist for such problems; see for example Ferris and Munson (2000), where a description is given of the PATH solver. Mathematical Programs with Complementarity Constraints A mathematical program with complementarity constraints is the following: x R n,y Rmf(x, y) (6) s.t. g(x, y) 0 (7) 0 y h(x, y) 0. (8) The objective function (6) needs no further description, except to state that the solution techniques we are intending to apply require that f (g and h) are at least once differentiable, and for some solvers twice differentiable. The constraints (7) are intended to represent standard nonlinear programg constraints. Clearly, these could involve equalities with a slight increase in exposition complexity. The constraints that are of interest here are the complementarity constraints (8). Essentially, these are parametric constraints (parameterized by x) on the variable y, and encode the structure that y is a solution to the nonlinear complementarity problem defined by h(x, ). Within the GAMS modeling system, this can be written simply and directly as: model mpecmod / deff, defg, defh.y /; option mpec=nlpec; solve mpecmod using mpec imizing obj; Here it is assumed that the objective (6) is defined in the equation deff, the general constraints (7) are defined in defg and the function h is described by defh. The complementarity relationship is defined by the bounds on y and the orthogonality relationship shown in the model declaration using.. AMPL provides a slightly different but equivalent syntax for this, see Ferris et al. (1999). The problem is frequently called a mathematical program with complementarity constraints (MPCC). Some solvers can process complementarity constraints explicitly. In many cases, this is achieved by a reformulation of the constraints (8) into the classical form given within (1). Dirkse and Ferris (2008) outlines a variety of ways to carry this out, all of which have been encoded in a solver package called NLPEC. While there are large numbers of different reformulations possible, the following parametric approach, coupled with the use of the nonlinear programg solver CONOPT or SNOPT, has proven effective in a large number of applications: x R n,y R m,s Rmf(x, y) s.t. g(x, y) 0 s = h(x, y) y 0, s 0 y i s i µ, 1 = 1,..., m. Note that a series of approximate problems are produced, parameterized by µ > 0; each of these approximate problems have stronger theoretical properties than the problem with µ = 0 (Ralph and Wright, 2004). A solution procedure whereby µ is successively reduced can be implemented as a simple option file

4 to NLPEC, and this has proven remarkably effective. Further details can be found in the NLPEC documentation (Dirkse and Ferris, 2008). The approach has been used to effectively optimize the rig in a sailboat design (Wallace et al., 2006). It is also possible to generalize the above complementarity condition to a mixed complementarity condition; details can be found in Ferris et al. (2005). Underlying the NLPEC solver package is an automatic conversion of the original problem into a standard nonlinear program which is carried out at the scalar model level. The technology to perform this conversion forms the core of the codes that we use to implement the model extensions of the sequel. Variational Inequalities A variational inequality VI(F, X) is to find x X: F (x) T (z x) 0, for all z X. Here X is a closed (frequently assumed convex) set, defined for example as X = {x x 0, g(x) 0}. Note that the first-order (imum principle) conditions of a nonlinear program f(z) z X are precisely of this form with F (x) = f(x). For concreteness, these conditions are necessary and sufficient for optimality of the linear programg. Solving the linear program (2) is equivalent to solving the variational inequality given by F (x) = c, X = {x Ax b, x 0}. (9) In this case, F is simply a constant function. While there are a large number of instances of the problem that arise from optimization applications, there are many cases where F is not the gradient of any function f. For example, asymmetric traffic equilibrium problems have this format, where the asymmetry arises for example due to different costs associated with left or right hand turns. A complete treatment of the theory and algorithms in this domain can be found in Facchinei and Pang (2003). Variational inequalities are intimately connected with the concept of a normal cone to a set S, for which a number of authors have provided a rich calculus. Instead of overloading a reader with more notation, however, we simply refer to the seal work in this area, Rockafellar and Wets (1998). Such a problem can be reformulated as a complementarity problem by introducing multipliers λ on the constraints g: 0 F (x) + λ T g(x) x 0 0 g(x) λ 0. If X has a different representation, this construction would be modified appropriately. In the linear programg example above (9), these conditions are precisely those given as (3). When X is the nonnegative orthant, the VI is just an alternative way to state a complementarity problem. However, when X is a more general set, it may be possible to treat it differently than simply introducing multipliers, see for example Cao and Ferris (1996). For example when X is a polyhedral set, algorithms may wish to generate iterates via projection onto X. A simple example two dimensional may be useful. Let [ F (x) = x x 1 + x 2 3 ], X = {x 0 x 1 + x 2 1}, so that F is an affine function, but F is not the gradient of any function f : R 2 R. For this particular data, VI(F, X) has a unique solution x = (0, 1). Such a variational inequality can be described in GAMS via the model statement model vi / deff, defg /; combined with an annotation file that indicates certain equations are to be treated differently by the EMP tool. In this case, the empinfo file states that the model equations deff define a function F that is to be part of a variational inequality, while the equations defg define constraints on X. Details on specific syntax can be found in Jagla et al. (2008). Mathematical Programs with Equilibrium Constraints Mathematical programs with equilibrium constraints are a generalization of the aforementioned MPCC problem class. The difference is that the lower level problem, instead of being a complementarity problem, is now a variational inequality. Coupling the two approaches mentioned above, results in the ability to model and solve such problems. Bilevel Programs Heirarchical optimization has recently become important for a number of different applications. New codes are being developed that exploit this structure,

5 at least for simple heirarchies, and attempt to define and implement algorithms for their solution. The simplest case is that of bilevel programg, where an upper level problem depends on the solution of a lower level optimization. For example: x,y f(x, y) s.t. g(x, y) 0, y solves s v(x, s) s.t. h(x, s) 0. This problem can be reformulated as an MPEC by replacing the lower level optimization problem by its optimality conditions: x,y f(x, y) s.t. g(x, y) 0, y solves VI( s v(x, s), X), where X = {s h(x, s) 0}. This approach then allows such problems to be solved using the NLPEC code, for example. However, there are several possible deficiencies that should be noted. Firstly, the optimality conditions encompassed in the VI may not have a solution, or the solution may only be necessary (and not sufficient) for optimality. Secondly, the MPEC solver may only find local solutions to the problem. The quest for practical optimality conditions and robust global solvers remains an active area of research. Importantly, the EMP tool will provide the underlying structure of the model to a solver if these advances detere appropriate ways to exploit this. We can model this bilevel program in GAMS by model bilev /deff,defg,defv,defh/; along with some extra annotations to a subset of the model defining equations. Specifically, within an empinfo file we state that the bilevel problem involves the objective v which is to be imized subject to the constraints specified in defh. The specific syntax is described in Jagla et al. (2008). A point that has been glossed over here but which is described carefully in the user manual is the process whereby multiple lower level problems are specified. Extended Nonlinear Programs Optimization models have traditionally been of the form (1). Specialized codes have allowed certain problem structures to be exploited algorithmically, for example simple bounds on variables. In a series of papers, Rockafellar and colleagues have introduced the notion of extended nonlinear programg, where the (primal) problem has the form: f 0(x) + θ(f 1 (x),..., f m (x)). (10) x X In this setting, X is assumed to be a nonempty polyhedral set, and the functions f 0, f 1,..., f m are smooth. The function θ can be thought of as a generalized penalty function. Specifically, when θ has the following form θ(u) = sup{y T u k(y)}, (11) y Y we obtain a computationally exploitable and theoretically powerful framework. A key point for computation and modeling is that the function θ can be fully described by defining the set Y and the function k. Furthermore, as we show below, different choices lead to a rich variety of functions θ, many of which are extremely useful for modeling. In the above setting θ can take on the value of and may well be nonsmooth, but it is guaranteed to be convex (proper and lower semicontinuous when Y and k is smooth and convex). Furthermore, from a modeling perspective, an extended nonlinear program can be specified simply by defining the functions f 0, f 1,..., f m as is done already, with the additional issue of simply defining Y and k. Conceptually, this is not much harder that what is done already, but leads to significant enhancements to the types of models that are available. This paper outlines an approach to do this within the GAMS modeling system for a number of different choices of Y and k. The tool we provide works in this setting by providing a library of functions θ that specify a library of choices for k and Y. Once a modeler deteres which constraints are treated via which choice of k and Y, the tool automatically forms an equivalent variational inequality or complementarity problem. As we show later, there may be alternative formulations that are computationally more appealing; such reformulations can be generated using different options to our tool. Forms of θ The EMP tool makes this problem format available to users in GAMS. As special cases, we can model piecewise linear penalties, least squares and L 1 approximation problems, as well as the notion of soft

6 and hard constraints. We allow modelers to utilize cone constraints and pass on the underlying geometric structure to solvers. Particular examples show enormous promise both from a modeling and solution perspective. For ease of exposition, we now describe a subset of the types of functions θ that can be generated by particular choices of Y and k. In many cases, the function θ is separable, that is θ(u) = m θ i (u i ). i=1 so we can either specify θ i or θ itself. Extended nonlinear programs include the classical nonlinear programg form (1) as a special case. This follows from the observation that if K is a closed convex cone, and we let ψ K denote the indicator function of K defined by: ψ K (u) = { 0 if u K then (1) can be rewritten as: else, x f(x) + ψ K ((g(x), h(x)), K = R m {0} p, where m and p are the dimensions of g and h respectively and R m = {u R m u 0}. An elementary calculation shows that ψ K (u) = sup v K u T v, where K = { u u T v 0, v K } is the polar cone of the given cone K. Thus, when θ(u) = ψ K (u) we simply take k 0 and Y = K. (12) In our example, K = R m + R p. To some extent, this is just a formalism that allows us to claim the classical case as a specialization; however when we take the cone K to be more general than the polyhedral cone used above, we can generate conic programs (see below) for example. The second example involves a piecewise linear function θ: Formally, for u R, { ρu if u 0 θ(u) = σu else. In this case, simple calculations prove that θ has the form (11) for the choices: k 0 and Y = [σ, ρ]. The special case where σ = ρ results in θ(u) = ρ u. This allows us to model nonsmooth L 1 approximation problems. Another special case results from the choice of σ = 0, whereby θ(u) = ρ max{u, 0}. This formulation corresponds to a soft penalization on an inequality constraint, namely if θ(f 1 (x)) is used then nothing is added to the objective function if f 1 (x) 0, but ρf 1 (x) is added if the constraint f i (x) 0 is violated. Contrast this to the classical setting above, where is added to the objective if the inequality constraint is violated. It is interesting to see that truncating the set Y, which amounts to bounding the multipliers, results in replacing the classical constraint by a linearized penalty. The third example involves a more interesting choice of k. If we wish to replace the absolute value penalization given above by a quadratic penalization (as in classical least squares analysis), that is θ(u) = γu 2 then a simple calculation shows that we should take k(y) = 1 4γ y2 and Y = R. By simply specifying this different choice of k and Y we can generate such models easily and quickly within the modeling system: note however that the reformulation we would use in these two cases are very different as we shall explain in the simple example below. Furthermore, in many applications it has become popular to penalize violations using a quadratic penalty only within a certain interval, afterwards switching to a linear penalty (chosen to make the penalty fucntion θ continuously differentiable Huber (1981). That is:

7 γu 1 2 γ2 if u γ 1 i.e. θ(u) = 2 u2 if u [ γ, γ] γu 1 2 γ2 else. Such functions arise from quadratic k and simple bound sets Y. In particular, the function γβ 2 + ρ(u β) if u β θ(u) = γu 2 if u [α, β] γα 2 + σ(u α) else arises from the choice of k(y) = 1 4γ y2 and Y = [σ, ρ], with α = σ 2γ and β = ρ 2γ. The final example that we give is that of L penalization. This example is different to the examples given above in that θ is not separable. However, straightforward calculation can be used to show θ(u) = results from the choice of { k 0 and Y = max i=1,...,m u i y R m y 0, that is Y is the unit simplex. Underlying theory } m y i = 1, i=1 The underlying structure of θ leads to a set of extended optimality conditions and an elegant duality theory. This is based on an extended form of the Lagrangian: L(x, y) = f 0 (x) + m y i f i (x) k(y) i=1 x X, y Y Note that the Lagrangian L is smooth - all the nonsmoothness is captured in the θ function. The theory is an elegant combination of calculus arguments related to f i and its derivatives, and variational analysis for features related to θ. It is shown in Rockafellar (1993) that under a standard constraint qualification, the first-order conditions of (10) are precisely in the form of the following variational inequality: VI ([ x L(x, y) y L(x, y) ] ), X Y. (13) When X and Y are simple bound sets, this is simply a complementarity problem. Note that EMP exploits this result. In particular, if an extended nonlinear program of the form (10) is given to EMP, then the optimality conditions (13) are formed as a variational inequality problem and can be processed as outlined above. For a specific example, we cite the fact that if we use the (classical) choice of k and Y given in (12), then the optimality conditions of (10) are precisely the standard complementarity problem given as (5). While this is of interest, we believe that other choices of k and Y may be more useful and lead to models that have more practical significance. Under appropriate convexity assumptions on this Lagrangian, it can be shown that a solution of the VI (13) is a saddle point for the Lagrangian on X Y. Furthermore, in this setting, the saddle point generates solutions to the primal problem (10) and its dual problem: max g(y), y Y with no duality gap. A simple example where g(y) = inf L(x, y), x X As an example, consider the problem x 1,x 2,x 3 exp(x 1 ) + 5 log(x 1 ) max(x 2 2 2, 0) s.t. x 1 /x 2 = log(x 3 ), 3x 1 + x 2 5, x 1 0, x 2 0. In this problem, we would take X = { x R 3 3x 1 + x 2 5, x 1 0, x 2 0 }. The function θ essentially treats 3 separable pieces: f 1 (x) = log(x 1 ) 1, f 2 (x) = x 2 2 2, f 3 (x) = x 1 /x 2 log(x 3 ). A classical problem would force f 1 (x) = 0, f 2 (x) 0 and f 3 (x) = 0, while imizing f 0 (x) = exp(x 1 ). In our problem, we still force f 3 (x) = 0, but apply a (soft) least squares penalty on f 1 (x) and a smaller one-sided penalization on f 2 (x). The above formulation is nonsmooth due to the max term in the objective function; in practice we could replace this by: exp(x 1) + 5 log(x 1 ) w x 1,x 2,x 3,w s.t. x 1 /x 2 = log(x 3 ), 3x 1 + x 2 5, x 1 0, x 2 0 w x 2 2 2, w 0

8 and recover a standard form NLP. If the penalty on f 1 (x) would be replaced by a one-norm penalization (instead of least squares), we would have to play a similar game, moving the function f 1 (x) into the constraints and adding additional variable(s). To some extent, this seems unnatural - a modeler should be able to interchange the penalization without having to reformulate the problem from scratch. The proposed extended NLP would not be reformulated at all by the modeler, but allow all these generalized constraints to be treated in a similar manner within the modeling system. The actual formulation would take: where θ(u) = θ 1 (u 1 ) + θ 2 (u 2 ) + θ 3 (u 3 ) θ 1 (u 1 ) = 5u 2 1, θ 2 (u 2 ) = 2 max(u 2, 0), θ 3 (u 3 ) = ψ {0} (u 3 ). The discussion above allows us to see that Y = R [0, 2] R, k(y) = 1 20 y The corresponding Lagrangian is the smooth function: L(x, y) = f 0 (x) + 3 y i f i (x) k(y). i=1 The corresponding VI (13) can almost be formulated in GAMS (except that the linear constraint in X cannot be handled currently except by introducing a θ 4 (x)). Thus f 4 (x) = 3x 1 + x 2 5, θ 4 (u) = ψ R resulting in the following choices for Y and k: Y = R [0, 2] R R +, k(y) = 1 20 y Since X and Y are now simple bound sets, (13) is now a complementarity problem and can be solved for example using PATH. A simple empinfo file details the choices of Y and k from the implemented library. The full model and option files are available from the authors. Reformulation as a classical NLP Suppose θ(u) = sup{u T y 1 y Y 2 yt Qy, } for a polyhedral set Y R m and a symmetric positive semidefinite Q R m m (possibly Q = 0). Suppose further that X = {x Rx r}, Y = { y S T y s }, Q = DJ 1 D T, F (x) = (f 1 (x),..., f m (x)), where J is symmetric and positive definite (for instance J = I). Then the optimal solutions x of (10) are the x components of the optimal solutions ( x, z, w) to f 0 (x) + s T z wt Jw s.t. Rx r, z 0, F (x) Sz Dw = 0. The multiplier on the equality constraint in the usual sense is the multiplier associated with x in the extended Lagrangian for (10). (Note that a Cholesky factorization may be needed to detere D.) It may be better to solve this reformulated NLP than to solve (13). However, it is important that we can convey all types of nonsmooth optimization problems to a solver as smooth optimization problems, and hence it is important to communicate the appropriate structure to the solver interface. We believe that specifying Y and k is a theoretically sound way to do this. Another example showing formulation of an extended nonlinear program as a complementarity problem within GAMS can be found in Dirkse and Ferris (1995). Conic Programg A problem of significant recent interest involves conic constraints: x X pt x s.t. Ax b 0, x C, where C is a convex cone. Using the notation outlined above, this can be expressed as an EMP: x X pt x + ψ R m (Ax b) + ψ C (x) For specific cones such as the Lorentz (ice-cream) cone, or the rotated quadratic cone, there are efficient implementations of interior point algorithms for their solution (Andersen et al., 2003). It is also possible to reformulate the problem in the form (1). Annotating

9 the variables that must lie in a particular cone using a empinfo file allows solvers like MOSEK (Andersen and Andersen, 2000) to receive the problem as a cone program, while standard NLP solvers would see a reformulation of the problem as a nonlinear program. Details on this approach can be found in the EMP manual. Embedded Complementarity Systems A different type of embedded optimization model that arises frequently in applications is: x f(x, y) s.t. g(x, y) 0 ( λ 0) H(x, y, λ) = 0 ( y free) Note the difference here: the optimization problem is over the variable x, and is parameterized by the variable y. The choice of y is fixed by the (auxiliary) complementarity relationships depicted here by H. Within GAMS, this is modeled as: model emp /deff,defg,defh/; Again, so this model can be processed correctly as an EMP, the modeler provides additional annotations to the model defining equations in an empinfo file, namely that the function H that is defined in defh is complementary to the variable y (and hence the variable y is a parameter to the optimization problem), and furthermore that the dual variable associated with the equation defg in the optimization problem is one and the same as the variable λ used to define H. Armed with this additional information, the EMP tool automatically creates the following MCP: 0 = x L(x, y, λ) x free 0 λ L(x, y, λ) λ 0 0 = H(x, y, λ) y free, where the Lagrangian is defined as L(x, y, λ) = f(x, y) + λ T g(x, y). Perhaps the most popular use of this formulation is where competition is allowed between agents. A standard method to deal with such cases is via the concept of Nash Games. In this setting x is a Nash Equilibrium if x i arg x i X i l i (x i, x i, q), i I, where x i are other players decisions and the quantities q are given exogenously, or via complementarity: 0 H(x, q) q 0. This mechanism is extremely popular in economics, and Nash famously won the Nobel Prize for his contributions to this literature. This format is again an EMP, more general than the example given above in two respects. Firstly, there is more than one optimization problem specified in the embedded complementarity system. Secondly, the parameters in each optimization problem consist of two types. Firstly, there are the variables q that are tied down by the auxiliary complementarity condition and hence are treated as parameters by the ith Nash player. Also there are the variables x i that are treated as parameters by the ith Nash player, but are treated as variables by a different player j. While we do not specify the syntax here for these issues, the manual provides examples that outline how to carry out this matching within GAMS. Finally, two points of note: first it is clear that the resulting model is a complementarity problem and can be solved using PATH, for example. Secondly, perforg the conversion from an embedded complementarity system or a Nash Game automatically is a critical step in making such models practically useful. We note that there is a large literature on discretetime finite-state stochastic games: this has become a central tool in analysis of strategic interactions among forward-looking players in dynamic environments. The Ericson and Pakes (1995) model of dynamic competition in an oligopolistic industry is exactly in the format described above, and has been used extensively in applications such as advertising, collusion, mergers, technology adoption, international trade and finance. Conclusions A number of new modeling formats involving complementarity and variational inequalities have been described and a new tool, EMP, that allows such problems to be specified in the GAMS modeling systems has been outlined. We believe this will make a modelers task easier by allowing model structure to be described succinctly. Furthermore, model generation can be done more reliably and automatically, and algorithms can exploit model structure to improve solution speed and robustness. Acknowledgements This work is supported in part by Air Force Office of Scientific Research Grant FA , and National Science Foundation Grants DMI , DMS and IIS The authors wish to

10 thank Todd Munson and Tom Rutherford for insightful comments on the material presented here. References Andersen, E. D. and Andersen, K. D. (2000). The MOSEK interior point optimizer for linear programg: an implementation of the homogeneous algorithm. In H. Frank et al., editors, High Performance Optimization, pages Kluwer Academic Publishers, Dordrecht, The Netherlands. Andersen, E. D., Roos, C., and Terlaky, T. (2003). On implementing a primal-dual interior-point method for conic quadratic optimization. Mathematical Programg, 95(2): Bisschop, J. and Meeraus, A. (1982). On the development of a general algebraic modeling system in a strategic planning environment. Mathematical Programg Study, 20:1 29. Brooke, A., Kendrick, D., and Meeraus, A. (1988). GAMS: A User s Guide. The Scientific Press, South San Francisco, California. Cao, M. and Ferris, M. C. (1996). A pivotal method for affine variational inequalities. Mathematics of Operations Research, 21: Dirkse, S. and Ferris, M. (2008). NLPEC, User s Manual. Dirkse, S. P. and Ferris, M. C. (1995). MCPLIB: A collection of nonlinear mixed complementarity problems. Optimization Methods and Software, 5: Duran, M. A. and Grossman, I. E. (1986). An outer-approximation algorithm for a class of mixedinteger nonlinear programs. Mathematical Programg, 36: Ericson, R. and Pakes, A. (1995). Markov perfect industry dynamics: A framework for empirical analysis. Review of Economic Studies, 62: Facchinei, F. and Pang, J. S. (2003). Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer-Verlag, New York, New York. Ferris, M. C., Dirkse, S. P., and Meeraus, A. (2005). Mathematical programs with equilibrium constraints: Automatic reformulation and solution via constrained optimization. In Kehoe, T. J., Srinivasan, T. N., and Whalley, J., editors, Frontiers in Applied General Equilibrium Modeling, pages Cambridge University Press. Ferris, M. C. and Munson, T. S. (2000). Complementarity problems in GAMS and the PATH solver. Journal of Economic Dynamics and Control, 24: Ferris, M. C. and Pang, J. S. (1997). Engineering and economic applications of complementarity problems. SIAM Review, 39: Fourer, R., Gay, D. M., and Kernighan, B. W. (1990). A modeling language for mathematical programg. Management Science, 36: Fourer, R., Gay, D. M., and Kernighan, B. W. (1993). AMPL: A Modeling Language for Mathematical Programg. Duxbury Press, Pacific Grove, California. Huber, P. J. (1981). Robust Statistics. John Wiley & Sons, New York. Jagla, J., Meeraus, A., Dirkse, S., and Ferris, M. (2008). EMP, User s Manual. In preparation, Ralph, D. and Wright, S. J. (2004). Some properties of regularization and penalization schemes for MPECs. Optimization Methods and Software, 19(5): Rockafellar, R. T. (1993). Lagrange multipliers and optimality. SIAM Review, 35: Rockafellar, R. T. and Wets, R. J. B. (1998). Variational Analysis. Springer-Verlag. Rutherford, T. F. (1999). Applied general equilibrium modeling with MPSGE as a GAMS subsystem: An overview of the modeling framework and syntax. Computational Economics, 14:1 46. Tawarmalani, M. and Sahinidis, N. V. (2004). Global Optimization of Mixed Integer Nonlinear Programs: A Theoretical and Computational Study. Mathematical Programg, 99(3): Wächter, A. and Biegler, L. T. (2006). On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programg. Mathematical Programg, 106(1): Wallace, J., Philpott, A., O Sullivan, M., and Ferris, M. (2006). Optimal rig design using mathematical programg. In 2nd High Performance Yacht Design Conference, Auckland, February, 2006, pages Westerlund, T., Skrifvars, H., Harjunkoski, I., and Pörn, R. (1998). An extended cutting plane method for solving a class of non-convex lp problems. Computers and Chemical Engineering, 22(3): Ferris, M. C., Fourer, R., and Gay, D. M. (1999). Expressing complementarity problems and communicating them to solvers. SIAM Journal on Optimization, 9: