Multiple Optimization Problems with Equilibrium Constraints (MOPEC)

Size: px

Start display at page:

Download "Multiple Optimization Problems with Equilibrium Constraints (MOPEC)"

Eleanore Harrell
5 years ago
Views:

1 Multiple Optimization Problems with Equilibrium Constraints (MOPEC) Michael C. Ferris Joint work with: Roger Wets, and Andy Philpott, Wolfgang Britz, Arnim Kuhn, Jesse Holzer University of Wisconsin, Madison International Conference on Continuous Optimization, Lisbon, Portugal July 29, 2013 Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

2 Optimization in the 21st century A P P L I C A T I O N S Many applications driving visibility Optimization embedded within disciples (operations research, computer science, engineering, economics, business) Distinct renaissance in the last decade Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

3 Optimization in the 21st century A P P L I C A T I O N S T H E O R Y Many applications driving visibility Optimization embedded within disciples (operations research, computer science, engineering, economics, business) Distinct renaissance in the last decade Theory well established, new problem classes, practically motivated results Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

4 Optimization in the 21st century A P P L I C A T I O N S L G O RI T M ST H E O R Y Many applications driving visibility Optimization embedded within disciples (operations research, computer science, engineering, economics, business) Distinct renaissance in the last decade Theory well established, new problem classes, practically motivated results New algorithms and (commercial) solver development New interfaces to statistics and big data Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

5 Optimization in the 21st century A P P L I C A T I O N S L G O RI T M ST H E O R Y Many applications driving visibility Optimization embedded within disciples (operations research, computer science, engineering, economics, business) Distinct renaissance in the last decade Theory well established, new problem classes, practically motivated results New algorithms and (commercial) solver development New interfaces to statistics and big data Critical to develop all three pieces: the strength of our discipline This talk will stress competition (between models), stochasticity (within models), dynamics (over time) Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

6 Water rights pricing (Britz/F./Kuhn) Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

7 The model AO (cooperative firms) max q i,x i,wo i 0 q i p x i,f w f i f {int,lab} s.t. q i f (x i,f + e i,f ) ɛ i,f x i,land e i,land wo i 1 = x i,wat + wo i Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

8 The model AO (this is not an MPEC) max q i,x i,wo i 0 q i p x i,f w f i f {int,lab} s.t. q i f (x i,f + e i,f ) ɛ i,f x i,land e i,land wo i 1 = x i,wat + wo i 0 p i q i d(p) 0 0 w lab i e i,lab i x i,lab 0 Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

9 (M)OPEC max θ(x, p) s.t. g(x, p) 0 x 0 p h(x, p) 0 equilibrium max theta x g vi h p Solved concurrently (in a Nash manner) Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

10 (M)OPEC max θ(x, p) s.t. g(x, p) 0 x 0 = h(x, p) equilibrium max theta x g vi h p Solved concurrently (in a Nash manner) Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

11 (M)OPEC max θ(x, p) s.t. g(x, p) 0 x 0 p h(x, p) 0 x x θ(x, p) + λ T x g(x, p) 0 λ g(x, p) 0 0 p h(x, p) 0 equilibrium max theta x g vi h p Solved concurrently (in a Nash manner) Requires global solutions of agents problems (or theory to guarantee KKT are equivalent) Theory of existence, uniqueness and stability based in variational analysis Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

12 The model IO (independent firms) max q i,x i,wo i 0 s.t. ( q i p ) x i,f w f f q i (x i,f + e i,f ) ɛ i,f f x i,land e i,land wo i 1 = x i,wat + wo i 0 p i q i d(p) 0 0 w lab i e i,lab i x i,lab 0 Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

13 !" #" $%&%& IO vs AO (price of anarchy) '(&&& '%&&&) '&&&&) $%&&&) $&&&&) %&&&) &)!") #") Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

14 The model IO + trade (mechanism design) max q i,x i,wo i,wri b,wri s 0 s.t. ( q i p x i,f w f wri b f q i (x i,f + e i,f ) ɛ i,f f (w wr + τ) + wr s i w wr ) x i,land e i,land wo i 1 = x i,wat + wo i wr i + wri b x i,wat + wri s 0 p i q i d(p) 0 0 w lab i 0 w wr i e i,lab i wri s i x i,lab 0 wr b i 0 Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

15 Figure 4 Click here to download high resolution image Different Management Strategies Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

16 MOPEC min θ i (x i, x i, p) s.t. g i (x i, x i, p) 0, i x i p solves VI(h(x, ), C) equilibrium min theta(1) x(1) g(1)... min theta(m) x(m) g(m) vi h p cons Reformulate optimization problem as first order conditions (complementarity) Use nonsmooth Newton methods to solve complementarity problem Precondition using Trade/Policy Model (MCP) individual optimization with fixed externalities Split model (18,000 vars) via region = + Gauss-Seidel, Jacobi, Asynchronous Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

17 is Hydro-Thermal paper we will System illustrate (Philpott/F./Wets) the concepts using the voir and one thermal plant, as shown in Figure 1. Figure 1: Example hydro-thermal system. Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

18 Simple electricity system optimization problem SO: max d k,u i,v j,x i 0 s.t. W k (d k ) C j (v j ) + V i (x i ) k K j T i H U i (u i ) + v j d k, i H j T k K x i = xi 0 u i + hi 1, i H u i water release of hydro reservoir i H v j thermal generation of plant j T x i water level in reservoir i H prod fn U i (strictly concave) converts water release to energy C j (v j ) denote the cost of generation by thermal plant V i (x i ) future value of terminating with storage x (assumed separable) W k (d k ) utility of consumption d k Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

19 SO equivalent to CE Consumers k K solve CP(k): max d k 0 Thermal plants j T solve TP(j): max v j 0 Hydro plants i H solve HP(i): max u i,x i 0 W k (d k ) p T d k p T v j C j (v j ) p T U i (u i ) + V i (x i ) s.t. x i = xi 0 u i + hi 1 Perfectly competitive (Walrasian) equilibrium is a MOPEC CE: d k arg max CP(k), k K, v j arg max TP(j), j T, u i, x i arg max HP(i), i H, 0 p U i (u i ) + v j d k. i H j T k K Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

20 Agents have stochastic recourse? Two stage stochastic programming, x 1 is here-and-now decision, recourse decisions x 2 depend on realization of a random variable R is a risk measure (e.g. expectation, CVaR) SP: max c T x 1 + R[q T x 2 ] s.t. Ax 1 = b, x 1 0, A T W T (ω)x 1 + W (ω)x 2 (ω) d(ω), x 2 (ω) 0, ω Ω. EMP/SP extensions to facilitate these models T igure Constraints matrix structure of 15) problem by suitable subgradient methods in an outer loop. In the inner problem is solved for various righthand sides. Convexity of the master Ferris (Univ. Wisconsin) convexity MOPEC of the value function in linear programming. ICCOPT 2013 In dual 14 decom / 24

21 Two stage stochastic MOPEC CP(k): TP(j): HP(i): d 1 k v 1 j max max 0 0 W k ( d 1 k ) p 1 d 1 k p 1 v 1 j C j (v 1 j ) max p 1 U i (u 1 ui 1,x1 i 0 i ) s.t. x 1 i = x 0 i u 1 i + h 1 i, 0 p 1 i H U i ( u 1 i ) + j T v 1 j k K d 1 k Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

22 Two stage stochastic MOPEC CP(k): TP(j): HP(i): max d 1 k,d2 k (ω) 0 ( ) W k d 1 k p 1 dk 1 + R[W ( k d 2 k (ω) ) p 2 (ω)dk 2 (ω)] max p 1 v 1 j (ω) 0 j C j (vj 1 ) + R[p 2 (ω)vj 2 ( (ω) C j v 2 j (ω) ) ] v 1 j,v 2 max u 1 i,x1 i 0 u 2 i (ω),x2 i (ω) 0 p 1 U i (ui 1 ) + R[p 2 (ω)u i (ui 2 (ω)) + V i (xi 2 (ω))] s.t. x 1 i = x 0 i u 1 i + h 1 i, 0 p 1 i H x 2 i (ω) = x 1 i U i ( u 1 i ) + u 2 i (ω) + h 2 i (ω) j T v 1 j k K d 1 k Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

23 Two stage stochastic MOPEC CP(k): TP(j): HP(i): max d 1 k,d2 k (ω) 0 ( ) W k d 1 k p 1 dk 1 + R[W ( k d 2 k (ω) ) p 2 (ω)dk 2 (ω)] max p 1 v 1 j (ω) 0 j C j (vj 1 ) + R[p 2 (ω)vj 2 ( (ω) C j v 2 j (ω) ) ] v 1 j,v 2 max u 1 i,x1 i 0 u 2 i (ω),x2 i (ω) 0 p 1 U i (ui 1 ) + R[p 2 (ω)u i (ui 2 (ω)) + V i (xi 2 (ω))] s.t. x 1 i = x 0 i u 1 i + h 1 i, 0 p 1 i H 0 p 2 (ω) i H x 2 i (ω) = x 1 i U i ( u 1 i ) + u 2 i (ω) + h 2 i (ω) j T U i ( u 2 i (ω) ) + j T v 1 j k K d 1 k vj 2 (ω) dk 2 (ω), ω k K Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

24 Results Suppose every agent is risk neutral and has knowledge of all deterministic data, as well as sharing the same probability distribution for inflows. SO solution is same as CE solution Using coherent risk measure (weighted sum of expected value and conditional value at risk), 10 scenarios for rain 1 High initial storage: risk-averse central plan (RSO) and the risk-averse competitive equilibrium (RCE) have same solution (but different to risk neutral case) 2 Low initial storage: RSO and RCE are very different. Since the hydro generator and the system do not agree on a worst-case outcome, the probability distributions that correspond to an equivalent risk neutral decision will not be common. 3 Extension: Construct MOPEC models for trading risk Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

25 Contracts in MOPEC (F./Wets) Competing agents (consumers, or generators in energy market) Each agent minimizes objective independently (cost) Market prices are function of all agents activities Additional twist: model must hedge against uncertainty Facilitated by allowing contracts bought now, for goods delivered later Conceptually allows to transfer goods from one period to another (provides wealth retention or pricing of ancilliary services in energy market) Can investigate new instruments to mitigate risk, or move to system optimal solutions from equilibrium (or market) solutions Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

26 Example as MOPEC: agents solve a Stochastic Program Buy y i contracts in period 1, to deliver D(ω)y i in period 2, scenario ω Each agent i: min C(x 1 i ) + ω π ω C(x 2 i (ω)) s.t. p 1 x 1 i + vy i p 1 e 1 i (budget time 1) p 2 (ω)x 2 i (ω) p 2 (ω)(d(ω)y i + e 2 i (ω)) (budget time 2) 0 v i y i 0 (contract) 0 p 1 i ( e 1 i xi 1 ) 0 (walras 1) 0 p 2 (ω) i ( D(ω)yi + e 2 i (ω) x 2 i (ω) ) 0 (walras 2) Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

27 Observations Examples from literature solved using homotopy continuation seem incorrect - need transaction costs to guarantee solution Solution possible via disaggregation only seems possible in special cases When problem is block diagonally dominant When overall (complementarity) problem is monotone (Pang): when problem is a potential game Progressive hedging possible to decompose in these settings by agent and scenario Research challenge: develop reliable algorithms for large scale decomposition approaches to MOPEC Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

28 PJM buy/sell dynamic model Storage transfers energy over time (horizon = T ). PJM: given price path p t, determine charge q + t and discharge q t : max h t,q + t,q t T p t (qt q t + ) t=0 s.t. h t = eq t + qt 0 h t S 0 q t + Q 0 qt Q h 0, h T fixed Uses: price shaving, load shifting, transmission line deferral What about real-time storage, or different storage technologies? Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

29 Stochastic price paths (day ahead market) min x,h,q +,q c1 (x) + E ω [ T ] p ωt (q ωt + qωt) + c 2 (q ωt + + qωt) t=0 s.t. h ωt = eq + ωt q ωt 0 h ωt Sx 0 q + ωt, q ωt Qx h ω0, h ωt fixed First stage decision x: amount of storage to deploy. Second stage decision: charging strategy in face of uncertainty Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

30 Distribution of (multiple) storage types Determine storage facilities x k to build, given distribution of price paths: no entry barriers into market, etc. MOPEC: for all k solve a two stage SP k : min x k,h k,q + k,q k [ T ] ck 1 (x k) + E ω p ωt (q + ωkt q ωkt ) + c2 k (q+ ωkt + q ωkt ) t=0 s.t. h ωkt = eq + ωkt q ωkt 0 h ωkt Sx k 0 q + ωkt, q ωkt Qx k h ωk0, h ωkt fixed p ωt = f ( γ, D ωt + k (q + ωkt q ωkt ) ) Parametric function (γ) determined by regression. Storage operators react to shift in demand. Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

31 What is EMP? Annotates existing equations/variables/models for modeler to provide/define additional structure equilibrium vi (agents can solve min/max/vi) bilevel (reformulate as MPEC, or as SOCP) disjunction (or other constraint logic primitives) randvar dualvar (use multipliers from one agent as variables for another) extended nonlinear programs (library of plq functions) Currently available within GAMS Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

32 Conclusions Optimization helps understand what drives a system Modern optimization within applications requires multiple model formats, computational tools and sophisticated solvers EMP model type is clear and extensible, additional structure available to solver Stochastic MOPEC models capture behavioral effects (as an EMP) Policy implications addressable using MOPEC Extended Mathematical Programming available within the GAMS modeling system Modeling, optimization, statistics and computation embedded within the application domain is critical Ferris (Univ. Wisconsin) MOPEC ICCOPT / 24

Modeling, equilibria, power and risk

Modeling, equilibria, power and risk Michael C. Ferris Joint work with Andy Philpott and Roger Wets University of Wisconsin, Madison Workshop on Stochastic Optimization and Equilibrium University of Southern