Bilevel Optimization, Pricing Problems and Stackelberg Games Martine Labbé Computer Science Department Université Libre de Bruxelles INOCS Team, INRIA Lille Follower Leader CO Workshop - Aussois - January 2017 1
PART I: Bilevel optimization CO Workshop - Aussois - January 2017 2
Bilevel Optimization Problem max x,y s.t. f(x, y) (x, y) 2 where y 2 S(x) S(x) = argmax y s.t.(x, y) 2 Y g(x, y) CO Workshop - Aussois - January 2017 3
Adequate framework for Stackelberg game Leader: 1st level, Follower: 2nd level, Leader takes follower s optimal reaction into account. (1905-1946) CO Workshop - Aussois - January 2017 4
Early papers on bilevel optimization Falk (1973): Linear min-max problem Bracken & McGill (1973): First bilevel model, structural properties, military application. CO Workshop - Aussois - January 2017 5
Applications Economic game theory Production planning Revenue management Security CO Workshop - Aussois - January 2017 6
Example: a linear BP y f max x,y s.t. f 1 x + f 2 y max y g 1 x + g 2 y s.t.(x, y) 2 Y Y OS g x Inducible region (IR) CO Workshop - Aussois - January 2017 7
Coupling constraints The follower sees only the second level constraints y Y f Infeasible BP g x y f Y g CO Workshop - Aussois - January 2017 8 x
Multiple second level optima max xy x 0 s.t. y max(1 x)y y s.t.0 y 1 f Y x y y Optimistic Y f Pessimistic f Y x x CO Workshop - Aussois - January 2017 9
Multiple followers independent max x s.t. f(x, y 1,...,y n ) (x, y 1,...,y n ) 2 max y k g(x, y k ),k =1,...,n s.t.(x, y k ) 2 Y k CO Workshop - Aussois - January 2017 10
Multiple followers dependent max x s.t. f(x, y 1,...,y n ) (x, y 1,...,y n ) 2 max y k g(x, y k,y k ),k =1,...,n s.t.(x, y k,y k )) 2 Y k CO Workshop - Aussois - January 2017 11
Linear BP max x c 1 x + d 1 y s.t. A 1 x + A 1 y apple b 1 }S max y c 2 x + d 2 y, [ ] s.t.a 2 x + B 2 y apple b 2 }T CO Workshop - Aussois - January 2017 12
Linear BP Linear BP is strongly NP-hard (Hansen et al. 1992) MILP is a special case of Linear BP x 2 {0, 1}, v =0andv =argmax{w : w apple x, w apple 1 x, w 0} w IR is not convex and may be disconnected. CO Workshop - Aussois - January 2017 13
Linear BP (Bialas & Karwan(1982), Bard(1983)). IR is the union of faces of S T If Linear BP is feasible, then there exists an optimal solution which is a vertex of S T. K-th best algorithm CO Workshop - Aussois - January 2017 14
Linear BP- single level reformulation max x c 1 x + d 1 y s.t. A 1 x + B 1 y apple b 1 max y d 2 y, s.t. B 2 y apple b 2 A 2 x ( ) max x c 1 x + d 1 y s.t. A 1 x + B 1 y apple b 1 B 2 y apple b 2 A 2 x B 2 = d 2 (B 2 y b 2 + A 2 x)=0 0 Branch & Bound (Hansen et al.1992) Branch & Cut (Audet et al. 2007) CO Workshop - Aussois - January 2017 15
Linear BP - Other approaches Complementary pivoting (Bialas & Karwan 1984) Penalty functions (Anandalingam & White 1989) Reverse convex programming (Al-Khayyal 1992) CO Workshop - Aussois - January 2017 16
Mixed Integer Linear BP P P 2 -hard (Lodi et al. 2014) Branch & Bound (Moore & Bard 1990, Fischetti et al. 2016) Branch & Cut (DeNegre & Ralphs 2009, Fischetti et al. 2016) Branch & Cut & Price (Han & Zeng 2014) CO Workshop - Aussois - January 2017 17
Some references L.N. Vincente and P.H. Calamai (1994), Bilevel and multilevel programming : a bibiliography review, J. Global Optim. 5, 291-306. S. Dempe. Foundations of bilevel programming. In Nonconvex optimization and its applications, volume 61. Kluwer Academic Publishers, 2002. J. Bard. Practical Bilevel Optimisation: Algorithms and Applications. KluwerAcademic Publishers, 1998 B. Colson, P. Marcotte, and G. Savard. Bilevel programming: A survey. 4 OR, 3:87-107, 2005. M. Labbé and A. Violin. Bilevel programming and price setting problems. 4OR, 11:1-30, 2013. CO Workshop - Aussois - January 2017 18
PART II: Pricing problems CO Workshop - Aussois - January 2017 19
Adequate framework for Price Setting Problem max T 2,x,y s.t. F (T,x,y) min x,y f(t,x,y) s.t.(x, y) 2 CO Workshop - Aussois - January 2017 20
Applications CO Workshop - Aussois - January 2017 21
Price Setting Problem with linear constraints max T,x,y Tx s.t. TC f = {x, y : Ax + By b} is bounded min x,y (c + T )x + dy s.t. Ax + By b {(x, y) 2 : x =0} is nonempty CO Workshop - Aussois - January 2017 22
Example: 2 variables in second level max T,x,y s.t. min x,y s.t. Ty c 1 x +(c 2 + T )y (x, y) 2 CO Workshop - Aussois - January 2017 23
Example: 2 variables in second level CO Workshop - Aussois - January 2017 24
The first level revenue CO Workshop - Aussois - January 2017 25
Network pricing problem network with toll arcs (A 1 ) and non toll arcs (A 2 ) Costs c a on arcs Commodities (o k,d k,n k ) Routing on cheapest (cost + toll) path Maximize total revenue CO Workshop - Aussois - January 2017 26
Example 9 2 2 2 0 1 2 3 4 5 10 12 UB on (T 1 + T 2 ) = SPL(T = 1) SPL(T = 0) = 22 6 = 16 T 2,3 =5,T 4,5 = 10 CO Workshop - Aussois - January 2017 27
Example with negative toll arc 2 2 0 0 0 1 2 3 T = 4 T = 2 T = 4 12 23 34 4 6 CO Workshop - Aussois - January 2017 28
Network pricing problem (Labbé et al., 1998, Roch at al., 2005) Strongly NP-hard even for only one commodity. Polynomial for one commodity if lower level path is known one commodity if toll arcs with positive flows are known one single toll arc. Polynomial algorithm with worst-case guarantee of (log A1 )/2 + 1 CO Workshop - Aussois - January 2017 29
Network pricing problem max T 0 min x,y s.t. n k x k a a2a 1 T a k2k ( k2k (c a + T a )x k a + c a y a ) a2a 1 a2a 2 a2i + (x k a + y k a) (x k a + ya)=b k k i 8k, i a2i x k a,y k a 0, 8k, a CO Workshop - Aussois - January 2017 30
NPP: single level reformulation max T,x,y, s.t. k2k n k a2a k T a x k a a2i + (x k a + y k a) k i a2i k j apple c a + T a (x k a + y k a)=b k i 8k, i 8k, a 2 A 1,i,j k k i j apple c a 8k, a 2 A 2,i,j (c a + T a )x k a + c a y a = a2a 1 a2a 2 x k a,y k a 0 8k, a T a 0 8a 2 A 1 k o k k d k 8k CO Workshop - Aussois - January 2017 31 31
Solution approach by Branch & Cut Formulate NPP as MIP Tight bound (M a k,n a ) on tax, if arc used and if arc not used, very effective Add valid inequalities to strengthen LP relaxation CO Workshop - Aussois - January 2017 32
Product pricing Seller Consumers R k i n k p i R k i is the reservation price of consumer k for product i CO Workshop - Aussois - January 2017 33
Product pricing PPP is Strongly NP-hard even if reservation price is independent of product (Briest 2006) PPP is polynomial for one product or one customer. CO Workshop - Aussois - January 2017 34
PPP - bilevel formulation max p 0 s.t. n k p i x k i k2k i2i max (R k x k i p i )x k i, k 2 K s.t. i2i i2i x k i apple 1 x k i 0 CO Workshop - Aussois - January 2017 35
PPP - single level formulation max p 0 s.t. n k p i x k i k2k i2i (Ri k p i )x k i Rj k p j, j 2 I,k 2 K i2i (Ri k p i )x k i 0, k 2 K i2i i2i x k i apple 1 x k i 0 CO Workshop - Aussois - January 2017 36
PPP: MILP formulation (Heilporn et al., 2010, 2011) Linearize single level formulation MILP Add new FDI s convex hull for k=1 FDI s added divides the gap by 2 to 4 CO Workshop - Aussois - January 2017 37
PART III: Stakelberg games CO Workshop - Aussois - January 2017 38
Bimatrix game C Follower D Leader A B (2,1) (1,0) (4,0) (3,2) 0.5 0.5 Leader Mixed Strategy Follower Pure Strategy A Stackelberg solution to the game (B,D) yielding a payoff of (3.5,1) CO Workshop - Aussois - January 2017 39
Stackelberg vs Nash Player 2 - A Player 2 - B Player 1 - A (2,2) (4,1) Player 1 - B (1,0) (3,1) Nash equilibrium: Player 1-A and Player 2-A => (2,2) Stackelberg solution: Player 1-B and Player 2-B => (3,1) Nash equilibrium may not exist There is always an (optimistic) Stackelberg solution CO Workshop - Aussois - January 2017 40
Stackelberg Games Stackelberg Game p-followers Stackelberg Game (Conitzer & Sandblom, 2006) Leader Follower Leader Follower (R,C) Type 1 Type 2 Type 3 Type p Objective of the Game Reward-maximizing strategy for the Leader. Follower will best respond to observable Leader s strategy. CO Workshop - Aussois - January 2017 41
Applications (Tambe et al., USC) CO Workshop - Aussois - January 2017 42
The beauty of this approach comes from randomization CO Workshop - Aussois - January 2017 43
1-Follower General Stackelberg game Follower optimally chooses one strategy j with probability 1 For each possible strategy j of the follower, determine the probabilities x i that leader chooses strategy i by solving the LP: max s.t. i2i i2i R ij x i x i =1 x i 0 C ij x i i2i C il x i, 8l 2 J i2i CO Workshop - Aussois - January 2017 44
Modeling a p-followers General Stackelberg Game Follower type k 2 K and 2 [0, 1] R k,c k 2 R I J, 8k 2 K x 2 S I := {x 2 [0, 1] I : i2i x i =1} x i = probability with which the Leader plays pure strategy i q k 2 S J := {q 2 [0, 1] J : j2j q j =1}, 8k 2 K q k j = probability with which type k Follower plays pure strategy j CO Workshop - Aussois - January 2017 45
Bilevel formulation (BIL-p-G) Max x,q s.t. i2i j2j k2k x i =1, i2i k R k ijx i q k j x i 2 [0, 1] 8i 2 I, 8 9 < = q k = arg max r k C k : ijx i rj k 8k 2 K, ; i2i j2j rj k 2 [0, 1] 8j 2 J, 8k 2 K, rj k =1 8k 2 K. j2j CO Workshop - Aussois - January 2017 46
Bilinear formulation Paruchuri et al.(2008) (QUAD) max x,q,a s.t. k Rijx k i qj k i2i j2j k2k x i =1, i2i qj k =1 8k 2 K, j2j 0 apple (a k Cijx k i ) apple (1 qj k )M 8j 2 J, 8k 2 K, i2i x i 2 [0, 1] 8i 2 I, q k j 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January 2017 47
Linearize x i qj k = zk, 8i 2 I,j 2 J, k 2 K ij zij k 2 [0, 1], 8i 2 I,j 2 J, k 2 K x i = P, 8i 2 I,k 2 K j2j z k ij q k j = P i2i z k ij, 8j 2 J CO Workshop - Aussois - January 2017 12 48
MIP-p-G (MIP-p-G) max x,q s.t. k Rijz k ij k i2i i2i j2j j2j k2k zij k =1, 8k 2 K (Cij k Ci`)z k ij k 0 8j, ` 2 J, 8k 2 K, i2i zij k 0 8i 2 I,8j 2 J, 8k 2 K, zij k 2 {0, 1} 8j 2 J, 8k 2 K, i2i j2j zij k = zij 1 8i 2 I,8k 2 K. j2j CO Workshop - Aussois - January 2017 49
Computational comparison Casorran et al.(2016) Time vs. % of problems solved LP TIme vs. % of problems solved % of problems solved 100 90 80 70 60 50 40 30 20 10 0 0 0.01 0.1 1 Ln(Time) 10 100 1000 D2 0 0.01 0.1 1 FMD2 DOBSS Ln(LPTime) FMDOBSS MIPpG 100 90 80 70 60 50 40 30 20 10 Nodes vs. % of problems solved 0 1 10 100 1000 10000 100000 1000000 Ln(Nodes) % of problems solved % of problems solved 100 90 80 70 60 50 40 30 20 10 100 90 80 70 60 50 40 30 20 10 %Gap vs. % of problems solved 0 0.01 0.1 1 10 100 1000 Ln(%Gap) GSGs: I={10,20,30}, J={10,20,30}, K={2,4,6} with variability. (D2) (FMD2) (DOBSS) (FMDOBSS) (MIP-p-G) Mean Gap % 110.56 110.56 31.88 30.64 7.56 CO Workshop - Aussois - January 2017 50
Stackelberg security game Payo s depend only on which target is attacked and whether it is covered or not Covered Uncovered Defender D k (j c) D k (j u) Attacker A k (j c) A k (j u) CO Workshop - Aussois - January 2017 51
Compact representation of Stackelberg Security Games Resources-Targets settings can be modeled as a Stackelberg Game BUT if m ressources and n targets then n m pure strategies! Stackelberg Security Games can be more compactly represented. Solve for optimal coverage probabilities of the targets. CO Workshop - Aussois - January 2017 52
Stackelberg security game: extended formulation (QUAD) max x,q,a s.t. qj k (D k (j c) x i + D k (j u) x i ) k k2k j2j x i =1, i2i i2i:j2i i2i:j/2i qj k =1 8k 2 K, j2j 0 apple a k (A k (j c) x i + A k (j u) i2i:j2i i2i:j/2i x i ) apple (1 qj k )M 8j 2 J, 8k 2 K, x i 2 [0, 1] 8i 2 I, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January 2017 53
Stackelberg security game: extended formulation (QUAD) max x,q,a s.t. qj k (D k (j c) x i + D k (j u) x i ) k k2k j2j x i =1, i2i i2i:j2i i2i:j/2i cj 1 - cj qj k =1 8k 2 K, j2j 0 apple a k (A k (j c) x i + A k (j u) i2i:j2i i2i:j/2i x i ) apple (1 qj k )M 8j 2 J, 8k 2 K, x i 2 [0, 1] 8i 2 I, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January 2017 54
Stackelberg security game: compact formulation (QUAD) max x,q,a s.t. k qj k (D k (j c)c j + D k (j u)(1 c j )) k2k j2j x i =1, i2i i:j2i x i = c j 8j 2 J, x i 2 [0, 1] 8i 2 I, qj k =1 8k 2 K, j2j 0 apple a k (A k (j c)c j + A k (j u)(1 c j ) apple (1 qj k )M 8j 2 J, 8k 2 K, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January 2017 55
Stackelberg security game compact formulation (SECU-K-Quad) Max c p k (qj k (c j D k (j c)+(1 c j )D k (j u))) k2k j2j s.t. c j 2 [0, 1] 8j 2 J c j apple m, j2j qj k (c j A k (j c) (1 c j )A k (j u)) qj k (c t A k (t c) (1 c t )A k (t u)) 8k 2 K, qj k 2 {0, 1} 8j 2 J, 8k 2 K qj k =1 8k 2 K. j2j CO Workshop - Aussois - January 2017 56
Stackelberg security game: MIP3-compact (SECU-p-MIP) Max y p k (D k (j c)yjj k + D k (j u)(qj k yjj)) k s.t. k2k j2j ylj k apple mqj k 8k, j, l2j 0 apple ylj k apple qj k, 8k, j qj k =1, 8k, j2j A k (j c)yjj k + A k (j u)(qj k yjj) k A(l c)ylj k A(l u)(ql k ylj) k 0 8j, l, k, ylj k 2 {0, 1} 8l, k, l2j ylj k = ylj 1 8l, k. j2j j2j CO Workshop - Aussois - January 2017 57
Link between MIP-p-G and SECU-p-MIP Apurestrategyofthedefenderisasetofatmostmtargets y k hj = P i2i:h2i zk ij Proj(LP (P MIP3 )) LP (P SECU p MIP ) CO Workshop - Aussois - January 2017 58
Computational comparison (Casorran et al. 2016) Time vs. % of problems solved LP time vs. % of problems solved 100 90 80 70 60 50 40 30 20 10 0 0.1 1 10 100 1000 10000 Ln(Time) Nodes vs. % of problems solved % of problems solved ERASER SDOBSS MIPpS 100 90 80 70 60 50 40 30 20 10 0 0 0.01 0.1 1 10 100 Ln(LPTime) Gap% vs. % of problems solved 100 90 80 70 60 50 40 30 20 10 0 1 10 100 1000 10000 100000 1000000 10000000 Ln(Nodes) 100 90 80 70 60 50 40 30 20 10 0 0.1 1 10 100 1000 SSGs: K 2 {4, 6, 8, 12}, J 2 {30, 40, 50, 60, 70},m2 {0.25 J, 0.50 J, 0.75 J } % of problems solved Ln(Gap%) (ERASER) (SDBOSS) (MIP-p-S) Mean Gap % 204.82 28.76 1.72 CO Workshop - Aussois - January 2017 59
RECAP Bilevel bilinear optimization Pricing problems Second level: LP Stackelberg games Second level: shortest path Second level: One out of N CO Workshop - Aussois - January 2017 60
RECAP Bilevel optimization Pricing problems Single level nonlinear reformulation Second level: LP Stackelberg games Second level: shortest path Second level: One out of N CO Workshop - Aussois - January 2017 61 61
RECAP Bilevel optimization Pricing problems Single level nonlinear reformulation Second level: LP Stackelberg games MIP Second level: shortest path Second level: One out of N CO Workshop - Aussois - January 2017 62 62
2nd IWOBIP - International Workshop in Bilevel Programming Lille, June 18-22, 2018 CO Workshop - Aussois - January 2017 63