Bilevel Optimization, Pricing Problems and Stackelberg Games

Transcription

1 Bilevel Optimization, Pricing Problems and Stackelberg Games Martine Labbé Computer Science Department Université Libre de Bruxelles INOCS Team, INRIA Lille Follower Leader CO Workshop - Aussois - January

2 PART I: Bilevel optimization CO Workshop - Aussois - January

3 Bilevel Optimization Problem max x,y s.t. f(x, y) (x, y) 2 where y 2 S(x) S(x) = argmax y s.t.(x, y) 2 Y g(x, y) CO Workshop - Aussois - January

4 Adequate framework for Stackelberg game Leader: 1st level, Follower: 2nd level, Leader takes follower s optimal reaction into account. ( ) CO Workshop - Aussois - January

5 Early papers on bilevel optimization Falk (1973): Linear min-max problem Bracken & McGill (1973): First bilevel model, structural properties, military application. CO Workshop - Aussois - January

6 Applications Economic game theory Production planning Revenue management Security CO Workshop - Aussois - January

7 Example: a linear BP y f max x,y s.t. f 1 x + f 2 y max y g 1 x + g 2 y s.t.(x, y) 2 Y Y OS g x Inducible region (IR) CO Workshop - Aussois - January

8 Coupling constraints The follower sees only the second level constraints y Y f Infeasible BP g x y f Y g CO Workshop - Aussois - January x

9 Multiple second level optima max xy x 0 s.t. y max(1 x)y y s.t.0 y 1 f Y x y y Optimistic Y f Pessimistic f Y x x CO Workshop - Aussois - January

10 Multiple followers independent max x s.t. f(x, y 1,...,y n ) (x, y 1,...,y n ) 2 max y k g(x, y k ),k =1,...,n s.t.(x, y k ) 2 Y k CO Workshop - Aussois - January

11 Multiple followers dependent max x s.t. f(x, y 1,...,y n ) (x, y 1,...,y n ) 2 max y k g(x, y k,y k ),k =1,...,n s.t.(x, y k,y k )) 2 Y k CO Workshop - Aussois - January

12 Linear BP max x c 1 x + d 1 y s.t. A 1 x + A 1 y apple b 1 }S max y c 2 x + d 2 y, [ ] s.t.a 2 x + B 2 y apple b 2 }T CO Workshop - Aussois - January

13 Linear BP Linear BP is strongly NP-hard (Hansen et al. 1992) MILP is a special case of Linear BP x 2 {0, 1}, v =0andv =argmax{w : w apple x, w apple 1 x, w 0} w IR is not convex and may be disconnected. CO Workshop - Aussois - January

14 Linear BP (Bialas & Karwan(1982), Bard(1983)). IR is the union of faces of S T If Linear BP is feasible, then there exists an optimal solution which is a vertex of S T. K-th best algorithm CO Workshop - Aussois - January

15 Linear BP- single level reformulation max x c 1 x + d 1 y s.t. A 1 x + B 1 y apple b 1 max y d 2 y, s.t. B 2 y apple b 2 A 2 x ( ) max x c 1 x + d 1 y s.t. A 1 x + B 1 y apple b 1 B 2 y apple b 2 A 2 x B 2 = d 2 (B 2 y b 2 + A 2 x)=0 0 Branch & Bound (Hansen et al.1992) Branch & Cut (Audet et al. 2007) CO Workshop - Aussois - January

16 Linear BP - Other approaches Complementary pivoting (Bialas & Karwan 1984) Penalty functions (Anandalingam & White 1989) Reverse convex programming (Al-Khayyal 1992) CO Workshop - Aussois - January

17 Mixed Integer Linear BP P P 2 -hard (Lodi et al. 2014) Branch & Bound (Moore & Bard 1990, Fischetti et al. 2016) Branch & Cut (DeNegre & Ralphs 2009, Fischetti et al. 2016) Branch & Cut & Price (Han & Zeng 2014) CO Workshop - Aussois - January

18 Some references L.N. Vincente and P.H. Calamai (1994), Bilevel and multilevel programming : a bibiliography review, J. Global Optim. 5, S. Dempe. Foundations of bilevel programming. In Nonconvex optimization and its applications, volume 61. Kluwer Academic Publishers, J. Bard. Practical Bilevel Optimisation: Algorithms and Applications. KluwerAcademic Publishers, 1998 B. Colson, P. Marcotte, and G. Savard. Bilevel programming: A survey. 4 OR, 3:87-107, M. Labbé and A. Violin. Bilevel programming and price setting problems. 4OR, 11:1-30, CO Workshop - Aussois - January

19 PART II: Pricing problems CO Workshop - Aussois - January

20 Adequate framework for Price Setting Problem max T 2,x,y s.t. F (T,x,y) min x,y f(t,x,y) s.t.(x, y) 2 CO Workshop - Aussois - January

21 Applications CO Workshop - Aussois - January

22 Price Setting Problem with linear constraints max T,x,y Tx s.t. TC f = {x, y : Ax + By b} is bounded min x,y (c + T )x + dy s.t. Ax + By b {(x, y) 2 : x =0} is nonempty CO Workshop - Aussois - January

23 Example: 2 variables in second level max T,x,y s.t. min x,y s.t. Ty c 1 x +(c 2 + T )y (x, y) 2 CO Workshop - Aussois - January

24 Example: 2 variables in second level CO Workshop - Aussois - January

25 The first level revenue CO Workshop - Aussois - January

26 Network pricing problem network with toll arcs (A 1 ) and non toll arcs (A 2 ) Costs c a on arcs Commodities (o k,d k,n k ) Routing on cheapest (cost + toll) path Maximize total revenue CO Workshop - Aussois - January

27 Example UB on (T 1 + T 2 ) = SPL(T = 1) SPL(T = 0) = 22 6 = 16 T 2,3 =5,T 4,5 = 10 CO Workshop - Aussois - January

28 Example with negative toll arc T = 4 T = 2 T = CO Workshop - Aussois - January

29 Network pricing problem (Labbé et al., 1998, Roch at al., 2005) Strongly NP-hard even for only one commodity. Polynomial for one commodity if lower level path is known one commodity if toll arcs with positive flows are known one single toll arc. Polynomial algorithm with worst-case guarantee of (log A1 )/2 + 1 CO Workshop - Aussois - January

30 Network pricing problem max T 0 min x,y s.t. n k x k a a2a 1 T a k2k ( k2k (c a + T a )x k a + c a y a ) a2a 1 a2a 2 a2i + (x k a + y k a) (x k a + ya)=b k k i 8k, i a2i x k a,y k a 0, 8k, a CO Workshop - Aussois - January

31 NPP: single level reformulation max T,x,y, s.t. k2k n k a2a k T a x k a a2i + (x k a + y k a) k i a2i k j apple c a + T a (x k a + y k a)=b k i 8k, i 8k, a 2 A 1,i,j k k i j apple c a 8k, a 2 A 2,i,j (c a + T a )x k a + c a y a = a2a 1 a2a 2 x k a,y k a 0 8k, a T a 0 8a 2 A 1 k o k k d k 8k CO Workshop - Aussois - January

32 Solution approach by Branch & Cut Formulate NPP as MIP Tight bound (M a k,n a ) on tax, if arc used and if arc not used, very effective Add valid inequalities to strengthen LP relaxation CO Workshop - Aussois - January

33 Product pricing Seller Consumers R k i n k p i R k i is the reservation price of consumer k for product i CO Workshop - Aussois - January

34 Product pricing PPP is Strongly NP-hard even if reservation price is independent of product (Briest 2006) PPP is polynomial for one product or one customer. CO Workshop - Aussois - January

35 PPP - bilevel formulation max p 0 s.t. n k p i x k i k2k i2i max (R k x k i p i )x k i, k 2 K s.t. i2i i2i x k i apple 1 x k i 0 CO Workshop - Aussois - January

36 PPP - single level formulation max p 0 s.t. n k p i x k i k2k i2i (Ri k p i )x k i Rj k p j, j 2 I,k 2 K i2i (Ri k p i )x k i 0, k 2 K i2i i2i x k i apple 1 x k i 0 CO Workshop - Aussois - January

37 PPP: MILP formulation (Heilporn et al., 2010, 2011) Linearize single level formulation MILP Add new FDI s convex hull for k=1 FDI s added divides the gap by 2 to 4 CO Workshop - Aussois - January

38 PART III: Stakelberg games CO Workshop - Aussois - January

39 Bimatrix game C Follower D Leader A B (2,1) (1,0) (4,0) (3,2) Leader Mixed Strategy Follower Pure Strategy A Stackelberg solution to the game (B,D) yielding a payoff of (3.5,1) CO Workshop - Aussois - January

40 Stackelberg vs Nash Player 2 - A Player 2 - B Player 1 - A (2,2) (4,1) Player 1 - B (1,0) (3,1) Nash equilibrium: Player 1-A and Player 2-A => (2,2) Stackelberg solution: Player 1-B and Player 2-B => (3,1) Nash equilibrium may not exist There is always an (optimistic) Stackelberg solution CO Workshop - Aussois - January

41 Stackelberg Games Stackelberg Game p-followers Stackelberg Game (Conitzer & Sandblom, 2006) Leader Follower Leader Follower (R,C) Type 1 Type 2 Type 3 Type p Objective of the Game Reward-maximizing strategy for the Leader. Follower will best respond to observable Leader s strategy. CO Workshop - Aussois - January

42 Applications (Tambe et al., USC) CO Workshop - Aussois - January

43 The beauty of this approach comes from randomization CO Workshop - Aussois - January

44 1-Follower General Stackelberg game Follower optimally chooses one strategy j with probability 1 For each possible strategy j of the follower, determine the probabilities x i that leader chooses strategy i by solving the LP: max s.t. i2i i2i R ij x i x i =1 x i 0 C ij x i i2i C il x i, 8l 2 J i2i CO Workshop - Aussois - January

45 Modeling a p-followers General Stackelberg Game Follower type k 2 K and 2 [0, 1] R k,c k 2 R I J, 8k 2 K x 2 S I := {x 2 [0, 1] I : i2i x i =1} x i = probability with which the Leader plays pure strategy i q k 2 S J := {q 2 [0, 1] J : j2j q j =1}, 8k 2 K q k j = probability with which type k Follower plays pure strategy j CO Workshop - Aussois - January

46 Bilevel formulation (BIL-p-G) Max x,q s.t. i2i j2j k2k x i =1, i2i k R k ijx i q k j x i 2 [0, 1] 8i 2 I, 8 9 < = q k = arg max r k C k : ijx i rj k 8k 2 K, ; i2i j2j rj k 2 [0, 1] 8j 2 J, 8k 2 K, rj k =1 8k 2 K. j2j CO Workshop - Aussois - January

47 Bilinear formulation Paruchuri et al.(2008) (QUAD) max x,q,a s.t. k Rijx k i qj k i2i j2j k2k x i =1, i2i qj k =1 8k 2 K, j2j 0 apple (a k Cijx k i ) apple (1 qj k )M 8j 2 J, 8k 2 K, i2i x i 2 [0, 1] 8i 2 I, q k j 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January

48 Linearize x i qj k = zk, 8i 2 I,j 2 J, k 2 K ij zij k 2 [0, 1], 8i 2 I,j 2 J, k 2 K x i = P, 8i 2 I,k 2 K j2j z k ij q k j = P i2i z k ij, 8j 2 J CO Workshop - Aussois - January

49 MIP-p-G (MIP-p-G) max x,q s.t. k Rijz k ij k i2i i2i j2j j2j k2k zij k =1, 8k 2 K (Cij k Ci`)z k ij k 0 8j, ` 2 J, 8k 2 K, i2i zij k 0 8i 2 I,8j 2 J, 8k 2 K, zij k 2 {0, 1} 8j 2 J, 8k 2 K, i2i j2j zij k = zij 1 8i 2 I,8k 2 K. j2j CO Workshop - Aussois - January

50 Computational comparison Casorran et al.(2016) Time vs. % of problems solved LP TIme vs. % of problems solved % of problems solved Ln(Time) D FMD2 DOBSS Ln(LPTime) FMDOBSS MIPpG Nodes vs. % of problems solved Ln(Nodes) % of problems solved % of problems solved %Gap vs. % of problems solved Ln(%Gap) GSGs: I={10,20,30}, J={10,20,30}, K={2,4,6} with variability. (D2) (FMD2) (DOBSS) (FMDOBSS) (MIP-p-G) Mean Gap % CO Workshop - Aussois - January

51 Stackelberg security game Payo s depend only on which target is attacked and whether it is covered or not Covered Uncovered Defender D k (j c) D k (j u) Attacker A k (j c) A k (j u) CO Workshop - Aussois - January

52 Compact representation of Stackelberg Security Games Resources-Targets settings can be modeled as a Stackelberg Game BUT if m ressources and n targets then n m pure strategies! Stackelberg Security Games can be more compactly represented. Solve for optimal coverage probabilities of the targets. CO Workshop - Aussois - January

53 Stackelberg security game: extended formulation (QUAD) max x,q,a s.t. qj k (D k (j c) x i + D k (j u) x i ) k k2k j2j x i =1, i2i i2i:j2i i2i:j/2i qj k =1 8k 2 K, j2j 0 apple a k (A k (j c) x i + A k (j u) i2i:j2i i2i:j/2i x i ) apple (1 qj k )M 8j 2 J, 8k 2 K, x i 2 [0, 1] 8i 2 I, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January

54 Stackelberg security game: extended formulation (QUAD) max x,q,a s.t. qj k (D k (j c) x i + D k (j u) x i ) k k2k j2j x i =1, i2i i2i:j2i i2i:j/2i cj 1 - cj qj k =1 8k 2 K, j2j 0 apple a k (A k (j c) x i + A k (j u) i2i:j2i i2i:j/2i x i ) apple (1 qj k )M 8j 2 J, 8k 2 K, x i 2 [0, 1] 8i 2 I, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January

55 Stackelberg security game: compact formulation (QUAD) max x,q,a s.t. k qj k (D k (j c)c j + D k (j u)(1 c j )) k2k j2j x i =1, i2i i:j2i x i = c j 8j 2 J, x i 2 [0, 1] 8i 2 I, qj k =1 8k 2 K, j2j 0 apple a k (A k (j c)c j + A k (j u)(1 c j ) apple (1 qj k )M 8j 2 J, 8k 2 K, qj k 2 {0, 1} 8j 2 J, 8k 2 K, a k 2 R 8k 2 K. CO Workshop - Aussois - January

56 Stackelberg security game compact formulation (SECU-K-Quad) Max c p k (qj k (c j D k (j c)+(1 c j )D k (j u))) k2k j2j s.t. c j 2 [0, 1] 8j 2 J c j apple m, j2j qj k (c j A k (j c) (1 c j )A k (j u)) qj k (c t A k (t c) (1 c t )A k (t u)) 8k 2 K, qj k 2 {0, 1} 8j 2 J, 8k 2 K qj k =1 8k 2 K. j2j CO Workshop - Aussois - January

57 Stackelberg security game: MIP3-compact (SECU-p-MIP) Max y p k (D k (j c)yjj k + D k (j u)(qj k yjj)) k s.t. k2k j2j ylj k apple mqj k 8k, j, l2j 0 apple ylj k apple qj k, 8k, j qj k =1, 8k, j2j A k (j c)yjj k + A k (j u)(qj k yjj) k A(l c)ylj k A(l u)(ql k ylj) k 0 8j, l, k, ylj k 2 {0, 1} 8l, k, l2j ylj k = ylj 1 8l, k. j2j j2j CO Workshop - Aussois - January

58 Link between MIP-p-G and SECU-p-MIP Apurestrategyofthedefenderisasetofatmostmtargets y k hj = P i2i:h2i zk ij Proj(LP (P MIP3 )) LP (P SECU p MIP ) CO Workshop - Aussois - January

59 Computational comparison (Casorran et al. 2016) Time vs. % of problems solved LP time vs. % of problems solved Ln(Time) Nodes vs. % of problems solved % of problems solved ERASER SDOBSS MIPpS Ln(LPTime) Gap% vs. % of problems solved Ln(Nodes) SSGs: K 2 {4, 6, 8, 12}, J 2 {30, 40, 50, 60, 70},m2 {0.25 J, 0.50 J, 0.75 J } % of problems solved Ln(Gap%) (ERASER) (SDBOSS) (MIP-p-S) Mean Gap % CO Workshop - Aussois - January

60 RECAP Bilevel bilinear optimization Pricing problems Second level: LP Stackelberg games Second level: shortest path Second level: One out of N CO Workshop - Aussois - January

61 RECAP Bilevel optimization Pricing problems Single level nonlinear reformulation Second level: LP Stackelberg games Second level: shortest path Second level: One out of N CO Workshop - Aussois - January

62 RECAP Bilevel optimization Pricing problems Single level nonlinear reformulation Second level: LP Stackelberg games MIP Second level: shortest path Second level: One out of N CO Workshop - Aussois - January

63 2nd IWOBIP - International Workshop in Bilevel Programming Lille, June 18-22, 2018 CO Workshop - Aussois - January