The Knapsack Problem

Transcription

1 The Knapsack Problem René Beier Max-Planck-Institut für Informatik Saarbrücken, Germany René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 1

2 The Knapsack Problem Given n items with weights w 1,...,w n profits p 1,..., p n and a knapsack of capacity b. Find a subset K {1,2,...,n} such that the capacity is not exceeded, i.e., satisfy w i b, i K and the profit is maximized, i.e., maximize p i. i K René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 2

3 Is it difficult? Worst Case: Knapsack is difficult (NP-hard) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 3

4 Is it difficult? Worst Case: Knapsack is difficult (NP-hard) Average Case: Knapsack is easy... René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 3

5 Is it difficult? Worst Case: Knapsack is difficult (NP-hard) Average Case: Knapsack is easy... Example: Uniform Distribution, 1 Mio items Preprocessing: Finding optimal solution: 850 ms 10 ms René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 3

6 Is it difficult? Worst Case: Knapsack is difficult (NP-hard) Average Case: Knapsack is easy... Example: Uniform Distribution, 1 Mio items Preprocessing: Finding optimal solution: 850 ms 10 ms Key concepts: Core algorithms, Sparse Dynamic Programming René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 3

7 uniform random instance profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 4

8 ... find the best knapsack filling profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 5

9 ... find the best knapsack filling capacity 5 4 profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 5

10 ... find the best knapsack filling capacity 5 4 profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 5

11 ... find the best knapsack filling profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 5

12 Dominating Sets A subset K [n] is called dominating set if p(k) > p(j), for all J [n] with w(j) w(k). René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 6

13 Dominating Sets A subset K [n] is called dominating set if p(k) > p(j), for all J [n] with w(j) w(k). dominating set = Pareto-optimal solution René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 6

14 Dominating Sets A subset K [n] is called dominating set if p(k) > p(j), for all J [n] with w(j) w(k). dominating set = Pareto-optimal solution A list with all dominating sets can be calculated efficiently... René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 6

15 The Nemhauser/Ullmann algorithm (1969) after adding elements 1 to i 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 7

16 The Nemhauser/Ullmann algorithm (1969) now add element i w i p i René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 7

17 The Nemhauser/Ullmann algorithm (1969) compute upper envelope w i p i René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 7

18 The Nemhauser/Ullmann algorithm (1969) continue with next element René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 7

19 running time analysis q i... # dominating sets over items {1,...,i}. running time T = O ( n i=1q i 1 ) ( n i=1 ) E[T ] = O E[q i 1 ] René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 8

20 running time analysis q i... # dominating sets over items {1,...,i}. running time T = O ( n i=1q i 1 ) ( n i=1 ) E[T ] = O E[q i 1 ] Lemma: For uniform random instances it holds i 1,E[q i ] 16i 3 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 8

21 running time analysis q i... # dominating sets over items {1,...,i}. running time T = O ( n i=1q i 1 ) ( n i=1 ) E[T ] = O E[q i 1 ] Lemma: For uniform random instances it holds i 1,E[q i ] 16i 3 thus E[T ] = O ( n i=1i 3 ) = O ( n 4) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 8

22 average case analysis - uniform distribution proof idea: show that the expected step size is large K K René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 9

23 average case analysis - uniform distribution proof idea: show that the expected step size is large K n/2 K We show E[ K K > 0] 1 32n 2. René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 9

24 average case analysis - uniform distribution We show Pr [ K 1 16n 2 K > 0 ] 1 2. This implies E[ K K > 0] 1 32n 2. René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 10

25 average case analysis - uniform distribution We show Pr [ K 1 16n 2 K > 0 ] 1 2. This implies E[ K K > 0] 1 32n 2. Let S 1,...,S 2 n denote the subsets in non-decreasing order of weight. Let K be the k-th subset in this order, i.e., K = S k. profit subsets S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 S k René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 10

26 average case analysis - uniform distribution Pr [ K 1 16n 2 ] K > 0 = Pr [ j < k : p(s j ) p(s k ) 1 16n 2 j < k : p(s j ) p(s k ) ] = = René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 11

27 average case analysis - uniform distribution Pr [ K 1 16n 2 ] K > 0 = Pr [ j < k : p(s j ) p(s k ) 1 16n 2 j < k : p(s j ) p(s k ) ] = Pr [ j : p(s j \ S k ) P(S k \ S j ) 1 16n 2 j : p(s j \ S k ) P(S k \ S j ) ] = René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 11

28 average case analysis - uniform distribution [ Pr K 1 16n 2 ] K > 0 = Pr [ j < k : p(s j ) p(s k ) 1 16n 2 j < k : p(s j ) p(s k ) ] = Pr [ j : p(s j \ S k ) P(S k \ S j ) 1 16n 2 j : p(s j \ S k ) P(S k \ S j ) ] = Pr [ j : p(s j \ S k ) U j 1 16n 2 j < k : p(s j \ S k ) U j ] René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 11

29 average case analysis - uniform distribution [ Pr K 1 16n 2 ] K > 0 = Pr [ j < k : p(s j ) p(s k ) 1 16n 2 j < k : p(s j ) p(s k ) ] = Pr [ j : p(s j \ S k ) P(S k \ S j ) 1 16n 2 j : p(s j \ S k ) P(S k \ S j ) ] = Pr [ j : p(s j \ S k ) U j 1 16n 2 j < k : p(s j \ S k ) U j ] } {{ } polytope A } {{ } polytope B René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 11

30 average case analysis - uniform distribution Let l = S k. W.l.o.g. S k = {n l + 1,...,n}. We obtain the (n l)-dimensional polytopes: A = B = { (p 1... p n l ) [0,1] n l j < k : p i U j 1 i S j \S k 16n 2 { } (p 1... p n l ) [0,1] n l j < k : p i U j i S j \S k } René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 12

31 average case analysis - uniform distribution Let l = S k. W.l.o.g. S k = {n l + 1,...,n}. We obtain the (n l)-dimensional polytopes: A = B = { (p 1... p n l ) [0,1] n l j < k : p i U j 1 i S j \S k 16n 2 { } (p 1... p n l ) [0,1] n l j < k : p i U j i S j \S k } Pr [ ] [ ] j < k : U j 1 4n Pr i K : pi 1 4n 1 4. Fix U j 1 4n. René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 12

32 average case analysis - uniform distribution Let l = S k. W.l.o.g. S k = {n l + 1,...,n}. We obtain the (n l)-dimensional polytopes: A = B = { (p 1... p n l ) [0,1] n l j < k : p i U j 1 i S j \S k 16n 2 { } (p 1... p n l ) [0,1] n l j < k : p i U j i S j \S k } Pr [ ] [ ] j < k : U j 1 4n Pr i K : pi 1 4n 1 4. Fix U j 1 4n. We will argue that Pr[A B] = vol(a) vol(b) 3 4. René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 12

33 average case analysis - uniform distribution Let l = S k. W.l.o.g. S k = {n l + 1,...,n}. We obtain the (n l)-dimensional polytopes: A = B = { (p 1... p n l ) [0,1] n l j < k : p i U j 1 i S j \S k 16n 2 { } (p 1... p n l ) [0,1] n l j < k : p i U j i S j \S k } Pr [ ] [ ] j < k : U j 1 4n Pr i K : pi 1 4n 1 4. Fix U j 1 4n. We will argue that Pr[A B] = vol(a) vol(b) 3 4. This implies Pr [ K 1 16n 2 K > 0 ] 1 2. René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 12

34 the polytopes y 1 1 x René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

35 the polytopes y 1 x + y A A 1 4n 1 x René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

36 the polytopes y 1 x + y A δ δ δ = 1 16n 2 A 1 4n x + y A δ 1 x René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

37 the polytopes 1 A B 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

38 the polytopes B ε := B shrinked by factor 1 ε in each dimension. 1 1 ε A B B ε 1 ε 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

39 the polytopes B ε := B shrinked by factor 1 ε in each dimension. 1 1 ε A B B ε A for ε = 1 4n. B ε 1 ε 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

40 the polytopes B ε := B shrinked by factor 1 ε in each dimension. 1 1 ε A B B ε A for ε = 1 4n. B ε 1 ε 1 vol(a) vol(b ε ) = (1 ε) n k vol(b) (1 εn) vol(b) 3 4 vol(b) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 13

41 Complexity measures Worst case: Adversary has full control (e.g. p i := w i ) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 14

42 Complexity measures Worst case: Adversary has full control (e.g. p i := w i ) Average case: Adversary has no control (e.g. p i uniformly random) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 14

43 Complexity measures Worst case: Adversary has full control (e.g. p i := w i ) Average case: Adversary has no control (e.g. p i uniformly random) Smoothed complexity: Adversary has limited control 1. adversary fixes some instance X with norm at most 1 (w i [0,1]) 2. Instance X is randomly perturbed with parameter δ. (p i := p i + r[0,δ]) 3. Smoothed complexity is expected running time w.r.t. random perturbation. (function of n and δ) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 14

44 New Result General Distributions Suppose weights are chosen by an adversary. Suppose profits are chosen independently according to arbitrary, possibly different probability distributions with finite mean. Let µ denote the maximum expected profit. Let φ denote the supremum of the profit density functions. Then the Nemhauser/Ullmann algorithm has expected running time O(µφn 5 ). φ density function René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 15 x

45 The Core Concept René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 16

46 optimal fractional solution: Dantzig s relaxation (1957) profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 17

47 optimal fractional solution: Dantzig s relaxation (1957) Linear time (Balas and Zemel, 1980) profit weight René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 17

48 core algorithms (Balas and Zemel, 1980) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 18

49 core algorithms integrality gap = integrality gap René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 19

50 core algorithms loss of an element loss René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 20

51 Lueker s result (1982) Ε[ ] = Θ log 2 n ( n ) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 21

52 implications for core algorithms René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 22

53 implications for core algorithms Apply the Nemhauser/Ullmann algorithm to core elements Profits follow a uniform distribution with density 1/ k number of core elements expected running time of Nemhauser/Ullmann is O(k 5 / ) ( log 2 ) n = Θ ; k 2 n = Θ(log 2 n) n René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 23

54 implications for core algorithms Apply the Nemhauser/Ullmann algorithm to core elements Profits follow a uniform distribution with density 1/ k number of core elements expected running time of Nemhauser/Ullmann is O(k 5 / ) ( log 2 ) n = Θ ; k 2 n = Θ(log 2 n) n O(npolylog(n)) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 23

55 some problematic details... not all core items have sufficient randomness René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 24

56 some problematic details... not all core items have sufficient randomness 1 F G G 1 1/N profit A B F 1/N 0 G weight 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 24

57 some problematic details... not all core items have sufficient randomness number of core items René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 24

58 some problematic details... not all core items have sufficient randomness number of core items integrality gap is not known Choose large core region s.t. P [failure] n 5 in case of failure: compute second list of dominating sets for remaining items Combine two lists in linear time (Horowitz and Sahni 74) René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 24

59 δ-correlated instances 1 profit Q δ/2 0 weight 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 25

60 δ-correlated instances 1 profit δ/2 Q E[ ] = O( δ n log2 n δ ) running time: O( n δ polylog n δ ) 0 weight 1 René Beier Max-Planck-Institut, Germany The Knapsack Problem p. 25