Global Optima from Local Algorithms

Transcription

1 Global Optima from Local Algorithms Dimitris Achlioptas 1 Fotis Iliopoulos 2 1 UC Santa Cruz 2 UC Berkeley Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

2 The Lovász Local Lemma The LLL Setting Probability space + Set of m bad events B = {E 1, E 2,..., E m }. If {E i } are independent, Pr[Nothing bad happens] = m i=1 (1 p i). But what if avoiding some bad events boosts some other bad events? Example: Ω = {0, 1} 3 with uniform measure, F = (x 1 x 2 ) (x 2 x 3 ). Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

3 The Lovász Local Lemma The LLL Setting Probability space + Set of m bad events B = {E 1, E 2,..., E m }. If {E i } are independent, Pr[Nothing bad happens] = m i=1 (1 p i). But what if avoiding some bad events boosts some other bad events? Example: Ω = {0, 1} 3 with uniform measure, F = (x 1 x 2 ) (x 2 x 3 ). General LLL (Erdős, Lovász 75) If each E i is mutually independent of all events in B \ (Γ(i) E i ) and there exist {x i } [0, 1) such that x i Pr[E i ] j Γ(i) (1 x j) then Pr[Nothing bad happens] m i=1 (1 x i). Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

4 The Lovász Local Lemma The LLL Setting Probability space + Set of m bad events B = {E 1, E 2,..., E m }. If {E i } are independent, Pr[Nothing bad happens] = m i=1 (1 p i). But what if avoiding some bad events boosts some other bad events? Example: Ω = {0, 1} 3 with uniform measure, F = (x 1 x 2 ) (x 2 x 3 ). Asymmetric LLL (Very Handy Corrolary) If each E i is mutually independent of all events in B \ (Γ(i) E i ) and Pr[E j ] 1 4 j Γ(i) then Pr[Nothing bad happens] > 0. Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

5 The Lovász Local Lemma The LLL Setting Probability space + Set of m bad events B = {E 1, E 2,..., E m }. If {E i } are independent, Pr[Nothing bad happens] = m i=1 (1 p i). But what if avoiding some bad events boosts some other bad events? Example: Ω = {0, 1} 3 with uniform measure, F = (x 1 x 2 ) (x 2 x 3 ). Lopsided LLL (Erdős, Spencer 91) If each E i is never boosted by any conditioning on events in B \ (Γ(i) E i ) and there exist {x i } [0, 1) such that x i Pr[E i ] j Γ(i) (1 x j) then Pr[Nothing bad happens] m i=1 (1 x i). Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

6 The Lovász Local Lemma A Tight Example and a Breakthrough Example Every k-cnf formula where each clause shares variables with at most 2 k /e other clauses is satisfiable. Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

7 The Lovász Local Lemma A Tight Example and a Breakthrough Example Every k-cnf formula where each clause shares variables with at most 2 k /e other clauses is satisfiable. Theorem (Gebauer, Szabo, Tardos 11) There exist unsatisfiable formulas with = (1 + δ k )2 k /e, where δ k 0. Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

8 The Lovász Local Lemma A Tight Example and a Breakthrough Example Every k-cnf formula where each clause shares variables with at most 2 k /e other clauses is satisfiable. Theorem (Gebauer, Szabo, Tardos 11) There exist unsatisfiable formulas with = (1 + δ k )2 k /e, where δ k 0. Algorithmic LLL: a s.t.a can be found efficiently if 2 k/4 [Beck 91], [Alon 91], [Molloy, Reed 98], [Czumaj, Scheideler 00], [Srinivasan 08] Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

9 The Lovász Local Lemma A Tight Example and a Breakthrough Example Every k-cnf formula where each clause shares variables with at most 2 k /e other clauses is satisfiable. Theorem (Gebauer, Szabo, Tardos 11) There exist unsatisfiable formulas with = (1 + δ k )2 k /e, where δ k 0. Algorithmic LLL: a s.t.a can be found efficiently if 2 k/4 [Beck 91], [Alon 91], [Molloy, Reed 98], [Czumaj, Scheideler 00], [Srinivasan 08] Theorem (Moser 09) If (F ) < 2 k 5 a sat assignment can be found in O( V + C log C ). Moser s ideas, with more care, yield 2 k /e. [Messner, Thierauf 11] Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

10 Algorithmic Aspects The Moser-Tardos Variable Setting Assume that µ is a product probability space over n variables, i.e., µ(ω) = µ 1 (ω 1 ) µ n (ω n ) for all ω Ω. Theorem (Moser Tardos 09) If E 1, E 2,..., E m satisfy the conditions of the original LLL, an element ω Ω such that none of the {E i } holds can be found in time O(m). Resample 1: Sample ω Ω according to µ Easy since µ is product 2: while some bad event occurs do 3: Select any occurring bad event E i 4: Resample the variables of E i according to µ 5: return ω Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

11 Algorithmic Aspects The Moser-Tardos Variable Setting Assume that µ is a product probability space over n variables, i.e., µ(ω) = µ 1 (ω 1 ) µ n (ω n ) for all ω Ω. Theorem (Moser Tardos 09) If E 1, E 2,..., E m satisfy the conditions of the original LLL, an element ω Ω such that none of the {E i } holds can be found in time O(m). Covers majority of existing LLL applications It can be parallelized [Moser, Tardos 09] It can be derandomized [Chandrasekaran, Goyal, Haeupler 09] May work even if m is exponential [Haeupler, Saha, Srinivasan 10] Shearer s condition suffices (weaker than LLL) [Kolipaka, Szegedy 11] Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

12 Algorithmic Aspects Limitations of the Moser Tardos Proof The proof depends heavily on µ being a product measure Does not extend to Lopsided LLL Hard to extend even to simple non-product spaces e.g., uniform measure on permutations [Harris, Srinivasan 13] Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

13 Algorithmic Aspects Limitations of the Moser Tardos Proof The proof depends heavily on µ being a product measure Does not extend to Lopsided LLL Hard to extend even to simple non-product spaces e.g., uniform measure on permutations [Harris, Srinivasan 13] Product measures require a variable decomposition. Product measures are algorithmically limiting: Resamplings are state-independent Example LLL requires e colors on graphs with max degree instead of just + 1 colors Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

14 Algorithmic Aspects What to do? Get Rid of the Measure Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

15 No Probability Measure Required No Measure Let Ω be an arbitrary finite set, such as: {0, 1} n P n Ham(G) := Hamiltonian cycles of graph G. Col q (G) := Valid edge-q-colorings of graph G. Permutations of [n] Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

16 No Probability Measure Required No Measure Let Ω be an arbitrary finite set, such as: {0, 1} n P n Ham(G) := Hamiltonian cycles of graph G. Col q (G) := Valid edge-q-colorings of graph G. Permutations of [n] Let F = {f 1, f 2,..., f m } be arbitrary subsets of Ω called flaws. Examples (Flaw f i is the subset of...) {0, 1} n that violates clause c i. SAT P n in which π(i) = i. Col q (G) in which cycle C i is bichromatic. Derangements Acyclic Edge Coloring Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

17 No Probability Measure Required The LLL as a Random Walk How to Find Flawless Objects Specify a directed graph D on Ω such that: Every flawed object has outdegree at least 1. Every flawless object has outdegree 0. Start at an arbitrary σ 1 Ω Take a random walk on D until you reach a sink. Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

18 No Probability Measure Required The LLL as a Random Walk How to Find Flawless Objects Specify a directed graph D on Ω such that: Every flawed object has outdegree at least 1. Every flawless object has outdegree 0. Start at an arbitrary σ 1 Ω Take a random walk on D until you reach a sink. Definition (writing σ Ω instead of ω Ω to emphasize state view) For σ Ω, let U(σ) = {f i : σ f i }. σ is flawless iff U(σ) =. σ Ω, f σ define a non-empty set A(f, σ) Ω of actions. Each τ A(f, σ) becomes an arc σ f τ in D. D is a multidigraph Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

19 No Probability Measure Required The Classics k-cnf formula F = c 1 c m with n variables. Ω = {0, 1} n. f i = {σ Ω : σ violates clause c i }. A(f i, σ) = {The 2 k mutations of σ through var(c i )}. (F ) < 2 k /e Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

20 No Probability Measure Required The Classics k-cnf formula F = c 1 c m with n variables. Ω = {0, 1} n. f i = {σ Ω : σ violates clause c i }. A(f i, σ) = {The 2 k mutations of σ through var(c i )}. q-coloring graph G(V, E) with n vertices. Ω = [q] n. f u,v = {σ Ω : col(u) = col(v)} A(f u,v, σ) = {All q mutations of σ through v} (F ) < 2 k /e q > e Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

21 No Probability Measure Required The Classics k-cnf formula F = c 1 c m with n variables. Ω = {0, 1} n. f i = {σ Ω : σ violates clause c i }. A(f i, σ) = {The 2 k mutations of σ through var(c i )}. q-coloring graph G(V, E) with n vertices. Ω = [q] n. f u,v = {σ Ω : col(u) = col(v)} A(f u,v, σ) = {All q mutations of σ through v} (F ) < 2 k /e q > e A(f u,v, σ) = {Only conflict-free mutations} q + 1 Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

22 No Probability Measure Required The Key Quantities Measuring Flaws Persistence Let D i be the subgraph of arcs labeled f i. The persistence of f i is Potential Causality A i = max σ f i In(σ) Out(σ) Write i j if there is any σ f i τ such that f j U(τ) \ (U(σ) \ f i ) i i if f i is still present in τ and i j if f j is a new flaw in τ Transience The digraph D is transient if for every i [m], A j < 1 e. j i Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

23 No Probability Measure Required The Key Quantities Main Result Let σ 1 Ω be arbitrary. For t = 1, 2,... Let f i be a random flaw present in σ t. Address f i by taking a uniformly random action in A(f i, σ). Theorem Let T 0 = log Ω + U(σ 1 ). If D is transient the probability that the walk does not each a sink within t = O(T 0 + s) steps is less than 2 s. Features No need for a uniform sample to start off. Running time depends on U(σ 1 ), not F. Ω-sampling = T 0 = O( F ). Flaw-choice: we can do better than random Left-handed version Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

24 No Probability Measure Required The Key Quantities General LLL... If there exist positive real numbers {x i } such that for all i [m], Pr(A i ) (1 + x j ) x i... j {i} Γ(i) Theorem (Main result)... If there exist positive real numbers {x f } such that for every f F, A f In fact, it suffices to have (1 + x g ) < x f... g Γ(f) x g < x f... A f S Ind(Γ(f)) g S Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

25 No Probability Measure Required The Key Quantities Rainbow Perfect Matchings in a Complete Graph Input: an edge-colored K 2n. Ouput: a rainbow perfect matching - Let Ω be the set of all perfect matchings of K 2n. - For each pair of vertex-disjoint edges{e i, e j } with the same color let f i,j = {M Ω : {e i, e j } M}. Result: If no color is used more than n/(2e) times... O(n 2 log n) time. Algorithm Design via Atomicity Algorithm: swap both same-color edges out. Atomicity: Consider M f i,j M. Add to M edges {e i, e j } and then close the two cycles. Remark: Swapping out only one of the two edges does not work. Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

26 No Probability Measure Required The Key Quantities Proof Entropic Method Atomicity if each state transition emits the flaw addressed, the trajectory up to moment t can be reconstructed from: I(t) = f 1, f 2,..., f t, σ t Amenability Every t-step trajectory must consume at least L(t) bits Transcience H(I t ) < L(t) for t > T 0 = likely trajectories of length < t Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

27 No Probability Measure Required The Key Quantities Summary + Future Work No probability function No variables State-dependent actions Just Ω and subsets Arbitrary Ω and subsets Non-trivial Algorithms Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15

28 No Probability Measure Required The Key Quantities Summary + Future Work No probability function No variables State-dependent actions Just Ω and subsets Arbitrary Ω and subsets Non-trivial Algorithms Future Work Quantum version Backtracking algorithms Sampling Thanks! Dimitris Achlioptas (UC Santa Cruz) Global Optima from Local Algorithms IMA Graphical Models / 15