The Monte Carlo Method

Transcription

1 The Monte Carlo Method Example: estimate the value of π. Choose X and Y independently and uniformly at random in [0, 1]. Let Pr(Z = 1) = π 4. 4E[Z] = π. { 1 if X Z = 2 + Y 2 1, 0 otherwise,

2 Let Z 1,..., Z m be the values of m independent experiments. W = m i=1 Z i. [ m ] m E[W ] = E Z i = E[Z i ] = mπ 4, i=1 i=1 W = 4 mw is a natural estimate for π. Pr( W π ɛπ) = ( Pr W mπ 4 ɛmπ ) 4 = Pr ( W E[W ] ɛe[w ]) 2e 1 12 mπɛ2.(chernoff bound, Cor. 4.6)

3 (ɛ, δ)-approximation Definition A randomized algorithm gives an (ɛ, δ)-approximation for the value V if the output X of the algorithm satisfies Pr( X V ɛv ) 1 δ. The method for approximating π gives an (ɛ, δ)-approximation as long as ɛ < 1 and m is large enough to make 2e mπɛ2 /12 δ so we need m 12 ln (2/δ) πɛ 2

4 Theorem Let X 1,..., X m be independent and identically distributed indicator random variables, with µ = E[X i ]. If m 3 ln 2 δ ɛ 2 µ, then ( 1 Pr m ) m X i µ ɛµ δ. i=1 That is, m samples provide an (ɛ, δ)-approximation for µ.

5 Approximate Counting Example counting problems: 1 How many spanning trees in a graph? 2 How many perfect matchings in a graph?

6 DNF Counting DNF = Disjunctive Normal Form. Problem: How many satisfying assignments to a DNF formula? A DNF formula is a disjunction of clauses. Each clause is a conjunction of literals. (x 1 x 2 ) (x 2 x 3 ) (x 1 x 2 x 3 x 4 ) (x 3 x 4 ) Compare to CNF. (x 1 x 2 ) (x 1 x 3 ) m clauses, n variables Let s first convince ourselves that obvious approaches don t work!

7 DNF counting is hard Question: Why? We can reduce CNF satisfiability to DNF counting. The negation of a CNF formula is in DNF. 1 CNF formula f 2 get the DNF formula ( f ) 3 count satisfying assignments to f 4 This number is 2 n if and only if f is unsatisfiable.

8 DNF counting is #P complete #P is the counting analog of NP. Any problem in #P can be reduced (in polynomial time) to the DNF counting problem. Example #P complete problems: 1 How many Hamilton circuits does a graph have? 2 How many satisfying assignments does a CNF formula have? 3 How many perfect matchings in a graph? What can we do about a hard problem?

9 (ɛ, δ) FPRAS for DNF counting FPRAS = Fully Polynomial Randomized Approximation Scheme Notation: U: set of all possible assignments to variables U = 2 n. H U: set of satisfying assignments Want to estimate Y = H Give ɛ > 0, δ > 0, find estimate X such that 1 Pr[ X Y > ɛy ] < δ 2 Algorithm should be polynomial in 1/ɛ, 1/δ, n and m.

10 Monte Carlo method Here s the obvious scheme (Algorithm 1, page 256 in book). 1. Repeat N times: 1.1. Sample x randomly from U, that is, generate one of the 2 n possible assignments uniformly at random Count a success if x H (formula satisfied by x) 2. Return fraction of successes U. Question: How large should N be? We have to evaluate the probability of our estimate being good.

11 Let ρ = H U. Let the indicator random variable Z i = 1 if i-th trial was successful. Then { 1 with probability ρ Z i = 0 with probability 1 ρ Z = N Z i is a binomial random variable whose expected value is i=1 E[Z] = Nρ X = Z U is our estimate of H N

12 Probability that our algorithm succeeds Recall: X denotes our estimate of H. Pr[(1 ɛ) H < X < (1 + ɛ) H ] = Pr[(1 ɛ) H < Z U /N < (1 + ɛ) H ] = Pr[(1 ɛ)nρ < Z < (1 + ɛ)nρ] > 1 e Nρɛ2 /3 e Nρɛ2 /2 > 1 2e Nρɛ2 /3 where we have used Chernoff bounds. For an (ɛ, δ) approximation, this has to be greater than 1 δ, 2e Nρɛ2 /3 < δ N > 3 ρɛ 2 log 2 δ

13 Theorem Let ρ = H / U. Then the Monte Carlo method is an (ɛ, δ) approximation scheme for estimating H provided that N > 3 ρɛ 2 log 2 δ.

14 What s wrong? How large could 1 ρ be? ρ is the fraction of satisfying assignments. 1 The number of possible assignments is 2 n. 2 Maybe there are only a polynomial (in n) number of satisfying assignments. 3 So, 1 ρ could be exponential in n. Question: An example where formula has only a few assignments?

15 The trick: Skewed sampling Increase the hit rate (ρ)! Sample from a different universe, ρ is higher, and all elements of H still represented. What s the new universe? Notation: H i set of assignments that satisfy clause i. H = H 1 H 2... H m Define a new universe U = H 1 H2... Hm means multiset union.

16 Example - Partition by clauses (x 1 x 2 ) (x 2 x 3 ) (x 1 x 2 x 3 x 4 ) (x 3 x 4 ) x 1 x 2 x 3 x 4 Clause

17 More about the universe U 1 U contains only the satisfying assignments. 2 U is a multiset (contains the same element many times). 3 Element of U is (v, i) where v is an assignment, i is the satisfied clause. U = {(v, i) v H i } 4 Each satisfying assignment v appears in as many clauses as it satisfies.

18 One way of looking at U Partition by clauses. m partitions, partition i contains H i.

19 Another way of looking at U Partition by assignments (one region for each assignment v). Each partition corresponds to an assignment. Can we count the different (distinct) assignments?

20 Example - Partition by assignments (x 1 x 2 ) (x 2 x 3 ) (x 1 x 2 x 3 x 4 ) (x 3 x 4 ) x 1 x 2 x 3 x 4 Clause

21 Canonical element Crucial idea: For each assignment group, find a canonical element in U. An element (v, i) is canonical if f ((v, i)) = 1 { 1 if i = min{j : v H j } f ((v, i)) = 0 otherwise For every assignment group, exactly one canonical element. So, count the number of canonical elements! Note: could use any other definition as long as exactly one canonical element per assignment

22 Count canonical elements Reiterating: 1 Number of satisfying assignments = Number of canonical elements. 2 Count number of canonical elements. 3 Back to old random sampling method for counting!

23 What is ρ? Lemma ρ 1, (pretty large). m Proof. H = m i=1 H i, since H is a normal union. So H i H Recall U = H 1 H2... Hm U = m i=1 H i, since U is a multiset union. U m H ρ = H U 1 m

24 How to generate a random element in U? Look at the partition of U by clauses. Algorithm Select: 1 Pick a random clause weighted according to the area it occupies. Pr[i] = H i U = H i m 1 H j H i = 2 (n k i ) where k i is the number of literals in clause i. 2 Choose a random satisfying assignment in H i. Fix the variables required by clause i. Assign random values to the rest to get v (v, i) is the random element. Running time: O(n).

25 How to test if canonical assignment? Or how to evaluate f ((v, i))? Algorithm Test: 1 Test every clause to see if v satisfies it. cov(v) = {(v, j) v H j } 2 If (v, i) the smallest in cov(v), then f (v, i) = 1, else 0. Running time: O(nm).

26 Algorithm Coverage: Back to random sampling 1 s 0 (number of successes) 2 Repeat N times: Select (v, i) using Select. if f (v, i) = 1 (check using Test) then success, increment s. 3 Return s U /N. Number of samples needed is (from Theorem 3): N = 3 ɛ 2 ρ ln 2 δ 3m ɛ 2 ln 2 δ Sampling, testing: polynomial in n and m We have an FPRAS Theorem The Coverage algorithm yields an (ɛ, δ) approximation to H provided that the number of samples N 3m ɛ 2 log 2 δ.

27 Counting Independent Sets Input: a graph G = (V, E). V = n, E = m. Let e 1,..., e m be an arbitrary ordering of the edges. G i = (V, E i ), where E i = {e 1,..., e i } G = G m, G 0 = (V, ) and G i 1 is obtained from G i be removing a single edge. Ω(G i ) = the set of independent sets in G i. Ω(G) = Ω(G m) Ω(G m 1 ) Ω(G m 1) Ω(G m 2 ) Ω(G m 2) Ω(G m 3 ) Ω(G 1) Ω(G 0 ) Ω(G 0). r i = Ω(G i), i = 1,..., m. Ω(G i 1 )

28 Algorithm Estimating r i Input: Graphs G i 1 = (V, E i 1 ) and G i = (V, E i ). Output: r i = an approximation of r i. 1 X 0. 2 Repeat for M = 1296m 2 ɛ 2 ln 2m δ independent trials: 1 Generate an uniform sample from Ω(G i 1 ); 2 If the sample is an independent set in G i, let X X Return r i X M.

29 Lemma r i 1/2. Proof. Ω(G i ) Ω(G i 1 ). Suppose that G i 1 and G i differ in the edge {u, v}. An independent set in Ω(G i 1 ) \ Ω(G i ) contains both u and v. To bound the size of the set Ω(G i 1 ) \ Ω(G i ), we associate each I Ω(G i 1 ) \ Ω(G i ) with an independent set I \ {v} Ω(G i ). An independent set I Ω(G i ) is associated with no more than one independent set I {v} Ω(G i 1 ) \ Ω(G i ), and thus Ω(G i 1 ) \ Ω(G i ) Ω(G i ). It follows that r i = Ω(G i) Ω(G i 1 ) = Ω(G i ) Ω(G i ) + Ω(G i 1 ) \ Ω(G i ) 1/2.

30 Lemma When m 1 and 0 < ɛ 1, the procedure for estimating r i yields an estimate r i that is (ɛ/2m, δ/m)-approximation for r i. Our estimate is 2 n m i=1 r i The true number is Ω(G) = 2 n m i=1 r i. To evaluate the error in our estimate we need to bound the ratio m r i R =. r i i=1

31 Lemma Suppose that for all i, 1 i m, r i is an (ɛ/2m, δ/m)-approximation for r i. Then Pr( R 1 ɛ) 1 δ. Proof: For each 1 i m, we have ( Pr r i r i ɛ ) 2m r i 1 δ m. Equivalently, ( Pr r i r i > ɛ ) 2m r i < δ m.

32 By the union bound the probability that r i r i > ɛ 2m r i for any i is at most δ, and hence r i r i ɛ 2m r i for all i with probability at least 1 δ. Equivalently, 1 ɛ 2m r i 1 + ɛ r i 2m holds for all i with probability at least 1 δ. When these bounds hold for all i, we can combine them to obtain 1 ɛ ( 1 ɛ ) m m r ( i 1 + ɛ ) m (1 + ɛ), 2m r i 2m i=1

33 Estimating r i Input: Graphs G i 1 = (V, E i 1 ) and G i = (V, E i ). Output: r i = an approximation of r i. 1 X 0. 2 Repeat for M = 1296m 2 ɛ 2 ln 2m δ independent trials: 1 Generate an uniform sample from Ω(G i 1 ); 2 If the sample is an independent set in G i, let X X Return r i X M.

34 Definition Let w be the (random) output of a sampling algorithm for a finite sample space Ω. The sampling algorithm generates an ɛ-uniform sample of Ω if, for any subset S of Ω, S Pr(w S) Ω ɛ. A sampling algorithm is a fully polynomial almost uniform sampler (FPAUS) for a problem if, given an input x and a parameter ɛ > 0, it generates an ɛ-uniform sample of Ω(x), and it runs in time polynomial in ln ɛ 1 and the size of the input x.

35 Estimating r i Input: Graphs G i 1 = (V, E i 1 ) and G i = (V, E i ). Output: r i = an approximation of r i. 1 X 0. 2 Repeat for M = 1296m 2 ɛ 2 ln 2m δ independent trials: 1 Generate an ɛ 6m -uniform sample from Ω(G i 1); 2 If the sample is an independent set in G i, let X X Return r i X M. Lemma When m 1 and 0 < ɛ 1, the procedure for estimating r i yields an (ɛ/2m, δ/m)-approximation for r i

36 How do we Generate an ɛ 6m -uniform sample from Ω(G i 1)?

37 From Approximate Sampling to Approximate Counting Theorem Given a fully polynomial almost uniform sampler (FPAUS) for independent sets in any graph, we can construct a fully polynomial randomized approximation scheme (FPRAS) for the number of independent sets in a graph G with maximum degree at most.

38 The Markov Chain Monte Carlo Method Idea: define an ergodic Markov chain whose stationary distribution is the desired probability distribution. Let X 0, X 1, X 2,..., X n be the run of the chain. The Markov chain converges to its stationary distribution from any starting state X 0 so after some sufficiently large number r of steps, the distribution at of the state X r will be close to the stationary distribution π of the Markov chain. Now, repeating with X r as the starting point we can use X 2r as a sample etc. So X r, X 2r, x 3r,... can be used as almost independent samples from π.

39 N(x) set of neighbors of x. Let M max x Ω N(x). Lemma Consider a Markov chain where for all x and y with y x, P x,y = 1 M if y N(x), and P x,y = 0 otherwise. Also, P x,x = 1 N(x) M. If this chain is irreducible and aperiodic, then the stationary distribution is the uniform distribution. Proof. We show that the chain is time-reversible, and apply Theorem For any x y, if π x = π y, then π x P x,y = π y P y,x, since P x,y = P y,x = 1/M. It follows that the uniform distribution π x = 1/ Ω is the stationary distribution.

40 Sampling a uniform distribution on the independent sets Consider a Markov chain whose states are independent sets in a graph G = (V, E): 1 X 0 is an arbitrary independent set in G. 2 To compute X i+1 : 1 Choose a vertex v uniformly at random from V. 2 If v X i then X i+1 = X i \ {v}; 3 if v X i, and adding v to X i still gives an independent set, then X i+1 = X i {v}; 4 otherwise, X i+1 = X i. The chain is irreducible The chain is aperiodic (as G has at least one edge) For y x, P x,y = 1/ V or 0. The lemma implies that the stationary distribution is the uniform distribution.

41 The Metropolis Algorithm Assuming that we want to sample with non-uniform distribution. For example, we want the probability of an independent set of size i to be proportional to λ i. Consider a Markov chain on independent sets in G = (V, E): 1 X 0 is an arbitrary independent set in G. 2 To compute X i+1 : 1 Choose a vertex v uniformly at random from V. 2 If v X i then set X i+1 = X i \ {v} with probability min(1, 1/λ); 3 if v X i, and adding v to X i still gives an independent set, then set X i+1 = X i {v} with probability min(1, λ); 4 otherwise, set X i+1 = X i.

42 Lemma For a finite state space Ω, let M max x Ω N(x). For all x Ω, let π x > 0 be the desired probability of state x in the stationary distribution. Consider a Markov chain where for all x and y with y x, P x,y = 1 ( M min 1, π ) y π x if y N(x), and P x,y = 0 otherwise. Further, P x,x = 1 y x P x,y. Then if this chain is irreducible and aperiodic, the stationary distribution is given by the probabilities π x.

43 Proof. We show the chain is time-reversible. For any x y, if π x π y, then P x,y = 1 and P y,x = π x /π y. It follows that π x P x,y = π y P y,x. Similarly, if π x > π y, then P x,y = π y /π x and P y,x = 1, and it follows that π x P x,y = π y P y,x. Note that the Metropolis Algorithm only needs the ratios π x /π y s. In our construction, the probability of an independent set of size i is λ i /B for B = x λsize(x) although we may not know B.