Approximate Counting and Markov Chain Monte Carlo

Size: px

Start display at page:

Download "Approximate Counting and Markov Chain Monte Carlo"

Tabitha Doyle
5 years ago
Views:

1 Approximate Counting and Markov Chain Monte Carlo A Randomized Approach Arindam Pal Department of Computer Science and Engineering Indian Institute of Technology Delhi March 18, 2011 April 8, 2011 Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

2 Agenda Introduction to Monte Carlo Method Approximating the Value of π The Complexity Classes #P and #P -complete Fully Polynomial Randomized Approximation Scheme DNF Counting Problem Introduction to Markov Chains Random Walks on Graphs Markov Chain Monte Carlo Method Counting the Number of Knapsack Solutions Rapidly Mixing Markov Chains Mixing Time, Canonical Paths and Conductance Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

3 The Monte Carlo Method Uses random sampling to estimate the value of a quantity. Primarily used in numerical analysis and simulation. Widely used in Physical sciences, Engineering, Computational Biology and Statistics among many others. A classic example is to estimate the value of π. Other examples include evaluating a definite integral (area under a curve), solving optimization problems using random walks. Simulated annealing and the Metropolis algorithm are good heuristics for approximating the global optimum of a given function in a large search space. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

4 Algorithms for Approximate Counting Problems DNF counting problem. (Karp and Luby) Network reliability. (Karger) Counting the number of Knapsack solutions. (Dyer, Frieze, Kannan, Kapoor, Perkovic and Vazirani) Approximating the Permanent. (Jerrum and Sinclair) Estimating the volume of a convex body. (Kannan, Lovasz and Simonovits) Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

5 Approximating the Value of π Let S = {(x, y) R 2 : x 1, y 1} be the 2 2 square centered at (0, 0). Let C = {(x, y) R 2 : x 2 + y 2 1} be the unit circle centered at (0, 0). Choose a point (x, y) S uniformly at random. This is equivalent to choosing x and y independently from a uniform distribution on the interval [ 1, 1]. Define a random variable, { 1 if (x, y) C in the i th iteration, Z i = 0 otherwise. Clearly, E[Z i ] = Pr[Z i = 1] = (C) (S) = π 4. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

6 Analysis Suppose we do this experiment m times independently. Let Z = m i=1 Z i. E[Z] = m i=1 E[Z i] = mπ 4. Let X = ( ( 4 m) Z. Then, E[X] = 4 ) m E[Z] = 4 m mπ 4 = π. Applying Chernoff bound, [ Z mπ Pr[ X π επ] = Pr εmπ ] 4 4 = Pr[ Z E[Z] εe[z]] 2e mπε2 /12. Since the error probability decreases exponentially with the number of trials m, we can get a very good approximation to π with high probability. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

7 The complexity classes #P and #P -complete Definition A problem Π #P if there is a nondeterministic polynomial time Turing machine that for any instance I of Π has a number of accepting computations exactly equal to the number of distinct solutions to I. Π is #P -complete if any problem Γ #P can be reduced to Π by a polynomial time Turing reduction relating the cardinalities of solution sets. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

8 Importance of #P -completeness The class #P consists of all counting problems associated with the decision problems in NP. There are easy problems in #P, like counting the number of spanning trees of a graph, which can be solved in polynomial time using Kirchhoff s Matrix Tree Theorem. If a #P -complete problem can be solved in polynomial time, then P = NP. There is no known efficient deterministic approximation algorithm for any #P -complete problem. But there are efficient randomized approximation algorithms for many #P -complete problems. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

9 Kirchhoff s Matrix Tree Theorem Theorem The number of spanning trees τ(g) of a graph G on n vertices is the absolute value of the determinant of any (n 1) (n 1) submatrix (cofactor) of the Laplacian matrix L(G). Equivalently, if λ 1,..., λ n 1 are the non-zero eigenvalues of L(G), then τ(g) = 1 n n 1 i=1 λ i. L(G) = (G) A(G), where (G) and A(G) are the degree matrix and the adjacency matrix of G respectively. Using this, one can prove Cayley s formula: τ(k n ) = n n 2. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

10 Fully Polynomial Randomized Approximation Scheme (FPRAS) Definition An (ε, δ)-fully Polynomial Randomized Approximation Scheme (FPRAS) for a problem is a randomized algorithm A which, given an input x and two parameters ε, δ (0 < ε, δ < 1), outputs a value A(x) corresponding to the actual value V (x) such that Pr[ A(x) V (x) εv (x)] 1 δ, in time polynomial in x, 1 ε and log( 1 δ ). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

11 DNF Counting Problem F is a boolean formula in disjunctive normal form (DNF) over n boolean variables x 1,..., x n. F = C 1... C t is a disjunction of clauses C i. Each clause C i = l 1 l ri of r i literals. Each literal l j is either a variable x k or it s complement x k. #F is the number of distinct satisfying assignments of F. Our goal is to compute #F. Note that 0 #F 2 n. Computing #F is #P -complete. Design an (ε, δ)-fpras with running time polynomial in n, t, 1 ε and log( 1 δ ). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

12 A Naive Random Sampling Algorithm 1 X 0. 2 for k = 1,..., m 3 for i = 1,..., n 4 set x i 1 with probability end for 6 if this random assignment satisfies F 7 X X end for 9 return Y ( X m) 2 n. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

13 Analysis of the Naive Algorithm Define a random variable, { 1 if F is satisfied in the k th iteration, X k = 0 otherwise. X k -s are independent 0 1 random variables. X = m k=1 X k. E[X k ] = Pr[X k = 1] = #F 2. n ( ) 2 n E[Y ] = E[X] m ( ) 2 n m = E[X k ] m ( 2 n = m = #F. k=1 ) ( m 2 n ) #F Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

14 Problem with the Naive Algorithm X k is a Bernoulli distribution with parameter p = #F 2 n. X is a binomial distribution with parameters m and p. By Chernoff bound, Pr[(1 ε)#f Y (1 + ε)#f ] ( ) X = Pr[(1 ε)p2 n 2 n (1 + ε)p2 n ] m = Pr[(1 ε)mp X (1 + ε)mp] 1 2e mpε2 /4. For this to be at least 1 δ, we must have, m 4 pε 2 ln 2 δ = 4 2n #F ε 2 ln 2 δ. If #F is sub-exponential (e.g. a polynomial) in n, this is an exponential-time algorithm and hence is not an FPRAS. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

15 Towards an FPRAS for DNF Counting F = C 1... C t. If C i = r i, there are 2 n r i satisfying assignments for C i. Let SC i be the set of assignments satisfying C i. Let U = {(i, a) : 1 i t and a SC i }. U = t i=1 SC i can be computed efficiently. We want to estimate #F = t i=1 SC i. Note that, #F U, since an assignment can satisfy more than one clause and so can appear in more than one pair in U. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

16 Sampling More Carefully The idea is to define a set S U with S = #F. S = {(i, a) : 1 i t and a SC i, but a / SC j for j < i}. We can estimate S by estimating the ratio S U, since we know U. Sample uniformly at random from U and count how many times they are in S. S is relatively dense in U, since each assignment can satisfy at most t different clauses, S U 1 t. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

17 How to Sample Uniformly at Random from U? First choose i with probability SC i U. Then choose an assignment a SC i uniformly at random. This can be done by setting each literal l / SC i to 1 with probability (independently and uniformly at random). 1 2 Pr[(i, a) is chosen] = Pr[i is chosen] Pr[a is chosen i is chosen] = SC i U = 1 U. 1 SC i This method samples uniformly at random from U. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

18 An FPRAS for DNF Counting 1 X 0. 2 for k = 1,..., m 3 with probability SC i U, choose uniformly at random, an 4 assignment a SC i. 5 if a / SC j for all j < i 6 X X end for 8 return Y ( X m) U. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

19 Analysis of the FPRAS A similar analysis shows that m 4t ε 2 ln 2 δ. Since the running time is polynomial in n, t, 1 ε and log( 1 δ ), this is an (ε, δ)-fpras. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

20 Introduction to Markov Chains Definition A finite Markov chain M is a discrete-time stochastic process defined over a finite set of n states Ω and an n n matrix P of transition probabilities. If X t is the state of M at time t, then the memorylessness property states that, Pr[X t+1 = j X 0 = i 0,..., X t 1 = i t 1, X t = i] = Pr[X t+1 = j X t = i] = P ij. M makes state transitions at discrete time steps t = 1, 2,... P has one row and one column for each state in Ω. The entry P ij is the probability that the next state will be j, given that the current state is i. For all i, j Ω, 0 P ij 1 and n j=1 P ij = 1. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

21 Classification of States For n 0, the n-step transition probability is defined as Pij n = Pr[X t+n = j X t = i]. State j is accessible from state i (i j), if for some n 0, Pij n > 0. If i j and j i, then we say that i and j communicate (i j). This is an equivalence relation. A Markov chain is irreducible if all states belong to one communicating class. This happens if and only if its graph representation is strongly connected. Let rij t be the probability that starting at state i, the first transition to state j occurs at time t. More precisely, r t ij = Pr[X t = j, X s j for 1 s t 1 X 0 = i]. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

22 Classification of States continued... A state i is recurrent (persistent) if f ii = t=1 rt ii = 1. Else, it is transient. A Markov chain is recurrent if every state is recurrent. The expected time to first reach j, after starting at i is h ij = t=1 t rt ij. This is called the hitting time. A recurrent state i is positive recurrent (non-null persistent) if h ii <. Otherwise, it is null recurrent (null persistent). Note that f ii = 1 does not mean that h ii <. A state i is called periodic, if there exists an integer T > 1 such that Pii n = Pr[X t+n = i X t = i] = 0, unless T n. T is called the period of i. Otherwise, i is called aperiodic. A Markov chain is called aperiodic if all its states are aperiodic. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

23 Ergodic Markov Chains and Stationary Distributions An aperiodic and positive recurrent state is called an ergodic state. A Markov chain is called ergodic if all its states are ergodic. A probability distribution π is called a stationary distribution for M with transition matrix P, if πp = π. Fundamental Theorem of Markov Chains Any finite, irreducible and aperiodic Markov chain is ergodic and has a unique stationary distribution π. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

24 Random Walks on Graphs Let G = (V, E) be a connected, non-bipartite, undirected graph, where V = n and E = m. This induces a Markov chain M G. The states of M G are the vertices of G. For any two vertices u, v V, { 1 d(u) if (u, v) E, P uv = 0 otherwise. M G is ergodic with stationary distribution π given by, π v = d(v) 2m. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

25 Counting the Number of Knapsack Solutions Let a = (a 0,..., a n 1 ) N n be an n-dimensional integer vector and let b N be any integer. Given an inequality a x = n 1 i=0 a ix i b, where x {0, 1} n. Compute the number N of such vectors x. Suppose a 0,..., a n 1 are the sizes of n items that can be packed into a knapsack of size b. N is the number of combinations of items that can be fitted into the knapsack. We have to count the number of Knapsack solutions. This problem is #P -complete. Note that n 1 i=0 a i > b. Else, n 1 i=0 a ix i n 1 i=0 a i b, and the number of solutions is 2 n. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

26 A Naive Random Sampling Algorithm 1 X 0. 2 for k = 1,..., m 3 for i = 0,..., n 1 4 set x i 1 with probability end for 6 if this random assignment satisfies a x b 7 X X end for 9 return Y ( X m) 2 n. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

27 Problem with this Approach Take a = (1,..., 1) and b = n 3. The expected number of trials before the event a x b occurs for the first time is exponential in n. A sequence of trials of reasonable length will typically yield a mean close to 0, even though the actual number of Knapsack solutions may be exponentially large. The variance of the estimator is too large for it to be of any practical value. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

28 Analysis of the Naive Algorithm Define a random variable, { 1 if a x b in the k th iteration, X k = 0 otherwise. X k -s are independent 0 1 random variables. X = m k=1 X k. E[X k ] = Pr[X k = 1] = N 2. n ( ) 2 n E[Y ] = E[X] m ( ) 2 n m = E[X k ] m ( 2 n = m = N. k=1 ) ( m 2 n ) N Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

29 Problem with the Naive Algorithm X k is a Bernoulli distribution with parameter p = N 2 n. X is a binomial distribution with parameters m and p. By Chernoff bound, Pr[(1 ε)n Y (1 + ε)n] ( ) X = Pr[(1 ε)p2 n 2 n (1 + ε)p2 n ] m = Pr[(1 ε)mp X (1 + ε)mp] 1 2e mpε2 /4. For this to be at least 1 δ, we must have, m 4 pε 2 ln 2 δ = 4 2n N ε 2 ln 2 δ. If N is sub-exponential (e.g. a polynomial) in n, this is an exponential-time algorithm and hence is not an FPRAS. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

30 A Markov Chain Monte Carlo Algorithm Consider the Markov chain M Knapsack with state space Ω = {x {0, 1} n : a x b} and transitions from a state x = (x 0,..., x n 1 ) Ω to another state y = (y 0,..., y n 1 ) Ω is defined by the following rules: State transition rules for M Knapsack 1 set y = x with probability select i uniformly at random from the range 0 i n 1. 3 let y = (x 0,..., x i 1, 1 x i, x i+1,..., x n 1 ). 4 if a y b 5 y = y. 6 else 7 y = x. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

31 Properties of M Knapsack M Knapsack may be interpreted as a random walk (with stationary moves) on the n-dimensional boolean hypercube with vertex set {0, 1} n, truncated by the hyperplane a x b. It is ergodic, since all pairs of states intercommunicate via the state (0,..., 0), and the presence of loops ensures aperiodicity. It can be easily checked that the stationary distribution is uniform over Ω. Starting in state (0,..., 0), simulate M Knapsack for sufficiently many steps until the distribution over states is close to the uniform distribution. Return the current state as the result. This gives a procedure for sampling Knapsack solutions almost uniformly at random. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

32 Product Estimators Let V be the set of elements we want to enumerate. The size of V is typically exponentially large in terms of the natural size k of the problem. Suppose, we can find a chain of subsets V 0 V 1 V m = V such that for each i, V 0 is known (usually V 0 = 1). Vi+1 V i is polynomially bounded in k. m is polynomially bounded. There is a polynomial-time oracle to generate a random element uniformly distributed over V i, for each i : 1 i m. Then we can estimate the ratios V i+1 V i by generating a polynomial number of elements of V i+1 and counting how often we hit V i. The product of these estimates and of V 0 gives an estimate for V. This scheme typically results in a FPRAS. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

33 From Sampling to Estimation of Ω We keep the vector a fixed, but allow the bound b to vary. Let M Knapsack (b) and Ω(b) be the Markov chain and its state space as functions of b. Assume without loss of generality that a 0 a 1... a n 1. Define b 0 = 0 and b i = min{b, i 1 j=0 a j}, for 1 i n. Note that b n = b, since n 1 j=0 a j > b. It is easy to see that Ω(b i 1 ) Ω(b i ) (n + 1) Ω(b i 1 ), for 1 i n, since any element of Ω(b i ) can be converted to an element of Ω(b i 1 ) by changing the rightmost 1 to a 0. Conversely, from any element of Ω(b i 1 ) we can get at most n + 1 elements of Ω(b i ). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

34 Estimating the Value of Ω Ω(b) = Ω(b n ) = Ω(b 0 ) = n i=1 1 ρ i. n i=1 Ω(b i ) Ω(b i 1 ) Note that Ω(b 0 ) = 1. ρ i = Ω(b i 1) Ω(b i ) 1 n+1. ρ i can be estimated by sampling almost uniformly from Ω(b i ) using the Markov chain M Knapsack (b i ), and computing the fraction of the samples that are in Ω(b i 1 ). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

35 Analysis Consider the random variable X i associated with a single run of the Markov chain M Knapsack (b i ). { 1 if the final state is a member of Ω(b i 1 ), X i = 0 otherwise. If we were able to simulate M Knapsack (b i ) to infinity, then we have E[X i ] = ρ i. However, we must terminate the simulation at some point, thereby introducing a small bias. We will ignore this and assume that E[X i ] = ρ i and Var[X i ] = ρ i (1 ρ i ). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

36 Analysis continued... Suppose we perform t = 17n 2 ɛ 2 trials, and let X i be the sample mean. Var[X i ] E[X i ] 2 = ρ i (1 ρ i ) t ρ 2 i = 1 ρ i tρ i n t, since ρ i 1 n + 1 = ɛ2 17n. Suppose this process is repeated for each ρ i. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

37 Analysis continued... Let Z = n i=1 X i. Note that X i are independent. Hence, E[Z] = n i=1 E[X i] = n i=1 ρ i = 1 Ω(b). Var[Z] E[Z] 2 = E[Z2 ] E[Z] 2 E[Z] 2 = E[ n i=1 X2 i ] (E[ n i=1 X i]) 1 2 n i=1 = E[X2 i ] ( n i=1 E[X i]) 1 2 n = = i=1 n i=1 ( E[Xi ] 2 + Var[X i ] E[X i ] 2 ( 1 + Var[X i] E[X i ] 2 ) 1. ) 1 Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

38 Analysis continued... Var[Z] E[Z] 2 ) n (1 + ɛ2 1 17n ɛ2 16. By Chebyshev s inequality, we conclude that [ ( Pr 1 ɛ ) 1 (1 2 Ω(b) Z + ɛ ) ] Ω(b) 4. The number of trials (Markov chain simulations) used is nt = 17n 3 ɛ 2, which is polynomial in n and 1 ɛ. This is an FPRAS for the number of Knapsack solutions, provided that M Knapsack is rapidly mixing. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

39 Rapidly Mixing Markov Chains A Markov chain is rapidly mixing, if it converges to the stationary distribution after a polynomial number of steps in n. This is a non-trivial condition for M Knapsack, since the size of the state space Ω is exponential in n. It is not known if M Knapsack is rapidly mixing. Whether there exists an FPRAS of any kind for the Knapsack counting problem is still unresolved. There are some recent advances in this area. Three techniques for bounding the mixing time of a Markov chain are coupling, canonical paths and conductance. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

40 Variation Distance and Mixing Time Let M be an ergodic Markov chain on state space Ω with transition probabilities P : Ω 2 [0, 1]. Denote by P t (x, S) the distribution of the state S Ω at time t, given that x Ω is the initial state. Let π be the stationary distribution of M. The variation distance at time t with respect to x is defined as x (t) = max P t (x, S) π(s) S Ω = 1 P t (x, y) π(y). 2 y Ω The mixing time of the Markov chain M is given by τ x (ε) = min{t : x (t ) ε for all t t}. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

41 Time-reversible Markov Chains A Markov chain M is called time-reversible if The loop probabilities P (x, x) 1 2, for all x Ω. It satisfies the following detailed balance condition: Q(x, y) = π(x)p (x, y) = π(y)p (y, x), for all x, y Ω. The detailed balance condition is stronger than that required merely for a stationary distribution. There are Markov chains with stationary distributions that do not have detailed balance. Detailed balance implies that, around any closed cycle of states, there is no net flow of probability. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

42 Canonical Paths We think of M as an undirected graph G = (Ω, E), where E = {(x, y) Ω 2 : Q(x, y) > 0}. For each pair (x, y) Ω 2, we specify a canonical path p xy from x to y in the graph G. The canonical path p xy corresponds to a sequence of legal transitions in M from the initial state x to the final state y. Let P = {p xy : x, y Ω} be the set of all canonical paths. The maximum load (congestion) on any edge in P is defined as ρ(p ) = max e E 1 Q(e) p xy e π(x)π(y) p xy. Intuitively, we expect a Markov chain to be rapidly mixing, if it contains no bottlenecks, i.e., if it admits a choice of paths P for which ρ(p ) is not too large. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

43 Relationship between Mixing Time and Congestion Theorem Let M be a finite, reversible, ergodic Markov chain with loop probabilities P (x, x) 1 2 for all states x. Let P be a set of canonical paths with maximum edge congestion ρ. Then, for any choice of the initial state x, the mixing time of M satisfies ( τ x (ε) ρ ln 1 π(x) + ln 1 ). ε Good upper bounds on the congestion ρ translate to good upper bounds on the mixing time τ x (ε). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

44 Conductance The conductance Φ of a Markov chain M is defined as Q(S, S) = Q(x, y), Φ(M) = x S,y S (x,y) E Q(S, S) min S Ω π(s). 0<π(S) 1 2 The conductance Φ may be viewed as a weighted version of edge expansion of the graph G = (Ω, E) associated with M. Alternatively, the ratio can be interpreted as the conditional probability that the chain in equilibrium escapes from the subset S of the state space Ω in one step, given that it is initially in S. Φ measures the readiness of the chain to escape from any small enough region of Ω to make rapid progress towards equilibrium. Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

45 Relationship between Mixing Time and Conductance Theorem Let M be a finite, reversible, ergodic Markov chain with loop probabilities P (x, x) 1 2 for all states x. Let Φ be the conductance of M. Then, for any choice of the initial state x, the mixing time of M satisfies τ x (ε) 2 ( Φ 2 ln 1 π(x) + ln 1 ). ε Good lower bounds on the conductance Φ translate to good upper bounds on the mixing time τ x (ε). Arindam Pal (IIT Delhi) Approximate Counting and MCMC April 8, / 45

The Monte Carlo Method

The Monte Carlo Method Example: estimate the value of π. Choose X and Y independently and uniformly at random in [0, 1]. Let Pr(Z = 1) = π 4. 4E[Z] = π. { 1 if X Z = 2 + Y 2 1, 0 otherwise, Let Z 1,...,