26.1 Metropolis method

Transcription

1 CS880: Approximations Algorithms Scribe: Dave Anrzejewski Lecturer: Shuchi Chawla Topic: Metropolis metho, volume estimation Date: 4/26/07 The previous lecture iscusse they some of the key concepts of Markov Chain Monte Carlo (MCMC) methos, incluing the stationary istribution π an the mixing time τ ǫ. This lecture introuces the Metropolis metho for constructing Markov chains in orer to sample from some istribution. The use of sampling methos for volume estimation is also introuce. 26. Metropolis metho 26.. MCMC review Recall from last time the key properties of a ranom walk Markov chain. Ω = the state space n = Ω P = the transition matrix, P ij = Pr[i j] π = the stationary istribution such that π P = π τ ǫ = the mixing time, after which the l -norm of the ifference between the chain istribution an the stationary istribution is guarantee to be < ǫ. Also recall this important theorem concerning the existence an uniqueness of the stationary istribution π. Theorem 26.. An aperioic irreucible finite Markov chain is ergoic an has a unique stationary istribution. We can easily guarantee the aperioicity of our chain by simply aing self-loops to all vertices. This will increase the mixing time by no more than a factor of Metropolis filter But how o we actually construct a Markov chain with a stationary istribution equal to our target istribution? Also, we want this metho to have a goo (that is, small) mixing time. The Metropolis metho allows us achieve these goals by efining our Markov chain as a ranom walk over a suitably efine graph. We efine the approach as follows. Say we which to sample values i Ω from a istribution Q(i). Then we efine an unirecte -regular graph G on Ω, picking this graph in such a way that it has high conuctance. Then from noe v, pick the next noe u uniformly from the neighbors. Then:

2 If Q(u) Q(v), move to noe u Else move to noe u with probability Q(u) Q(u) Q(v), stay with probability ( Q(v) ). First we examine the graph itself. Since it is fully connecte an unirecte, it is irreucible. Since all noes have self-eges, it is aperioic. Therefore this ranom walk is guarantee to have a unique stationary istribution π. Now we must show that this stationary istribution is equal to our target istribution Q. Claim π = Q Proof: Say that our initial π = Q, then take one step. Consier any noe v, an calculate the probability of arriving at noe v after this one step. If it is equal to Q(v), then we have shown that QP = Q, an therefore π = Q. We nee to calculate the probability of starting at istribution Q, taking one step, an then ening up in state v. This can be ecompose into three cases: we move from a neighbor u into v where Q(u) Q(v), we move from a neighbor u into v where Q(u) < Q(v), or we are alreay in v an we choose a neighbor u such that Q(u) < Q(v) but we en up staying at v. Let n be the number of neighbors u such that Q(u) Q(v). Q (v) = Q(u) Q(v) Q(u)Q(v) Q(u) + Q(u)<Q(v) Q(u) + Q(u)<Q(v) Q(u) Q(v)( Q(v) ) (26..) = n Q(v) + n Q(u) + n Q(v) n Q(u) (26..2) = Q(v) (26..3) This shows that a ranom walk using the Metropolis metho is guarantee to converge to our target istribution Q. It is worth noting that our scheme of uniformly choosing a neighbor is a special case of the general Metropolis-Hastings sampler [3]. In the more general case, a proposal istribution is use to select the next caniate state conitione on the current state. This proposal istribution nee not be uniform over neighbors, an in fact nee not even be symmetric Volume estimation An interesting application of sampling techniques is the problem of estimating the volume of a convex shape K R n using an inclusion oracle which reveals whether a given point is containe in the shape or not. We are also given two balls, one completely enclosing K an one completely enclose by K. Call these K B(0,R) an K B(0,r). This technique that we use has interesting parallels to the concept of self-reucibility. What is the probability that a uniformly chosen point in the larger ball will be in K? We can use the smaller ball to boun this probability as vol(b(0,r)) vol(b(0,r)). However, for large n we will suffer the curse of imensionality [4], an this lower boun will be very small, in particular ( r R )n. 2

3 The key insight of our approach is that we can use a sample from a series of regions K i in orer to make the probability of fining a point in the region as large as possible. K K_i+ R r Figure 26..: An example of our volume estimation problem setup. K = K o K... K l = B(0,R) (26..4) These regions are constructe such that vol(k i+) vol(k i ) is small for all i. Furthermore we choose a small l n log( R r ), so we o not nee to iterate too many times. We can estimate the volume of K using estimates of the ratios of ajacent K i in the following way. vol(k) = vol(k o) vol(k ) vol(k ) vol(k 2 )...vol(k l ) vol(b(0,r)) (26..5) vol(k l ) So now we simply nee to figure out how to sample from K i. We efine K i as a re-scaling of K, subject to containment by B(0,R). This gives us a nice boun on the volume ratio of ajacent K i. K i = ( + /n) i K B(0,R) (26..6) vol(k i+ ) vol(k i ) ( + /n) n = e (26..7) 3

4 Note that ( + n )n log R/r K contains B(0,r), therefore e is inee O(n log R/r). To sample from K i, we then simply sample uniformly from K an then re-scale. But how to sample from K itself? To approach this problem, we employ the MCMC methos we have been iscussing. We efine our ranom walk, known as the Ball-walk, as follows. From any point u K, sample a point ranomly from the ball centere at u with raius δ, B(u,δ), an move to the new point if it is insie K. If the point is outsie K, stay at u. Note that the graph efine by this rule allows us to reach any point from any other point, an also allows self-loops. Therefore it is irreucible an aperioic, an must have a unique stationary istribution π. The resulting Markov chain is time-reversible. Therefore, the stationary istribution is uniform. For the practicality of this scheme, it is important to choose a goo value for δ in orer to get goo samples from K. Taken to the extreme, a huge δ value woul result in constantly picking points outsie K, an therefore remaining at the current point. Likewise, a very small δ woul result in taking very small steps, making it very slow to explore all of K. Also, if something is known about the geometry of K, it may be helpful to rescale the proposal ball to an ellipse, for example. This is accomplishe by putting the boy in an isotropic position via an affice transformation, so as to remove all sharp corners. Figure 26..2: Rescaling the proposal ball to an ellipse base on the geometry of K. The first approach base on this technique was polynomial in n, but with an unfortunate orer O(n 23 ) []. Newer approaches, ubbe hit an run, first choose a irection, an then sample uniformly from the line segment along that irection containe in K. This approach rastically improves mixing time, achieving Õ(n4 ) [2]. Figure 26..3: The hit an run technique. 4

5 The inapproximability result is that one cannot estimate volume within a constant factor in Ω(n 2 ) time. References [] Martin Dyer, Alan Frieze, Ravi Kannan. A ranom polynomial-time algorithm for approximating the volume of convex boies. JACM 99. [2] Laszlo Lovasz, Santosh Vempala. Simulate Annealing in Convex Boies an an O(n 4 ) Volume Algorithm FOCS [3] D. MacKay. Information Theory, Inference, an Learning Algorithms. Cambrige University Press, [4] Trevor Hastie, Robert Tibshirani, Jerome Frieman. The Elements of Statistical Learning: Data Mining, Inference, an Preiction. Springer-Verlag, 200. [5] V. Vazirani. Approximation Algorithms. Springer,