if j is a neighbor of i,

Size: px

Start display at page:

Download "if j is a neighbor of i,"

Harvey Welch
5 years ago
Views:

1 We see that if i = j the the coditio is trivially satisfied. Otherwise, T ij (i) = (i)q ij mi 1, (j)q ji, ad, (i)q ij T ji (j) = (j)q ji mi 1, (i)q ij. (j)q ji Now there are two cases, if (j)q ji (i)q ij the we have that both of these expressios are equal to (i)q ij ad if (j)q ji apple (i)q ij the both expressios are equal to (j)q ji. So we see that the detailed balace coditios are satisfied ad we coclude that is the limitig distributio of the Markov chai costructed by the MH algorithm. Lets re-visit the permutatio example. We described the matrix Q earlier, essetially: Q ij = 1 N(i) if j is a eighbor of i, where N(i) is the umber of eighbours of the permutatio i. So ow to complete the descriptio of the MH algorithm we simply ote that sice our target is the uiform distributio over the set of permutatios satisfyig the coditio (6.1) wehavethat, =mi 1, N(j), N(i) so the algorithm is quite simple, we start at some permutatio that satisfies the costrait ad propose a eighborig permutatio. If that eighbor has more eighbors, the =1ad we accept the proposal, otherwise we accept it with probability N(j)/N (i). This simple procedure has limitig distributio equal to the uiform distributio o the set of permutatios satisfyig the costrait! 6.4 Gibbs Samplig A very popular variat of the MH algorithm arises whe we are samplig multi-dimesioal radom vectors (both the permutatio example ad the Isig model example are examples of this). Gibbs samplig is just the MH algorithm with a very particular proposal distributio. The idea i Gibbs samplig is that we start i some state (x 1,...,x ) ad ru the followig algorithm: 1. Choose a coordiate to update, uiformly at radom, i.e. with probability 1/ we select co-ordiate j. 2. Update its value by samplig from the coditioal distributio keepig all other coordiates fixed, i.e. we sample a ew value for x j : x j (x j (x 1,...,x j 1,x j+1,...,x )). We repeat these steps. As before there are two miracles: (1) we ca ofte sample from the coditioal distributio eve whe we caot ormalize the distributio (2) the Gibbs samplig algorithm costructs a Markov chai with statioary distributio equal to. Gibbs samplig does ot make much sese for the permutatio example so lets uderstad how it works for the Isig model. 80

2 Fact 6.5. Ofte samplig from the coditioal distributio (x j (x 1,...,x j 1,x j+1,...,x )) is easy. This is difficult to make precise but lets see what happes i the Isig model: (x j =1 (x 1,...,x j 1,x j+1,...,x )) = ((x 1,...,x j 1, 1,x j+1,...,x )) ((x 1,...,x j 1,x j+1,...,x )) ((x 1,...,x j 1, 1,x j+1,...,x )) = ((x 1,...,x j 1, 1,x j+1,...,x )) + ((x 1,...,x j 1, +1,x j+1,...,x )) exp( E(x 1,...,x j 1, 1,x j+1,...,x )) = exp( E(x 1,...,x j 1, 1,x j+1,...,x )) + exp( E(x 1,...,x j 1, 1,x j+1,...,x )), ad this is a very easy distributio to sample from. I geeral, a coditioal distributio will be a ratio of joit distributios ad so the ormalizig costat will agai cacel (as it did i the MH algorithm). Fact 6.6. The limitig distributio of the Gibbs Markov chai is. We ca see that as before the Gibbs algorithm is costructig a Markov chai ad we eed to figure out what its trasitio matrix is. It is agai easy to check that i most iterestig cases the costructed chai will be irreducible ad aperiodic (try to do this for the Isig model as a exercise) so we just eed to check detailed balace coditios. We will do this by showig that Gibbs is a special case of the MH algorithm. Notice that there are o accept/reject steps i Gibbs but first what is the proposal distributio? We propose to move from (x 1,...,x ) to (x 0 1,...,x0 ) if they differ i oly oe co-ordiate. Lets deote this coordiate j (chose with probability 1/) the the proposal happes with probability: Q((x 1,...,x ), (x 0 1,...,x 0 )) = 1 (x0 j x 1,...,x j 1,x j+1,...,x ), ad similarly we ca calculate: Q((x 0 1,...,x 0 ), (x 1,...,x )) = 1 (x j x 1,...,x j 1,x j+1,...,x ). Now if this were really a MH algorithm, we should accept the proposal with probability: =mi 1, ((x0 1,...,x0 )) Q((x 0 1,...,x0 ), (x 1,...,x )) ((x 1,...,x )) Q((x 1,...,x ), (x 0 1 (,...,x0 )) ) =mi 1, ((x0 1,...,x0 )) (x j x 1,...,x j 1,x j+1,...,x ) ((x 1,...,x )) (x 0 j x. 1,...,x j 1,x j+1,...,x ) Now otice that, (x j x 1,...,x j 1,x j+1,...,x )= ((x 1,...,x )) Px ((x 1,...,x j 1,x,x j+1,...,x )), so we see that the ratio is always equal to 1. So if we use the coditioal distributio as a proposal distributio the we should always accept the ew sample which is exactly what Gibbs samplig does. So we coclude that Gibbs samplig has limitig distributio. 81

3 Chapter 7 PageRak Algorithm I this chapter we will discuss aother importat applicatio of Markov chais, ad some of the theory we have developed so far cocerig the limitig behaviour of Markov chais. 7.1 Motivatio ad history PageRak is a algorithm that was developed by Sergey Bri, Rajeev Motwai, Larry Page ad Terry Wiograd. The basic ituitio was that oe could try to measure the umber ad quality of liks to a webpage, ad use that to estimate how importat the webpage is. Agoodwebpageisoethatislikedtofrommayothergoodwebpages. Similar algorithms were proposed by various people aroud the same time. Notably, Jo Kleiberg suggested the HITS (Hyperlik-Iduced Topic Search) algorithm for the same problem. Today, Google uses may other sigals i webpage rakig but PageRak remais a very ifluetial idea. PageRak s first isight is relatively straightforward: we ca thik of the World-Wide-Web (WWW) as a directed graph. The odes are webpages. Differet webpages lik to each other: these are the edges. For today we have webpages umbered {1,...,}. We will attempt to assig each webpage a PageRak score. Deote these (suggestively) { 1, 2,..., }. 7.2 Some early attempts Give our graph o the webpages, a first attempt would be to treat all webpages equally. We the rak each webpage by the umber of pages that lik to it. So the top webpage is just the webpage with the highest i-degree, ad so o. Ituitively, this seems like a bad idea. More precisely, you could imagie that I ca create a webpage ad may sub-pages that all lik to this webpage ad this will result i the first page havig a very high-rakig eve though all the pages that liked to it are owed by me. More broadly, we wat the rakig to be difficult to maipulate locally. As a side commet ideas like these of tryig to artificially boost a webpage s rakig/reputatio has led i recet years to may startups attemptig Search Egie Optimizatio, ad o the other had compaies like Google costatly try to keep this i check. The ext attempt is recursive but is very close to the actual PageRak algorithm. We could weight each of the i-liks to a webpage. Suppose we are tryig to compute the PageRak score for a particular webpage i. Ituitively, 82

4 1. Webpages that lik to a particular i, ad have a high PageRak score, should be give more weight. 2. Webpages that lik to a particular i, ad have a low PageRak score, should be give less weight. This is a bit circular, but the key poit is that that is ot a problem. We have bee seeig may circular defiitios recetly. We will re-visit this i a bit. Lets coect this back to Markov chais. The way to do this is to thik of a so-called radom surfer, oe who visits webpages by clickig liks at radom. The radom surfer s curret locatio is really just the state i a Markov chai. Defie, the adjacecy matrix for a directed graph as 2 3 A 11 A A 1 A 21 A A 2 A = , A 1 A 2... A where A ij =1if there is a edge or lik from i! j. Now, we wat to tur this ito a trasitio matrix for our radom surfer. This is easy to do. We just defie: where 2 3 P 11 P P 1 P 21 P P 2 P = , P 1 P 2... P P ij = A P ij j=1 A. ij Lets go through a simple example. Cosider the followig graph: 83

5 We ca write dow the radom surfer trasitio matrix /2 1/2 0 P = Usig MATLAB, we ca fid the limitig distributio of the radom surfer, ad we obtai Is this ituitive? Observe the followig thigs: =[0.4, 0.2, 0.4, 0]. 1. If there are o liks i to a page it has a score of Page 3 has may i-liks ad gets a high-score. 3. Page 1 has oly oe i-lik but it is from a ifluetial page so it also gets a high-score. So we jumped from some ituitio about assigig high score to webpages which are liked to from other pages with high score to the radom surfer model, ad the limitig distributio. Lets take a step back. We kow that for a statioary distributio : T = T P. So i particular the PageRak score for a page i satisfies: i = X j P ji = j=1 X j=1 j A ji d j. Is this ituitive? What about the ormalizatio by degrees? This effectively cotrols the total vote of each page. So we have made some progress. We have formulated our desired characteristics, ito thigs that are satisfied by the limitig distributio of a particular Markov chai. Now, we just eed to make sure that the limitig distributio exists. There are a few problems we might ru ito. 84

6 Well we kow that for fiite Markov chais there is always at least oe statioary distributio. Why ot use oe of those (it may ot be uique)? Here is a cocrete example of what ca go wrog. Suppose we have the followig web graph: So the radom surfer matrix has two statioary distributios: [1/3 1/3 1/3 0 0] ad [0001/3 1/3]. If we iterpret these etries as PageRak scores the the two rakig are completely at odds with each other. Ituitively, what is happeig is that the rakig of the webpages depeds o where the radom surfer starts. 7.3 The real PageRak algorithm The real PageRak algorithm makes a slight modificatio. Suppose that our radom surfer occasioally gets bored ad clicks o a radom webpage. Our bored radom surfer also follows a Markov chai, with trasitio matrix: Q =(1 ) P We eed to set the parameter, ad eed to do so carefully. What ca go wrog if we pick a value that is too small? Too large? Google uses the value =0.85. This simple modificatio is icredibly importat. We have ow created a Markov chai with a uique statioary distributio, irrespective of the uderlyig graph. We ow have all the pieces i place. We ca represet the WWW as a (very large) matrix ad compute its uique limitig distributio. We ll discuss this i a bit more detail. 7.4 Computig PageRak scores The real cotributio of the PageRak paper was to show that oe could compute the PageRak scores o huge graphs. O a small graph we could use MATLAB. 85

7 Revisitig our earlier example we would ru the followig commads:» P = [0 1/2 1/2 0; ; ; ];» Q = (1-0.85)*oes(4,4)/ *P;» Q 1000 as = This is somewhat ielegat. The other way of doig this ivolves observig that the statioary distributio is just the top eigevector of the trasitio matrix. Thik this through. So we could use the followig istead:» [V,D] = eig(q );» V(:,1)/sum(V(:,1)) as = Computig the top eigevector of a ( ) matrix typically takes time O( 3 ), which is very slow. Oe could istead use a iterative method called the power method, to compute the limitig distributio. This is really very similar to what we tried first (i.e. raisig the trasitio matrix to some high power). It is called the power method of computig eigevectors. Essetially, we could use the followig iterative algorithm: 1. Start with some iitial distributio Compute iteratively: t+1 = t Q util the vector stabilizes. So, i MATLAB we would use:» p = oes(1,4)/4;» for i = 1:1000 p = p*q; ed» p p = The PageRak paper showed that they could compute PageRak scores o web-scale usig the power method. A importat observatio that they made was that oe ca compute these very fast whe the trasitio matrix (without the bored part) is sparse. 7.5 Summary It is worthwhile to thik through how oe would combie PageRak scores (which really igore the cotet of the webpage) with thigs like TF-IDF (term frequecy-iverse documet frequecy) that use the cotet of the webpage (ad the search query). 86

8 We saw how to cast the problem of rakig webpages as that of fidig the limitig distributio of a Markov chai. We saw how to use isights about the existece ad uiqueess of limitig distributios to fix the Markov chai, ad fially we saw how oe might compute PageRak scores at web-scale. I the years sice PageRak was proposed, several ice variats have also bee proposed to improve the basic algorithm (check out the Wikipedia page for PageRak to fid some refereces). 87

Notes for Lecture 11

Notes for Lecture 11 U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with