15-451/651: Design & Analysis of Algorithms October 23, 2018 Lecture #17: Prediction from Expert Advice last changed: October 25, 2018

Transcription

1 5-45/65: Design & Analysis of Algorithms October 23, 208 Lecture #7: Prediction from Exert Advice last changed: October 25, 208 Prediction with Exert Advice Today we ll study the roblem of making redictions based on exert advice. There are n exerts (who are just eole with oinions, the uotes suggesting that they may or may not have any exertise in the matter). Each day the following seuence of events haens: We see the n exerts redictions of the outcome. We make our own rediction about the outcome. In arallel, the actual outcome is revealed. We are correct if we made the right rediction, and make a mistake otherwise. This rocess goes on indefinitely. Our goal: at any time (say after some T days), if the best of the exerts so far has made only few mistakes, we want to have made not too many more mistakes. We want to bound the number of mistakes, and hence this is called the mistake-bound model. 2 Warmu: Simle Strategies To start off, say we have just U/Down redictions. (If the market will go u or down, or if the weather will be fair or not.) Suose we know the best exert makes no mistakes. Can we hoe to make only a few mistakes? Here s a strategy that makes only log 2 n mistakes. Just redict what the majority of the remaining exerts redicts. (In case of a tie, choose arbitrarily.) When an exert is incorrect, discard him. So each time we make a mistake, we reduce by the number of exerts by at least /2. This means after log 2 n mistakes we will be left with the erfect exert. Exercise : Argue that you cannot get an algorithm that makes fewer than log 2 n mistakes in this setting. Suose the best exert makes at most M mistakes on some seuence. Can we hoe to make only a few mistakes? Here s a strategy that makes only (M + )(log 2 n + ) mistakes. Run the above majority-and-halving strategy, but when you have discarded all the exerts, bring them all back (call this the beginning of a new hase ), and continue. Note that in each hase each exert makes at least one mistake, and you made log 2 n+ mistakes. Hence, if the best exert makes only M mistakes, there would be at most M finished hases (lus the last unfinished one), and hence at most (M + )(log 2 n + ) mistakes in all. Exercise 2: Suose the redictions belonged to some set of K items. (E.g., you could say cloudy, sunny, rain, snow.) Give algorithms showing the bound of O(M log n) mistakes still holds. Let s assume n is a ower of 2.

2 3 The Weighted Majority/Multilicative Weights Algorithm Throwing away an exert when he makes a mistake seems too drastic. Suose we instead assign weights w j to the exerts, sum the weights of the exert saying U, sum the weights of the of the exert saying Down, and redict the outcome with greater weight. (This is called the weighted majority rule, since we are following the advice of the exerts that form the weighted majority.) Then once we see the outcome, we can reduce the weight of the exerts who were wrong. In the above algorithm, we were zeroing out the weight, but suose we are gentler? The (basic) deterministic weighted majority algorithm does the following: Start with each exert having weight. Each time an exert makes a mistake, half his weight. Outut the rediction of the exerts who form the weighted majority. Remarkably, we can get a much stronger result now. Theorem If on some seuence of days, the best exert makes M mistakes. The basic deterministic weighted majority algorithm makes 2.4(M + log 2 n) mistakes. Proof: Let Φ := n i= w i be the sum of weights of the n exerts. Note that initially Φ = n. Moreover, we claim that each time we make a mistake Φ new 3 4 Φ old. Indeed, at least half the weight (which was making the majority rediction) gets halved (because it made a mistake), so we lose at least a uarter of the weight with each mistake. (Also if we don t make a mistake, Φ does not increase.) So if we ve made m mistakes at some oint, the total weight is at most Φ final (3/4) m Φ init = (3/4) m n. Moreover, if the best exert i has made M mistakes, then Φ final w i = (/2) M. So (/2) M (3/4) m n (4/3) m n2 M. Taking logs (base 2) and noting that log 2 (4/3) = comletes the roof. This is retty cool! If the best exert makes mistakes 0% of the time we err about 24% of the time (lus O(log n), but since this deends only on the number of exerts, it will be a negligible fraction as M log n). We can imrove the 2.4 factor down to as close to 2 as we want, as the next exercise shows. Exercise 3: Show that if we reduce the weight not by /2 but by ( ɛ) for some ɛ /2, then the number of mistakes is at most 2( + ɛ)m + O( log n ). (Hint: check out the aroximations we use in the ɛ roof of Theorem 2.) However, you can show that no deterministic rediction algorithm can make fewer than 2M mistakes. How? Two exerts: one says U all the time, the other Down all the time. Fix any rediction algorithm. Since this algorithm is deterministic, you know what rediction it will make on any day, if you know what haened on all revious days. As the adversary, you can now make the real outcome on this day be the oosite of the algorithm s rediction for this day. So algorithm makes mistake on all days. But one of the exerts must be right at least 50% of the days. 2

3 4 Randomized Weighted Majority Given the lower bound above for deterministic algorithms, randomization seems like a natural source of hel. (At least that examle would fail.) Let s define the Randomized Weighted Majority algorithm as follows: Start with unit weights. At each time, redict U with robability j says U w j j w, j and Down otherwise. Whenever an exert makes a mistake, multily its weight by ( ɛ). Note that the weights are retty much like in the basic MW algorithm, just reduced more gently. (Moreover, note that the rediction we make does not alter the weight reduction ste.) A slightly different, but euivalent view is the following: Start with unit weights. At each time, ick a random exert, where exert i is icked w with robability i, and redict that icked exert s rediction. And each time an j w j exert makes a mistake, multily its weight by ( ɛ). The analysis very similar to Theorem, we just need to handle exectations. Theorem 2 Let ɛ /2. If on some seuence of days, the best exert makes M mistakes. The exected number of mistakes the randomized weighted majority algorithm makes is at most ( + ɛ)m + ln n Proof: Again, let us look at the otential Φ = j w j, the total weight in the system. Having fixed the outcome on all days, this otential varies deterministically. Let F t be the fraction of the total weight on the t th day that is on the exerts who make a mistake on that day. On day t our robability of making a mistake is F t. Hence The exected number of mistakes we make is t F t. On the t th day, Φ new = Φ old ( ɛf t ). So, Φ final = n ( ɛf t ) n e ɛ t Ft, where we used that ( + x) e x for all x R. Again, Φ final ( ɛ) M, so t ( ɛ) M n e ɛ t Ft ɛ t F t M ln + ln n. ( ɛ) We can now use that ln ( ɛ) = ln( ɛ) ɛ + ɛ2 for ɛ [0, 2 ] to get the exected number of mistakes = t 3 F t M( + ɛ) + ln n

4 Pretty sweet, right? The number of mistakes we make is almost the same as the best exert, lus this additive term that just deends on n and ɛ, but is indeendent of the inut seuence. Since the otimal number of mistakes M is at most T, the number of days, we can say our # of errors otimal # of errors + ɛt + ln n ɛ So dividing both sides by T, we get the error rate (i.e., errors er unit time): () our error rate otimal error rate + ɛ + ln n ɛt ln n And now setting ɛ = T : (2) ln n our error rate otimal error rate + 2 T. (3) This last term is called the regret. So as we do the rediction for longer and longer (i.e., T ), our error rate gets as close as we want to the otimal error rate. (We have vanishing regret.) 4. Some Extensions (otional material) Reeated Zero-Sum Game. The interretation of choosing a random exert (according to the robability distribution ) allows us to view the rocess as laying a two-layer zero-sum game w i j w j reeatedly. Let the cost matrix contain only 0s and s. Each column is an exert. Each day the adversary lays a row (which defines the costs for us), we lay a column (which is like choosing an exert). Theorem 2 says that we can lay almost as best as the best column in hindsight. Fractional Costs. Suose each day t, instead of costs that are zero (no mistake) or one (mistake), we have costs c t i in [0, ]. We can extend the result to that setting with a very slight change: udate the weight of an exert i by w i w i ( ɛc t i ). With small changes in the analysis, the guarantee of Theorem 2 goes through unchanged! (Exercise: Show this!) 4.2 A Proof of the Minimax Theorem (otional material) Recall the minimax theorem: it says that in any zero-sum game (given by the row layer s ayoff matrix M), if we define the row layer s guaranteed ayoff (which is a lower bound on what she s guaranteed to earn regardless of what the column layer does): V R = max min j T Me j = max min T M and the column layer s guaranteed ayoff (which is an uer bound on what the row layer can earn, regardless of what he does): V C = min max i e T i M = min max T M, then V C = V R. Clearly, V R V C. The tricky art is the other direction, which we now use the exerts theorem to rove. Suose not. There is some zero-sum game where V R = V C δ for δ > 0, with a strict ineuality. Remember, 4

5 If the column layer commits to some strategy first, there is some row that the row layer can lay to get ayoff at least V C. On the other hand, if the row layer commits first, then the column the column layer can lay so that row layer get at most V R = V C δ. By scaling the ayoffs, imagine they are in [0, ]. Now imagine we use Randomized Weighted Majority (RWM) to lay the columns. Since the distribution from which RWM draws is udated in a deterministic fashion, let the row layer lay otimally against this distribution at each ste. In T stes, RWM has cost at most that of the best column (exert) in hindsight +ɛt + ln n The best column in hindsight can ensure cost at most V R T, since it s like the row layer has layed first. But at each time, the row layer knows your strategy, so his exected ayoff is at least V C T. Putting these together or V C T row layer s ayoff V R T + ɛt + ln n ɛ, δt ɛt + ln n But we were free to choose ɛ and T as we want, so if we choose ɛ = δ/2 and T ln n, then we get ɛ 2 a contradiction. Hence, there cannot be any ga δ, and V R = V C. 5