CS 388R: Radomzed Algorthms Fall 205 Prof. Erc Prce Lecture 4 Sep 9, 205 Scrbe: Xagru Huag & Chad Voegele Overvew I prevous lectures, we troduced some basc probablty, the Cheroff boud, the coupo collector problem, ad game tree evaluato. I ths lecture, we wll troduce cocetrato equaltes. 2 Coupo Collector Problem Draw umbers (coupos) depedetly from ] = {, 2,..., }. How log does t take to see all of the umbers? Suppose T s the umber of draws to get the -th ew umber. Let T = T. Fact. The T s are depedet of each other. Fact 2. T follows geometrc dstrbuto wth success probablty, p = +. Fact 3. If X Geometrc(p), EX] = p + ( p) (EX X 2]) EX] = p = p + ( p)( + EX]) EX 2 ] = p 2 + ( p) EX 2 X 2] = p + ( p)e(x + ) 2 ] = p + ( p)(ex 2 ] + 2EX] + ) = p + ( p)ex 2 ] + 2( p)/p + ( p) EX 2 ] = 2 p p 2 V ar(x) = E(X E(X)) 2 ] = EX 2 ] (EX]) 2 V ar(x) = p p 2 p 2
Therefore the Coupo Collector Problem, ET ] = ET ] = = = + = H log V art ] ( p 2 = + V art ] = ) 2 = V art ] 2 ( = ) 2 2 π2 6 = O(2 ) 3 Cocetrato Iequaltes, P rt + α] Assume s some falure probablty. Settg α = have Defto 4. Uo Boud ( + ( +, P rt + α ] ) log ) α ad because ( x )x < e, we P rx X 2... X ] P rx ] Usg a uo boud, we have P r T + log log ] = P r T + ] α = P rt + α... T + α ] P rt + α ] Defto 5. Wth Hgh Probablty (w.h.p.) X O(y) w.h.p. c 2, c, s.t. P rx c y] c 2 T = O( log 2 ) wth hgh probablty. 2
3. Markov s Iequalty For a o-egatve radom varable T ad ay o-egatve α, I the Coupo Collector Problem, ET ] P rt α] α P rt α] ET ] α α = ET ] P r = H T H ] 3.2 Chebyshev s Iequalty For a radom varable, X, let µ = EX] deote the expectato ad σ 2 = V arx] deote the varace. Startg from Markov s Iequalty, we fd Settg α ασ P r(x µ) 2 α 2 ] E(X µ)]2 α 2 = σ2 α 2 P r(x µ) 2 α 2 σ 2 ] α 2 Takg the square root, we fd P rx µ + ασ] α 2 P rx µ ασ] α 2 Usg ths result the Coupo Collector Problem, gves us P rt H + O()] Settg = log 2 Most of the tme, the typcal devato s O(σ). ( ) P rt H + O( log )] O log 2 P r x µ O(σ)] 3
3.3 Momet Method If f s o-egatve, by Markov s equalty, P rf(x µ) f(α)] Ef(X µ)] f(α) For f creasg, Set f = t k, For oe sde, P rx µ α] Ef(X µ)] f(α) P r X µ k α k ] E x µ k ] α k P rx µ + α] E x µ k ] α k Settg = E x µ k ], we have α k P r X µ + E x µ k ] /k ( ) ] /k If we cosder X N(0, σ 2 ), we kow E x k ] (kσ 2 ) k/2 k > 0 whch meas Settg k = log, we get P r X µ + O P r X µ + O ( ( ) )] k /k σ ( log )] 3.4 Momet Geeratg Fucto Defto 6. The momet geeratg fucto, parameterzed by λ, s defed as MGF X (λ) = Ee λ(x µ) ] Assume X s cetered (EX] = 0). e λx = + λx + (λx)2 2 + (λx)3 3! +... + (λx)k k! We ca use parameter λ to adjust the weghts o each term. Whe λ s larger, more weght s o hgher order terms. 4
From the dervato of the Momet Method, settg f(x) = e λx, P rx µ + α] MGF X(λ) e λα Fact 7. If X N(0, σ 2 ), MGF X (λ) = Ee λx ] e λ2 σ 2 Usg ths, Set λ = α σ 2, we get If = e α2 2σ 2, we have λ R P rx µ + α] MGF x(λ) e λα = e λ2 σ 2 2 λα = e 2(λσ α σ ) 2 α2 2σ 2 P rx µ + α] e α2 2σ 2 α = σ 2log ( ) Note that ths s the same O log boud as we foud the method of momets, except that ow we kow the costat. 3.5 Subgaussa Varables Clam 8. The followg three statemets are equvalet f we oly care up to a costat for σ (.e., j {, 2, 3}, σ = θ(σ j )) X s subgaussa wth parameter σ,.e. λ R, MGF X (λ) e λ 2 σ 2 2 () P rx µ + t] e t 2 ) E x k ] /k O (σ 3 k Fact 9. The sum of subgaussa varables are subgaussa. 2σ 2 2 (2) X = X +... + X MGF X (λ) = E e λx] ] = E e λ( x ) = E = ] E e λx (by depedece) e λx ] (3) = MGF X (λ) e λ2 σ 2 /2 (by subgaussa) = e λ2 2 ( σ2 ) 5
Ths mples X s subgaussa wth parameter σ2. Fact 0. If X 0, ], the X s subgaussa wth σ = /2 by Hoeffdg s Lemma. Let X = X where X 0, ]. X s subgaussa wth σ = /2. Plug ths to (2), we have whch s exactly the Cheroff boud. P rx µ + α] e 2α2 4 Next Class I the Coupo Collector Problem, we had P rt α] ( ) α e α Ths s ot of the form e α2 so we caot use the subgaussa results. We wll relax the subgaussa requremet to subexpoetal ad subgamma. Ths wll lead to Berste s equalty. MGF X (λ) = Ee λx ] e λ2 σ 2 2 λ B Refereces MR] Rajeev Motwa, Prabhakar Raghava Radomzed Algorthms. Cambrdge Uversty Press, 0-52-47465-5, 995. RV] Roma Vershy Itroducto to the o-asymptotc aalyss of radom matrces. CoRR, abs-0-3027, 200. https://e.wkpeda.org/wk/hoeffdg%27s_equalty 6