Skip lists: A randomized dictionary
|
|
- Marcia Adams
- 6 years ago
- Views:
Transcription
1 Discrete Math for Bioiformatics WS 11/12:, by A. Bocmayr/K. Reiert, 31. Otober 2011, 09: Sip lists: A radomized dictioary The expositio is based o the followig sources, which are all recommeded readig: 1. Pugh: Sip lists: a probabilistic alterative to balaced treed. Proceedigs WADS, LNCS 382, 1989, pp Sedgewic: Algorithme i C++, 2002, Pearsos, (Chapter 13.5) 3. Lecture Script from Michiel Smid, Uiversity of the Saarlad. 4. Motwai, Raghava: Radomized algorithms, Chapters 8.3 ad Kleiberg, Tardos: Algorithm desig, Chapter 13.9 Itroductio Here a little refresher of sum formulas you will eed: (x + y) = ( ) x y ad for 0 < r < 1: r = 1 r +1 1 r r 1 = 1 r Itroductio We cosider the so called dictioary problem. Give a set S of real umbers, store them i a data structure such that the followig three opertios ca be performed efficietly: Search(x): Give the real umber x, report the maximal elemet of S { } that is at most equal to x. Isert(x): Give a real umber x isert it ito the data structure. Delete(x): Give a real umber x delete it from the data structure. The stadard data structures for this problem is the balaced biary tree. It supports all the above operatios i worst case time O(log) ad uses O() space. Well ow classes of balaced trees are for example AVL-trees, BB[α]-trees ad red-blac-trees. I order to maitai their worst case time behaviour all those data structures eed more or less elaborate rebalacig operatios which mae a implemetatio o-trivial, which i tur leads ot to the best practical ru times. We will itroduce i this lecture a alterative, radomized data structure, the sip list. It uses i expectatio liear space ad supports the above dictioary operatio i expected time O(log ) with high probability. Why do we do this? We will see that the data structure is coceptually much simpler ad more elegat tha balaced trees. Nevertheless we will exchage a worst case rutime agaist a expected ru time. However, the aalysis will show that sip lists behave very well ad are very fast i practice (the differece is similar to the determiistic merge sort ad the radomized quicsort algorithm). The goal of this lecture is to
2 3002 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09:53 itroduce you to the data structure show you how to aalyze the radomized ru time itroduce you to tail estimates usig Cheroff bouds. Sip lists Throughout the lecture we assume that we ca geerate radom, idepedet bits i uit time. Let S be a set of real umbers. The we costruct a sequece of sets S 1,S 2,... as follows: 1. For each elemet x S, flip a coi util zero comes up. 2. For each i 1, S i is the set of elemets i S for which we flipped the coi at least i times. Let h be the umber of sets that are costructed. The it is clear that /0 = S h S h 1 S h 2 S 2 S 1 = S The sip list for S cosists the of the followig: 1. For each 1 i h, the elemets of S i { } are stored i a sorted lied list L i. 2. For each 1 < i h, there is a poiter from each x L i to its occurrece i L i 1. Here is a example. Suppose S = {1,2,5,7,8,9,11,12,14,17,19,20}. Flippig cois might lead to S 1 = S, S 2 = {1,2,5,8,11,17,20}, S 3 = {2,5,11,20}, S 4 = {11}, ad S 5 = /0. Searchig sip lists We ca ow implemet the search for x as follows: 1. Let y h be the oly elemet i L h. 2. For i = h,h 1,...,2 (a) Follow the poiter from y i i L i to its occurece i L i 1.
3 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09: (b) Startig i y i 1, wal to the right alog L i 1, util a elemet is reached that is larger tha x or the ed of L i 1 is reached. Let ow y i 1 be the last ecoutered elemet i L i 1 that is at most equal to x. 3. Output y 1. The followig figure illustrates the search step (we search for elemet 10): It is ot hard to imagie how the isertio ad deletio operatios wor o a sip list. Isertig ito a sip list For isertig a elemet x ito the dictioary we proceed as follows: 1. Ru the search algorithm for x. Let y 1,y 2,...,y h be the elemets of L 1,L 2,...,L h that are computed while searchig. If x = y 1, the x S ad othig has to be doe. Hece assume that x y Flip a coi util a zero comes up. Let l be the umber of coi flips. 3. For each 1 i mi(l,h), add x to the list L i immediately after y i. 4. If l h, the create ew lists L h+1,...,l l+1 storig the sets S h+1 { }, where each set cotais x except for S l+1 which is empty. 5. For each 1 < i l, give x i L i a poiter to its occurrece i L i If l h, the for each h + 1 i l + 1, give i L i a poiter to its occurrece i L i Set h = max(h,l + q).
4 3004 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09:53 Deletig from a sip list 1. Ru the search algorithm for x. Let y 1,y 2,...,y h be the elemets of L 1,L 2,...,L h that are computed while searchig. If x y 1, the x / S ad othig has to be doe. Hece assume that x = y For each 1 i h such that x = y i delete y i from the list L i. 3. For i = h,h 1,...: if L i 1 oly stores, delete the list L i ad set h = h 1. Why are sip lists efficiet? The ituitio We have see, that mostly what we do i sip lists is to search. The rebalacig is doe by throwig a coi a few times ad maig local chages alog the search path. How expesive is the search? It is the sum over all traversed path legth at each level. We expect there to be log levels. At each level we travel to the right. However for a fixed level we do ot expect to do this log, sice this would imply, that all the elemets are ot i the level above. Hece we expect to sped a costat amout of time at each level which would add up to a total search time of O(log). We will ow prove this more formally. Why are sip lists efficiet? The proofs The size of a sip list ad the ruig times of the search ad update algorithms are radom variables. We will prove that their expected values are boud by O() ad O(log ) respectively. Recall that h deotes the umber of sets S i that result from our probabilistic costructio. How ca we derive a upper boud for h? Let x be a elemet of S ad h(x) be the umber of sets S i that cotai x. The h(x) is a radom variable distributed acccordig to a geometric distributio with p = 1/2. Hece Pr(H(x) = ) = (1/2) ad E(h(x)) = 2. That meas if we loo at a specific elemet we oly expect it to be i S 1 ad S 2. Clearly h = 1+max{h(x) : x S}. From E(h(x)) = 2 for ay x S, however, we caot coclude that the expected value of h is three. We ca estimate E(h) as follows. Agai cosider a fixed x S. It follows that for ay 1, h(x) if ad oly if the first 1 coi flips produced a oe. That is Pr(h(x) ) = (1/2) 1. I additio it is clear that h + 1 if ad oly if there is a x S such that h(x). Hece Pr(h + 1) Pr(h(x) ) = 2 1 This estimate does ot mae sese for < 1 + log. For those values of we ca use the trivial upper boud Pr(h + 1) 1. The E(h) equals: log Pr(h + 1) = Pr(h + 1) + + log Pr(h + 1). (exercise: proof the first equality, that is E(X) = Pr(X ) for a radom variable X that taes values {0,1,2,...}.) The first summatio o the right had side is at most 1 + log. The secod sum ca be bouded from above by: 2 1 = (1/2) log 1 (1/2) log log Hece we have prove that E(h) 3 + log.
5 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09: The expected size of a sip list ca easily be computed. Let M deote the total size of the sets S 1,S 2,...,S h. The M = x S h(x) ad by liearity of expectatio: E(M) = E(h(x)) = 2 = 2. x S x S If M deotes the total umber of odes i a sip list, the M is equal to M plus h. Hece E(M ) = E(M + h) = E(M) + E(h) log. What is left to do is to estimate the search costs. Let x be a real umber ad let C i deote the umber of elemets i the list L i that are ispected whe searchig for x (We do ot cout the elemet of L i at which the algorithm starts walig to the right. Hece, C i couts comparisos betwee x ad elemets of S.) The search cost is the proportioal to h i=1 (1 + C i). Agai we caot use liearity of expectatio sice h is a radom variable. Agai the tric is to fix a iteger A ad aalyze the search cost up to a level A ad above level A separately (ad differetly). We first estimate the search level above A, i.e., the total costs i the lists L A+1,L A+2,...,L h. Sice the cost is at most equal to the total size of these lists, its expected value is at most equal to the expected value of M A := h i=a+1 L i. How do we estimate this value? We first ote that the lists L i, A + 1 i h, form a sip list for S A+1. Hece we have: E(M A ) = E(M A S A+1 = ) Pr( S A+1 = ) where E(M A S A+1 = ) is the expected size of a sip list with elemets. We have already see that this is O(). Hece we oly eed to compute Pr( S A+1 = ). Sice S A+1 = if ad oly if out of the elemets of S exactly reach the level A + 1, we have: ( ) Pr( S A+1 = ) = ( 1 2 )A (1 ( 1 2 )A ). Settig p = 1 2 A, we ifer that the expected value of MA is proportioal to: ( ) ( ) p (1 p) 1 = p (1 p) 1 ( ) 1 = p p (1 p) 1 1 = p(p + (1 p)) 1 = p Hece the expected search cost above level A is bouded by O(/2 A ). Next we estimate the expected search cost i the lists L 1,L 2,...,L A. Recall that C i is the umber of elemets searched whe searchig for x. We use agai coditioal expectatio. Let l i (x) be the umber of elemets i L i that are at most equal to x. The E(C i ) = E(C i l i (x) = ) Pr(l i (x) = ). Assume that l i () =. Also assume that there is a elemet i L i that is larger tha x.
6 3006 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09:53 The C i = j if ad oly if the largest j 1 elemets of L i that are at most equal to x do ot appear i L i+1, but the elemet that immediately precedes these j 1 elemets does appear i L i+1. Hece Pr(C i = j l i (x) = ) ( 1 2 )j 1, 0 j. This iequality also holds if x is at least equal to the maximal elemet of L i. From this we obtai: E(C i l i (x) = ) = j=0 4. j Pr(C i = j l i (x) = ) j j=0 2 j 1 (exercise. Hit: write the sum j=0 j x j 1 as a derivative of j=0 x j, apply boudig) This, i tur implies that E(C i ) 4 Pr(l i (x) = ) = 4 It follows that the expected search cost up to level A is proportioal to: E( A i=1 (1 + C i )) = A (1 + E(C i )) 5A Summarizig we have show that the expected search time for elemet x is bouded by: O( 2 A + A). Settig A to log we obtai the required boud of O(log). Tail estimates: Cheroff bouds So far we proved bouds o the expected size, search time ad update time for a sip list. I this sectio we coder so called tail estimates. That is, we estimate the probability that the actual search time deviates sigificatly from its expected value. For example assume for a momet that the costat i the O(log) term for the search time is oe. The we wat to estimate the probability that the actual search time is at least t log. We could derive a estimate usig Marov s iequality. Lemma 1. Let X be a radom variable that taes o-egative values, ad let µ be the expected value of X. The for ay t > 0, Pr(X tµ) 1 t. Proof: Let s = tµ. The µ = x Pr(X = x) (3.1) x x Pr(X = x) x s s Pr(X = x) x s = s Pr(X s)
7 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09: Hece the probability that the actual search time is at least t log is less tha or equal to 1/t. This is ot very impressive. The probablity that the search time is more tha 100 times its expected value is at most 1/100. So if this bouds was tight oe search i a hudred taes more tha 100 times the time of the average search. I this sectio we will see that Cheroff bouds give a much tighter estimate. We will prove that the probablity that the search time exceeds t log is less tha or equal to t/8 for t 5. Hece i a sip list of 1000 elemets, the probability that the search time is more tha 100 times its expected value is which i practice meas, it will ever occur. (Eve for t = 50 the boud is still 10 19, ad for t = 10 the probability is still oly ). Marov s iequality holds for ay o-egative radom variable. The Cheroff techique applies to radom variables X that ca be writte as the sum i=1 X i of mutually idepedet radom variables X i. (Variables are called (mutually) idepedet if their joit desity fuctio is the product of the idividual desity fuctios. Beware that mutual idepedece is differet tha pairwise idepedece! (exercise)). I such cases much better bouds ca be obtaied. So let X 1,X 2,X 3...,X be a sequece of mutually idepedet radom variables ad let X = i=1 X i. The momet geeratig fuctio (mgf) for a (discrete) radom variable Y is defied as m Y (λ) = E(e λy ) = e λy Pr(Y = y) y As the ame suggests the fuctio is used to easily geerate the momets of the radom variable Y. Clearly m(0) = 1 ad it is easy to show that µ = m (0) ad σ 2 = m (0) µ 2 (exercise). I the case of X, which is a sum of idepedet variables, m X (λ) = i=1 m X i (λ) ad of course the mea value of X is the derivate of the mgf at positio 0 which is simply the product of all meas of the X i. Or writte dow: E(e λx ) = E(e λ(x 1+ +X ) ) = i=1 E(e λx i ). Now let s > 0 ad λ > 0. Sice X s if ad oly if e λx e λs, we have Pr(X s) = Pr(e λx e λs ). By applyig Marov s iequality to the o egative radom variable e λx, we get This yields: Pr(X s) = Pr(e λx e λs ) e λs E(e λx ). Pr(X s) e λs i=1 E(e λx i ), for s > 0 ad λ > 0. This is the basic iequality we wor with. To estimate Pr(X s) we eed boud o E(e λx i ). Of course those bouds deped o the probability distributio of X i. We will ow illustrate the techique usig the geometric distributio with parameter p = 1/2. Let T be the umber of flips we eed util a oe comes up i a series of coi flips. The Pr(T = ) = (1/2) for 1 ad E(T ) = 2. Now assume we are iterested i T which is the umber of flips we eed util we obtai a oe exactly times (i.e. T = T 1 ). If we defie the radom variables X i as the umber of flips betwee the (i 1) st (excludig) ad the i-th oe (icludig), the X i is distributed accordig to a geometric distributio. (This property is also called the memoryless property of the geometric or expoetial distributio). The T = i=1 X i, where each X i is distributed accordig to a geometric distributio ad the expected value is E(T ) = 2, ad Marov s iequality gives Pr(T (2 + t) 2 2+t ). For 0 < λ < log2 we have E(e λx i ) = e λ Pr(X i = ) = (e λ /2) e λ = 2 e λ
8 3008 Siplists: A radomized dictioary, by Kut Reiert, 31. Otober 2011, 09:53 We ow apply our basic iequality with s = (2 + t), where t > 0 ad get Pr(T (2 + t)) e λ(2+t) ( e λ 2 e λ ) = ( e λ(1+t) 2 e λ ) Now we choose λ such that the term o the right had side is miimized (exercise) ad fid λ = log(1 + t 2+t ). Hece we have Sice 1 x e x for all x, we have Pr(T (2 + t)) (1 + t/2) (1 t 2 + 2t )(1+t). (1 t 2 + 2t )1+t (e t 2+2t ) 1+t = e t/2. Moreover, 1 + t/2 e t/4 for t 3. This proves that for t 3 Pr(T (2 + t)) e t/4 e t/2 = e t/4. Compare this with the boud obtaied from Marov s iequality (which was 2 2+t )! We ca subsume our fidig i the followig theorem: Theorem 2. Let X 1,X 2,...,X be mutually idpedet radom variables ad assume that each X i is distributed accordig to a geometric distributio. Let T = X i, the E(T ) = 2 ad for ay t 3 holds: Pr(T (2 + t)) e t/4 e t/2 = e t/4. Corollary 3. Let c 1 be a costat ad let m be a positive iteger. Further let = c lm. The for ay s 5 it holds Pr(T s) m (s 2)c 4.
An Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationHashing and Amortization
Lecture Hashig ad Amortizatio Supplemetal readig i CLRS: Chapter ; Chapter 7 itro; Sectio 7.. Arrays ad Hashig Arrays are very useful. The items i a array are statically addressed, so that isertig, deletig,
More information1 Hash tables. 1.1 Implementation
Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a
More informationSkip Lists. Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015 S 3 S S 1
Presetatio for use with the textbook, Algorithm Desig ad Applicatios, by M. T. Goodrich ad R. Tamassia, Wiley, 2015 Skip Lists S 3 15 15 23 10 15 23 36 Skip Lists 1 What is a Skip List A skip list for
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationRandomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
More informationCS 330 Discussion - Probability
CS 330 Discussio - Probability March 24 2017 1 Fudametals of Probability 11 Radom Variables ad Evets A radom variable X is oe whose value is o-determiistic For example, suppose we flip a coi ad set X =
More informationLecture 2: April 3, 2013
TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 2: April 3, 203 Scribe: Shubhedu Trivedi Coi tosses cotiued We retur to the coi tossig example from the last lecture agai: Example. Give,
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationOPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES
OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES Peter M. Maurer Why Hashig is θ(). As i biary search, hashig assumes that keys are stored i a array which is idexed by a iteger. However, hashig attempts to bypass
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More information11. Hash Tables. m is not too large. Many applications require a dynamic set that supports only the directory operations INSERT, SEARCH and DELETE.
11. Hash Tables May applicatios require a dyamic set that supports oly the directory operatios INSERT, SEARCH ad DELETE. A hash table is a geeralizatio of the simpler otio of a ordiary array. Directly
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More information4.3 Growth Rates of Solutions to Recurrences
4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.
More informationCS / MCS 401 Homework 3 grader solutions
CS / MCS 401 Homework 3 grader solutios assigmet due July 6, 016 writte by Jāis Lazovskis maximum poits: 33 Some questios from CLRS. Questios marked with a asterisk were ot graded. 1 Use the defiitio of
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationDefinitions: Universe U of keys, e.g., U N 0. U very large. Set S U of keys, S = m U.
7 7 Dictioary: S.isertx): Isert a elemet x. S.deletex): Delete the elemet poited to by x. S.searchk): Retur a poiter to a elemet e with key[e] = k i S if it exists; otherwise retur ull. So far we have
More informationRademacher Complexity
EECS 598: Statistical Learig Theory, Witer 204 Topic 0 Rademacher Complexity Lecturer: Clayto Scott Scribe: Ya Deg, Kevi Moo Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved for
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationDesign and Analysis of Algorithms
Desig ad Aalysis of Algorithms Probabilistic aalysis ad Radomized algorithms Referece: CLRS Chapter 5 Topics: Hirig problem Idicatio radom variables Radomized algorithms Huo Hogwei 1 The hirig problem
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationLecture 9: Hierarchy Theorems
IAS/PCMI Summer Sessio 2000 Clay Mathematics Udergraduate Program Basic Course o Computatioal Complexity Lecture 9: Hierarchy Theorems David Mix Barrigto ad Alexis Maciel July 27, 2000 Most of this lecture
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationThe Maximum-Likelihood Decoding Performance of Error-Correcting Codes
The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,
More informationHOMEWORK 2 SOLUTIONS
HOMEWORK SOLUTIONS CSE 55 RANDOMIZED AND APPROXIMATION ALGORITHMS 1. Questio 1. a) The larger the value of k is, the smaller the expected umber of days util we get all the coupos we eed. I fact if = k
More informationAnalysis of Algorithms. Introduction. Contents
Itroductio The focus of this module is mathematical aspects of algorithms. Our mai focus is aalysis of algorithms, which meas evaluatig efficiecy of algorithms by aalytical ad mathematical methods. We
More information( ) = p and P( i = b) = q.
MATH 540 Radom Walks Part 1 A radom walk X is special stochastic process that measures the height (or value) of a particle that radomly moves upward or dowward certai fixed amouts o each uit icremet of
More informationCS284A: Representations and Algorithms in Molecular Biology
CS284A: Represetatios ad Algorithms i Molecular Biology Scribe Notes o Lectures 3 & 4: Motif Discovery via Eumeratio & Motif Represetatio Usig Positio Weight Matrix Joshua Gervi Based o presetatios by
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More informationDiscrete Mathematics and Probability Theory Fall 2016 Walrand Probability: An Overview
CS 70 Discrete Mathematics ad Probability Theory Fall 2016 Walrad Probability: A Overview Probability is a fasciatig theory. It provides a precise, clea, ad useful model of ucertaity. The successes of
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More information6.895 Essential Coding Theory October 20, Lecture 11. This lecture is focused in comparisons of the following properties/parameters of a code:
6.895 Essetial Codig Theory October 0, 004 Lecture 11 Lecturer: Madhu Suda Scribe: Aastasios Sidiropoulos 1 Overview This lecture is focused i comparisos of the followig properties/parameters of a code:
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationUC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 17 Lecturer: David Wagner April 3, Notes 17 for CS 170
UC Berkeley CS 170: Efficiet Algorithms ad Itractable Problems Hadout 17 Lecturer: David Wager April 3, 2003 Notes 17 for CS 170 1 The Lempel-Ziv algorithm There is a sese i which the Huffma codig was
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio
More informationLecture 9: Expanders Part 2, Extractors
Lecture 9: Expaders Part, Extractors Topics i Complexity Theory ad Pseudoradomess Sprig 013 Rutgers Uiversity Swastik Kopparty Scribes: Jaso Perry, Joh Kim I this lecture, we will discuss further the pseudoradomess
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationSorting Algorithms. Algorithms Kyuseok Shim SoEECS, SNU.
Sortig Algorithms Algorithms Kyuseo Shim SoEECS, SNU. Desigig Algorithms Icremetal approaches Divide-ad-Coquer approaches Dyamic programmig approaches Greedy approaches Radomized approaches You are ot
More information19.1 The dictionary problem
CS125 Lecture 19 Fall 2016 19.1 The dictioary proble Cosider the followig data structural proble, usually called the dictioary proble. We have a set of ites. Each ite is a (key, value pair. Keys are i
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationFinal Review for MATH 3510
Fial Review for MATH 50 Calculatio 5 Give a fairly simple probability mass fuctio or probability desity fuctio of a radom variable, you should be able to compute the expected value ad variace of the variable
More informationLecture 11: Pseudorandom functions
COM S 6830 Cryptography Oct 1, 2009 Istructor: Rafael Pass 1 Recap Lecture 11: Pseudoradom fuctios Scribe: Stefao Ermo Defiitio 1 (Ge, Ec, Dec) is a sigle message secure ecryptio scheme if for all uppt
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationIP Reference guide for integer programming formulations.
IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 19
CS 70 Discrete Mathematics ad Probability Theory Sprig 2016 Rao ad Walrad Note 19 Some Importat Distributios Recall our basic probabilistic experimet of tossig a biased coi times. This is a very simple
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMath 216A Notes, Week 5
Math 6A Notes, Week 5 Scribe: Ayastassia Sebolt Disclaimer: These otes are ot early as polished (ad quite possibly ot early as correct) as a published paper. Please use them at your ow risk.. Thresholds
More informationRandom Models. Tusheng Zhang. February 14, 2013
Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationCS161: Algorithm Design and Analysis Handout #10 Stanford University Wednesday, 10 February 2016
CS161: Algorithm Desig ad Aalysis Hadout #10 Staford Uiversity Wedesday, 10 February 2016 Lecture #11: Wedesday, 10 February 2016 Topics: Example midterm problems ad solutios from a log time ago Sprig
More informationPosted-Price, Sealed-Bid Auctions
Posted-Price, Sealed-Bid Auctios Professors Greewald ad Oyakawa 207-02-08 We itroduce the posted-price, sealed-bid auctio. This auctio format itroduces the idea of approximatios. We describe how well this
More informationECE 6980 An Algorithmic and Information-Theoretic Toolbox for Massive Data
ECE 6980 A Algorithmic ad Iformatio-Theoretic Toolbo for Massive Data Istructor: Jayadev Acharya Lecture # Scribe: Huayu Zhag 8th August, 017 1 Recap X =, ε is a accuracy parameter, ad δ is a error parameter.
More informationShannon s noiseless coding theorem
18.310 lecture otes May 4, 2015 Shao s oiseless codig theorem Lecturer: Michel Goemas I these otes we discuss Shao s oiseless codig theorem, which is oe of the foudig results of the field of iformatio
More information7.7 Hashing. 7.7 Hashing. Perfect Hashing. Direct Addressing
Dictioary: S.isertx): Isert a elemet x. S.deletex): Delete the elemet poited to by x. S.searchk): Retur a poiter to a elemet e with key[e] = k i S if it exists; otherwise retur ull. So far we have implemeted
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationSequences, Mathematical Induction, and Recursion. CSE 2353 Discrete Computational Structures Spring 2018
CSE 353 Discrete Computatioal Structures Sprig 08 Sequeces, Mathematical Iductio, ad Recursio (Chapter 5, Epp) Note: some course slides adopted from publisher-provided material Overview May mathematical
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationLecture 2 Long paths in random graphs
Lecture Log paths i radom graphs 1 Itroductio I this lecture we treat the appearace of log paths ad cycles i sparse radom graphs. will wor with the probability space G(, p) of biomial radom graphs, aalogous
More informationClassification of problem & problem solving strategies. classification of time complexities (linear, logarithmic etc)
Classificatio of problem & problem solvig strategies classificatio of time complexities (liear, arithmic etc) Problem subdivisio Divide ad Coquer strategy. Asymptotic otatios, lower boud ad upper boud:
More informationLecture 5: April 17, 2013
TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 5: April 7, 203 Scribe: Somaye Hashemifar Cheroff bouds recap We recall the Cheroff/Hoeffdig bouds we derived i the last lecture idepedet
More information1 Statement of the Game
ANALYSIS OF THE CHOW-ROBBINS GAME JON LU May 10, 2016 Abstract Flip a coi repeatedly ad stop wheever you wat. Your payoff is the proportio of heads ad you wish to maximize this payoff i expectatio. I this
More informationIntro to Learning Theory
Lecture 1, October 18, 2016 Itro to Learig Theory Ruth Urer 1 Machie Learig ad Learig Theory Comig soo 2 Formal Framework 21 Basic otios I our formal model for machie learig, the istaces to be classified
More informationDisjoint set (Union-Find)
CS124 Lecture 7 Fall 2018 Disjoit set (Uio-Fid) For Kruskal s algorithm for the miimum spaig tree problem, we foud that we eeded a data structure for maitaiig a collectio of disjoit sets. That is, we eed
More informationLecture 4 February 16, 2016
MIT 6.854/18.415: Advaced Algorithms Sprig 16 Prof. Akur Moitra Lecture 4 February 16, 16 Scribe: Be Eysebach, Devi Neal 1 Last Time Cosistet Hashig - hash fuctios that evolve well Radom Trees - routig
More informationTHE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS
THE ASYMPTOTIC COMPLEXITY OF MATRIX REDUCTION OVER FINITE FIELDS DEMETRES CHRISTOFIDES Abstract. Cosider a ivertible matrix over some field. The Gauss-Jorda elimiatio reduces this matrix to the idetity
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 15
CS 70 Discrete Mathematics ad Probability Theory Summer 2014 James Cook Note 15 Some Importat Distributios I this ote we will itroduce three importat probability distributios that are widely used to model
More informationRecursive Algorithm for Generating Partitions of an Integer. 1 Preliminary
Recursive Algorithm for Geeratig Partitios of a Iteger Sug-Hyuk Cha Computer Sciece Departmet, Pace Uiversity 1 Pace Plaza, New York, NY 10038 USA scha@pace.edu Abstract. This article first reviews the
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationThe Random Walk For Dummies
The Radom Walk For Dummies Richard A Mote Abstract We look at the priciples goverig the oe-dimesioal discrete radom walk First we review five basic cocepts of probability theory The we cosider the Beroulli
More informationf X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36
Probability Distributios A Example With Dice If X is a radom variable o sample space S, the the probablity that X takes o the value c is Similarly, Pr(X = c) = Pr({s S X(s) = c} Pr(X c) = Pr({s S X(s)
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationSolutions to Tutorial 5 (Week 6)
The Uiversity of Sydey School of Mathematics ad Statistics Solutios to Tutorial 5 (Wee 6 MATH2962: Real ad Complex Aalysis (Advaced Semester, 207 Web Page: http://www.maths.usyd.edu.au/u/ug/im/math2962/
More informationLinear Regression Demystified
Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to
More informationLONG SNAKES IN POWERS OF THE COMPLETE GRAPH WITH AN ODD NUMBER OF VERTICES
J Lodo Math Soc (2 50, (1994, 465 476 LONG SNAKES IN POWERS OF THE COMPLETE GRAPH WITH AN ODD NUMBER OF VERTICES Jerzy Wojciechowski Abstract I [5] Abbott ad Katchalski ask if there exists a costat c >
More informationLecture 7: October 18, 2017
Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More information