An Introduction to Randomized Algorithms
|
|
- Jared Russell
- 5 years ago
- Views:
Transcription
1 A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis of radomized algorithms. For a quick overview of probability ad terms associated with it, the reader is advised to see the Appedix. 1 Radomized QuickSort I this sectio, we preset the classic quick sort algorithm ad compute the expected ruig time of the algorithm. We assume that the elemets of the set are all distict. Below is the radomized quick sort algorithm. Algorithm RadQuickSort(S) Choose a pivot elemet x i u.a.r from S = {x 1, x 2,, x } Split the set S ito two subsets S 1 = {x j x j < x i } ad S 2 = {x j x j > x i } by comparig each x j with the chose x i Recurse o sets S 1 ad S 2 Output the sorted set S 1 the x i ad the sorted S 2. ed Algorithm Aalysis Let T() be the umber of steps take by the RadQuickSort algorithm o a set of size. Note that the maximum value of T() occurs whe the pivot elemet x i is the largest/smallest elemet of the remaiig set durig each recursive call of the algorithm. I this case, T() = + ( 1) = O( 2 ). This value of T() is reached with a very low probability of = 1!. Also, the best case occurs whe the pivot elemet splits the set S ito two equal sized subsets ad the T() = O( l ). This implies that T() has a distributio betwee O( l ) ad O( 2 ). Now we derive the expected value of T(). Note that if the i th smallest elemet is chose as the pivot elemet the S 1 ad S 2 will be of sizes i 1 ad i 1 respectively ad this choice has a probability of 1. The recurrece relatio for T() is: T() = + T(X) + T( 1 X) (1) where, Pr[X = i] = 1 for i {0, 1,, 1}. Hece, Pr[ 1 X = i] = Pr[X = 1 i] = 1. However, it is icorrect to write T( 1 X) = T(X). Takig expectatios o both sides of (1), E[T()] = E[T(i)] j=1 E[T(j)] = E[T(i)]. Let f(i) = E[T(i)]. The, f() = f(i). Simplifyig, Substitutig 1 for i (2), f() = 2 + 2(f(1) + f(2) + + f( 1)) (2) ( 1)f( 1) = ( 1) 2 + 2(f(1) + f(2) + + f( 2)) (3) Subtractig (3) from (2), we get f() ( 1)f( 1) = (2 1)+2f( 1) or f() = f( 1)+. We prove by iductio that f() 2 l. 1
2 1.0.1 Claim: f() 2 l Proof: Iductio basis: = 1, the claim holds. Let the claim hold for all values up to 1. The, f() = f( 1) ( 1)l( 1) + = 2(2 1) = 2(2 1) l( 1) (l + l( )) + We make use of the stadard iequality stated below. by iductio hypothesis Fact x e x for x R. For example, e 0.2 ad e 0.2. f() 2(2 1) (l 1 ) = 2 l 2 l l, establishig the iductive step. Hece, the expected ruig time of the radomized quick sort algorithm is O( l ). But oe of the limitatios of the recurrece relatio approach is that we do ot how the ruig time of the algorithm is spread aroud its expected value. Ca this aalysis be exteded to aswer questios such as, with what probability does the algorithm RadQuickSort eeds more tha 12 l time steps? Later o, we apply a differet techique ad establish that this probability is very small. Similarly, for the case of the Maximum algorithm, by solvig the recurrece relatio for the expected ruig time, we will ot be able to aswer questios of the form, what is the probability that the ru time is greater tha 3? Aswers to such questios give how close the radom variable is spread aroud its expected value or how varied the distributio is. To be able to aswer such queries, we study Tail iequalities i the ext sectio. 2 Tail Iequalities I this sectio, we study three ways to estimate the tail probabilities of radom variables. It will be oted that, the more iformatio we kow about the radom variable the better the estimate we ca derive about a give tail probability. 2.1 Markov Iequality Theorem 2.1 Markov Iequality If X is a o-egative valued radom variable with a expectatio of µ, the Pr[X cµ] 1 c. Proof. By defiitio, µ = a apr[x = a] = a<cµ apr[x = a] + a cµ apr[x = a] 0 + a cµ cµpr[x = a] as X is o-egative valued = cµ a cµ Pr[X = a] = cµpr[x cµ] Hece, Pr[X cµ] µ cµ = 1 c. The kowledge of the stadard deviatio of the radom variable X would most ofte give a better boud. 2.2 Chebychev Iequality We first defie the terms stadard deviatio ad variace of a radom variable X. Defiitio 2.2 Let X be a radom variable with a expectatio of µ. The variace of X, deoted by var(x), is defied as var(x) = E[(X µ) 2 ]. The stadard deviatio of X, deoted by σ X, is defied as σ X = var(x). Note that by defiitio, var(x) = E[(X µ) 2 ] = E[X 2 2Xµ+µ 2 ] = E[X 2 ] µ 2. The secod equality follows from the liearity of expectatios. 2
3 Theorem 2.3 Chebychev Iequality Let X be a radom variable with expectatio µ X ad stadard deviatio σ X. The, Pr[ X µ X cσ X ] 1 c 2. Proof. Let radom variable Y = (X µ X ) 2. The, E[Y ] = E[(X µ X ) 2 ] = σx 2 by defiitio ad also Y is a o-egative valued radom variable. Now, Pr[ X µ X cσ X ] = Pr[(X µ X ) 2 c 2 σx 2 ] = Pr[Y c2 σx 2 ]. Applyig Markov Iequality to the radom variable Y, Pr[Y c 2 σx 2 ] = Pr[Y c2 µ Y ] 1 c. Hece the 2 theorem. 2.3 Cheroff Bouds The tail estimates give by Theorem 2.1, Theorem 2.3 work for radom variables i geeral. But if the radom variable X ca be expressed as a sum of idepedet radom variables each of which is 0, 1 valued, the we ca obtai very tight bouds o the tail estimates. This is expressed i the followig theorem ad the bouds are commoly called as Cheroff Bouds. Theorem 2.4 Cheroff Bouds: Let X be a radom variable defied as X = X 1 + X X where each X i, 1 i, is a 0, 1 valued radom variable ad all X i s ( are idepedet. ) Also, let E[X] = µ ad Pr[X i = 1] = p i, 1 i. The for ay µ. e δ > 0, Pr[X µ(1 + δ)] δ (1+δ) 1+δ Proof. The ormal strategy employed to prove tail estimates of sums of idepedet radom variables is to make use of expoetial momets. While provig Chebychev iequality (Theorem 2.3), we made use of secod-order momet. It ca be observed that usig higher order momets would geerally improve the boud o the tail iequality. But usig expoetial momets would result i a vast improvemet i the boud as we shall see later i some examples. Observe that, µ = p i (4) by defiitio of X ad usig liearity of expectatios. Let Y i = e txi for some parameter t to be chose later. Note that Y i s are also idepedet as X i s are. Defie the radom variable Y = Y 1 Y 2 Y. The, E[Y i ] = E[e txi ] = p i e t + (1 p i )e 0 = 1 p i + p i e t (5) E[Y ] = E[Y 1 Y 2 Y ] = Π E[Y i] = Π (1 p i + p i e t ) (6) where the secod equality follows from idepedece of Y i s ad the last equality follows from (5). Now, Pr[X µ(1 + δ)] = Pr[Y e tµ(1+δ) ] E[Y ] e tµ(1+δ) = Π (1 p i + p i e t ) e tµ(1+δ) (7) by usig Markov iequality from Theorem 2.1 ad equatio (6). We ow make use of the iequality from Fact 1.1 i equatio (7) which the reduces to, Pr[X µ(1 + δ)] = Π e pi(1 et ) e tµ(1+δ) = e µ(1 e t ) = e µ[(e t 1) t(1+δ)] e tµ(1+δ) where the secod equality follows from equatio (4). I (8), we ca choose a value of t that miimizes the probability estimate. To fid the miimum let f(t) = le µ[(et 1) t(1+δ)] = µ(1 e t ) tµ(1 + δ). Differetiatig f(t) with respect to t ad equatig it to zero gives us µe t µ(1 + δ) = 0 or t = l(1 + δ). Usig this value of t i (8), That completes the proof. Pr[X µ(1 + δ)] e µ(1 (1+δ)) (1 + δ) = e µδ ( µ(1+δ) (1 + δ) = e δ ) µ. (9) (1+δ) (1 + δ) (1+δ) But the form of the iequality i equatio (9) is ot very coveiet to hadle. I additio this form is hard to ivert, i.e. give the probability boud, choose a appropriate δ. Istead, we use the followig form most ofte. (8) 3
4 Theorem 2.5 Let X be defied as i Theorem 2.4. The, 2.4 Applicatio of Tail Iequalities Pr[X (1 + δ)µ] We cosider the use of tail iequalities to two problems Balls ad Bis { e µδ 2 /4 if δ 1 e µδ l δ if δ > 1 Cosider throwig balls idepedetly ad uiformly at radom ito bis. We are iterested i the probability that bi 1 cotais more tha 4 balls. Defie 0, 1 valued radom variables X i, 1 i, defied as X i = 1 if ball i falls ito bi 1 ad 0 otherwise. By uiformity, Pr[X i = 1] = 1. Defie the radom variable X = X 1 + X 2 + X. Thus X deotes the umber of balls that fall i bi 1. By liearity of expectatios, E[X] = E[ X i] = E[X i] = 1 = 1. Usig Markov iequality from Theorem 2.1, we get Pr[X 4] 1 4 (10) Before usig Chebychev iequality (Theorem 2.3), we first compute the stadard deviatio of X as follows. var(x i ) = E[X 2 i ] E[X i ] 2 = (11) var(x) = var(x 1 ) = ( ) = 1 1 where the first equality i (12) follows from the idepedece of X i s. Hece, σ X = 1 1 (12) (13) Applyig Chebychev iequality to the radom variable X, Pr[X 4] = Pr[X 1 3] Pr[ X 1 3] ( ) 2 = (14) Usig Cheroff bouds from Theorem 2.4, Pr[X 4] = Pr[X (1 + 3)1] e 1 3l 3 = 1 27 where i we used the simpler form of Cheroff bouds as stated i Theorem 2.5. Now comparig equatios (10), (14) ad (15) we ca see that usig Chebychev iequality gives a better boud tha that of Markov iequality ad a much better boud is obtaied usig Cheroff bouds. As aother example, let us cosider the probability that bi 1 has more tha l balls. Usig Markov iequality, we get Pr[X l]. Usig Chebychev iequality we get that Pr[X l] Pr[ X 1 10 l] l l 100l 2. Whereas, usig Cheroff bouds, we ca get Pr[X l] e 10l l (10l ) = 10l 10 l 1. With a tighter aalysis usig Cheroff bouds it ca be show that for the case of balls ad 10 bis, the probability that bi 1 has more tha c l l l for some costat c is very small, say 1. 2 (15) 4
5 2.4.2 l balls ad bis I this sceario we have l balls ad bis where balls are throw ito the bis idepedetly ad uiformly at radom. Defie the radom variables X i, 1 i l ad X as above. We have Pr[X i = 1] = 1/ ad E[X] = l. Let us estimate the probability that bi 1 has more tha 10 l balls. Usig Markov iequality, we get that Pr[X 10 l] 1/10. Istead, usig Cheroff bouds, we get Pr[X 10 l] = Pr[X (1 + 9)l] e 9l l 9 1/ 20 which is expoetially small. I geeral, it ca be observed that if the expectatio of the radom variable is small the we pay a higher pealty to derive a w.h.p boud. A Probability Theory We start by defiig probability ad the itroduce some well-kow iequalities that we ofte use. Let Ω be a arbitrary set, called the sample space. We start by defiig a σ field, also sometimes called a σ algebra. Defiitio A.1 (σ field) A collectio F of subsets of Ω is called a σ field if it satisfies: 1. Ω F 2. A F implies A c F, ad 3. For ay coutable sequece A 1, A 2,..., if A 1, A 2,... F the A 1 A 2... F. Defiitio A.2 A set fuctio Pr o a σ field F of subsets of Ω such that Pr : F [0, 1] is called a probability measure if it satisfies: 1. 0 Pr(A) 1, A F. 2. Pr(Ω) = 1, ad 3. If A 1, A 2,... is a disjoit sequece of sets i F the ( ) Pr A i = Pr(A i ) The triad (Ω, F, Pr) is ofte called a probability space. For equivalet ad alterate defiitios, examples, ad a more complete itroductio, we refer the reader to stadard textbooks o probability [?]. I the followig, if o probability space is metioed the ay space (Ω, F, Pr) ca be take. We ofte use the followig iequality called Boole s iequality which is part of a geeral Boole-Boferroi iequalities [?] ad this is also sometimes referred to as the uio boud as it provides a boud o the probability of a uio of evets. This iequality is also referred to as the (fiite) sub-additivity property of the probability measure. Propositio A.3 (Boole s iequality) For ay arbitrary evets A 1, A 2,...A, ( ) Pr A i Pr(A i ) The otio of idepedece is a importat cocept i the study of probability. Defiitio A.4 (Idepedece) A collectio of evets {A i : i I} is said to be idepedet if for all S I, Pr( i S A i ) = Π i S Pr(A i ). We ow defie radom variable, which is ay measurable fuctio from Ω to IR. Let R deote the stadard Borel σ field associated with IR, which is the σ field geerated by left-ope itervals of IR [?]. Defiitio A.5 (Radom Variable) Give a probability space (Ω, F, Pr), a mappig X : Ω IR is called a radom variable if it satisfies the coditio that X 1 (R) F for every R R. 5
6 We represet as {X x} as the set {ω Ω X(ω) x} for x IR ad also write Pr(X x) as the probability of the above evet. Similar defiitio ca be made for represetig the set {ω Ω X(ω = x} as {X = x}. The otio of idepedece also exteds to radom variables. Two radom variables X ad Y are said to be idepedet if the evets {X x} ad {Y y} are idepedet for x, y R. The defiitio exteds to multiple radom variables just as i Defiitio A.4. Associated with ay radom variable is a distributio fuctio defied as follows. Defiitio A.6 (Distributio fuctio) The distributio fuctio F : IR [0, 1] for a radom variable X is defied as F X (x) = Pr(X x). A radom variable X is said to be a discrete radom variable if the rage of X is a fiite or coutably ifiite subset of IR. For discrete radom variables, the followig defiitio ca be provided for the desity of a radom variable. Defiitio A.7 (Desity) Give a radom variable X, the desity fuctio f X : IR [0, 1] of X is defied as f X (x) = Pr(X = x). The above defiitio ca be exteded to all types of radom variables also with proper care. I the rest of this sectio, we focus o discrete radom variables oly ad hece the defiitios are made for the case of discrete radom variables. With proper care, the defiitios however ca be exteded [?]. A importat quatity of iterest of a radom variable is its expectatio. Defiitio A.8 (Expectatio) Give a probability space (Ω, F, Pr) ad a radom variable X, the expectatio of X, deoted E[X], is defied as E[X] = x IRxPr[X = x] with the covetio that 0 = 0 = 0. 6
Convergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationLecture 6: Coupon Collector s problem
Radomized Algorithms Lecture 6: Coupo Collector s problem Sotiris Nikoletseas Professor CEID - ETY Course 2017-2018 Sotiris Nikoletseas, Professor Radomized Algorithms - Lecture 6 1 / 16 Variace: key features
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationLecture 2: April 3, 2013
TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 2: April 3, 203 Scribe: Shubhedu Trivedi Coi tosses cotiued We retur to the coi tossig example from the last lecture agai: Example. Give,
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationEmpirical Process Theory and Oracle Inequalities
Stat 928: Statistical Learig Theory Lecture: 10 Empirical Process Theory ad Oracle Iequalities Istructor: Sham Kakade 1 Risk vs Risk See Lecture 0 for a discussio o termiology. 2 The Uio Boud / Boferoi
More informationCS 330 Discussion - Probability
CS 330 Discussio - Probability March 24 2017 1 Fudametals of Probability 11 Radom Variables ad Evets A radom variable X is oe whose value is o-determiistic For example, suppose we flip a coi ad set X =
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationRandomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationMassachusetts Institute of Technology
Solutios to Quiz : Sprig 006 Problem : Each of the followig statemets is either True or False. There will be o partial credit give for the True False questios, thus ay explaatios will ot be graded. Please
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationMath 216A Notes, Week 5
Math 6A Notes, Week 5 Scribe: Ayastassia Sebolt Disclaimer: These otes are ot early as polished (ad quite possibly ot early as correct) as a published paper. Please use them at your ow risk.. Thresholds
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 19
CS 70 Discrete Mathematics ad Probability Theory Sprig 2016 Rao ad Walrad Note 19 Some Importat Distributios Recall our basic probabilistic experimet of tossig a biased coi times. This is a very simple
More informationLecture 15: Learning Theory: Concentration Inequalities
STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 5: Learig Theory: Cocetratio Iequalities Istructor: Ye-Chi Che 5. Itroductio Recall that i the lecture o classificatio, we have see that
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationDiscrete Mathematics and Probability Theory Spring 2012 Alistair Sinclair Note 15
CS 70 Discrete Mathematics ad Probability Theory Sprig 2012 Alistair Siclair Note 15 Some Importat Distributios The first importat distributio we leared about i the last Lecture Note is the biomial distributio
More informationChapter 0. Review of set theory. 0.1 Sets
Chapter 0 Review of set theory Set theory plays a cetral role i the theory of probability. Thus, we will ope this course with a quick review of those otios of set theory which will be used repeatedly.
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 15
CS 70 Discrete Mathematics ad Probability Theory Summer 2014 James Cook Note 15 Some Importat Distributios I this ote we will itroduce three importat probability distributios that are widely used to model
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS Cotets 1. A few useful discrete radom variables 2. Joit, margial, ad
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationCS / MCS 401 Homework 3 grader solutions
CS / MCS 401 Homework 3 grader solutios assigmet due July 6, 016 writte by Jāis Lazovskis maximum poits: 33 Some questios from CLRS. Questios marked with a asterisk were ot graded. 1 Use the defiitio of
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationCS 171 Lecture Outline October 09, 2008
CS 171 Lecture Outlie October 09, 2008 The followig theorem comes very hady whe calculatig the expectatio of a radom variable that takes o o-egative iteger values. Theorem: Let Y be a radom variable that
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationLecture 2 February 8, 2016
MIT 6.854/8.45: Advaced Algorithms Sprig 206 Prof. Akur Moitra Lecture 2 February 8, 206 Scribe: Calvi Huag, Lih V. Nguye I this lecture, we aalyze the problem of schedulig equal size tasks arrivig olie
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationIntroduction to Probability. Ariel Yadin. Lecture 2
Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take
More informationFinal Review for MATH 3510
Fial Review for MATH 50 Calculatio 5 Give a fairly simple probability mass fuctio or probability desity fuctio of a radom variable, you should be able to compute the expected value ad variace of the variable
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More information1 Hash tables. 1.1 Implementation
Lecture 8 Hash Tables, Uiversal Hash Fuctios, Balls ad Bis Scribes: Luke Johsto, Moses Charikar, G. Valiat Date: Oct 18, 2017 Adapted From Virgiia Williams lecture otes 1 Hash tables A hash table is a
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationLecture 4: April 10, 2013
TTIC/CMSC 1150 Mathematical Toolkit Sprig 01 Madhur Tulsiai Lecture 4: April 10, 01 Scribe: Haris Agelidakis 1 Chebyshev s Iequality recap I the previous lecture, we used Chebyshev s iequality to get a
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationf X (12) = Pr(X = 12) = Pr({(6, 6)}) = 1/36
Probability Distributios A Example With Dice If X is a radom variable o sample space S, the the probablity that X takes o the value c is Similarly, Pr(X = c) = Pr({s S X(s) = c} Pr(X c) = Pr({s S X(s)
More informationDisjoint Systems. Abstract
Disjoit Systems Noga Alo ad Bey Sudaov Departmet of Mathematics Raymod ad Beverly Sacler Faculty of Exact Scieces Tel Aviv Uiversity, Tel Aviv, Israel Abstract A disjoit system of type (,,, ) is a collectio
More informationLecture 2 Long paths in random graphs
Lecture Log paths i radom graphs 1 Itroductio I this lecture we treat the appearace of log paths ad cycles i sparse radom graphs. will wor with the probability space G(, p) of biomial radom graphs, aalogous
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationHere are some examples of algebras: F α = A(G). Also, if A, B A(G) then A, B F α. F α = A(G). In other words, A(G)
MATH 529 Probability Axioms Here we shall use the geeral axioms of a probability measure to derive several importat results ivolvig probabilities of uios ad itersectios. Some more advaced results will
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationBHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13
BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationTopic 5 [434 marks] (i) Find the range of values of n for which. (ii) Write down the value of x dx in terms of n, when it does exist.
Topic 5 [44 marks] 1a (i) Fid the rage of values of for which eists 1 Write dow the value of i terms of 1, whe it does eist Fid the solutio to the differetial equatio 1b give that y = 1 whe = π (cos si
More informationLinear Regression Demystified
Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationVariance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom
Variace of Discrete Radom Variables Class 5, 18.05 Jeremy Orloff ad Joatha Bloom 1 Learig Goals 1. Be able to compute the variace ad stadard deviatio of a radom variable.. Uderstad that stadard deviatio
More informationAgnostic Learning and Concentration Inequalities
ECE901 Sprig 2004 Statistical Regularizatio ad Learig Theory Lecture: 7 Agostic Learig ad Cocetratio Iequalities Lecturer: Rob Nowak Scribe: Aravid Kailas 1 Itroductio 1.1 Motivatio I the last lecture
More information