CS166 Handout 02 Spring 2018 April 3, 2018 Mathematical Terms and Identities

Transcription

1 CS166 Hadout 02 Sprig 2018 April 3, 2018 Mathematical Terms ad Idetities Thaks to Ady Nguye ad Julie Tibshirai for their advice o this hadout. This hadout covers mathematical otatio ad idetities that may be useful over the course of CS166. Feel free to refer to this hadout for referece o a variety of topics. If you have ay suggestios o how to improve this hadout, please let us kow! Set Theory The set N cosists of all atural umbers. That is, N = { 0, 1, 2, 3, } The set Z cosists of all itegers: Z = {, -3, -2, -1, 0, 1, 2, 3, } The set R cosists of all real umbers. The set Ø is the empty set cosistig of o elemets. If x belogs to set S, we write x S. If x does ot belog to S, we write x S. The uio of two sets S₁ ad S₂ is deoted S₁ S₂. Their itersectio is deoted S₁ S₂, differece is deoted S₁ S₂ or S₁ \ S₂, ad symmetric differece is deoted S₁ Δ S₂. If S₁ is a subset of S₂, we write S₁ S₂. If S₁ is a strict subset of S₂, we deote this by S₁ S₂. The power set of a set S (deoted (S)) is the set of all subsets of S. The Cartesia product of two sets S₁ ad S₂ is the set S₁ S₂ = { (a, b) a S₁ ad b S₂ } First-Order Logic The egatios of the basic propositioal coectives are as follows: ( p) p (p q) p q (p q) p q (p q) p q (p q) p q The egatios of the ad quatifers are as follows: x. φ x. φ x. φ x. φ The statemet iff abbreviates if ad oly if.

2 2 / 7 Summatios The sum of the frst atural umbers ( ) is give by 1 i= ( 1) 2 The sum of the frst terms of the arithmetic series a, a + b, a + 2b,, a + ( 1)b is 1 1 (a+i b)=a i =0 1 1+b i=a+ b( 1) 2 The sum of the frst terms of the geometric series 1, r, r 2, r 3,, r -1 is give by As a useful special case, whe r = 2, we have 1 r i = r 1 r i =2 1 I the case that r < 1, the sum of all ifite terms of the geometric series is give by r i = 1 1 r The followig summatio ofte arises i the aalysis of algorithms: whe r < 1, we have i r i = r (1 r) 2 Iequalities The followig idetities are useful for maipulatig iequalities: If A B ad B C, the A C If A B ad C 0, the CA CB If A B ad C 0, the CA CB If A B ad C D, the A + C B + D If A, B Z, the A B iff A < B + 1 If f is ay mootoically icreasig fuctio ad A B, the f(a) f(b) If f is ay mootoically decreasig fuctio ad A B, the f(a) f(b) The followig iequalities are ofte useful i algorithmic aalysis: x 1 x 2... x e x 1 + x x 1 + x x

3 3 / 7 Floors ad Ceiligs The floor fuuctio x deotes the largest iteger less tha or equal to x. The ceilig fuuctio x deotes the smallest iteger greater tha or equal to x. These fuctios obey the rules x x < x + 1 ad x Z x 1 < x x ad x Z Additioally, x + = x + ad x + = x + for ay Z. Asymptotic Notatio Let f, g : N N. The f() = O(g()) if ₀ N. c R. N. ( ₀ f() cg()) f() = Ω(g()) if ₀ N. c > 0 R. N. ( ₀ f() cg()) f() = Θ(g()) if f() = O(g()) f() = Ω(g()) Whe multiple variables are ivolved i a expressio, big-o otatio geeralizes as follows: we say that f(x₁,, xₙ) = O(g(x₁,, xₙ)) if there are costats N ad c such that for ay x₁ N, x₂ N,, xₙ N, we have f(x₁,, xₙ) c g(x₁,, xₙ). The followig rules apply for O otatio: If f() = O(g()) ad g() = O(h()), the f() = O(h()) (also Ω, Θ, o, ω) If f₁() = O(g()) ad f₂() = O(g()), the f₁() + f₂() = O(g()) (also Ω, Θ, o, ω) If f₁() = O(g₁()) ad f₂() = O(g₂()), the f₁()f₂() = O(g₁()g₂()) (also Ω, Θ, o, ω) We ca use o ad ω otatios to deote strict bouds o growth rates: f ()=o( g()) if lim f () g() =0 Polyomials, expoets, ad logarithms are related as follows: f ()=ω( g()) if lim log a = Θ(log b ) for ay fxed costats a, b > 1 f () g() = Ay polyomial of degree k with positive leadig coefciet is Θ( k ) log b = o( k ) for ay k > 0 k = o(b ) for ay b > 1 b = o(c ) for ay 1 < b < c I a graph, deotes the umber of odes ( V ) ad m deotes the umber of edges ( E ). I ay graph, m = O( 2 ). I a dese graph, m = Θ( 2 ); a sparse graph is oe where m = o( 2 ).

4 4 / 7 The Master Theorem If a, b, ad d are costats, the the recurrece relatio T() = at( / b) + O( d ) solves as follows: T()={O( d ) O( d log ) O( log ba ) if log b a < d if log b a = d if log b a > d Logarithms ad Expoets Logarithms ad expoets are iverses of oe aother: b log b x =log b b x =x The chage-ofu-base fuormula for logarithms states that log b a= log c a log c b Sums ad differeces of logarithms traslate ito logarithms of products ad quotiets: The power rule for logarithms states log b xy=log b x+log b y log b ( x/ y)=log b x log b y I some cases, expoets may be iterchaged: log b x y = y log b x (a b ) c =a bc =(a c ) b We ca chage the base of a expoet usig the fact that logarithms ad expoets are iverses: a c =b clog ba

5 5 / 7 Probability If E₁ ad E₂ are mutually exclusive evets, the P(E₁) + P(E₂) = P(E₁ E₂) For ay evets E₁, E₂, E₃,, icludig overlappig evets, the uio boud states that P ( i =1 E i ) P (E i ) i=1 The probability of E give F is deoted P(E F) ad is give by The chai rule for coditioal probability is P (E F )= P (E F ) P (F ) P(E ₙ Eₙ -₁ E₁) = P(E E -1 E₁) P( E -1 E -2 E₁) P(E₁) Two evets E₁ ad E₂ are called idepedet if P(E₁ E₂) = P(E₁) P(E₂) For ay evet E, the complemet of that evet (deoted E) represets the evet that E does ot occur. E ad E are mutually exclusive, ad Expected Value P(E) + P(E) = 1 The expected value of a discrete radom variable X is defed as E [ X ]= (i P ( X =i)) The expected value operator is liear: for ay a, b R ad ay radom variable X: E[aX + b] = ae[x] + b More geerally, if X₁, X₂, X₃, Xₙ are ay radom variables, the E [ i =1 If X ad Y are idepedet radom variables, the X i ]= E[ X i ] i=1 E[XY] = E[X]E[Y]

6 6 / 7 Variace ad Covariace The variace of a radom variable X is defed as Equivaletly: Var[X] = E[(X - E[X]) 2 ] Var[X] = E[X 2 ] - E[X] 2 Give two radom variables X ad Y, the covariace of X ad Y is defed as Equivaletly: Accordigly: Variace is ot a liear operator: Cov[X, Y] = E[(X E[X])(Y E[Y])] Cov[X, Y] = E[XY] E[X]E[Y] Var[X] = Cov[X, X] Var[aX + by] = a 2 Var[X] + 2ab Cov[X, Y] + b 2 Var[Y] The variace of a summatio of radom variables, icludig depedet variables, ca be simplifed usig the followig rule: Var [ i=1 X i ]= Var [ X i ]+ Cov [ X i, X j ] i =1 i j If X ad Y are idepedet radom variables, the Cov[X, Y] = 0. However, if Cov[X, Y] = 0, it is ot ecessarily the case that X ad Y are idepedet. Cocetratio Iequalities Markov's iequality says that for ay oegative radom variable X with fite expected value ad ay c > 0, we have both P (X c E [X ]) 1 c ad P (X c ) E[ X ] c Chebyshev's iequality states that for ay radom variable X with fite expected value that P ( X E[ X ] c Var [X ] ) 1 c 2 ad P ( X E[X ] c ) The Cheroff boud says that if X ~ Biom(, p) for p < ½, that P(X 2 (1/ 2 p) 2 ) e 2 p I the case where p is a fxed costat, otice that the right-had side is e -O(1).. Var [X ] c 2.

7 Useful Probability Equalities ad Iequalities A idicator radom variable is a radom variable X where X ={ 1 if evet F occurs 0 otherwise For ay idicator variable, E[X] = P(F). Idicator variables are Beroulli radom variables, so if X is a idicator variable, the Var[X] = P(F)(1 P(F)). If X₁, X₂,, Xₙ are radom variables, the P (max{ X 1, X 2,..., X } k)=p ( X 1 k X 2 k... X k) P (mi {X 1, X 2,..., X } k)=p ( X 1 k X 2 k... X k) O expectatio, repeatedly fippig a biased coi that comes up heads with probability p requires 1 / p trials before the coi will come up heads. 7 / 7 Harmoic Numbers The th harmoic umber, deoted Hₙ, is give by H = i =1 The harmoic umbers are close i value to l : for ay 1, we have so H ₙ = Θ(log ) 1 i l ( + 1) Hₙ l + 1,