CS161 Hadout 05 Summer 2013 July 10, 2013 Mathematical Terms ad Idetities Thaks to Ady Nguye ad Julie Tibshirai for their advice o this hadout. This hadout covers mathematical otatio ad idetities that may be useful over the course of CS161. Feel free to refer to this hadout for referece o a variety of topics. If you have ay suggestios o how to improve this hadout, please let us kow! Set Theory The set N cosists of all atural umbers. That is, N = { 0, 1, 2, 3, } The set Z cosists of all itegers: Z = {, -3, -2, -1, 0, 1, 2, 3, } The set R cosists of all real umbers. The set Ø is the empty set cosistig of o elemets. If x belogs to set S, we write x S. If x does ot belog to S, we write x S. The uio of two sets S ₁ ad S ₂ is deoted S ₁ S ₂. Their itersectio is deoted ece is deoted S ₁ S ₂ or S ₁ \ S ₂, ad symmetric differece is deoted S ₁ Δ S ₂. If S ₁ is a subset of S ₂, we write S ₁ S ₂, differ- S ₁ S ₂. If S ₁ is a strict subset of S ₂, we deote this by S ₁ S ₂. The power set of a set S (deoted ( S)) is the set of all subsets of S. The Cartesia product of two sets S ₁ ad First-Order Logic S ₂ is the set S ₁ S ₂ = { ( The egatios of the basic propositioal coectives are as follows: (p (p ( p) p q) p q q) p q (p q) p q (p q) p q The egatios of the ad quatifiers are as follows: The statemet iff abbreviates if ad oly if. x. φ x. φ x. φ x. φ a, b) a S ₁ b S ₂ }
2 / 5 Summatios The sum of the first atural umbers (0 + 1 + 2 + + 1) is give by i= () 2 The sum of the first terms of the arithmetic series a, a + b, a + 2b,, a + ( 1)b is (a+i b)=a i =0 b () 1+b i=a+ 2 The sum of the first terms of the geometric series 1, r, r 2, r 3,, r -1 is give by As a useful special case, whe r = 2, we have r i = r 1 r 1 2 i =2 1 I the case that r < 1, the sum of all ifiite terms of the geometric series is give by r i = 1 1 r The followig summatio ofte arises i the aalysis of algorithms: whe r < 1: i r i = r (1 r) 2 Iequalities The followig idetities are useful for maipulatig iequalities: If A B ad B C, the A C If A B ad C 0, the CA CB If A B ad C 0, the CA CB If A B ad C D, the A + C B + D If A, B Z, the A B iff A < B + 1 If f is ay mootoically icreasig fuctio ad A B, the f(a) f(b) If f is ay mootoically decreasig fuctio ad A B, the f(a) f(b) The followig iequalities are ofte useful i algorithmic aalysis: e x 1 + x x 1 x 2... x x 1 + x 2 +...+x
3 / 5 Floors ad Ceiligs The floor fuctio x returs the largest iteger less tha or equal to x. The ceilig fuctio x returs the smallest iteger greater tha or equal to x. These fuctios obey the rules x x < x + 1 ad x 1 < x x ad x Z x Z Additioally, x + = x + ad x + = x + for ay Z. Asymptotic Notatio Let f, g : N N. The f() = O(g()) iff ₀ N. c R. N. ( ₀ f() cg()) f() = Ω(g()) iff ₀ N. c > 0 R. N. ( ₀ f() cg()) f() = Θ(g()) iff f() = O(g()) f() = Ω(g()) To show that f ad g are ot asymptotically related, you ca use these equivalet defiitios: f() O(g()) iff ₀ N. c R. N. ( ₀ f() > cg()) f() Ω(g()) iff ₀ N. c > 0 R. N. ( ₀ f() < cg()) The followig rules apply for O otatio: f() Θ(g()) iff f() O(g()) f() Ω(g()) If f() = O(g()) ad g() = O(h()), the f() = O(h()) (also Ω, Θ, o, ω) If f ₁( ) = O(g()) ad f ₂( ) = O(g()), the f ₁( ) + f ₂( ) = O(g()) (also Ω, Θ, o, ω) If f ₁( ) = O(g ₁( )) ad f ₂( ) = O(g ₂( )), the f ₁( )f ₂( ) = O(g ₁( )g ₂( )) (also Ω, Θ, o, ω) We ca use o ad ω otatios to deote strict bouds o growth rates: f ()=o(g ()) iff lim f () =0 f ()=ω(g ()) iff lim g () Polyomials, expoets, ad logarithms are related as follows: f () g() = Ay polyomial of degree k with positive leadig coefficiet is Θ( k ) log b = o( k ) for ay k > 0 k = o(b ) for ay b > 1 b = o(c ) for ay 1 < b < c I a graph, deotes the umber of odes ( V ) ad m deotes the umber of edges ( E ). I ay graph, m = O( 2 ). I a dese graph, m = Θ( 2 ); a sparse graph is oe where m = o( 2 ). A graph algorithm rus i liear time if it rus i time O(m + ).
4 / 5 Logarithms ad Expoets Logarithms ad expoets are iverses of oe aother: b log b x =log b b x =x The chage-of-base formula for logarithms states that log b a= log c a log c b Sums ad differeces of logarithms traslate ito logarithms of products ad quotiets: The power rule for logarithms states log b xy=log b x+log b y log b ( x/ y)=log b x log b y I some cases, expoets may be iterchaged: log b x y = y log b x (a b ) c =a bc =(a c ) b We ca chage the base of a expoet usig the fact that logarithms ad expoets are iverses: Probability If E ₁ ad E₂ are mutually exclusive evets, the a c =b clog ba P(E₁) + P(E₂) = P(E₁ E₂) For ay evets E ₁, E ₂, E ₃,, icludig overlappig evets, the uio boud states that P ( i=1 E i) P ( E i ) i=1 The probability of E give F is deoted P(E F) ad is give by The chai rule for coditioal probability is P (E F )= P (E F ) P (F ) P(E ₙ Eₙ - ₁ E ₁) = P(E ₙ Eₙ - ₁ E ₁) P( Eₙ - ₁ Eₙ - ₂ E ₁) P(E ₁) Two evets E ₁ ad E ₂ are called idepedet iff P(E ₁ E ₂) = P(E ₁) P(E ₂) For ay evet E, the complemet of that evet (deoted E) represets the evet that E does ot occur. E ad E are mutually exclusive, ad P(E) + P(E) = 1
5 / 5 Expected Value The expected value of a discrete radom variable X is defied as E [ X ]= (i P ( X =i)) The expected value operator is liear: for ay a, b E[aX + b] = ae[x] + b R ad ay radom variable X: More geerally, if X ₁, X ₂, X ₃, Xₙ are ay radom variables, icludig depedet variables: E [ i =1 If X ad Y are idepedet radom variables, the X i ]= E[ X i ] i=1 E[XY] = E[X]E[Y] Useful Probability Equalities ad Iequalities A idicator radom variable is a radom variable X where For ay idicator variable, E[X] = P(F). X ={ 1 if evet F occurs 0 otherwise Markov's iequality states that for ay radom variable X ad costat c, that If X ₁, X ₂,, Xₙ are radom variables, the P ( X c E[ X ]) 1 c P (max{ X 1, X 2,..., X } k)=p ( X 1 k X 2 k... X k) P (mi { X 1, X 2,..., X } k)=p ( X 1 k X 2 k... X k) O expectatio, repeatedly flippig a biased coi that comes up heads with probability p requires 1 / p trials before the coi will come up heads. Harmoic Numbers The th harmoic umber, deoted Hₙ, is give by H = i =1 The harmoic umbers are close i value to l : for ay 1, we have Ad so H ₙ = Θ(log ) 1 i l ( + 1) Hₙ l + 1