Lecture : Mathematical Prelimiaries Obective: Reviewig mathematical cocepts ad tools that are frequetly used i the aalysis of algorithms. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # I this ad the ext two lectures, our mai obective is to give a review of a variety of mathematical cocepts ad tools that are frequetly used i the aalysis of algorithms. Most of these cocepts ad tools should be familiar to the studet. They are provided here maily as a referece ad for completeess.
Topics reviewed i this lecture Sets Tuples ad Cartesia Products Relatios ad Fuctios Proof Methods - Direct Proof - Idirect Proof - Proof by Cotradictio - Proof by Couterexample - Mathematical Iductio Logarithms Floor ad Ceilig Fuctios Factorial ad Biomial coefficiets The Pigeohole Priciple Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 2 The topics briefly reviewed i this lecture iclude the followig: - Sets - Tuples ad Cartesia Products - Relatios ad Fuctios - Five mathematical Proof methods which iclude:. The Direct Proof method 2. Idirect Proof 3. Proof by Cotradictio 4. Proof by Couterexample 5. Mathematical Iductio - The we will review logarithms - Floor ad Ceilig Fuctios - Factorial ad Biomial coefficiets, ad fially - The Pigeohole Priciple First we will itroduce Sets
Sets Why sets are importat to the algorithm course? Defiitio: A set is a collectio of distict elemets, which are called members or elemets of the set. This collectio of elemets is treated as a sigle obect. Two commoly used methods to represet a set:. By listig its elemets betwee a pair of curly braces, or 2. By describig its elemets betwee curly braces Lecture # <BUTTONS AREA: Do ot write aythig here> Why sets are importat to the algorithm course? We study set cocepts because iput of a algorithm may be cosidered as a set draw from some particular domai like the set of itegers. We will briefly preset basic cocepts ad a few elemetary properties of sets used i the desig ad aalysis of algorithms. But first let us defie sets. Defiitio: A set is a collectio of distict elemets, which are called members or elemets of the set. This collectio of elemets is treated as a sigle obect. The two commoly used methods to represet a set are: The first method is by listig list elemets betwee a pair of curly braces. The secod method is by describig list elemets betwee curly braces
Sets (Cot.) Examples. Example : The set of itegers betwee ad ca be represeted as {, 2, 3,..., }. This same set ca be represeted usig the secod method as S {x x ad x is a iteger}. The secod method is the oly way used to describe ucoutable set. Example 2: S {x x is a real umber ad x }. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 4 Examples o Sets: Example : Usig the first method of represetatio, the set of itegers betwee ad ca be represeted as {, 2, 3,..., }. O the other had, this same set ca be represeted usig the secod method by specifyig some properties as follows: S {x x ad x is a iteger}. The expressio for S is read the set of all elemets x such that x is equal or greater tha ad less tha or equal to ad x is a iteger. The secod method is the oly way used to describe ucoutable set. Example 2: The set otatio S {x x is a real umber ad x } is used to represet the set of real umbers betwee ad.
Sets (Cot.) Set Properties A set is fiite if it cotais elemets for some costat >, ad ifiite otherwise. - Example above represets a fiite set. - Examples of ifiite sets: Set of atural umbers {, 2, } Set of itegers Set of ratioals ad Set of reals. Ifiite set is coutable if its elemets ca be listed as the first elemet, secod elemet, ad so o; otherwise it is called ucoutable. The set of real umbers is ucoutable. Lecture # <BUTTONS AREA: Do ot write aythig here> Elemetary properties of sets A set is said to be fiite if it cotais elemets for some costat >, ad ifiite otherwise. - The set i Example above is a fiite set. - Examples of ifiite sets are: Set of atural umbers {, 2, } Set of itegers Set of ratioals ad Set of reals. A ifiite set is called coutable if its elemets ca be listed as the first elemet, secod elemet, ad so o; otherwise it is called ucoutable. For example, the set of real umbers is ucoutable.
Sets (Cot.) Set properties (Cot.) The empty set is deoted by {} or by φ, ad it is a subset of every set. The cardiality of a fiite set A deoted by A, is the umber of elemets i A. x A idicates that x is a member of A. A set B is a subset of a set A, deoted by B A, if each elemet of B is a elemet of A. A is said to be a superset of the set B. B A deotes that B is a proper subset of A. That is, i additio to B A, B A. For ay set A, A A, ad φ A. If A ad B are two sets, the A B ad B A, meas A B. The Uio of two sets A ad B, deoted by A U B, is the set {x x A or x B} Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 6 The empty set is deoted by {} or by φ, ad it is a subset of every set. The cardiality of a fiite set A deoted by A, is the umber of elemets i A. x A idicates that x is a member of A. A set B is a subset of a set A, deoted by B A, if each elemet of B is a elemet of A. A is said to be a superset of the set B. B A deotes that B is a proper subset of A. That is, i additio to B A, B A. For ay set A, A A, ad φ A. If A ad B are two sets, the A B ad B A, meas A B. The Uio of two sets A ad B, deoted by A U B, is the set {x x A or x B}
Sets (Cot.) Set properties (Cot.) The itersectio of two sets A ad B, deoted by A B, is the set {x x A ad x B}. The differece of a set A from a set B, deoted by A - B, is the set {x x A ad x B}. The complemet of a set A, deoted by A, is defied as U - A, where U is the uiversal set cotaiig A, which is usually uderstood from the cotext. If A, B ad C are sets, the A U ( B U C) (A U B) U C, ad A (B C) (A B) C. Two sets A ad B are said to be disoit if A B φ. The power set of a set A, deoted by P(A), or 2 A is the set of all subsets of A. If A, the P (A) 2. Note that φ P(A) ad A P(A). To describe the umber of distict subsets of cardiality k that a fiite set of elemets have, we use the special otatio, read as umber of combiatios of k items take k at a time, or choose k. Also, the otatio C (, k) is sometimes used. These quatities are called biomial coefficiets, as will be discussed later. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 7 The itersectio of two sets A ad B, deoted by A B, is the set {x x A ad x B}. The differece of a set A from a set B, deoted by A - B, is the set {x x A ad x B}. The complemet of a set A, deoted by A, is defied as U - A, where U is the uiversal set cotaiig A, which is usually uderstood from the cotext. If A, B ad C are sets, the A U ( B U C) (A U B) U C, ad A (B C) (A B) C. Two sets A ad B are said to be disoit if A B φ. The power set of a set A, deoted by P(A), or 2 A is the set of all subsets of A. If A, the P (A) 2. Note that φ P(A) ad A P(A). To describe the umber of distict subsets of cardiality k that a fiite set of elemets have, we use the special otatio, read as umber of combiatios k of items take k at a time, or choose k. Also, the otatio C (, k) is sometimes used. These quatities are called biomial coefficiets, as will be discussed later.
Tuples ad the Cartesia Products A ordered -tuple is a ordered collectio (a, a 2,, a ) that has a as its first elemet, a 2 as its secod elemet,, ad a as its th elemet. A tuple ofte do ot have the same type. Short tuples have special ames: ordered pairs refer for 2-tuples, triples for 3-tuples, quadruple for 4-tuples, quituple for 5-tuples, ad so o. The Cartesia (or cross) product of two sets A ad B, deoted by A * B, is the set of all ordered pairs (a, b) where a A ad b B. I set otatio: A * B {(a, b) a A ad b B}. I geeral, the Cartesia product of sets is: A * A 2 * * A {(a, a 2,, a ) a i A i, i }. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 8 Tuples ad the Cartesia Products A ordered -tuple is a ordered collectio (a, a 2,, a ) that has a as its first elemet, a 2 as its secod elemet,, ad a as its th elemet. A tuple ofte do ot have the same type. Short tuples have special ames: ordered pairs refer for 2-tuples, triples for 3- tuples, quadruple for 4-tuples, quituple for 5-tuples, ad so o. The Cartesia (or cross) product of two sets A ad B, deoted by A * B, is the set of all ordered pairs (a, b) where a A ad b B. I set otatio: A * B {(a, b) a A ad b B}. I geeral, the Cartesia product of sets is: A * A 2 * * A {(a, a 2,, a ) a i A i, i }.
Relatios ad Fuctios Why we study fuctios ad relatios? Elemetary properties of relatios ad fuctios. - Let A ad B be two oempty sets, a biary relatio, or simply a relatio, R from A to B is a set of ordered pairs (a, b) where a A ad b B. That is R A * B. - If A B, we say that R is a relatio o the set A. - The domai of R writte as Dom (R) {a for some b B, (a, b) R}. - The rage of R writte as Ra (R) {b for some a A, (a, b) R}. Examples o domai ad rage S {(2, 5), (3, 3)}, S 2 {(x, y) x, y are positive itegers ad x y} ad S 3 {(x, y) x, y are real umbers ad x 2 y 2 } the, Dom (S ) {2, 3}; Ra (S ) {5, 3); Dom (S 2 ) Ra (S 2 ) is the set of atural umbers, ad Dom (S 3 ) Ra (S 3 ) is the set of real umbers i the iterval [-.. ]. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 9 Relatios ad Fuctios Why we study fuctios ad relatios? We study relatios ad fuctios because a algorithm ca be cosidered as a fuctio (or costraied relatio) that maps each possible iput to a specific output. Elemetary properties of relatios ad fuctios used i the desig ad aalysis of algorithms: - Let A ad B be two oempty sets, a biary relatio, or simply a relatio, R from A to B is a set of ordered pairs (a, b) where a A ad b B. That is R A * B. - If A B, we say that R is a relatio o the set A. - The domai of R writte as Dom (R) {a for some b B, (a, b) R}. - The rage of R writte as Ra (R) {b for some a A, (a, b) R}. Example o domai ad rage Assume the three sets S {(2, 5), (3, 3)}, S 2 {(x, y) x, y are positive itegers ad x y} ad S 3 {(x, y) x, y are real umbers ad x 2 y 2 } the, Dom (S ) {2, 3}; Ra (S ) {5, 3); Dom (S 2 ) Ra (S 2 ) is the set of atural umbers, ad Dom (S 3 ) Ra (S 3 ) is the set of real umbers i the iterval [-.. ].
Relatios ad Fuctios (Cot.) Elemetary properties of relatios ad fuctios (Cot.) - Let R be a relatio o a set A: the R is said to be: reflexive if (a, a) R a A. symmetric if (a, b) R implies (b, a) R. trasitive if (a, b) R ad (b, c) R implies (a, c) R. irreflexive if (a, a) R a A asymmetric if (a, b) R implies (b, a) R atisymmetric if (a, b) R ad (b, a) R implies a b. - A relatio that is reflexive, atisymmetric ad trasitive is called a partial order. Example o a partial order relatio: Let S {(x, y) x, y are positive itegers ad x divides y} S 2 {(x, y) x, y are itegers ad x y} the: both S ad S 2 are reflexive, atisymmetric ad trasitive ad hece both are partial order. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # Elemetary properties of relatios ad fuctios (Cot.) - Let R be a relatio o a set A: the R is said to be: reflexive if (a, a) R a A. symmetric if (a, b) R implies (b, a) R. trasitive if (a, b) R ad (b, c) R implies (a, c) R. irreflexive if (a, a) R a A asymmetric if (a, b) R implies (b, a) R atisymmetric if (a, b) R ad (b, a) R implies a b. - A relatio that is reflexive, atisymmetric ad trasitive is called a partial order. Example o a partial order relatio: Let S {(x, y) x, y are positive itegers ad x divides y} S 2 {(x, y) x, y are itegers ad x y} the: both S ad S 2 are reflexive, atisymmetric ad trasitive ad hece both are partial order.
Relatios ad Fuctios (Cot.) Equivalece Relatios It is a relatio that is, reflexive, trasitive ad symmetric. If R is a relatio o a set A, R partitios A ito equivalece classes such that: - For ay C i, i k, if x C i ad y C i, the (x, y) R. - If x C i ad y C i ad i, the (x, y) R. Example: x y (mod ) if x - y k * for some iteger k. That is, x y (mod ) if both x ad y leave the same remaider whe divided by. For istace: 3 8 (mod 5) If we defie the relatio: R {(x, y) x, y are itegers ad x y (mod )}, the R is a equivalece relatio. R partitios the set of itegers ito classes: C, C,, C - such that x C i ad y C i if ad oly if x y (mod ). Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # Equivalece Relatios A equivalece relatio is a relatio that is, reflexive, trasitive ad symmetric. If R is a relatio o a set A, R partitios A ito equivalece classes C, C 2,, C k such that ay two elemets i oe equivalece class are related by R. That is: - For ay C i, i k, if x C i ad y C i, the (x, y) R. - If x C i ad y C i ad i, the (x, y) R. Example o Equivalece relatio: Let x ad y be two itegers, ad let be a positive iteger, the x ad y are said to be cogruet modulo, deoted by x y (mod ) if x - y k * for some iteger k. Thus we say that x ad y are cogruet modulo if both x ad y leave the same remaider whe divided by. For istace: 3 8 (mod 5) as 3 8 * 5, or both 3 ad 8 both have the same remaider (3) whe divided by 5. If we defie the relatio: R {(x, y) x, y are itegers ad x y (mod )}, the R is a equivalece relatio. R partitios the set of itegers ito classes: C, C,, C - such that x C i ad y C i if ad oly if x y (mod ).
Relatios ad Fuctios (Cot.) Fuctios: Defiitio We write this as f (x) y Examples:. The relatio {(, 2), (3, 4), (2, 4) is a fuctio, 2. The relatio {(, 2), (, 4) is ot sice for the elemet i the domai there is more tha o elemet i the rage. 3. The relatio {(x, y)} x, y are positive itegers ad x y 3 } is a fuctio 4. The relatio {(x, y) x is a positive iteger, y is iteger ad x y 2 } is ot a fuctio. Importat properties of fuctios Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 2 Fuctios: A fuctio f is a (biary) relatio such that for every elemet x Dom (f) there exists oe elemet y Ra (f) with (x, y) f. Usually we write this as f (x) y istead of (x, y) f ad says that y is the value or image of f at x. Examples:. The relatio {(, 2), (3, 4), (2, 4) is a fuctio, 2. The relatio {(, 2), (, 4) is ot sice for the elemet i the domai there is more tha o elemet i the rage. 3. The relatio {(x, y)} x, y are positive itegers ad x y 3 } is a fuctio 4. The relatio {(x, y) x is a positive iteger, y is iteger ad x y 2 } is ot a fuctio. Properties of fuctios - Let f be a fuctio with Dom (f)a ad Ra (f) B for oempty sets A ad B: We say that f is oe to oe if f (x) f (y) implies x y, that is if for o differet elemets x ad y i A, f (x) f (y). - f is said to be oto B if Ra (f) B - f is said to be oe to oe correspodece betwee A ad B (also called biectio) if it is both oe to oe ad oto B.
Proof Techiques. Direct Proof 2. Idirect Proof 3. Proof by Cotradictio 4. Proof by Couterexample 5. Proof by Mathematical Iductio Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 3 Proof Techiques Mathematical proofs is cosidered as a essetial compoet ad prerequisite to algorithm desig ad aalysis. We will give here a brief review of five proof methods that are commoly used i algorithm desig ad aalysis.. Direct Proof 2. Idirect Proof 3. Proof by Cotradictio 4. Proof by Couterexample 5. Proof by Mathematical Iductio
Notatios used i mathematical proofs A propositio or assertio. The egatio symbol. The implicatio symbol. The if ad oly if symbol. Thus: - P Q stads for P implies Q or if P the Q. - The statemet P Q, stads for: P if ad oly if Q. It cosists of two parts (i.e., two implicatios) each of which should be prove separately: The if part (implicatio): P Q. The oly if part (implicatio): Q P. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 4 Notatios used i mathematical proofs The followig otatio is used i mathematical proof. A statemet that ca be either true or false but ot both is called a propositio or assertio. The symbol is the egatio symbol. The symbol is the implicatio symbol, ad is the if ad oly if symbol. Thus: - P Q stads for P implies Q or if P the Q. - The statemet P Q, stads for: P if ad oly if Q. That is: P is true if ad oly if Q is true. It cosists of two parts (two implicatios) each of which should be prove separately: The if part (implicatio): P Q. The oly if part (implicatio): Q P.
Direct proof method We wat to prove that if P the Q, that is: P Q. We start the proof by assumig that P is true, ad the deduce the truth of Q from the truth of P. 2 Example: If N is a eve iteger, the prove that N is also eve iteger. Solutio: Sice N is eve, N 2k for some iteger k. So, N 2 (2K) 2 4K 2 2 (2K 2 ). It follows that N 2 is a eve iteger. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 5 Direct Proof method We wat to prove that if P the Q, that is: P Q We start the proof by assumig that P is true, ad the deduce the truth of Q from the truth of P. Example: Use the direct proof method to prove that if N is a eve iteger, the prov that N 2 is also eve iteger. Solutio: Sice N is eve, N 2k for some iteger k. So, N 2 (2K) 2 4K 2 2 (2K 2 ). It follows that N 2 is a eve iteger.
Idirect proof method Idirect proof if ot Q the ot P P Q is logically equivalet to if ot Q the ot P The statemet if it is raiig the it is cloudy is equivalet to the statemet if it is ot cloudy the it is ot raiig. Example: Prove the assertio that if 2 is a eve iteger, the is a eve iteger Solutio: This assertio is logically equivalet to if is a odd iteger, the 2 is a odd iteger. Sice is odd the 2k for some iteger k. Hece, 2 (2k ) 2 4k 2 4k 2(2k 2 2k). Which implies that 2 is a odd iteger. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 6 Idirect proof method It is some times much easier to prove if ot Q the ot P tha usig a direct proof for the statemet if P the Q. P Q is logically equivalet to the cotrapositive implicatio if ot Q the ot P, which is also writte as Q P. For istace, the statemet if it is raiig the it is cloudy is equivalet to the statemet if it is ot cloudy the it is ot raiig. Example: Prove the assertio that if 2 is a eve iteger, the is a eve iteger usig idirect proof. Solutio: This assertio is logically equivalet to if is a odd iteger, the 2 is a odd iteger which is proved usig the direct method as follows: Sice is odd the 2k for some iteger k. Hece, 2 (2k ) 2 4k 2 4k 2(2k 2 2k). Which implies that 2 is a odd iteger.
Proof by cotradictio Widely used ad extremely powerful. To prove the statemet if P the Q : - Start with the assumptio that P is true but Q is false Logical reasoig for proof by cotradictio. Other cotradictios may be reached as well. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 7 Proof by cotradictio Widely used ad extremely powerful. Its power comes from makig proofs cocise ad short as will be see i the ext example. To prove the statemet if P the Q : - start with the assumptio that P is true but Q is false: If this leads to a cotradictio, it implies that our assumptio Q is false must be wrog ad hece Q must follow from P. Logical reasoig for proof by cotradictio: - If we kow that P Q is true ad Q is false, the P must be false. - So, if we assume at the begiig that P is true, Q is false, ad reach the coclusio that P is false, the we have that P is both true ad false. - But P caot be both true ad false - a cotradictio. - We coclude our assumptio that Q is false is wrog Q must be true. Other cotradictios may be reached as well: For istace, after assumig that P is true ad Q is false, we may reach the coclusio that, say, -.
Proof by cotradictio (Cot.) Example. Prove the theorem that: If balls are distributed ito m boxes, the oe box must cotai at least / m balls. Solutio: If all boxes have less tha / m balls, the the total umber of balls is at most m m m m m < m m m which is a cotradictio. This is a excellet example that shows its power i producig very short proofs. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 8 Example o proof by cotradictio: Prove by cotradictio the pigeohole pricipal theorem that: If balls are distributed ito m boxes, the oe box must cotai at least / m balls. The defiitio of this priciple will be give later i this lecture. Solutio: If all boxes have less tha / m balls, the the total umber of balls is at most m m m m m < m m m which is a cotradictio. This is a excellet example that shows the power of this proof method i producig very short proofs.
Proof by couterexample method Geeral Itroductio Example: Cosider the fuctio f () 2 4 defied o the set of oegative itegers: - f () 4, f () 43,, f (39) 6 are all primes. - Assertio: f () is always a prime umber. - We use Proof by couterexample, to falsify this assertio: cosider f (4) 68 (4) 2 which is ot a prime. We coclude that the assertio is false. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 9 Proof by couterexample method Geeral Itroductio: - This method is used to show that a propositio that holds true quite ofte is ot always true. - Also, it is usually used whe we are faced with a problem that requires provig or disprovig a give assertio. I this case, we may start by tryig to disprove the assertio with a couterexample. - I algorithm aalysis, this techique is frequetly used to show that a algorithm does ot always produce a result with certai properties. Example: Let f () 2 4 be a fuctio defied o the set of oegative itegers: - f () 4, f () 43,, f (39) 6 are all primes. - Assertio: f () is always a prime umber. - We use Proof by couterexample, to falsify this assertio: cosider f (4) 68 (4) 2 which is ot a prime. We coclude that the assertio is false.
Proof by mathematical iductio Proof By Mathematical Iductio: A powerful proof techique. The truth of a umber of statemets (propositios) ca be iferred from the truth of a few specific istaces. Suppose a sequece of statemets S, S 2,..., we wat to prove to be true. Suppose also that:. For some i, we kow that S, S 2,, S i are true. 2. The problem is such that for ay i, the truths of S, S 2,... S imply the truth of S. We ca the use iductio to show that every statemet i this sequece is true. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 2 Proof By Mathematical Iductio: It is a powerful proof techique for provig that a property holds for a sequece of atural umbers,, 2,. Typically, is take to be or, but it ca be ay atural umber. Also, the sequece,, 2, ca be ay liearly ordered set. However, i the maority of proofs by iductio, the set of atural umbers is assumed. I this techique, the truth of a umber of statemets (propositios) ca be iferred from the truth of a few specific istaces. Suppose we have a sequece of statemets S, S 2,..., we wat to prove to be true. Suppose also that the followig holds:. For some i, we kow that S, S 2,, S i are true. 2. The problem is such that for ay i, the truths of S, S 2,... S imply the truth of S. We ca the use iductio to show that every statemet i this sequece is true.
Proof by mathematical iductio (Cot.) Example Prove that a biary tree of height has at most 2 leaves: Proof: Let L () represet the umber of leaves of a biary tree of height. Show that L () 2 : Iductio Basis: Clearly L () 2 sice a tree of height has at most oe leaf. Iductio Hypothesis: Assume L (k) 2 k for all k, k. Iductio Step: Let T be a tree of height. By iductio hypothesis, the umber of leaves i the left ad right subtrees of the root is 2. It follows that the umber of leaves i T is 2 2 2 {}. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 2 Example o Proof by mathematical iductio Usig the Priciple of Mathematical Iductio, Prove that a biary tree of height has at most 2 leaves: Proof: Let L () represet the umber of leaves of a biary tree of height. We wat to show that L () 2 usig mathematical iductio: Iductio Basis: Clearly L () 2 sice a tree of height ca have o odes other tha the root. That is, it has at most oe leaf. Iductio Hypothesis: Assume L (k) 2 k for all k, k. Iductio Step: Let T be a tree of height. By iductio hypothesis, the umber of leaves i the left ad right subtrees of the root is 2. It follows that the umber of leaves i T is 2 2 2 {}. Thus, the property is true for ( ). Sice T is arbitrary, the proof is complete.
Logarithms Importace of the logarithm fuctio. Defiitio of the logarithm fuctio The base is omitted whe it is 2. The atural umber e, defied by e lim... 2.78288...! 2! 3! log e x is writte as l x, ad is called the atural logarithm of x. l x is also defied by: x l x dt. t Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 22 Logarithms The logarithm fuctio, ad specifically to the base 2, is used very extesively i the aalysis of may algorithms. Defiitio. Let b >, be a positive real umber, x a real umber, ad suppose that for some positive real umber y we have: y b x. The, x is called the logarithm of y to the base b ad is writte as x log y. Sice the log to the base 2 is used most ofte i computatioal complexity, therefore, we omit the base whe it is 2. Aother useful base is the atural umber e, which is defied by e lim... 2.78288...! 2! 3! Geerally, log e x is writte as l x, ad is called the atural logarithm of x. l x is also defied by: x l x dt. t b
Logarithms (Cot.) Properties of logarithms log b xy log b x log b y, ad log b ( c ) y log b c, if c >. log b is a strictly icreasig fuctio. log b is a oe-to-oe fuctio. log. b log e.443 ad log 3.32. The derivative of l (x) is /x. The derivative of log (x) log (e) / x. log b a b a. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 23 Properties of logarithms For ay real umbers x ad y greater tha, the followig properties of logarithms follow from the defiitio: log b xy log b x log b y, ad log b ( c y ) y log b c, if c >. log is a strictly icreasig fuctio, that is, if x > y, the log x > log y. b log b is a oe-to-oe fuctio, that is, if log b x log b y, the x y. log b. log e.443. log 3.32. The derivative of l (x) is /x. The derivative of log (x) log (e) / x. log b a b a. b b
Logarithms (Cot.) Properties of logarithms (Cot.) Covertig betwee bases: log a x log b x. log a b For example: l x log x ad l 2 log x l x. log e Importat idetity: x log b y logb x y, x, y >. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 24 More properties of logarithms To covert from oe base to aother, we use the followig chai rule: log a x log b x. log a b For example: l x log x ad l 2 log x l x. log e The followig idetity ca be proved by takig the logarithms of both sides: x log b y logb x y, x, y >.
Floor ad Ceilig Fuctios Defiitios: - Mootoically icreasig fuctio - Mootoically decreasig fuctio. - Strictly icreasig fuctio. - Strictly decreasig fuctio. The floor ad ceilig fuctios to be defied later are mootoically icreasig. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 25 Floor ad Ceilig Fuctios Defiitios: - A fuctio is mootoically icreasig if m implies f (m) f (). - A fuctio is mootoically decreasig if m implies f (m) f (). - A fuctio is strictly icreasig if m < implies f (m) < f (). - A fuctio is strictly decreasig if m < implies f (m) > f (). The floor ad ceilig fuctios to be defied later are mootoically icreasig.
Floor ad Ceilig Fuctios (Cot.) Let x be ay real umber, the: - x is defied as the greatest iteger x. - x is defied as the smallest iteger x. Examples: 2.7 2 2.7 3 2.5 3 2.5 2 x / 2 x / 2 x. x x. x x. x- < x x / m < x. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 26 Let x be ay real umber, the: - The floor of x, deoted by x is defied as the greatest iteger x. - The ceilig of x, deoted by x is defied as the smallest iteger x. Examples: 2.7 2 2.7 3 2.5 3 2.5 2 Few more importat properties are: x / 2 x / 2 x. x x x x.. x- < x x / m < x.
Floor ad Ceilig Fuctios (Cot.) Theorem. Let f (x) be a mootoically icreasig fuctio such that if f (x) is iteger, the x is iteger. The, ( x ) f ( x ad f ( x ) f ( x. f ) ) Examples: for ay positive iteger ad itegers a ad b : -. x / x / x / x / / a / b / ab / a / b / ab / 2 / 2 / 2 / 4 / 2 / 8 -. -. -. -. If is ot a power of 2, the there is a iteger k such that 2 k < < 2 k. I this case, k ad. log log k The followig two iequalities hold: - 2 log < 2. - < 2 log 2 Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 27 Theorem. Let f (x) be a mootoically icreasig fuctio such that if f (x) is iteger, the x is iteger. The, ( x ) f ( x ad f ( x ) f ( x. f ) ) Examples: for ay positive iteger ad itegers a ad b : -. x / x / x / x / / a / b / ab / a / b / ab / 2 / 2 / 2 / 4 / 2 / 8 -. -. -. -. If is ot a power of 2, the there is a iteger k such that 2 k < < 2 k. I this case, k ad. log log k The followig two iequalities hold: - 2 log < 2. - < 2 log 2
Factorial ad Biomial coefficiets Geeral Factorials - Permutatio of a set of distict obects - k-permutatio - Whe k, the quatity becomes P * 2 * *, ad is called factorial ad deoted by!. - Useful approximatio to! is Stirlig s formula ( 2π *( / e). - Usig Stirlig s formula, we compute 3! with relative error of about.27% oly. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 28 Factorial ad Biomial coefficiets We briefly list some of the importat combiatorial properties that are frequetly used i the aalysis of algorithm. These properties are used with combiatorial problems. I particular, we will limit our review to permutatios ad combiatios, which lead to the defiitios of factorials ad biomial coefficiets. Factorials - A permutatio of a set of distict obects is defied as a ordered arragemet of these obects. For example, i the 3-elemets set {a, b, c} there are 6 permutatios: abc, acb, bac, bca, cab, cba. - We may be also iterested i ordered arragemets of some of the elemets of a set. I this case, we use the term k-permutatio to refer to a ordered arragemet of k elemets of a set. The umber of ways to choose k obects i a ordered arragemet (also, called the umber of permutatios of obects take k at a time) is: (-) (-2) ( k ) ad deoted by P or P (, k) k
This is ot a ew slide Please ote that the spoke part below belogs to the previous slide. There was o space over there. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 29 - Whe k, the quatity becomes P * 2 * *, ad is called the umber of permutatios of obects. Because of its importace, this quatity is desigated as factorial ad deoted by!. By covetio,!, which gives the followig simple recursive defiitio of!: -!,! ( )! if. - A useful approximatio to! is Stirlig s formula ( 2π *( / e). - For example, usig Stirlig s formula, we compute 3! with relative error of about.27% oly.
Biomial coefficiets Combiatios (also, referred to as k-combiatios or choose k ) deoted by C. k Example: the combiatios of the four letters a, b, c ad d take three at a time This is i fact the uordered selectio of 3 elemets from the 4-elemets set. Pk! Ck, k k! k!( k)! Other commo otatios for C k are C (, k) ad Biomial coefficiet. k This umber is equivalet to the umber of k-elemet subsets i a set of elemets. Example, Subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 3 Biomial coefficiets Aother importat cocept is the combiatio of obects take k at a time (also, referred to as k-combiatios or choose k ). It refers to the possible ways to choose k obects out of obects, disregardig order, ad deoted by C k. Example: the combiatios of the four letters a, b, c ad d take three at a time are: a b c, a b d, a c d, b c d. This is i fact the uordered selectio of 3 elemets from the 4-elemets set. Sice order does ot matter here, the combiatios of obects take k at a time is equal to the umber of permutatios of obects take k at a time divided by k!. That is Pk! Ck, k k! k!( k)! Other commo otatios for Ck are C (, k) ad k This umber is also called the Biomial coefficiet. This ame is used because these umbers occur as coefficiets i the expasio of powers of biomial expressios such as (x y). This umber is equivalet to the umber of k-elemet subsets i a set of elemets. For example, the 3-elemet subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}.
Biomial coefficiets (Cot.) Importat properties of Biomial coefficiets: - The umber of ways to choose k elemets out of elemets is equal to the umber of ways ot to choose -k elemets out of elemets. This implies that: k k k k k, i particular - The followig idetity is also importat oe: Lecture # Slide # 3 <BUTTONS AREA: Do ot write aythig here> Some of the importat properties of biomial coefficiets: - The # of ways to choose k elemets out of elemets is equal to the umber of ways ot to choose -k elemets out of elemets. This implies that: k k k k k, i particular This meas that the -combiatio ( the umber of obects take at a time) equals the -combiatio ( the umber of combiatios of obects take at a time) ad equals to. - The followig idetity is also importat oe: That is, the umber of combiatios of obects take k at a time equals to the umber of combiatios of - obects take k at a time the umber of combiatios of - obects take k - at a time.
Biomial coefficiets (Cot.) Properties of Biomial coefficiets (Cot.) - Pascal s Idetity. Used i the geometric arragemet of the biomial coefficiets i Pascal triagle. - It is stated as follows. Let ad k be positive itegers with k. The k k k The th row i Pascal s triagle cosists of the biomial coefficiets, k, k,..., as follows: Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 32 Properties of biomial coefficiets (Cot.) - Pascal s Idetity. It is used i the geometric arragemet of the biomial coefficiets i a triagle called Pascal triagle show later. - Pascal s Idetity is stated as follows. Let ad k be positive itegers with k. The k k k The th row i Pascal s triagle cosists of the biomial coefficiets, k,,..., as show i the ext slide. k
Biomial coefficiets (Cot.) Properties of Biomial coefficiets (Cot.) 2 2 2 2 2 3 3 3 3 2 3 3 3 4 6 4 4 4 3 4 2 4 4 4 The first 5 rows i Pascal s Triagle. Lecture # Slide # 33 <BUTTONS AREA: Do ot write aythig here> 2 2 2 2 2 3 3 3 3 2 3 3 3 4 6 4 4 4 3 4 2 4 4 4 The first 5 rows i Pascal s Triagle.
Biomial coefficiets (Cot.) Properties of Biomial coefficiets (Cot.) - Biomial theorem. It gives coefficiets of the expasio of powers of biomial expressios. - Biomial theorem statemet: Let be a positive iteger, ad x, y be two variables the:. y x y x ) (. y xy y x y x x 2 2... 2 Lecture # Slide # 34 <BUTTONS AREA: Do ot write aythig here> Properties of biomial coefficiets (Cot.) - A fudametal tool i the aalysis of algorithms is the biomial theorem. It gives coefficiets of the expasio of powers of biomial expressios, where a biomial expressio is simply the sum of two terms, such as x y. - The biomial theorem is stated as follows: Let be a positive iteger, ad x, y be two variables the:. y x y x ) (. y xy y x y x x 2 2... 2
Biomial coefficiets (Cot.) Examples o Biomial theorem: Example : Fid the expasio of (x y) 3. Solutio: Usig the biomial theorem, we get: (x y) 3 3 3 3 x y. C (3, ) x 3 C(3, ) x 2 y C (3, 2) xy 2 C (3, 3) y 3 x 3 3x 2 y 3xy 2 y 3 Example 2: What is the coefficiet of xy 3 i the expasio (x y) 4? Solutio: Usig the biomial theorem, we have the coefficiet as follows: 4 4! 4! 4 3 3!(4 3)! 3!! Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 35 Examples o Biomial theorem: Example : Fid the expasio of (x y) 3. Solutio: Usig the biomial theorem, we get: (x y) 3 3 3 3 x y. C (3, ) x 3 C(3, ) x 2 y C (3, 2) xy 2 C (3, 3) y 3 x 3 3x 2 y 3xy 2 y 3 Example 2: What is the coefficiet of xy 3 i the expasio (x y) 4? Solutio: Usig the biomial theorem, we have the coefficiet as follows: 4 4! 4! 4 3 3!(4 3)! 3!!
Biomial coefficiets (Cot.) If x, the we get:. ) ( y y Makig x ad y, the we get, () ) ( that is 2... This states that the umber of all subsets of a set of size is equal to 2. If we let x - ad y, the we get: ) ( () (-) ) ) (( That is or.... ± odd eve Lecture # Slide # 36 <BUTTONS AREA: Do ot write aythig here> If we let x i the Biomial theorem above, the we get the followig importat idetity:. ) ( y y Makig x ad y, i the above theorem, the we get, () ) ( that is 2... I terms of combiatios, this idetity states that the umber of all subsets of a set of size is equal to 2, as expected. If we let x - ad y i the biomial theorem, the we get: ) ( () (-) ) ) (( That is, or.... ± odd eve
Biomial coefficiets (Cot.) From the Biomial theorem above, we have: ( x) 2 2x x 2 ; (x) 3 3x 3x 2 x 3 ad so o. Cotiuig this way idefiitely, we get the elemets of the Pascal triagle, where: Each row after the first is computed from the previous row usig the equatio k k k The Biomial coefficiets are the digits of the umber ( b) writte i base b otatio, if b is sufficietly large (e.g., ). Example: Cosider base, where ( ), we have: (), (), () 2 2*() 2 2, () 3 33, () 4 464 ad so o. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 37 From the Biomial theorem above, we have: ( x) 2 2x x 2 ; (x) 3 3x 3x 2 x 3 ad so o. Cotiuig this way idefiitely, we get the elemets of the Pascal triagle show previously, where: Each row after the first is computed from the previous row usig the equatio k k k It is observed that the biomial coefficiets are the digits of the umber ( b) writte i base b otatio, if b is sufficietly large (e.g., ). Example: Cosider base, where ( ), we have: (), (), () 2 2*() 2 2, () 3 33, () 4 464 ad so o., where the digits,4,6,4, i the last umber for example are the etries i the fifth row of Pascal s triagle!!!
The Pigeohole Priciple A easy, ituitive ad extremely powerful. States that: If k or more obects are placed ito k boxes, the there is at least oe box cotaiig two or more of the obects. Theorem: If obects are distributed ito m boxes, the: - Oe box cotai at least / m obects ad oe box must cotai at most / m obects. Example: Suppose that G (V, E) be a coected udirected graph with m vertices. Also, let p be a path i G that visits > m vertices. Usig the Pigeohole Priciple, we show that p must cotai a cycle. Solutio: Sice the / m 2, there is at least oe vertex, say v that is visited by p more tha oce. Thus, the portio of the path that starts ad eds at v costitutes a cycle. Lecture # <BUTTONS AREA: Do ot write aythig here> Slide # 38 The Pigeohole Priciple (also called: the Dirichlet drawer priciple) It is a easy, ituitive ad extremely powerful priciple ad is essetial i the aalysis of algorithms. I the 9 th cetury, the Germa mathematicia Dirichlet used this priciple i his work. The Pigeohole Priciple states that: If k or more obects are placed ito k boxes, the there is at least oe box cotaiig two or more of the obects. Theorem: If obects are distributed ito m boxes, the: - Oe box cotai at least / m obects ad oe box must cotai at most / m obects. Example o the Pigeohole Priciple: Suppose that G (V, E) be a coected udirected graph with m vertices. Also, let p be a path i G that visits > m vertices. Usig the Pigeohole Priciple, we show that p must cotai a cycle. Solutio: Sice the / m 2, there is at least oe vertex, say v that is visited by p more tha oce. Thus, the portio of the path that starts ad eds at v costitutes a cycle.