Deviations from the Mean The Markov inequality for non-negative RVs Variance Definition The Bienaymé Inequality For independent RVs The Chebyeshev Inequality
Markov s Inequality For any non-negative random variable X p(x a) apple E(X) a
Variance Deviation: The deviation of X at s S is X(s) E(X), the difference between the value of X at s, and the mean of X Variance: Let X be a random variable on the sample space S. The variance of X, denoted by V(X), is the weighted average of the square of the deviation of X ( ) = X ( s) E ( X ) V X s S ( ). ( ) 2 p s Standard Deviation: The standard deviation of X, denoted by σ(x), is defined to be V ( X).
Variance Theorem : If X is a random variable on a sample space S, then V(X) = E(X 2 ) E(X) 2 Theorem: If X is a random variable on a sample space S and E(X) = μ, then V(X) = E((X μ) 2 )
Variance Example: What is the variance of the random variable X, where X(t) = 1 if a coin toss results in a HEAD and 0 if it results in a TAIL, where p is the probability of HEAD and q = 1-p is the probability of TAIL? Solution: Because X takes only the values 0 and 1, it follows that X 2 (t) = X(t). Hence, 2 2 V X = E X - E X = p - p 2 = p 1 - p = pq. ( ) ( ) ( ) ( ) Variance of the Value of a Die: What is the variance of a random variable X, where X is the number that comes up when a fair die is rolled? 2 Solution: We have ( ) ( ) ( ) 2 V X = E X - E X. We seen earlier that E(X) = 7/2. Note that 2 2 2 2 2 2 2 ( ) ( ) E X = 1/ 6 1 + 2 + 3 + 4 + 5 + 6 = 91/ 6. We conclude that V ( X) 91 æ7ö 35 = - ç =. 6 è 2ø 12 2
Independent Random Variables Definition: The events E and F on a sample space S are independent if and only if p EÇ F = p E p F ( ) ( ) ( ). Definition: Random variables X and Y on a sample space S are independent if and only if p(x = r 1 and Y = r 2 ) = p(x = r 1 ) p(y = r 2 ) for all r 1, r 2 Theorem: If X and Y are independent RVs on a sample space S, then E(XY) = E(X)E(Y).
Variance Bienaymé s Formula: If X and Y are two independent random variables on a sample space S, then V(X + Y) = V(X) + V(Y) Generalization: if X i, i = 1,2,,n, with n a positive integer, are pairwise independent random variables on S, then V( X 1 + X 2 +!+ X ) n = V( X ) 1 + V( X ) 2 +!+ V( X n ). Linearity of variance only for independent RVs
Variance Example: Find the variance of the number of HEADS when a coin is tossed n times, where on each toss, p is the probability of HEADS and q is the probability of TAILS. Solution: Let X i be the random variable with X i ((t 1, t 2,., t n )) = 1 if the i th toss is a HEADS 0 if it is a TAILS. Let X = X 2 + X 3 +. X n. Then X counts the number of successes in the n trials. By Bienaymé s Formula, it follows that V(X)= V(X 1 ) + V(X 2 ) + + V(X n ). By the previous example,v(x i ) = pq for i = 1,2,,n. Hence, V(X) = npq.
Chebyshev s Inequality Chebyschev s Inequality: Let X be a random variable on a sample space S. 2 ( ( ) - ( ) ³ ) ( ) p X s E X r V X r. for any positive real number r.
Chebyshev s Inequality Example: Suppose that X is a random variable that counts the number of tails when a fair coin is tossed n times. (Hence E(X) = n/2 and V(X) = n/4, according to our previous examples.) Chebyschev s inequality with r = n: ( ( ) ) ( ) ( ) 2 p X s - n/ 2 ³ n n/ 4 n = 1/ 4. This means that the probability that the number of tails that come up on n tosses deviates from the mean, n/2, by more than n is no larger than ¼.
RELATIONS (From Chapter 9)
Topics we will cover Binary Relations definition, notation, and terminology We won t do n-ary relations for n>2 (useful in databases) The reflexive, symmetric, and transitive properties Equivalence relations and equivalence classes Combining relations Composing relations
Binary Relations Definition: A binary relation R from a set A to a set B is a subset RÍ A B. Example: Let A = {0,1,2} and B = {a,b} {(0, a), (0, b), (1,a), (2, b)} is a relation from A to B. We can represent relations from a set A to a set B graphically or using a table Relations are more general than functions. A function is a relation where exactly one element of B is related to each element of A.
Binary Relations on a (single) Set Definition: A binary relation R on a set A is a subset of A A or a relation from A to A. Example: Let A = {a,b,c}. Then R = {(a,a),(a,b), (a,c)} is a relation on A. Let A = {1, 2, 3, 4}. The ordered pairs in the relation R = {(a,b) a divides b} are (1,1), (1, 2), (1,3), (1, 4), (2, 2), (2, 4), (3, 3), and (4, 4).
Question: How many different relations are there on a set A with A = n? A relation on A is a subset of A X A How many elements in A X A? n 2 How many distinct subsets of A X A? 2^(n 2 ) Hence, 2^(n 2 ) different relations can be defined on A
Binary Relations on a Set Example: Consider these relations on the set of integers: 1 2 3 ( ) { } R = a, b a b, ( ) { } R = a, b a > b, ( ) { } R = a, b a = b or a =-b, 4 5 6 ( ) { } R = a, b a = b, ( ) { } R = a, b a = b+ 1, ( ) { } R = a, b a+ b 3. (Note that these relations are on an infinite set and each of these relations is an infinite set.) Which of these relations contain each of the pairs (1,1), (1, 2), (2, 1), (1, 1), and (2, 2)? Solution: Checking the conditions that define each relation, we see that the pair (1,1) is in R 1, R 3, R 4, and R 6 : (1,2) is in R 1 and R 6 : (2,1) is in R 2, R 5, and R 6 : (1, 1) is in R 2, R 3, and R 6 : (2,2) is in R 1, R 3, and R 4.
Properties A binary relation R from a set A to a set B is a subset of A X B A binary relation R on a (single) set A is a subset of A X A Properties of binary relations on a set A: Reflexive Symmetric Transitive Antisymmetric
Properties Which of these relations are reflexive, symmetric, transitive, and anti-symmetric? Reflexive: R 1, R 3, R 4 Symmetric: R 3, R 4, R 6 Transitive: R 1 R 4 Anti-symmetric: R 1, R 2, R 4, R 5
Combining Relations Since relations are sets, we can combine any relations R 1 and R 2 using basic set operations to form new relations such as R 1 R 2, R 1 R 2, R 1 R 2, and R 2 R 1. Example: Let A = {1,2,3} and B = {1,2,3,4}. The relations R 1 = {(1,1),(2,2),(3,3)} and R 2 = {(1,1),(1,2),(1,3),(1,4)} can be combined using basic set operations to form new relations R 1 R 2 =? R 1 R 2 =? R 1 R 2 =? R 2 R 1 =?
Composition Definition: Suppose R 1 is a relation from a set A to a set B. R 2 is a relation from B to a set C. Then the composition (or composite) of R 2 with R 1, denoted R 2 R 1, is a relation from A to C where (x,z) is a member of R 2 R 1 if and only if (x,y) is a member of R 1 and (y,z) is a member of R 2 Example. Let S 1 = {a, b, c}, S 2 = {m, n, o, p} and S 3 = {w, x, y, z} Let R 1 = {(a, p), (b, m)} and R 2 = {(m,x), (m, z), (n, w)} Determine R 2 R 1 Answer: {(b,x), (b, z)}
Powers of a Relation (Used in defining Equivalence Classes) Definition: The powers R n of binary relation R on a set A is defined inductively by: Basis Step: R 1 = R Inductive Step: R n+1 = R n R The powers of a transitive relation are subsets of the relation: Theorem: The relation R on a set A is transitive iff R n R for n = 1,2,3. (Easily proved by mathematical induction)
Equivalence Relations Definition: A relation on a set A is called an equivalence relation if it is reflexive, symmetric, and transitive. Definition: Two elements a, and b that are related by an equivalence relation are called equivalent. The notation a b is often used to denote that a and b are equivalent elements with respect to a particular equivalence relation.
Strings Example: Suppose that R is the relation on the set of strings of English letters such that arb if and only if l(a) = l(b), where l(x) is the length of the string x. Is R an equivalence relation? Solution: Do the the properties of an equivalence relation hold? Reflexivity: Because l(a) = l(a), it follows that ara for all strings a. Symmetry: Suppose that arb. Since l(a) = l(b), l(b) = l(a) also holds and bra. Transitivity: Suppose that arb and brc. Since l(a) = l(b),and l(b) = l(c), l(a) = l(a) also holds and arc. Hence, R is an equivalence relation
Congruence Modulo m Example: Let m be an integer > 1. Is the relation R = {(a,b) a b (mod m)} an equivalence relation on the set of integers? Solution: Recall that a b (mod m) if and only if m divides a b. Reflexivity: a a (mod m) since a a = 0 is divisible by m since 0 = 0 m. Symmetry: Suppose that a b (mod m). Then a b is divisible by m, and so a b = km, where k is an integer. It follows that b a = ( k) m, so b a (mod m). Transitivity: Suppose that a b (mod m) and b c (mod m). There are integers k 1 and k 2 with a b = k 1 m and b c = k 2 m. We obtain by adding the equations: a c = (a b) + (b c) = k 1 m + k 2 m = (k 1 + k 2 ) m. Hence, R is an equivalence relation
Divides Example: Is the divides relation on the set of positive integers an equivalence relation? Solution: Reflexivity: a a for all a. Symmetry: For example, 2 4, but 4 2. Hence, the relation is not symmetric. Transitivity: Suppose that a divides b and b divides c. Then there are positive integers k and l such that b = ak and c = bl. Hence, c = a(kl), so a divides c. Therefore, the relation is transitive. Hence, divides is not an equivalence relation.
Equivalence Classes Definition: Let R be an equivalence relation on a set A. The set of all elements that are related to an element a of A is called the equivalence class of a. The equivalence class of a with respect to R is denoted by [a] R. Note that { } [ ] ( ) a = s a, s ÎR. R When only one relation is under consideration, we can write [a], without the subscript R, for this equivalence class. If b [a] R, then b is called a representative of this equivalence class. Any element of a class can be used as a representative of the class.
Equivalence Classes