If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r).

Size: px

Start display at page:

Download "If the objects are replaced there are n choices each time yielding n r ways. n C r and in the textbook by g(n, r)."

Berenice Morrison
6 years ago
Views:

1 Caveat: Not proof read. Corrections appreciated. Combinatorics In the following, n, n 1, r, etc. will denote non-negative integers. Rule 1 The number of ways of ordering n distinguishable objects (also called permutations) is n!. This follows from the fact that there are n choices for the first object, n 1 for the second, and so on down to one choice for the nth object. Note that distingishability is crucial. Consider 3 objects a, b, c. There are 3! = 6 ways of ordering them: abc, acb, bac, bca, cab, and cba. Rule 2 From a group of n distinguishable objects the number of ways of choosing an ordered set of r objects without replacement is n!/(n r)!. As before there are n choices for the first object, n 1 for the second, and so on down to n (r 1) choices for the rth object. This yields n(n 1) (n (r 1)) = n(n 1) (n (r 1)) (n r)(n (r + 1)) 1 (n r)(n (r + 1)) 1 = n! (n r)! If the objects are replaced there are n choices each time yielding n r ways. Rule 3 The number of ways of choosing r objects out ( of ) n without reference to order is n!/r!(n r)!. n This is denoted in general by the symbol or by r n C r and in the textbook by g(n, r). This result is obtained from the rules above as follows: The number of ways of choosing r ordered objects out of n i.e., n!/(n r)! from Rule 2; since the order of the r objects is not to be included, the number of permutations of the r objects, given by Rule 1, r! must be divided out yielding the result. Check this explicitly for n = 5 and r = 2 for example. Rule 4 Consider N objects out of which n 1 are of type 1 and are indistinguishable and the rest n 2 = N n 1 are of type 2 and are indistinguishable among themselves. The number of ways in which the N objects can be ordered is N!/n 1!n 2!. Justify this to yourself! 1

2 Rudimentary Probability Theory Consider an experiment and its possible outcomes. The set of all possible outcomes is called the sample space. The possible observations, in general, are termed events. The simple (indecomposable) events that constitute the sample space are also called sample points. Example I: If a die is cast(thrown) the possible outcomes are 1,2,3,4,5, or 6. The sample space is the set {1, 2, 3, 4, 5, 6} and each of the possible outcomes {1}, {2} etc. is a simple event. We must distinguish simple events from compound events that arise when one asks more complicated questions: Consider the event that the result of throwing the die is even. This event corresponds to {2,4,6} and can be decomposed into 3 simple events and is a a compound event. Exercise: How many sample points are there in the sample space for tossing three (distinguishable) coins? As an intuitive basis for assigning probabilities to events we will invoke a relative frequency interpretation. This can be done by repeating the (idealized) experiment n times (n ) and counting the number of times the event occurs. Probability for an event = (number of times the event occurs)/(number of attempts n) in the limit of infinite number of attempts. One can also employ an ensemble approach in which one considers many (n ) identical copies of the same system on which the experiment is performed and counting the number of copies in which the event occurs. This is one way of ascribing meaning to the statement the probability for obtaining H(eads) when a coin is tossed is one-half. Example I: For a fair die the probability of obtaining any one of the numbers between 1 and 6 is 1/6. We have assumed that each of the sample points is equally probable. Note that the sum of the probabilities of all the sample points is one. Note that answering questions in probability theory requires a clear specification of the sample space (what are the allowed outcomes?) and assigning probabilities to simple events. Given the sample space defined in Example I above what is the probability that the result of throwing a die is even? This compound event defined earlier corresponds to {2,4,6}. The probability for its occurrence is 1/6 + 1/6 + 1/6 = 1/2. This intuitively obvious result can be elevated to the following proposition: Proposition I. If two events A and B with probabilities p(a) and p(b) respectively are mutually exclusive the probability that either A or B occurs is p(a) + p(b). What is the probability that when two (fair) dice are thrown the resultant sum is 6? This event (let us denote it by S 6 ) can occur in five ways: {1,5}, {2,4}, {3,3}, {4,2}, or {5,1}. This is another example of a compound event that can be decomposed into 5 simple events. 2

3 What is the probability for this compound event S 6? First, what is the probability of obtaining {1, 5}? This is clearly equal to 1/6 1/6 = 1/36. One can justify the result by noting that the sample space consists of 36 simple events: {1,1},{1,2},{1,3},......{6,4},{6,5},{6,6}, and each of these is equally probable. A more general way of justifying this is to use the following proposition: II. If two events A and B with probabilities p(a) and p(b) respectively are independent, the probability for the occurrence of both A and B is p(a) p(b). Actually the above is the definition of independence! Given the result that the probability of {1,5} etc., is 1/36 the probability of S 6 is 5/36 by using Proposition I. Example III. Consider drawing one card from each of two decks of cards. What is the probability that at least one will be an ace? Answer: Assuming the obvious sample space, the probability is 25/169. Solution: It is useful to remember that to calculate the probability, p, that something occurs at least once it is best to compute the probability that the event never occurs, q, and noting that p = 1 q. The probability that the card drawn is not an ace is 12/13 for a given deck and thus q = = Thus the required probability is =

4 Set Theoretic Miscellany: Let the entire sample space be the set denoted by E. Each event corresponds to a subset S with probability p(s). Clearly p(e) = 1 and p(s) 0. Let S 1 and S 2 be events with probabilities p(s 1 ) and p(s 2 ) respectively given a sample space and a certain assignment of probabilities for the simple events. The event S 1 S 2 corresponds to either S 1 or S 2 or both occurring. The event S 1 S 2 corresponds to both S 1 and S 2 occurring. RESULT I : If S 1 and S 2 are independent then p(s 1 S 2 ) = p(s 1 ) p(s 2 ). Proposition II. This is RESULT II : p(s 1 S 2 ) = p(s 1 ) + p(s 2 ) p(s 1 S 2 ) 4

5 Continuous Probability Distributions Consider a continuous random variable x with a probability density denoted by p(x); i.e., the probability of the variable assuming a value between x and x + dx is given by p(x)dx. The cumulative distribution function is defined by P (x) x dy p(y) (1) so that we have p(x) = dp. The moments of the distribution are defined as follows: the dx mth moment, µ m, is given by µ m, dx x m p(x) ; (2) It is important to observe that the moments may not exist since integrals need not converge. It is convenient to define the characteristic function or the Fourier transform of the probability density function: ˆp(k) dx e ikx p(x) e ikx. (3) Expanding the exponential in the definition of the characteristic function formally (without worrying about convergence) we obtain ˆp(k) = m=0 (ik) m m! µ m. (4) We note in passing that ˆp(k) is said to be the generator of the moments. Generating functions constitute a useful device in many mathematical applications. Recall, for example, that the generating function for Legendre polynomials is given by 1 = 1 2tx + x 2 n=0 t n P n (x). (5) Useful Aside: Another useful set of quantities that characterize the probability distribution is the set of cumulants defined as follows; expand the logarithm of the characteristic function in powers of the variable k: ln ˆp(k) = j=1 (ik) j C j, (6) j! where the jth coefficient is denoted by C j referred to as the jth cumulant. This is in fact one definition of the cumulants. We can relate the cumulants to the moments as follows. Write the right-hand side of the above equation using Eq. (4) as ln (ik) j 1 + µ k j! j=1 5

6 and expand ln[ 1 + ikµ 1 + (ik) 2 µ 2 /2 + ] as a power series in ik and compare with the corresponding coefficients on the right-hand side of Eq. (6). For example, on obtains for the second cumulant the useful result C 2 = µ 2 µ 2 1. Please check this! Here are a few more for your reference: C 3 = µ 3 3µ 2 µ 1 + 2µ 2 2. (7) C 4 = µ 4 4µ 3 µ 1 3µ µ 2 µ 2 1 6µ 4 1. (8) There is no simple formula for the general cumulant in terms of the moments. The moments can be expressed in terms of the cumulants more elegantly. We will compute these for specific examples and they will be useful in statistical mechanics where the first two moments/cumulants suffice or one needs to consider the entire probability distribution. Many random variables: Define independent variables. recall independent events in basic probability theory. If the events {X a} and {Y b} are independent for all a and b the corresponding distributions are said to be independent. For our purposes it is sufficient to say the following. If the probability density for x 1 and x 2 r 2 (x 1, x 2 ) obeys then the two random variables are independent. r 2 (x 1, x 2 ) = p(x 1 ) q(x 2 ) (9) If two random variables are not independent they are said to be correlated. If they are independent then x 1 x 2 = x 1 x 2 and thus x 1 x 2 x 1 x 2 = (x 1 x 1 ) (x 2 x 2 ) (10) is a measure of the correlation and is known as the correlation between the random variables. This is called covariance in statistics. The generating function for the cross-moments µ m,n = x m 1 x n 2 is given by e i(k 1x 1 + k 2 x 2 ) = m=0 n=0 (ik 1 ) m m! (ik 2 ) n µ m,n. (11) n! The generating function of the moments of two independent random variables is the product of the generating functions while that of the cumulants is the sum of the generating functions. Note that if y = x 1 + x 2 then we have the useful result that the cumulants of the sum is the sum of the cumulants. Show this explicitly. We wish to investigate the probability distribution of a sum of independent random variables. Consider two random variables x 1 and x 2 that are independent and identically distributed with probability density p(x). We wish to find the probability density Q(y), of the variable y = x 1 + x 2. Clearly, r(y) = dx 1 dx 2 p(x 1 ) p(x 2 ) δ(y (x 1 + x 2 )) (12) 6

7 This is a fundamental equation in continuous probability theory and should be clearly understood: The probability that the sum has a value between y and y + dy, r(y)dy, can be computed as follows: r(y) is given by the probability that the first variable assumes a value between x 1 and x 1 + dx 1, p(x 1 )dx 1 multiplied by the probability that the second variable has a value between x 2 and x 2 + dx 2, p(x 2 )dx 2, with the constraint that the sum should be y (imposed by the delta function) and then integrated over all possible values of x 1 and x 2. Fourier transforming the equation for r(y) we obtain ˆr(k) = [ˆp(k)] 2. (13) This can be generalized easily to the sum of n independent random variables with different n densities p j (x): ˆr(k) = ˆp j (k). j=1 7

Executive Assessment. Executive Assessment Math Review. Section 1.0, Arithmetic, includes the following topics:

Executive Assessment. Executive Assessment Math Review. Section 1.0, Arithmetic, includes the following topics: Executive Assessment Math Review Although the following provides a review of some of the mathematical concepts of arithmetic and algebra, it is not intended to be a textbook. You should use this chapter