. Introduction The quick summary, going forwards: Start with random variable X. 2 Compute the mean EX and variance 2 = varx. 3 Approximate X by the normal distribution N with mean µ = EX and standard deviation. That is, we make the approximation Pa < X < b Pa < N < b. It's not always good, but we can always make it. 4 Convert the normal distribution N to the standard normal distribution Z. Specically Z = N µ and so a µ Pa < N < b = P < Z < b µ 5 Look up the appropriate values in a table and you're done. Sometimes you just follow these steps going backwards a bit. 2. Getting an intuition for the Central Limit Theorem The following pictures are meant to illustrate the idea of the central limit theorem. Let's put say for the sake of discussion, that we are ipping an unfair coin which has the probability p = of coming up heads. If we perform just one trial, then we expect to get no heads 0 nine-tenths of the time and a heads one-tenth of the time. Figure 2. gives a histogram representation of the distribution. Notice that the height of the rectangle centered at 0 is Figure 2.. Histogram of binomial distribution with trials and probabilty p of success blue. Normal distribution with mean p and variance p p red. nine-tenths and the height of the rectangle centered at is one-tenth. If we were to use a normal distribution to approximate the rst box, we'd nd the area under the normal distribution from 0.5 to 0.5. This is illustrated as computing the green shaded area in Figure 2.2.
2 Figure 2.2. The green shaded area represents the approximation of the probability of getting no heads. Now what happens when we increase the number of trials? As we ip n coins, we have the following information: Pk heads show up = n p k p n k, k for k ranging between 0 and n. The mean number of heads to show up is ES n = np and the variance is vars n = np p. For example, when we ip 50 coins, we'd expect heads to come up, on average, 50 0 = 5 times. Let's look at the corresponding mass density functions of the binomial distribution for n = 5 Figure 2.3, n = 50 Figure 2.4, and n = 00 Figure 2.5. As before, we also plot the probability density function for the normal distribution of corresponding mean and variance. For n = 5, notice that there is a probability of about 0.6 of getting no heads. We could approximate this by nding the area under the corresponding normal distribution from x = 0.5 to 0.5. Then the probability of getting more heads decreases. But then for n = 50, there is less than a probability of 0.0 for getting no heads and the probability of getting more heads increases at rst, peaking at the probability to get 5 heads, and then decreasing. In general, as n increases, the normal distribution becomes a better and better approximation for S n. Note also that the approximation of the normal distribution to the binomial distribution for n = above is relatively bad compared to the case for n = 00. Let's continue to the next section with an example.
3 Figure 2.3. Mass density function of the binomial distribution with 5 trials and probabilty p of success blue truncated at n = 3. Probability density function of the normal distribution with mean 5p and variance 5p p red. Figure 2.4. Mass density function of the binomial distribution with 50 trials and probabilty p of success blue truncated at n = 3. Probability density function of the normal distribution with mean 50p and variance 50p p red. Figure 2.5. Mass density function of the the binomial distribution with 00 trials and probabilty p of success blue truncated at n = 22. Probability density function of the normal distribution with mean 00p and variance 00p p red.
4 Figure 3.. Histogram of binomial distribution with 200 trials and probabilty p of success blue. Normal distribution with mean 200p and variance 200p p red. Shaded in blue is the probability of getting at least 23 heads in 200 ips of a coin with probability of success p. Figure 3.2. Histogram of binomial distribution with 200 trials and probabilty p of success blue. Normal distribution with mean 200p and variance 200p p red. Shaded in red is the approximation of getting at least 23 heads in 200 ips of a coin with probability of success p. 3. Doing an Example Let's nd the probability of getting at least 23 heads after ipping a coin 200 times provided the coin has a probability of success coming up heads p =. Thus we're looking for the 0 shaded blue area in Figure 3.. But instead of using the formula for binomial distribution and adding up many terms, we'll use the normal curve to approximate. Thus, we'll be looking for the shaded red area in Figure 3.2. Note that the red region starts at 22.5 to improve the approximation. But what are we to do, because all we have is a table for the standard normal distribution! The key is in computing z-values! z = x µ,
where µ is the mean and is the standard deviation. You can choose to read some theory in Section 3. or head straight to Section 3.2 to continue with the example. 3.. Theory Behind z-value. We start o with a normal distribution X with mean µ and standard deviation, and want to nd the probability that X lies between a and b. For example, we have Figure 3.3a. By denition we have Pa X b = = ˆ b a ˆ b a fxdx 2π e x µ2 /22 dx. First let's make a u-substitution, u = x µ. Then du = dx and we have ˆ b a 2π e x µ2 /22 dx = ˆ b µ a µ 2π e u2 /22 du. Next, let's make another substitition, v = u. Thus dv = du and we have ˆ b µ a µ ˆ b µ/ /22 du = 2π e u2 a µ/ a µ = P 2π e v2 /2 dv Z b µ, where Z is the standard normal distribution mean 0 and standard deviation. Summarizing, given a normal distribution X with mean µ and standard deviation, a µ Pa X b = P X µ b µ a µ = P Z b µ where Z = X µ is the standard normal distribution. We graph the new labeling in Figure 3.3b. Notice that the shape of the curve and the area we're looking to compute remains the same as Figure 3.3a. 3.2. Resuming the Example. At the beginning of the example, we established that the probability of ipping at least 23 heads after 200 coin ips with probality is approximately 0 the area under the normal distribution X with mean 200 = 20 and variance 200 9 = 8 0 0 0 so standard deviation 3 2. In short, we have By using z-values, see theory above, we have PS 200 23 PX 22.5. PX 22.5 = P Z 22.5 20 3 2 5
6 a Normal distribution X with mean µ = 0 and standard deviation = 3. Shaded in red is probability that X lies between a = 7 and b = 5 b Standard normal distribution Z mean 0 and standard deviation. Shaded in red is probability that X lies between 7 0 3 = and 5 0 3 = 5 3. Note that to fully imitate the look of Figure 3.3a, the y-axis has been shifted from intersecting the x-axis at 0 to intersecting it at 0 0 3 = 0 3. Figure 3.3
Approximating the z-value 22.5 20 3 as 0.589..., we look in the chart to nd PZ 0.59 = 2 0.7224 and we conclude 27.5 20 P Z 3 0.7224 = 0.2776. 2 If we were to replace the numbers in this example with letters, we'd have the following. PS n k PX k 0.5 where X has mean np and variance np p and so standard deviation np p. Using z-values, we have k 0.5 np PX k 0.5 = P Z np p 7 where Z = X np np p is the standard normal distribution. 3.3. Some more theory. The end of the last example leads up the next discussion, where we're interested in X = X, where X is the total number of heads that appear after n n tosses. We have and µ = EX = n EX = n np = p varx = n varx = p p np p = 2 n2 n so that p p =. n Then we approximate, for large n, a µ Pa X b P Z b µ n a p = P Z b p n, p p p p where Z = n X p p p In any case, all we're ever doing is approximating rst by a normal distribution with the mean and standard deviation of the original distribution. Then we convert the normal distribution to a standard normal distribution.
4. Example, Going Backwards Let's apply what we've learned going backwards. Instead of being given the probability, we want to determine how often we have to ip a coin to know the probability of heads coming up within 0. of its true value with probability at least 0.8. That is, for what n do we have P Xn p 0. 0.8. Well let's rewrite the left-hand side to match what we've been discussing. P Xn p 0. = P 0. Xn p 0. = P 0. + p X n 0. + p Great, now we have our random variable between two values and we're ready to approximate it by a normal distribution. We subtract its mean p and divide by its standard deviation p p n and obtain n 0. P Z 0. n 0.8. p p p p Now part of going backwards is to nd the value c such that P c Z c = 0.8. We can use our table to do this. Using a table that gives values from to z for z 0, we have 0.2 = 0.8 and dividing by two gives us 0., so that we look for the value 0.9 in the table and nd.29 rounding up, instead of down. We have Then we need P.29 Z.29 = 0.905 0.0985 = 0.8030. n 0. p p.29 8 which is the same as n 2.9 p p or n 2.9 2 p p. Since we don't know p, the worst-case scenario is when p =, so we need 2 n 2.9 2 4.6 2 2