XIV. BASICS OF ROBABILITY Somewhere out there is a set of all possile event (or all possile sequences of events which I call Ω. This is called a sample space. Out of this we consider susets of events which I will usually denote y E. Example: The set Ω is the set of all possile starting salaries of new hds in economics. We re interested in a particular suset E eing equal to the range from $58500 to $63400; or we could e interested in the suset consisting of the element $360. Example: The set Ω is the possile outcomes of three coin tosses. The event E could e the outcomes with two heads and one tails. Each element of this set is assigned a proaility. There are three asic rules for proailities:. ( S ( 0. 0 ( E E Ω 3. ( E E ( E E ( E+ ( E The proaility that something in the set happens is one. The proaility that nothing in the set happens is zero. Second the proaility of an event happening is etween zero and one. Finally for two disjoint sets the proaility that the outcome is in the union of these sets equals the sum of the proaility of eing in either one. From these three axioms we also derive these results:. c ( E ( E. ( E E ( E+ ( E ( E E ( E+ ( E 3. E E E3 E E E3 ( E E + + ( E E ( E E + ( E E E 3 3 3 The process continues similarly for finding the proaility of eing in the union of four or more sets. Two events E and E are called independent if ( E E ( E ( E. For instance the proaility that the sun shines in Mongolia tomorrow is aout 3 4. The proaility that I eat an ice cream cone on any given day is 5 6. If these are independent events (and they are then the proaility that tomorrow I eat an ice cream cone and the sun shines in Mongolia is going to e 5 8. On the other hand the weather forecast for Austin says that on any given day the chance of the weather getting over 00 Fahrenheit is 5 6. Incidentally I eat an ice cream cone on exactly those days when the temperature gets over 00 so the proaility that tomorrow Summer 00 math class notes page 98
the it is over 00 in Austin and I eat an ice cream cone is 5 8. These are not independent events. The conditional proaility that event E occurs given that event E is ( E E ( E E E ( Using the definition of independence we see that two events are independent if and only if ( E E ( E. What is the proaility that I eat an ice cream cone conditional on each of the weathers? From this we have that: E E E E E E E E ( E E ( E ( E E ( E E E E C C ( E ( E E+ ( E ( E E This (in either of the two forms given is known as Bayes Law. Example: My friend Edmund is walking though a neighorhood filled with very large very aggressive squirrels. One out of every ten of these animals are raid. A person walking through this neighorhood stands a one in four chance of eing itten y a raid squirrel; one in fifteen of eing itten y the ordinary kind. Edmund gets itten. He s worried; he knows that raid squirrels are much more likely to ite (ut then again there are fewer of them. What s the chance that he was itten y a raid one? Let s call E raid and E ites. We want to find: ( raid ites It appears that the odds are in his favor. raid ites raid ( 0 ( 4 + 4 + + 5 9 0 4 0 5 4 9 5 5 + ( raid ( ites raid+ ( not raid ( ites not raid 5 reviously I introduced proaility density functions for continuous distriutions. A random variale X is defined as a real-valued function defined on the sample space. Essentially the numeric outcome of some random event (squirrels eing raid is not a numeric outcome. I ll write ( X [ a ] for the proaility that the random variale takes on a value in the interval from a to. In this context a proaility density function f has the property that for any suset a [ ] R 5 7 Summer 00 math class notes page 99
a a X f x dx A proaility density function must give proailities that satisfy the rules mentioned aove. Another important function is the cumulative density function F a function that tells the proaility of the outcome of the random variale eing less than or equal to a particular value in the proaility space: x F( x ( X x f( x dx Because the DF is the derivative of the CDF sometimes DFs are represented as the differential quantity df( x. This has exactly the same meaning. Recall that the expected value or mean of X is defined as: [ ] E X xf x dx xdf x The mean is often denoted y µ. The expected value of a real-valued function h of X is defined as: [ ] E h X h x f x dx h x df x These are the same definitions as what given earlier only with slightly different notation. Because integrals are linear expectations have this property: E[ ah( X+ ] ae[ h( X ]+ Rememer the example from the first week of class that the starting salaries of new economics hds are distriuted according to this DF: y f( ( µ y exp σ π σ with µ 564 and σ 873. (All the numers in this example are purely fictitious. My friend just completed the program and got a jo ut he wouldn t tell me how much he s getting paid. I can calculate the expected value of his salary though as we did efore. However he did let it slip that he was getting paid more than $. According to the graduate advisor noody graduating last year had a starting salary over $00. So now I have a it more information what can I say his expected salary is? (Surely it s more than the usual expectation $564. I use a version of Bayes law to calculate the conditional cumulative density function: F( x X [ a ] ( X x X [ a ] ( a X x F( x F a ( X [ a ] F Fa Summer 00 math class notes page 00
[ ] This is the proaility that X is less than or equal to a particular value x a conditional on the knowledge that X is in this interval. Taking the derivative of this we have the conditional proaility density function: f( xx [ a ] f( x F Fa df x F Fa Essentially all the conditional DF does is divide y the size of the region we are conditioning on; this ensures that ( X [ a ] X [ a ] F Fa a f x dx Fa Fa F F which makes it a good DF for the space on which it is defined. We can use the conditional DF to find the conditional expected value of X which is defined as: E[ XX [ a ]] xf x dx F Fa a F Fa The conditional expected value of a function h is defined as: E[ hx X [ a ]] F Fa a hx f xdx xdf x Going ack to the prolem of trying to figure out the expected starting salary of my friend what we have to do is to evaluate the integral: y µ y exp σ π σ 00 dy. The expected value of his salary conditional and divide y F 00 F on eing in this range is: E[ YY [ k k] ] 70 00 00 yf y dy 70 00 f y dy 00 y µ y exp dy σ π σ 00 ( y µ exp dy σ π 70 σ Okay looks like fun. (Rememer: integration y parts as well as tales of values of the CDF of the standard normal distriution are your friends! We might e curious risky a random variale is. There are many different stocks which might have the same expected value ut some have values that are much more volatile than others. erhaps we could ask how much we expect the value of the random variale to deviate from its mean (or expected value: E[ X ] E[ X E[ X] ] E[ X] E[ X] µ 0 Rememer that the expected value within the rackets is treated as a constant. Summer 00 math class notes page 0
So the expected deviation is always zero ecause expected value means that it s just as likely to e aove it as elow it so these all cancel out. An easy way to get rid of the prolem of the positives canceling with the negatives is to square everything. The variance of a random variale is defined as the expected value of its squared deviation from the mean: [ ] [ ] Var( X E ( X E X E X µ With some manipulation we can show that this is equal to: E[ ] ( E[ ] Var X X X Sometimes the variance will e called the second central moment. The square root of the variance is called the standard deviation of X: σ Var( X It turns out that if you evaluate the distriution function I gave for salaries µ is the mean of the distriution and σ is the standard deviation. That particular distriution is known as the normal distriution. And that s all the fun proaility and statistics you get for now. Mean median mode Summer 00 math class notes page 0