pectations - Chapter. pectations.. Introduction In this chapter we define five mathematical epectations the mean variance standard deviation covariance and correlation coefficient. We appl these general formula to an arra of situations involving discrete and continuous random variables obeing single and joint probabilit distribution functions to evaluate epectations of both random variables and functions of random variables. Ideall the reader observes the common analog in the application of the five concepts epressed in a variet of different was... Mean of a Random Variable The mean is another name for the average. A third snonm for mean is the epected value. Let be a random variable with probabilit distribution f. The mean of is f..a if is a discrete random variable and f d..b if is a continuous random variable. We observe that this epectation weights each value of b its corresponding probabilit. ample.: You all ma be prett upset that I suggest that the complicated formulae above give the average when ou have been taught since elementar school that the average is given b: n i n i
pectations - where ou just sum up all the elements in the set and divide b the number of elements. Well let me reassure ou that what ou learned in elementar school is not wrong. This formula above is one case of equation. where the probabilit distribution f /n. This is called the discrete uniform probabilit distribution in the following chapter. Since in this case f is not a function of it can be pulled out of the summation giving the familiar result for the mean. However the uniform distribution is just one of an infinite number of probabilit distributions. The general formula will appl for an probabilit distribution... Mean of a Function of a Random Variable quation. gives the epected value of the random variable. In general however we need the epected value of a function of that random variable. In the general case our equations which define the mean become: h h h f..a if is discrete and h h h f d..b if is continuous where h is some arbitrar function of. You should see that equation. is an eample of the case of equation. where h. This is one kind of function. However there is no point in learning a formula for one function when the formula for all functions is at hand. So equation. is the equation to remember. ample.: In a gambling game three coins are tossed. A man is paid $ when all three coins turn up the same and he will lose $ otherwise. What is the epected gain? In this problem the random variable is the number of heads. The distribution function f is a uniform distribution for the 8 possible outcomes of the gambling game. The function h is the paout or forfeit for outcome. outcome h f HHH $ /8 HHT -$ /8 HTH -$ /8 HTT -$ /8 THH -$ /8 THT -$ /8 TTH -$ /8 TTT $ /8
pectations - We know that the probabilit distribution f /8 since there are 8 random equall probable outcomes. Using equation. we find: 8 8 8 8 8 i i h h h f h The average outcome is that the gambler loses a dollar. We can also work this same problem another wa and make the distribution over the number of heads rather than all possible outcomes. In this case the table looks like: outcome h f HHH $ /8 HHTHTHTHH -$ /8 TTH THT TTH -$ /8 TTT $ /8 Then using equation. again we find: 8 8 8 8 i h f h f h In fact there are other was to define the problem. Choose a distribution that makes sense to ou. The will all give the same answer so long as our distribution agrees with the phsical realit of the problem... Mean of a Function of Two Random Variables quation. gives the epected value of a function of one random variable. This equation can be simpl etended to a function of two random variables. h f h h..a if and are discrete random variables and d d f h h h..b
pectations - if and are continuous random variables. You should see that equation. is entirel analogous to equation.. ample.: Given the joint probabilit densit function f for otherwise find the mean of h. Using equation..b we have h h X Y h f dd dd dd d d h h 6 What this sas is that the average value of for this joint probabilit distribution function is.6... Variance of a Random Variable The mean is one parameter of a distribution of data. It gives us some indication of the location of the random variable. It does not however give us an information about the distribution of the random variable. For eample in Figure.. we observe the visuall three PDFs with different values of the mean but the same value of the variance. The location of each PDF is different but the spread of the PDF remains the same. In Figure.. we observe three PDFs with different values of the variance but the same value of the mean. Here the location of the PDF remains constant but the spread of the PDF increases with increasing variance. The variance is a statistical measure of this spread.
pectations -.. f.... -. -.... 7. Figure.. Three continuous probabilit densit functions with common variance but different values of the mean. Figure.. Three continuous probabilit densit functions with common mean but different values of the variance. We define the variance as follows. Let be a random variable with PDF f and mean. The variance of is defined as [ ] f..a if is a discrete random variable and
[ ] pectations - 6 f d..b if is a continuous random variable. You should see that equation. is just another case of equation. where h. What the variance gives is the average of the square of the deviation from the mean. The square is in there so that all the deviations are positive and the variance is a positive number. Some tricks with the variance: The definition for the variance given above if evaluated properl will alwas give the correct value of the variance. However there is another shortcut formula that is often used. We derive the shortcut here. [ ] Thus we have f d f d f d f [ ] [ ] ] [ ] [ ] [ ] [ ] [ ] f d d f d f d. People frequentl epress the variance as the difference between the mean of the squares and the square of the mean of the random variable. The do this because sometimes ou have [ ] and [ ] so the variance is often easier to calculate from equation. than it is from equation.. ample..: Calculate the mean and variance of the discrete data set of numbers containing {6789}.
pectations - 7 The mean is calculated from equation..a where the probabilit distribution is uniform i.e. f /n. So.. The variance is calculated b squaring each number in the set so that ou have a new set of containing {966968}. Then the mean of this set of numbers using equation..a is 8.. Now using equation. we have: [ ] [ ] 8.. 8. The variance is alwas positive. If ou don t get a positive answer using this formula then ou have most certainl done something wrong. Warning on using equation. the shortcut for the variance The equation. ma look ver friendl but it comes with dangers. You often use this equation to obtain a small variance from the difference of two large numbers. Therefore the answer ou obtain ma contain round-off errors. You need to keep all our insignificant figures in the averages in order to obtain the variance to the same number of significant figures. ample..: Use [ ] [ ] f /. s [9.97978.67686 9.9869898.786788.69869.76788 9.96998 9.96977 9.96 9.986899] to obtain the variance of the following numbers using Below we give a table that reports the means and variance when keeping different number of significant figures: sig figs: 9.99e 9.98e.99e- sig figs: 9.99e 9.986e.99e- 6 sig figs: 9.99e 9.986e.79e- 7 sig figs: 9.99e 9.9866e.8677e- 8 sig figs: 9.997e 9.9866e.669e- 9 sig figs: 9.997e 9.98667e.79e- sig figs: 9.997e 9.98667e.9886e- all sig figs: 9.9978e 9.98667e.e-
pectations - 8 You can see that when we onl keep significant figures our calculated variance is off b 8%! You need to keep additional significant figures in the mean and the mean of the squares in order to get the variance with an accurac. For our information the Matlab code that I used to generate the data is provided in the appendi of this chapter..6. Standard Deviation The standard deviation is the positive square root of the variance. ample.6.: Calculate the standard deviation of the discrete data set of numbers containing {6789}. We calculated the variance in the eample.. above. The standard deviation is the square root of the variance 8..878 The standard deviation gives us a number in the same units as the random variable which describes the spread of the data..7. Variance of a Function of a Random Variable Let be a random variable with PDF f. Let g be an arbitrar function of. We know that the mean of g is and mean g from equation.. The variance of a function g is [ g ] g g g g f.6.a if is a discrete random variable and [ g g ] d.6.b g g g f if is a continuous random variable. You should see that equation.6 is just another case of h g. As in the case equation. where the function of the random variable is g where the function was a second form: h in equation. equation.6 can also be reduced to
pectations - 9 [ g ] [ g ].7 g Beware: we have defined functions f g and h. f is the probabilit distribution. g is the arbitrar function of the random variable that we would like to know about. h is the function with a mean that provides the variance of its argument. In other words if h g then g h. g ample.7.: Given the joint probabilit densit function f for - < < otherwise find the variance of g -. [ ] We will use equation.7. To do so we must find g and [ g ]. [ g ] of g and can be calculated from the formula for the mean equation.. is the mean g g g f d..b d 6 6 6 Now we repeat the calculation for the square of g [ ] [ ] d Then we substitute into equation.7 [ g ] [ g ] g
pectations -.8. Variance & Covariance of a Function of Two Random Variables B analogous methods we can etend the variance definition to a function of two variables. [ g ] g f.8. a g g g if and are discrete random variables and [ g g ] g g f dd.8.b if and are continuous random variables. You should see that equation.8 is just another case h g of equation. where the function of the random variable let s call it g. Again equation.8 can be rewritten [ g ] [ g ] h g g.9 Now let s think about equation.8. If g then h - and we calculate the variance of from equation.8. If g then h - and we have the variance of from equation.8. h then we can use equation.8 to calculate the Now if COVARIANC. The covariance has the units of. There is no function g defined for the covariance so equation.9 does not appl to the calculation of the covariance. But if ou h into equation. and solve as we did to arrive with equation substitute. ou find: [ ] [ ] [ ] X Y. The qualitative significance of the covariance is the dependenc between variables and. qualitative significance > as increases increases and are independent < as increases decreases
pectations - ample.8.: Given the joint probabilit densit function otherwise for f find the covariance of and. To find the covariance we need: [ ] [ ] and [ ]. We alread calculated [ ] in eample.. and we found [ ].6. Using a similar procedure we calculate the epected value of [ ] [ ] d d dd f [ ] d d [ ] 7 In an analogous fashion we calculate [ ] [ ] d d dd f [ ] d d [ ] 6 6 6 6 so using equation. we find:
pectations - 7 6 [ ] [ ] [ ]. 6667.9. Correlation Coefficients The magnitude of does not sa anthing regarding the strength of the relationship between and because. depends on the values taken b and. A scaled version of the covariance called the correlation coefficient is much more useful. The correlation coefficient is defined as. X Y This variable ranges from - to and is when is zero. A negative correlation coefficient means that when increases decreases and vice versa. A positive correlation coefficient means that when increases also increases and vice versa for decreasing. qualitative significance > as increases increases and are independent < as increases decreases - ample.9.: Given the joint PDF in eample.8. find the correlation coefficient and make a statement about whether is strongl or weakl correlated to relative to the variance of and. To do this we need the variance of and which means we need [ ] and [ ] [ ] f dd dd [ ] d d
pectations - [ ] so [ ] [ ] 789. 7 X Now for : [ ] d d dd f g [ ] d d dd [ ] so [ ] [ ] 7. 6 Y so the correlation coefficient is.877.7.789.667 Y X The small value of the correlation coefficient indicates that the random variables and are not strongl correlated but the are weakl negativel correlated... Means and Variances of linear combinations of Random Variables These are several rules for means and variances. These rules have their basis in the theor of linear operators. A linear operator L[] performs some operation on such that: ] [ ] [ ] [ bl al b a L. where and are variables and a and b are constants. This is the fundamental rule which all linear operators must follow.
pectations - d Consider the differential operator: L [ ] [ ]. Is it a linear operator? To prove or disprove dt the linearit of the differential operator ou must substitute it into equation. to verif it. d dt d? [ a b] a d dt [ ] b d dt [ ] d? d d [ a] [ b] a [ ] b [ ] dt dt dt dt d d d d a [ ] b [ ] a [ ] b [ ] This is an identit. dt dt dt dt So we have shown that the differential operator is a linear operator. What about the integral operator L [ ] dt? [ a b] dt a dt? b dt adt bdt a dt a? b dt dt b dt a dt? b dt This is an identit. So we have shown that the differential operator is a linear operator. What about the square operator L [ ]?? [ a b] a b? a ab b a b a a ab b b? we can use the quadratic equation to solve for : ab ± a b a a b b a a For an given value of the solution to this quadratic formula are the onl solutions which satisf equation.. In order for the operator to be linear equation. must be satisfied for all. Therefore the square operator is not a linear operator.
pectations - Now let s see if the mean is a linear operator we will do this just for the continuous case but the result could also be shown for the discrete case:? [ a b] a[ ] b[ ] Substitute in the definition of the mean from equation.? [ a b] f d a f d b f d The integral of a sum is the sum of the integrals. Constants can be pulled outside the integral so a f d b f d a f d b f d This is an identit. The mean is a linear operator. As a result we have a few simplifications for the mean. In the equations below we assume that a and b are constants. The mean of a constant is the constant. a a Adding a constant to a random variable adds the same constant to the mean. a b a b The mean of the sum is the sum of the means. g h g h These rules also appl to joint PDFs. g ± h g h If and onl if and are independent random variables then onl for independent and
pectations - 6 We can show that the variance is not a linear operator. However b substituting in for the definition of the variance equation. we can come up with several short-cuts for computing some variances of functions. Again we assume that a and b are constants. The variance of a constant is zero. b Adding a constant to a random variable does not change the variance of the random variable. a b a The variance of the sum is not the sum of the variances. a b ab a b If and onl if and are independent then a b a b onl for independent and We did not just make an of these theorems up. The can all be derived. As an eample we now derive a b ab a b We begin b direct substitution of ab into the definition of the variance: [ g g ] g f g g g a b a b f dd dd g a b ab a a b b a b a b g a f dd b f dd abf f dd a a b f dd b a b f dd a b dd f dd
pectations - 7 g a a a b f dd b f dd b a b f dd ab f dd a b g a b ab a a b b a b a b a b a b g a b a b ab a ab b g ab b ab a g a b ab Q..D. f dd f dd ab.. An tended ample for Discrete Random Variables Consider the isomerization reaction: A B This reaction takes place in a plant which relies on raw material solution which unfortunatel is supposed to have a concentration of reactant of. mol/liter but in realit varies /- %. The reactor is jacketed and is supposed to be isothermal. Da to da observation of the thermocouples in the reactor indicates that temperature swings about % around its set point of K. The reaction rate is given as r b kc A k o e a RT C A where k is the rate constant k o is the pre-eponential factor of the rate constant a is the activation energ R is the gas constant T is the temperature and C A is the concentration of the reactant. In one such reaction liters kj J k o a and R 8.. Over a min mol mol K month spot measurements are made of the reactor measuring the concentration of the reactant and the temperature.
pectations - 8 Consider that the probabilit of obtaining an of the data points was uniform. Therefore f where n is the number of measurements taken. n The tabulated data and the functions of that data are shown below: runs C A T r B C A T r C T B A CA rb T rb mol liter K mol min. 96.9.8. 879.8. 7.9..9. 7.8.. 79.8.6 7.6. 67.6. 7...6 7..6 78.. 6.96.8...67..6 6.9..7.8 7.87..7 76.9. 8.67.7 6.6 6. 7..8. 78.9.8.. 7.7 7.8 99.6.9.6 897.67.8 9.9. 86.6 8.8...7 79.6.7 7.6..9 9.89.9.7.78 96.. 7.7..7.6 7.78.8. 786..8.. 7.. 98...7 88878.6.6 6.. 8.9..6.. 979..9 7...77. 7..6.7 789..7.. 7.8. 8..9.9 78.87.8 9.. 79.9. 6.7.6. 977.9..7. 9.67 6.87 9.6..76 86..6 77.89. 8. 7.8.6..68 9778.97..76.6 97.7 8.6... 96.8.6.6..6 9.8 89.8.6.69 886..7.. 7.7.89...79 968.97. 67..9 98.8 sum 9.66 896.8 6.66 9.68 76.6. 776. 6.7 98.89 mean.98 9.8..98 87.8. 88.8. 99. variance..7. covariance -...9 standard deviation. 7.9.7 correlation -...8 We use the definition of the mean f to obtain epectation values for the following functions: C A T r B C A T r B C A T CA rb and T r B. The epectations are shown in the table above in the row marked mean. The variances of C A T and rb are calculated using the difference between the mean of the square and the square of the mean rule. [ g ] [ g ] g Those variances are shown in the first three columns in the row marked variance. The covariances are obtained using the formula:
pectations - 9 [ ] [ ] [ ] and are shown in the last three columns for C A T C r and T r A B B. The standard deviations and correlation coefficients are given in the bottom row obtained from: and g g. Phsical eplanation of statistical results: The mean and the standard deviation of the concentration show that statisticall speaking: C A.98 ±. mol liter mol Similarl T 9. ± 7. 9K and r B. ±.7. min The phsical meaning of the correlation coefficients are as follows. The concentration of A and the temperature two independent random variables should not be correlated. The correlation coefficient should be zero. It is -.. This non-zero value is a result of onl having data points. More data points would eventuall average out to a correlation coefficient of zero. The C A rb correlation coefficient should be positive because as the concentration increases the reaction rate increases. It is positive. The C A rb correlation coefficient is small because the relationship is a linear weak relationship. The T rb correlation coefficient should be positive because as the temperature increases the reaction rate increases. It is positive. The T rb correlation coefficient is large because the relationship is an eponential strong relationship... An tended ample for Continuous Random Variables A construction compan has designed a distribution function which describes the area of their construction sites. The sites are all rectangular with dimensions a and b. The Joint PDF of the dimensions a and b are: ab f a b for a < and b otherwise
pectations - The compan is interested in determining pre-construction site costs including fencing and clearing land. The amount of fencing gives rise to a perimeter cost. The Perimeter Costs PC are $ per meter of fencing required: a b PC a b The amount of land cleared is proportional to the area of the site and gives rise to an area cost. The Area Costs AC are $ per square meter of the lot: AC a b ab a Are a and b independent? b Find the mean of a b PC and AC. c Find the variance of a b PC and AC. d Find the covariance of a b a PC a AC b PC b AC and PC AC. e Find the correlation coefficient of a b a PC a AC b PC b AC and PC AC. a a and b are independent if f g h where the marginal distributions are defined in Chapter as g f d and h f d We evaluate the marginal distributions. g a f a b db h b f a b da abdb abda b a a b f ab g h a b 7 Therefore a and b are independent. b Find the mean of a b PC and AC. The general formula for the mean is: a b 7 ab
pectations - h X Y h f dd h b a 7 98.6 a a a abdbda a da m 6 b a 7 7. b b b abdbda a da m We can use the definition of the mean to compute the mean value of the perimeter costs. PC 8 a 7 a 7 8 PC a b abdbda a 8 9 6 $.9 6 6 6 b b a da OR remember that the mean is a linear operator a b a b PC PC a b a b.6. $.6 For the area cost we can use the definition of the mean. AC 8 AC ab abdbda a b 8 a da 7 7 $9.6 89 OR remember for independent variables. AC AC ab a b.6. $9.6 because a and b are independent. c Find the variance of a b PC and AC. The working equation to calculate the variance of a function is: [ g ] [ g ] g
pectations - For these variables we have calculated the mean necessar to evaluate the function in the second term on the right hand side. We must net calculate the mean of the square the first term on the right hand side before we can calculate the variance. a a abdbda a b a da 7 68 a. a [ ] [ ]..6. 8 a a b b b abdbda a a 7 da b. b [ ] [ ]..8. 88 b b To calculate the variance of PC a b a b remember ax by a X b Y ab Then we onl need to calculate the covariance of a and b. The working formula for the covariance is: [ ] [ ] [ ] So we need the epectation value of ab ab ab ab abdbda a b da a 7 6.8 89 Then ab.8.6.. The covariance of a and b is zero. We should have known that because we showed in part a that a and b were statisticall independent. PC 6.. a b a b ab
pectations - Lastl for the area cost AC 6 AC ab abdbda a 6 a 7 6. b da AC [ AC ] [ ]. 9.6 8. AC d Find the covariance of a b a PC a AC b PC b AC and PC AC. In part c we found the covariance of a b to be. because the were statisticall independent. For the rest of these quantities we use the rule [ ] [ ] [ ] where we alread have the epectation values of the two factors in the second term on the r.h.s. We onl need to find the first term on the r.h.s. to find the covariance. For the covariance between a and the perimeter cost we have apc 8 a 8 apc a ab abdbda a 7 a 7 8 9 8 9 6 9.6 b a b da apc [ apc] [ a] [ ] 9.6.6.9. 6 PC For the covariance between b and the perimeter cost we have bpc 8 a 8 bpc ab b abdbda a 7 a 7 8 9 9 8 76 b b a da 9.6 bpc [ ] [ b] [ ] 9.6.8.9. 6 bpc PC
pectations - For the covariance between a and the area cost we have aac 8 aac a b abdbda a 8 a 7 76.9 b da aac [ aac] [ a] [ ] 76.9.69.6. 6 AC For the covariance between b and the area cost we have bac 8 bac ab abdbda a 8 a 7 98 88.89 b da bac [ ] [ b] [ ] 88.89.9.6. bac AC For the covariance between the perimeter cost and the area cost we have ACPC 6 a 6 ACPC a b ab abdbda a 7 a 7 6 88.6 b a b da ACPC [ ACPC] [ AC] [ ].6 9.6.9 6. PC e Find the correlation coefficient of The general formula for the correlation coefficient is: X Y a b a PC a AC b PC b AC and PC AC. ab ab a b..6..
pectations - 6. PCAC PCAC PCAC 6. 8.6 apc apc apc.6 6..6 bpc bpc bpc. 6..6 aac aac aac.6 8.. bac bac bac. 8.9....7 These correlations with the eception of a and b are all positive. The should be because as ou increase one side of the lot either a or b ou should increase both the perimeter and the area. Also as ou increase the perimeter on average ou increase the area given our distribution function... Subroutines Code.. Variance as a function of truncation This simple code illustrates the need to keep all significant figures in the intermediate calculations of averages before computing the variance using equation.. n; r randn; s.**r - s s.^; f /n; format long mu_s sumf*s; mu_s sumf*s; var_s mu_s - mu_s^; for i ::8 mu_s_cuti roundmu_s*^i/^i; mu_s_cuti roundmu_s*^i/^i; var_s_cuti mu_s_cuti - mu_s_cuti^; fprintf'%i sig figs: mu_s %6.9e mu_s %6.9e var_s %6.9e\n' i mu_s_cutimu_s_cutivar_s_cuti; end
pectations - 6 fprintf'all sig figs: mu_s %6.9e mu_s %6.9e var_s %6.9e\n' mu_smu_svar_s;.. Problems Homework problems are posted on the course website.