PROBABILITY AND MATHEMATICAL STATISTICS Prasaa Sahoo Departmet of Mathematics Uiversity of Louisville Louisville, KY 409 USA
THIS BOOK IS DEDICATED TO AMIT SADHNA MY PARENTS, TEACHERS AND STUDENTS v
vi
Copyright c 003. All rights reserved. This book, or parts thereof, may ot be reproduced i ay form or by ay meas, electroic or mechaical, icludig photocopyig, recordig or ay iformatio storage ad retrieval system ow kow or to be iveted, without writte permissio from the author. vii
viii
ix PREFACE This book is both a tutorial ad a textbook. I this book we preset a itroductio to probability ad mathematical statistics ad it is iteded for studets already havig some elemetary mathematical backgroud. It is iteded for a oe-year seior level udergraduate ad begiig graduate level course i probability theory ad mathematical statistics. The book cotais more material tha ormally would be taught i a oe-year course. This should give the teacher flexibility with respect to the selectio of the cotet ad level at which the book is to be used. It has arise from over 5 years of lectures i seior level calculus based courses i probability theory ad mathematical statistics at the Uiversity of Louisville. Probability theory ad mathematical statistics are difficult subjects both for studets to comprehed ad teachers to explai. A good set of examples makes these subjects easy to uderstad. For this reaso aloe we have icluded more tha 350 completely worked out examples ad over 65 illustratios. We give a rigorous treatmet of the fudametals of probability ad statistics usig mostly calculus. We have give great attetio to the clarity of the presetatio of the materials. I the text theoretical results are preseted as theorems, propositio or lemma, of which as a rule rigorous proofs are give. I the few exceptios to this rule refereces are give to idicate where details ca be foud. This book cotais over 450 problems of varyig degrees of difficulty to help studets master their problem solvig skill. To make this less wordy we have There are several good books o these subjects ad perhaps there is o eed to brig a ew oe to the market. So for several years, this was circulated as a series of typeset lecture otes amog my studets who were preparig for the examiatio 0 of the Actuarial Society of America. May of my studets ecouraged me to formally write it as a book. Actuarial studets will beefit greatly from this book. The book is writte i simple Eglish; this might be a advatage to studets whose ative laguage is ot Eglish.
I caot claim that all the materials I have writte i this book are mie. I have leared the subject from may excellet books, such as Itroductio to Mathematical Statistics by Hogg ad Craig, ad A Itroductio to Probability Theory ad Its Applicatios by Feller. I fact, these books have had a profoud impact o me, ad my explaatios are iflueced greatly by these textbooks. If there are some resemblaces, the it is perhaps due to the fact that I could ot improve the origial explaatios I have leared from these books. I am very thakful to the authors of these great textbooks. I am also thakful to the Actuarial Society of America for lettig me use their test problems. I thak all my studets i my probability theory ad mathematical statistics courses from 988 to 003 who helped me i may ways to make this book possible i the preset form. Lastly, if it was t for the ifiite patiece of my wife, Sadha, for last several years, this book would ever gotte out of the hard drive of my computer. The etire book was typeset by the author o a Macitosh computer usig TEX, the typesettig system desiged by Doald Kuth. The figures were geerated by the author usig MATHEMATICA, a system for doig mathematics desiged by Wolfram Research, ad MAPLE, a system for doig mathematics desiged by Maplesoft. The author is very thakful to the Uiversity of Louisville for providig may iteral fiacial grats while this book was uder preparatio. x Prasaa Sahoo, Louisville
xi
xii TABLE OF CONTENTS. Probability of Evets..................... Itroductio.. Coutig Techiques.3. Probability Measure.4. Some Properties of the Probability Measure.5. Review Exercises. Coditioal Probability ad Bayes Theorem....... 7.. Coditioal Probability.. Bayes Theorem.3. Review Exercises 3. Radom Variables ad Distributio Fuctios....... 45 3.. Itroductio 3.. Distributio Fuctios of Discrete Variables 3.3. Distributio Fuctios of Cotiuous Variables 3.4. Percetile for Cotiuous Radom Variables 3.5. Review Exercises 4. Momets of Radom Variables ad Chebychev Iequality. 73 4.. Momets of Radom Variables 4.. Expected Value of Radom Variables 4.3. Variace of Radom Variables 4.4. Chebychev Iequality 4.5. Momet Geeratig Fuctios 4.6. Review Exercises
xiii 5. Some Special Discrete Distributios........... 07 5.. Beroulli Distributio 5.. Biomial Distributio 5.3. Geometric Distributio 5.4. Negative Biomial Distributio 5.5. Hypergeometric Distributio 5.6. Poisso Distributio 5.7. Riema Zeta Distributio 5.8. Review Exercises 6. Some Special Cotiuous Distributios......... 4 6.. Uiform Distributio 6.. Gamma Distributio 6.3. Beta Distributio 6.4. Normal Distributio 6.5. Logormal Distributio 6.6. Iverse Gaussia Distributio 6.7. Logistic Distributio 6.8. Review Exercises 7. Two Radom Variables................. 85 7.. Bivariate Discrete Radom Variables 7.. Bivariate Cotiuous Radom Variables 7.3. Coditioal Distributios 7.4. Idepedece of Radom Variables 7.5. Review Exercises 8. Product Momets of Bivariate Radom Variables.... 3 8.. Covariace of Bivariate Radom Variables 8.. Idepedece of Radom Variables 8.3. Variace of the Liear Combiatio of Radom Variables 8.4. Correlatio ad Idepedece 8.5. Momet Geeratig Fuctios 8.6. Review Exercises
xiv 9. Coditioal Expectatios of Bivariate Radom Variables 37 9.. Coditioal Expected Values 9.. Coditioal Variace 9.3. Regressio Curve ad Scedastic Curves 9.4. Review Exercises 0. Fuctios of Radom Variables ad Their Distributio. 57 0.. Distributio Fuctio Method 0.. Trasformatio Method for Uivariate Case 0.3. Trasformatio Method for Bivariate Case 0.4. Covolutio Method for Sums of Radom Variables 0.5. Momet Method for Sums of Radom Variables 0.6. Review Exercises. Some Special Discrete Bivariate Distributios..... 89.. Bivariate Beroulli Distributio.. Bivariate Biomial Distributio.3. Bivariate Geometric Distributio.4. Bivariate Negative Biomial Distributio.5. Bivariate Hypergeometric Distributio.6. Bivariate Poisso Distributio.7. Review Exercises. Some Special Cotiuous Bivariate Distributios.... 37.. Bivariate Uiform Distributio.. Bivariate Cauchy Distributio.3. Bivariate Gamma Distributio.4. Bivariate Beta Distributio.5. Bivariate Normal Distributio.6. Bivariate Logistic Distributio.7. Review Exercises
xv 3. Sequeces of Radom Variables ad Order Statistics.. 35 3.. Distributio of Sample Mea ad Variace 3.. Laws of Large Numbers 3.3. The Cetral Limit Theorem 3.4. Order Statistics 3.5. Sample Percetiles 3.6. Review Exercises 4. Samplig Distributios Associated with the Normal Populatio................. 389 4.. Chi-square distributio 4.. Studet s t-distributio 4.3. Sedecor s F -distributio 4.4. Review Exercises 5. Some Techiques for Fidig Poit Estimators of Parameters............... 407 5.. Momet Method 5.. Maximum Likelihood Method 5.3. Bayesia Method 5.3. Review Exercises 6. Criteria for Evaluatig the Goodess of Estimators..................... 447 6.. The Ubiased Estimator 6.. The Relatively Efficiet Estimator 6.3. The Miimum Variace Ubiased Estimator 6.4. Sufficiet Estimator 6.5. Cosistet Estimator 6.6. Review Exercises
xvi 7. Some Techiques for Fidig Iterval Estimators of Parameters............... 487 7.. Iterval Estimators ad Cofidece Itervals for Parameters 7.. Pivotal Quatity Method 7.3. Cofidece Iterval for Populatio Mea 7.4. Cofidece Iterval for Populatio Variace 7.5. Cofidece Iterval for Parameter of some Distributios ot belogig to the Locatio-Scale Family 7.6. Approximate Cofidece Iterval for Parameter with MLE 7.7. The Statistical or Geeral Method 7.8. Criteria for Evaluatig Cofidece Itervals 7.9. Review Exercises 8. Test of Statistical Hypotheses............. 53 8.. Itroductio 8.. A Method of Fidig Tests 8.3. Methods of Evaluatig Tests 8.4. Some Examples of Likelihood Ratio Tests 8.5. Review Exercises 9. Simple Liear Regressio ad Correlatio Aalysis.. 575 9.. Least Squared Method 9.. Normal Regressio Aalysis 9.3. The Correlatio Aalysis 9.4. Review Exercises 0. Aalysis of Variace.................. 6 0.. Oe-way Aalysis of Variace with Equal Sample Sizes 0.. Oe-way Aalysis of Variace with Uequal Sample Sizes 0.3. Pair wise Comparisos 0.4. Tests for the Homogeeity of Variaces 0.5. Review Exercises
xvii. Goodess of Fits Tests................. 643.. Chi-Squared test.. Kolmogorov-Smirov test.3. Review Exercises Refereces......................... 659 Aswers to Selected Review Exercises........... 665
Probability ad Mathematical Statistics 35 Chapter 3 SEQUENCES OF RANDOM VARIABLES AND ORDER STASTISTICS I this chapter, we geeralize some of the results we have studied i the previous chapters. We do these geeralizatios because the geeralizatios are eeded i the subsequet chapters relatig mathematical statistics. I this chapter, we also examie the weak law of large umbers, the Beroulli s law of large umbers, the strog law of large umbers, ad the cetral limit theorem. Further, i this chapter, we treat the order statistics ad percetiles. 3.. Distributio of sample mea ad variace Cosider a radom experimet. Let X be the radom variable associated with this experimet. Let f(x) be the probability desity fuctio of X. Let us repeat this experimet times. Let X k be the radom variable associated with the k th repetitio. The the collectio of the radom variables { X,X,..., X } is a radom sample of size. From here after, we simply deote X,X,..., X as a radom sample of size. The radom variables X,X,..., X are idepedet ad idetically distributed with the commo probability desity fuctio f(x). For a radom sample, fuctios such as the sample mea X, the sample variace S are called statistics. I a particular sample, say x,x,..., x,we
Sequeces of Radom Variables ad Order Statistics 35 observed x ad s. We may cosider ad X = S = X i ( Xi X ) as radom variables ad x ad s are the realizatios from a particular sample. I this sectio, we are maily iterested i fidig the probability distributios of the sample mea X ad sample variace S, that is the distributio of the statistics of samples. Example 3.. Let X ad X be a radom sample of size from a distributio with probability desity fuctio { f(x) = 6x( x) if 0 <x< 0 otherwise. What are the mea ad variace of sample sum Y = X + X? Aswer: The populatio mea µ X = E (X) = =6 0 0 x 6x( x) dx x ( x) dx = 6B(3, ) (here B deotes the beta fuctio) Γ(3) Γ() =6 Γ(5) ( ) =6 =. Sice X ad X have the same distributio, we obtai µ X = = µ X. Hece the mea of Y is give by E(Y )=E(X + X ) = E(X )+E(X ) = + =.
Probability ad Mathematical Statistics 353 Next, we compute the variace of the populatio X. The variace of X is give by Var(X) =E ( X ) E(X) ( ) = 6x 3 ( x) dx 0 =6 x 3 ( x) dx 0 ( ) =6B(4, ) 4 ( ) Γ(4) Γ() =6 Γ(6) 4 ( ) ( ) =6 0 4 ( 4 = 6 0 5 0 = 0. Sice X ad X have the same distributio as the populatio X, we get ) Var(X )= 0 = Var(X ). Hece, the variace of the sample sum Y is give by Var(Y )=Var(X + X ) = Var(X )+Var(X )+Cov (X,X ) = Var(X )+Var(X ) = 0 + 0 = 0. Example 3.. Let X ad X be a radom sample of size from a distributio with desity f(x) = { 4 for x =,, 3, 4 0 otherwise. What is the distributio of the sample sum Y = X + X?
Sequeces of Radom Variables ad Order Statistics 354 Aswer: Sice the rage space of X as well as X is {,, 3, 4}, the rage space of Y = X + X is R Y = {, 3, 4, 5, 6, 7, 8}. Let g(y) be the desity fuctio of Y. We wat to fid this desity fuctio. First, we fid g(), g(3) ad so o. g() = P (Y =) = P (X + X =) = P (X = ad X =) = P (X =)P (X = ) (by idepedece of X ad X ) = f() f() = ( )( ) = 4 4 6. g(3) = P (Y =3) = P (X + X =3) = P (X = ad X =)+P (X = ad X =) = P (X =)P (X =) + P (X =)P (X = ) (by idepedece of X ad X ) = f() f() + f() f() = ( 4 )( ) + 4 ( 4 )( ) = 4 6.
Probability ad Mathematical Statistics 355 g(4) = P (Y =4) = P (X + X =4) = P (X = ad X =3)+P (X = 3 ad X =) + P (X = ad X =) = P (X =3)P (X =)+P (X =)P (X =3) + P (X =)P (X = ) (by idepedece of X ad X ) = f() f(3) + f(3) f() + f() f() ( )( ) ( )( ) ( )( ) = + + 4 4 4 4 4 4 = 3 6. Similarly, we get g(5) = 4 6, g(6) = 3 6, g(7) = 6, g(8) = 6. Thus, puttig these ito oe expressio, we get g(y) =P (Y = y) y = f(k) f(y k) = k= 4 y 5, y =, 3, 4,..., 8. 6 y Remark 3.. Note that g(y) = f(k) f(y k) is the discrete covolutio k= of f with itself. The cocept of covolutio was itroduced i chapter 0. The above example ca also be doe usig the momet geeratig fuc-
Sequeces of Radom Variables ad Order Statistics 356 tio method as follows: M Y (t) =M X+X (t) = M X (t) M X (t) ( e t + e t + e 3t + e 4t )( e t + e t + e 3t + e 4t ) = 4 4 ( e t + e t + e 3t + e 4t ) = 4 = et +e 3t +3e 4t +4e 5t +3e 6t +e 7t + e 8t. 6 Hece, the desity of Y is give by g(y) = 4 y 5, y =, 3, 4,..., 8. 6 Theorem 3.. If X,X,..., X are mutually idepedet radom variables with desities f (x ),f (x ),..., f (x ) ad E[u i (X i )], i =,,..., exist, the [ ] E u i (X i ) = E[u i (X i )], where u i (i =,,..., ) are arbitrary fuctios. Proof: We prove the theorem assumig that the radom variables X,X,..., X are cotiuous. If the radom variables are ot cotiuous, the the proof follows exactly i the same maer if oe replaces the itegrals by summatios. Sice ( ) E u i (X i ) = E(u (X ) u (X )) = = = u (x ) u (x )f(x,..., x )dx dx u (x ) u (x )f (x ) f (x )dx dx u (x )f (x )dx = E (u (X )) E (u (X )) = E (u i (X i )), u (x )f (x )dx
Probability ad Mathematical Statistics 357 the proof of the theorem is ow complete. Example 3.3. Let X ad Y be two radom variables with the joit desity { e (x+y) for 0 <x,y< f(x, y) = 0 otherwise. What is the expected value of the cotiuous radom variable Z = X Y + XY + X + X? Aswer: Sice f(x, y) =e (x+y) = e x e y = f (x) f (y), the radom variables X ad Y are mutually idepedet. Hece, the expected value of X is E(X) = xf (x) dx = 0 0 xe x dx = Γ() =. Similarly, the expected value of X is give by E ( X ) = = 0 0 = Γ(3) =. x f (x) dx x e x dx Sice the margials of X ad Y are same, we also get E(Y ) = ad E(Y )=. Further, by Theorem 3., we get E [Z] =E [ X Y + XY + X + X ] = E [( X + X )( Y + )] = E [ X + X ] E [ Y + ] (by Theorem 3.) = ( E [ X ] + E [X] )( E [ Y ] + ) =(+)(+) =9.
Sequeces of Radom Variables ad Order Statistics 358 Theorem 3.. If X,X,..., X are mutually idepedet radom variables with respective meas µ,µ,..., µ ad variaces σ,σ,..., σ, the the mea ad variace of Y = a i X i, where a,a,..., a are real costats, are give by µ Y = a i µ i ad σy = Proof: First we show that µ Y = a i µ i. Sice µ Y = E(Y ) ( ) = E a i X i = = a i E(X i ) a i µ i a i σi. we have asserted result. Next we show σ Y = a i σ i. Cosider σy = Var(Y ) = Var(a i X i ) = a i Var(X i ) = a i σi. This completes the proof of the theorem. Example 3.4. Let the idepedet radom variables X ad X have meas µ = 4 ad µ = 3, respectively ad variaces σ = 4 ad σ =9. What are the mea ad variace of Y =3X X? Aswer: The mea of Y is µ Y =3µ µ =3( 4) (3) = 8.
Probability ad Mathematical Statistics 359 Similarly, the variace of Y is σy = (3) σ +( ) σ =9σ +4σ = 9(4) + 4(9) =7. Example 3.5. Let X,X,..., X 50 be a radom sample of size 50 from a distributio with desity { θ e x θ for 0 x< f(x) = 0 otherwise. What are the mea ad variace of the sample mea X? Aswer: Sice the distributio of the populatio X is expoetial, the mea ad variace of X are give by µ X = θ, ad σx = θ. Thus, the mea of the sample mea is E ( X ) ( ) X + X + + X 50 = E 50 = 50 = 50 50 50 E (X i ) = 50 θ = θ. 50 The variace of the sample mea is give by θ Var ( X ) = Var = = 50 50 =50 = θ 50. ( 50 ) 50 X i ( ) σx 50 i ( ) θ 50 ) θ ( 50
Sequeces of Radom Variables ad Order Statistics 360 Theorem 3.3. If X,X,..., X are idepedet radom variables with respective momet geeratig fuctios M Xi (t), i =,,...,, the the momet geeratig fuctio of Y = a ix i is give by Proof: Sice M Y (t) = M Xi (a i t). M Y (t) =M aixi(t) = M aix i (t) = M Xi (a i t) we have the asserted result ad the proof of the theorem is ow complete. Example 3.6. Let X,X,..., X 0 be the observatios from a radom sample of size 0 from a distributio with desity f(x) = π e x, <x<. What is the momet geeratig fuctio of the sample mea? Aswer: The desity of the populatio X is a stadard ormal. Hece, the momet geeratig fuctio of each X i is M Xi (t) =e t, i =,,..., 0. The momet geeratig fuctio of the sample mea is Hece X N ( 0, 0). M X (t) =M 0 = = = 0 (t) 0 Xi M Xi ( 0 t ) 0 e t 00 [ e t 00 ] 0 = e ( 0 t ).
Probability ad Mathematical Statistics 36 The last example tells us that if we take a sample of ay size from a ormal populatio, the the sample mea also has a ormal distributio. The followig theorem says that a liear combiatio of radom variables with ormal distributios is agai ormal. Theorem 3.4. If X,X,..., X are mutually idepedet radom variables such that X i N ( µ i,σi ), i =,,...,. The the radom variable Y = a i X i is a ormal radom variable with mea µ Y = a i µ i ad σy = a i σi, that is Y N ( a iµ i, ) a i σ i. Proof: Sice each X i N ( ) µ i,σi, the momet geeratig fuctio of each X i is give by M Xi (t) =e µit+ σ i t. Hece usig Theorem 3.3, we have M Y (t) = M Xi (a i t) = e µit+ σ i t µit+ = e ( Thus the radom variable Y N a i µ i, σ i t. ) a i σi. The proof of the theorem is ow complete. Example 3.7. Let X,X,..., X be the observatios from a radom sample of size from a ormal distributio with mea µ ad variace σ > 0. What are the mea ad variace of the sample mea X? Aswer: The expected value (or mea) of the sample mea is give by E ( X ) = E (X i ) = = µ. µ
Sequeces of Radom Variables ad Order Statistics 36 Similarly, the variace of the sample mea is Var ( X ) = ( ) Xi Var = ( ) σ = σ. This example alog with the previous theorem says that if we take a radom sample of size from a ormal populatio with mea µ ad variace σ, the the ( sample ) mea is also ormal with mea µ ad variace σ, that is X N µ, σ. Example 3.8. Let X,X,..., X 64 be a radom sample of size 64 from a ormal distributio with µ = 50 ad σ = 6. What are P (49 <X 8 < 5) ad P ( 49 < X<5 )? Aswer: Sice X 8 N(50, 6), we get P (49 <X 8 < 5) = P (49 50 <X 8 50 < 5 50) ( 49 50 = P < X ) 8 50 5 50 < 4 4 4 ( = P 4 < X 8 50 < ) 4 4 ( = P ) 4 <Z< 4 ( =P Z< ) 4 = 0.974 (from ormal table). By the previous theorem, we see that X N ( 50, 64) 6. Hece P ( 49 < X<5 ) = P ( 49 50 < X 50 < 5 50 ) = P 49 50 6 64 = P < X 50 6 64 < X 50 6 64 < < 5 50 6 64 = P ( <Z<) =P (Z <) = 0.9544 (from ormal table).
Probability ad Mathematical Statistics 363 This example tells us that X has a greater probability of fallig i a iterval cotaiig µ, tha a sigle observatio, say X 8 (or i geeral ay X i ). Theorem 3.5. Let the distributios of the radom variables X,X,..., X be χ (r ),χ (r ),..., χ (r ), respectively. If X,X,..., X are mutually idepedet, the Y = X + X + + X χ ( r i). Proof: Sice each X i χ (r i ), the momet geeratig fuctio of each X i is give by M Xi (t) =( t) r i. By Theorem 3.3, we have M Y (t) = M Xi (t) = ( t) r i =( t) ri. Hece Y χ ( r i) ad the proof of the theorem is ow complete. The proof of the followig theorem is a easy cosequece of Theorem 3.5 ad we leave the proof to the reader. Theorem 3.6. If Z,Z,..., Z are mutually idepedet ad each oe is stadard ormal, the Z + Z + + Z χ (), that is the sum is chi-square with degrees of freedom. The followig theorem is very useful i mathematical statistics ad its proof is beyod the scope of this itroductory book. Theorem 3.7. If X,X,..., X are observatios of a radom sample of size from the ormal distributio N ( µ, σ ), the the sample mea X = X i ad the sample variace S = (X i X) have the followig properties: (A) X ad S are idepedet, ad (B) ( ) S σ χ ( ). Remark 3.. At first sight the statemet (A) might seem odd sice the sample mea X occurs explicitly i the defiitio of the sample variace S. This remarkable idepedece of X ad S is a uique property that distiguishes ormal distributio from all other probability distributios. Example 3.9. Let X,X,..., X deote a radom sample from a ormal distributio with variace σ > 0. If the first percetile of the statistics W = (X i X) σ is.4, where X deotes the sample mea, what is the sample size?
Sequeces of Radom Variables ad Order Statistics 364 Aswer: = P (W.4) 00 ( ) (X i X) = P σ.4 ) = P (( ) S σ.4 = P ( χ ( ).4 ). Thus from χ -table, we get =7 ad hece the sample size is 8. Example 3.0. Let X,X,..., X 4 be a radom sample from a ormal distributio with ukow mea ad variace equal to 9. Let S = 4 ( 3 Xi X ).IfP ( S k ) =0.05, the what is k? Aswer: 0.05 = P ( S k ) ( 3S = P 9 3 ) 9 k = P (χ (3) 39 ) k. From χ -table with 3 degrees of freedom, we get 3 9 k =0.35 ad thus the costat k is give by k = 3(0.35) =.05. 3.. Laws of Large Numbers I this sectio, we maily examie the weak law of large umbers. The weak law of large umbers states that if X,X,..., X is a radom sample of size from a populatio X with mea µ, the the sample mea X rarely deviates from the populatio mea µ whe the sample size is very large. I other words, the sample mea X coverges i probability to the populatio mea µ. We begi this sectio with a result kow as Markov iequality which is eed to establish the weak law of large umbers.
Probability ad Mathematical Statistics 365 Theorem 3.8 (Markov Iequality). Suppose X is a oegative radom variable with mea E(X). The P (X t) E(X) t for all t>0. Proof: We assume the radom variable X is cotiuous. If X is ot cotiuous, the a proof ca be obtaied for this case by replacig the itegrals with summatios i the followig proof. Sice E(X) = = = t t t t t xf(x)dx xf(x)dx + xf(x)dx t xf(x)dx tf(x)dx because x [t, ) f(x)dx = tp(x t), we see that P (X t) E(X). t This completes the proof of the theorem. I Theorem 4.4 of the chapter 4, Chebychev iequality was treated. Let X be a radom variable with mea µ ad stadard deviatio σ. The Chebychev iequality says that P ( X µ <kσ) k for ay ozero positive costat k. This result ca be obtaied easily usig Theorem 3.8 as follows. By Markov iequality, we have P ((X µ) t ) E((X µ) ) t for all t>0. Sice the evets (X µ) t ad X µ t are same, we get P ((X µ) t )=P ( X µ t) E((X µ) ) t
Sequeces of Radom Variables ad Order Statistics 366 for all t>0. Hece P ( X µ t) σ t. Lettig k = σ t i the above equality, we see that P ( X µ kσ) k. Hece P ( X µ <kσ) k. The last iequality yields the Chebychev iequality P ( X µ <kσ) k. Now we are ready to treat the weak law of large umbers. Theorem 3.9. Let X,X,... be a sequece of idepedet ad idetically distributed radom variables with µ = E(X i ) ad σ = Var(X i ) < for i =,,...,. The lim P ( S µ ε) =0 for every ε. Here S deotes X+X+ +X. Proof: By Theorem 3. (or Example 3.7) we have E(S )=µ ad Var(S )= σ. By Chebychev s iequality P ( S E(S ) ε) Var(S ) ε for ε>0. Hece P ( S µ ε) σ ε. Takig the limit as teds to ifiity, we get lim P ( S σ µ ε) lim ε which yields lim P ( S µ ε) =0 ad the proof of the theorem is ow complete.
Probability ad Mathematical Statistics 367 It is possible to prove the weak law of large umbers assumig oly E(X) to exist ad fiite but the proof is more ivolved. The weak law of large umbers says that the sequece of sample meas { } S from a populatio X stays close to populatio mea E(X) most = of the times. Let us cosider a experimet that cosists of tossig a coi ifiitely may times. Let X i beifthei th toss results i a Head, ad 0 otherwise. The weak law of large umbers says that S = X + X + + X as (3.0) but it is easy to come up with sequeces of tosses for which (3.0) is false: H H H H H H H H H H H H H H T H H T H H T H H T The strog law of large umbers (Theorem 3.) states that the set of bad sequeces like the oes give above has probability zero. Note that the assertio of Theorem 3.9 for ay ε>0ca also be writte as lim P ( S µ <ε)=. The type of covergece we saw i the weak law of large umbers is ot the type of covergece discussed i calculus. This type of covergece is called covergece i probability ad defied as follows. Defiitio 3.. Suppose X,X,... is a sequece of radom variables defied o a sample space S. The sequece coverges i probability to the radom variable X if, for ay ε>0, lim P ( X X <ε)=. I view of the above defiitio, the weak law of large umbers states that the sample mea X coverges i probability to the populatio mea µ. The followig theorem is kow as the Beroulli law of large umbers ad is a special case of the weak law of large umbers. Theorem 3.0. Let X,X,... be a sequece of idepedet ad idetically distributed Beroulli radom variables with probability of success p. The, for ay ε>0, lim P ( S p <ε)=
Sequeces of Radom Variables ad Order Statistics 368 where S deotes X+X+ +X. The fact that the relative frequecy of occurrece of a evet E is very likely to be close to its probability P (E) for large ca be derived from the weak law of large umbers. Cosider a repeatable radom experimet repeated large umber of time idepedetly. Let X i =ife occurs o the i th repetitio ad X i =0ifE does ot occur o i th repetitio. The µ = E(X i )= P (E)+0 P (E) =P (E) for i =,, 3,... ad X + X + + X = N(E) where N(E) deotes the umber of times E occurs. Hece by the weak law of large umbers, we have ( ) ( ) lim P N(E) P (E) >ε = lim P X + X + + X µ >ε = lim P ( S µ ) >ε =0. Hece, for large, the relative frequecy of occurrece of the evet E is very likely to be close to its probability P (E). Now we preset the strog law of large umbers without a proof. Theorem 3.. Let X,X,... be a sequece of idepedet ad idetically distributed radom variables with µ = E(X i ) ad σ = Var(X i ) < for i =,,...,. The lim P ( S µ) ε) =0 for every ε. Here S deotes X+X+ +X. The type covergece i Theorem 3. is called almost sure covergece. The otio of almost sure covergece is defied as follows. Defiitio 3. Suppose the radom variable X ad the sequece X,X,..., of radom variables are defied o a sample space S. The sequece X (w) coverges almost surely to X(w) if P ({ w S lim X (w) =X(w) }) =. It ca be show that the covergece i probability implies the almost sure covergece but ot the coverse.
Probability ad Mathematical Statistics 369 3.3. The Cetral Limit Theorem Cosider a radom sample of measuremet {X i }. The X i s are idetically distributed ad their commo distributio is the distributio of the populatio. We have see that if the populatio distributio is ormal, the the sample mea X is also ormal. More precisely, if X,X,..., X is a radom sample from a ormal distributio with desity f(x) = σ π e ( x µ σ ) the ) X N (µ, σ. The cetral limit theorem (also kow as Lideberg-Levy Theorem) states that eve though the populatio distributio may be far from beig ormal, still for large sample size, the distributio of the stadardized sample mea is approximately stadard ormal with better approximatios obtaied with the larger sample size. Mathematically this ca be stated as follows. Theorem 3. (Cetral Limit Theorem). Let X,X,..., X be a radom sample of size from a distributio with mea µ ad variace σ <, the the limitig distributio of Z = X µ σ is stadard ormal, that is Z coverges i distributio to Z where Z deotes a stadard ormal radom variable. The type of covergece used i the cetral limit theorem is called the covergece i distributio ad is defied as follows. Defiitio 3.3. Suppose X is a radom variable with cumulative desity fuctio F (x) ad the sequece X,X,... of radom variables with cumulative desity fuctios F (x),f (x),..., respectively. The sequece X coverges i distributio to X if lim F (x) =F (x) for all values x at which F (x) is cotiuous. The distributio of X is called the limitig distributio of X.
Sequeces of Radom Variables ad Order Statistics 370 Wheever a sequece of radom variables X,X,... coverges i distributio to the radom variable X, it will be deoted by X d X. Example 3.. Let Y = X + X + + X 5 be the sum of a radom sample of size 5 from the distributio whose desity fuctio is f(x) = { 3 x if <x< 0 otherwise. What is the approximate value of P ( 0.3 Y.5) whe oe uses the cetral limit theorem? Aswer: First, we fid the mea µ ad variace σ for the desity fuctio f(x). The mea for this distributio is give by µ = 3 x3 dx [ ] x 4 = 3 =0. 4 Hece the variace of this distributio is give by Var(X) =E(X ) [ E(X)] = 3 x4 dx [ ] x 5 = 3 = 3 5 =0.6. 5 P ( 0.3 Y.5) = P ( 0.3 0 Y 0.5 0) ( ) 0.3 = P Y 0.5 5(0.6) 5(0.6) 5(0.6) = P ( 0.0 Z 0.50) = P (Z 0.50) + P (Z 0.0) =0.695 + 0.5398 =0.33.
Probability ad Mathematical Statistics 37 Example 3.. Let X,X,..., X be a radom sample of size = 5 from a populatio that has a mea µ =7.43 ad variace σ =56.5. Let X be the sample mea. What is the probability that the sample mea is betwee 68.9 ad 7.97? Aswer: The mea of X is give by E ( X ) =7.43. The variace of X is give by Var ( X ) = σ = 56.5 =.5. 5 I order to fid the probability that the sample mea is betwee 68.9 ad 7.97, we eed the distributio of the populatio. However, the populatio distributio is ukow. Therefore, we use the cetral limit theorem. The cetral limit theorem says that X µ σ N (0, ) as approaches ifiity. Therefore P ( 68.9 X 7.97 ) ( 68.9 7.43 = X 7.43 ) 7.97 7.43.5.5.5 = P ( 0.68 W 0.36) = P (W 0.36) + P (W 0.68) =0.594. Example 3.3. Light bulbs are istalled successively ito a socket. If we assume that each light bulb has a mea life of moths with a stadard deviatio of 0.5 moths, what is the probability that 40 bulbs last at least 7 years? Aswer: Let X i deote the life time of the i th bulb istalled. The 40 light bulbs last a total time of S 40 = X + X + + X 40. By the cetral limit theorem 40 X i µ N(0, ) σ as. Thus S 40 (40)() N(0, ). (40)(0.5)
Sequeces of Radom Variables ad Order Statistics 37 That is Therefore S 40 80.58 N(0, ). P (S 40 7()) ( S40 80 = P.58 = P (Z.530) =0.0057. ) 84 80.58 Example 3.4. Light bulbs are istalled ito a socket. Assume that each has a mea life of moths with stadard deviatio of 0.5 moth. How may bulbs should be bought so that oe ca be 95% sure that the supply of bulbs will last 5 years? Aswer: Let X i deote the life time of the i th bulb istalled. The light bulbs last a total time of The total average life spa S has S = X + X + + X. E (S )= ad Var(S )= 6. By the cetral limit theorem, we get Thus, we seek such that S E (S ) 4 N(0, ). 0.95 = P (S 60) ( ) S 60 = P 4 4 ( ) 40 8 = P Z ( ) 40 8 = P Z. From the stadard ormal table, we get 40 8 =.645
Probability ad Mathematical Statistics 373 which implies.645 +8 40=0. Solvig this quadratic equatio for, we get = 5.375 or 5.58. Thus =3.5. So we should buy 3 bulbs. Example 3.5. America Airlies claims that the average umber of people who pay for i-flight movies, whe the plae is fully loaded, is 4 with a stadard deviatio of 8. A sample of 36 fully loaded plaes is take. What is the probability that fewer tha 38 people paid for the i-flight movies? Aswer: Here, we like to fid P (X < 38). Sice, we do ot kow the distributio of X, we will use the cetral limit theorem. We are give that the populatio mea is µ = 4 ad populatio stadard deviatio is σ =8. Moreover, we are dealig with sample of size = 36. Thus ( X 4 P (X <38) = P < 8 6 = P (Z < 3) = P (Z <3) = 0.9987 =0.003. ) 38 4 8 6 Sice we have ot yet see the proof of the cetral limit theorem, first let us go through some examples to see the mai idea behid the proof of the cetral limit theorem. Later, at the ed of this sectio a proof of the cetral limit theorem will be give. We kow from the cetral limit theorem that if X,X,..., X is a radom sample of size from a distributio with mea µ ad variace σ, the X µ σ d Z N(0, ) as. However, the above expressio is ot equivalet to X d Z N ) (µ, σ as
Sequeces of Radom Variables ad Order Statistics 374 as the followig example shows. Example 3.6. Let X,X,..., X be a radom sample of size from a gamma distributio with parameters θ = ad α =. What is the distributio of the sample mea X? Also, what is the limitig distributio of X as? Aswer: Sice, each X i GAM(, ), the probability desity fuctio of each X i is give by { e x if x 0 f(x) = 0 otherwise ad hece the momet geeratig fuctio of each X i is M Xi (t) = t. First we determie the momet geeratig fuctio of the sample mea X, ad the examie this momet geeratig fuctio to fid the probability distributio of X. Sice M X (t) =M Xi(t) ( ) t = M Xi = ( ) t = ( ) t, therefore X GAM (,). Next, we fid the limitig distributio of X as. This ca be doe agai by fidig the limitig momet geeratig fuctio of X ad idetifyig the distributio of X. Cosider lim M X (t) = lim = ( ) t ( lim t = e t = e t. )
Probability ad Mathematical Statistics 375 Thus, the sample mea X has a degeerate distributio, that is all the probability mass is cocetrated at oe poit of the space of X. Example 3.7. Let X,X,..., X be a radom sample of size from a gamma distributio with parameters θ = ad α =. What is the distributio of X µ σ as where µ ad σ are the populatio mea ad variace, respectively? Aswer: From Example 3.7, we kow that M X (t) = ( ) t. Sice the populatio distributio is gamma with θ = ad α =, the populatio mea µ is ad populatio variace σ is also. Therefore M X (t) =M X (t) = e t M X ( t ) = e t ( ) t = ( ). t e t The limitig momet geeratig fuctio ca be obtaied by takig the limit of the above expressio as teds to ifiity. That is, lim M X (t) = lim e t ( ) t = e t (usig MAPLE) = X µ σ N(0, ). The followig theorem is used to prove the cetral limit theorem. Theorem 3.3 (Lévy Cotiuity Theorem). Let X,X,... be a sequece of radom variables with distributio fuctios F (x),f (x),... ad momet geeratig fuctios M X (t),m X (t),..., respectively. Let X be a
Sequeces of Radom Variables ad Order Statistics 376 radom variable with distributio fuctio F (x) ad momet geeratig fuctio M X (t). If for all t i the ope iterval ( h, h) for some h>0 lim M X (t) =M X (t), the at the poits of cotiuity of F (x) lim F (x) =F (x). The proof of this theorem is beyod the scope of this book. The followig limit lim [ + t + d() ] = e t, if lim d() =0, (3.) whose proof we leave it to the reader, ca be established usig advaced calculus. Here t is idepedet of. Now we proceed to prove the cetral limit theorem assumig that the momet geeratig fuctio of the populatio X exists. Let M X µ (t) be the momet geeratig fuctio of the radom variable X µ. We deote M X µ (t) asm(t) whe there is o dager of cofusio. The M(0) =, M (0) = E(X µ) =E(X) µ = µ µ =0, M (0) = E ( (3.) (X µ) ) = σ. By Taylor series expasio of M(t) about 0, we get M(t) =M(0) + M (0) t + M (η) t where η (0,t). Hece usig (3.), we have M(t) =+ M (η) t =+ σ t + M (η) t σ t =+ σ t + [ M (η) σ ] t. Now usig M(t) we compute the momet geeratig fuctio of Z. Note that Z = X µ σ = σ (X i µ).
Probability ad Mathematical Statistics 377 Hece ( ) t M Z (t) = M Xi µ σ ( ) t = M X µ σ [ ( )] t = M σ [ = + t + (M (η) σ ) t σ for 0 < η < σ t. Note that sice 0 < η < σ t, we have ] lim t σ =0, lim η =0, ad lim M (η) σ =0. (3.3) Lettig d() = (M (η) σ ) t σ ad usig (3.3), we see that lim d() = 0, ad Usig (3.) we have M Z (t) = [ + t + d() ]. (3.4) [ lim M Z (t) = lim + t + d() ] = e t. Hece by the Lévy cotiuity theorem, we obtai lim F (x) =Φ(x) where Φ(x) is the cumulative desity fuctio of the stadard ormal distributio. Thus Z Z ad the proof of the theorem is ow d complete. Remark 3.3. I cotrast to the momet geeratig fuctio, sice the characteristic fuctio of a radom variable always exists, the origial proof of the cetral limit theorem ivolved the characteristic fuctio (see for example A Itroductio to Probability Theory ad Its Applicatios, Volume II by Feller). I 988, Brow gave a elemetary proof usig very clever Taylor series expasios, where the use characteristic fuctio has bee avoided.
Sequeces of Radom Variables ad Order Statistics 378 3.4. Order Statistics Ofte, sample values such as the smallest, largest, or middle observatio from a radom sample provide importat iformatio. For example, the highest flood water or lowest witer temperature recorded durig the last 50 years might be useful whe plaig for future emergecies. The media price of houses sold durig the previous moth might be useful for estimatig the cost of livig. The statistics highest, lowest or media are examples of order statistics. Defiitio 3.4. Let X,X,..., X be observatios from a radom sample of size from a distributio f(x). Let X () deote the smallest of {X,X,..., X }, X () deote the secod smallest of {X,X,..., X }, ad similarly X (r) deote the r th smallest of {X,X,..., X }. The the radom variables X (),X (),..., X () are called the order statistics of the sample X,X,..., X. I particular, X (r) is called the r th -order statistic of X,X,..., X. The sample rage, R, is the distace betwee the smallest ad the largest observatio. That is, R = X () X (). This is a importat statistic which is defied usig order statistics. The distributio of the order statistics are very importat whe oe uses these i ay statistical ivestigatio. The ext theorem gives the distributio of a order statistic. Theorem 3.4. Let X,X,..., X be a radom sample of size from a distributio with desity fuctio f(x). The the probability desity fuctio of the r th order statistic, X (r),is g(x) = where F (x) deotes the cdf of f(x).! (r )! ( r)! [F (x)]r f(x) [ F (x)] r, Proof: Let h be a positive real umber. Let us divide the real lie ito three segmets, amely IR =(, x) [x, x + h] (x + h, ).
Probability ad Mathematical Statistics 379 The probability, say p, of a sample value falls ito the first iterval (, x] ad is give by x p = f(t) dt. Similarly, the probability p of a sample value falls ito the secod iterval ad is x+h p = f(t) dt. x I the same toke, we ca compute the probability of a sample value which falls ito the third iterval p 3 = x+h f(t) dt. The the probability that (r ) sample values fall i the first iterval, oe falls i the secod iterval, ad ( r) fall i the third iterval is ( ) p r p p r! 3 = r,, r (r )! ( r)! pr p p r 3 =: P h (x). Sice P h (x) g(x) = lim h 0 h, the probability desity fuctio of the r th statistics is give by P h (x) g(x) = lim h 0 h [ ]! p = lim h 0 (r )! ( r)! pr h p r 3 [ (! x ) ] r = lim f(t) dt (r )! ( r)! h 0 [ [ x+h ( ) ] r lim f(t) dt] lim f(t) dt h 0 h h 0 x x+h! = [F (x)] r f(x) [ F (x)] r. (r )! ( r)! The secod limit is obtaied as f(x) due to the fact that lim h 0 h x+h x = lim h 0 = d dx = f(x) f(t) dt x+h a x a f(t) dt x a h f(t) dt f(t) dt, where x<a<x+ h
Sequeces of Radom Variables ad Order Statistics 380 Example 3.8. Let X,X be a radom sample from a distributio with desity fuctio f(x) = { e x for 0 x< 0 otherwise. What is the desity fuctio of Y = mi{x,x } where ozero? Aswer: The cumulative distributio fuctio of f(x) is F (x) = x 0 e t dt = e x I this example, = ad r =. Hece, the desity of Y is g(y) =! 0!! [F (y)]0 f(y) [ F (y)] =f(y) [ F (y)] =e y ( +e y) =e y. Example 3.9. Let Y <Y < <Y 6 be the order statistics from a radom sample of size 6 from a distributio with desity fuctio f(x) = What is the expected value of Y 6? Aswer: { x for 0 <x< 0 otherwise. f(x) =x F (x) = x = x. The desity fuctio of Y 6 is give by 0 tdt g(y) = 6! [F (y)] 5 f(y) 5! 0! =6 ( y ) 5 y =y.
Probability ad Mathematical Statistics 38 Hece, the expected value of Y 6 is E (Y 6 )= = 0 0 = 3 = 3. yg(y) dy y y dy [ y 3 ] 0 Example 3.0. Let X, Y ad Z be idepedet uiform radom variables o the iterval (0,a). Let W = mi{x, Y, Z}. What is the expected value of ( ) W? a Aswer: The probability distributio of X (or Y or Z) is f(x) = { a if 0 <x<a 0 otherwise. Thus the cumulative distributio of fuctio of f(x) is give by 0 if x 0 x F (x) = a if 0 <x<a if x a. Sice W = mi{x, Y, Z}, W is the first order statistic of the radom sample X, Y, Z. Thus, the desity fuctio of W is give by g(w) = 3! [F (w)] 0 f(w) [ F (w)] 0!!! =3f(w) [ F (w)] ( =3 w ) ( a a) = 3 ( w ). a a Thus, the pdf of W is give by ( ) 3 a w a if 0 <w<a g(w) = 0 otherwise.
Sequeces of Radom Variables ad Order Statistics 38 The expected value of W is [ ( E W ) ] a a ( = w ) g(w) dw a = = 0 a 0 a 0 = 3 5 = 3 5. ( w ) 3 a a 3 a ( w a ) 4 dx [ ( w a ) 5 ] a ( w a ) dw Example 3.. Let X,X,..., X be a radom sample from a populatio X with uiform distributio o the iterval [0, ]. What is the probability distributio of the sample rage W := X () X ()? Aswer: To fid the distributio of W, we eed the joit distributio of the radom variable ( X (),X () ). The joit distributio of ( X(),X () ) is give by h(x,x )=( )f(x )f(x )[F (x ) F (x )], where x x ad f(x) is the probability desity fuctio of X. To determie the probability distributio of the sample rage W, we cosider the trasformatio } U = X () which has a iverse 0 W = X () X () } X () = U X () = U + W. The Jacobia of this trasformatio is ( ) 0 J = det =. Hece the joit desity of (U, W ) is give by g(u, w) = J h(x,x ) = ( )f(u)f(u + w)[f (u + w) F (u)]
Probability ad Mathematical Statistics 383 where w 0. Sice f(u) ad f(u+w) are simultaeously ozero if 0 u ad 0 u + w. Hece f(u) ad f(u + w) are simultaeously ozero if 0 u w. Thus, the probability of W is give by j(w) = = = where 0 w. w 0 g(u, w) du ( )f(u)f(u + w)[f (u + w) F (u)] du w du = ( )( w) w 3.5. Sample Percetiles The sample media, M, is a umber such that approximately oe-half of the observatios are less tha M ad oe-half are greater tha M. Defiitio 3.5. Let X,X,..., X be a radom sample. The sample media M is defied as X ( + ) if is odd M = [ ] X ( ) + X ( + ) if is eve. The media is a measure of locatio like sample mea. Recall that for cotiuous distributio, 00p th percetile, π p,isaumber such that πp p = f(x) dx. Defiitio 3.6. The 00p th sample percetile is defied as π p = X ([p]) X (+ [( p)]) if p<0.5 if p>0.5. where [b] deote the umber b rouded to the earest iteger. Example 3.. Let X,X,..., X be a radom sample of size. What is the 65 th percetile of this sample?
Sequeces of Radom Variables ad Order Statistics 384 Aswer: 00p =65 p =0.65 ( p) = ()( 0.65) = 4. [( p)] = [4.] = 4 Hece by defiitio of 65 th percetile is π 0.65 = X (+ [( p)]) = X (3 4) = X (9). Thus, the 65 th percetile of the radom sample X,X,..., X is the 9 th - order statistic. For ay umber p betwee 0 ad, the 00p th sample percetile is a observatio such that approximately p observatios are less tha this observatio ad ( p) observatios are greater tha this. Defiitio 3.7. The 5 th percetile is called the lower quartile while the 75 th percetile is called the upper quartile. The distace betwee these two quartiles is called the iterquartile rage. Example 3.3. If a sample of size 3 from a uiform distributio over [0, ] is observed, what is the probability that the sample media is betwee 4 ad 3 4? Aswer: Whe a sample of ( + ) radom variables are observed, the ( +) th smallest radom variable is called the sample media. For our problem, the sample media is give by X () = d smallest {X,X,X 3 }. Let Y = X (). The desity fuctio of each X i is give by f(x) = { if 0 x 0 otherwise. Hece, the cumulative desity fuctio of f(x) is F (x) =x.
Probability ad Mathematical Statistics 385 Thus the desity fuctio of Y is give by g(y) = 3! [F (y)] f(y) [ F (y)] 3!! =6F (y) f(y) [ F (y)] =6y ( y). Therefore P ( ) 4 <Y <3 = 4 = 3 4 4 3 4 4 [ y =6 = 6. g(y) dy 6 y ( y) dy y3 3 ] 3 4 4 3.6. Review Exercises. Suppose we roll a die 000 times. What is the probability that the sum of the umbers obtaied lies betwee 3000 ad 4000?. Suppose Kathy flip a coi 000 times. What is the probability she will get at least 600 heads? 3. At a certai large uiversity the weight of the male studets ad female studets are approximately ormally distributed with meas ad stadard deviatios of 80, ad 0, ad 30 ad 5, respectively. If a male ad female are selected at radom, what is the probability that the sum of their weights is less tha 80? 4. Seve observatios are draw from a populatio with a ukow cotiuous distributio. What is the probability that the least ad the greatest observatios bracket the media? 5. If the radom variable X has the desity fuctio ( x) for 0 x f(x) = 0 otherwise, what is the probability that the larger of idepedet observatios of X will exceed?
Sequeces of Radom Variables ad Order Statistics 386 6. Let X,X,X 3 be a radom sample from the uiform distributio o the iterval (0, ). What is the probability that the sample media is less tha 0.4? 7. Let X,X,X 3,X 4,X 5 be a radom sample from the uiform distributio o the iterval (0, θ), where θ is ukow, ad let X max deote the largest observatio. For what value of the costat k, the expected value of the radom variable kx max is equal to θ? 8. A radom sample of size 6 is to be take from a ormal populatio havig mea 00 ad variace 4. What is the 90 th percetile of the distributio of the sample mea? 9. If the desity fuctio of a radom variable X is give by x for e <x<e f(x) = 0 otherwise, what is the probability that oe of the two idepedet observatios of X is less tha ad the other is greater tha? 0. Five observatios have bee draw idepedetly ad at radom from a cotiuous distributio. What is the probability that the ext observatio will be less tha all of the first 5?. Let the radom variable X deote the legth of time it takes to complete a mathematics assigmet. Suppose the desity fuctio of X is give by e (x θ) for θ<x< f(x) = 0 otherwise, where θ is a positive costat that represets the miimum time to complete a mathematics assigmet. If X,X,..., X 5 is a radom sample from this distributio. What is the expected value of X ()?. Let X ad Y be two idepedet radom variables with idetical probability desity fuctio give by f(x) = { e x for x>0 0 elsewhere. What is the probability desity fuctio of W = max{x, Y }?
Probability ad Mathematical Statistics 387 3. Let X ad Y be two idepedet radom variables with idetical probability desity fuctio give by 3 x θ for 0 x θ f(x) = 3 0 elsewhere, for some θ>0. What is the probability desity fuctio of W = mi{x, Y }? 4. Let X,X,..., X be a radom sample from a uiform distributio o the iterval from 0 to 5. What is the limitig momet geeratig fuctio of X µ σ as? 5. Let X,X,..., X be a radom sample of size from a ormal distributio with mea µ ad variace. If the 75 th percetile of the statistic W = ( Xi X ) is 8.4, what is the sample size? 6. Let X,X,..., X be a radom sample of size from a Beroulli distributio with probability of success p =. What is the limitig distributio the sample mea X? 7. Let X,X,..., X 995 be a radom sample of size 995 from a distributio with probability desity fuctio f(x) = e λ λ x x! x =0,,, 3,...,. What is the distributio of 995X? 8. Suppose X,X,..., X is a radom sample from the uiform distributio o (0, ) ad Z be the sample rage. What is the probability that Z is less tha or equal to 0.5? 9. Let X,X,..., X 9 be a radom sample from a uiform distributio o the iterval [, ]. Fid the probability that the ext to smallest is greater tha or equal to 4? 0. A machie eeds 4 out of its 6 idepedet compoets to operate. Let X,X,..., X 6 be the lifetime of the respective compoets. Suppose each is expoetially distributed with parameter θ. What is the probability desity fuctio of the machie lifetime?. Suppose X,X,..., X + is a radom sample from the uiform distributio o (0, ). What is the probability desity fuctio of the sample media X (+)?
Sequeces of Radom Variables ad Order Statistics 388. Let X ad Y be two radom variables with joit desity f(x, y) = { x if 0 <y<x < 0 otherwise. What is the expected value of the radom variable Z = X Y 3 +X X Y 3? 3. Let X,X,..., X 50 be a radom sample of size 50 from a distributio with desity f(x) = { Γ(α) θ α x α e x θ for 0 <x< 0 otherwise. What are the mea ad variace of the sample mea X? 4. Let X,X,..., X 00 be a radom sample of size 00 from a distributio with desity f(x) = { e λ λ x x! for x =0,,,..., 0 otherwise. What is the probability that X greater tha or equal to?
Probability ad Mathematical Statistics 389 Chapter 4 SAMPLING DISTRIBUTIONS ASSOCIATED WITH THE NORMAL POPULATIONS Give a radom sample X,X,..., X from a populatio X with probability distributio f(x; θ), where θ is a parameter, a statistic is a fuctio T of X,X,..., X, that is T = T (X,X,..., X ) which is free of the parameter θ. If the distributio of the populatio is kow, the sometimes it is possible to fid the probability distributio of the statistic T. The probability distributio of the statistic T is called the samplig distributio of T. The joit distributio of the radom variables X,X,..., X is called the distributio of the sample. The distributio of the sample is the joit desity f(x,x,..., x ; θ) =f(x ; θ)f(x ; θ) f(x ; θ) = f(x i ; θ) sice the radom variables X,X,..., X are idepedet ad idetically distributed. Sice the ormal populatio is very importat i statistics, the samplig distributios associated with the ormal populatio are very importat. The most importat samplig distributios which are associated with the ormal
Samplig Distributios Associated with the Normal Populatio 390 populatio are the followigs: the chi-square distributio, the studet s t- distributio, the F-distributio, ad the beta distributio. I this chapter, we oly cosider the first three distributios, sice the last distributio was cosidered earlier. 4.. Chi-square distributio I this sectio, we treat the Chi-square distributio, which is oe of the very useful samplig distributios. Defiitio 4.. A cotiuous radom variable X is said to have a chisquare distributio with r degrees of freedom if its probability desity fuctio is of the form f(x; r) = Γ( r ) x r r e x if 0 x< 0 otherwise, where r > 0. If X has chi-square distributio, the we deote it by writig X χ (r). Recall that a gamma distributio reduces to chi-square distributio if α = r ad θ =. The mea ad variace of X are r ad r, respectively. Thus, chi-square distributio is also a special case of gamma distributio. Further, if r, the chi-square distributio teds to ormal distributio. Example 4.. If X GAM(, ), the what is the probability desity fuctio of the radom variable X? Aswer: We will use the momet geeratig method to fid the distributio of X. The momet geeratig fuctio of a gamma radom variable is give by M(t) =( θt) α, if t< θ.
Probability ad Mathematical Statistics 39 Sice X GAM(, ), the momet geeratig fuctio of X is give by M X (t) = t, t <. Hece, the momet geeratig fuctio of X is M X (t) =M X (t) = t = ( t) = MGF of χ (). Hece, if X is GAM(, ) or is a expoetial with parameter, the X is chi-square with degrees of freedom. Example 4.. If X χ (5), the what is the probability that X is betwee.45 ad.83? Aswer: The probability of X betwee.45 ad.83 ca be calculated from the followig: P (.45 X.83) = P (X.83) P (X.45) =.83 0.83 f(x) dx.45 0 f(x) dx = 0 Γ ( ) 5 5 =0.975 0.050 (from χ table) =0.95. x 5 e x dx.45 0 Γ ( ) 5 5 x 5 e x dx This above itegrals are hard to evaluate ad thus their values are take from the chi-square table. Example 4.3. If X χ (7), the what are values of the costats a ad b such that P (a <X<b)=0.95? Aswer: Sice 0.95 = P (a <X<b)=P (X <b) P (X <a), we get P (X <b)=0.95 + P (X <a).