PROBABILITY AND MATHEMATICAL STATISTICS. Prasanna Sahoo Department of Mathematics University of Louisville Louisville, KY USA
|
|
- Ruth Doyle
- 6 years ago
- Views:
Transcription
1 PROBABILITY AND MATHEMATICAL STATISTICS Prasaa Sahoo Departmet of Mathematics Uiversity of Louisville Louisville, KY 409 USA
2 THIS BOOK IS DEDICATED TO AMIT SADHNA MY PARENTS, TEACHERS AND STUDENTS v
3 vi
4 Copyright c 003. All rights reserved. This book, or parts thereof, may ot be reproduced i ay form or by ay meas, electroic or mechaical, icludig photocopyig, recordig or ay iformatio storage ad retrieval system ow kow or to be iveted, without writte permissio from the author. vii
5 viii
6 ix PREFACE This book is both a tutorial ad a textbook. I this book we preset a itroductio to probability ad mathematical statistics ad it is iteded for studets already havig some elemetary mathematical backgroud. It is iteded for a oe-year seior level udergraduate ad begiig graduate level course i probability theory ad mathematical statistics. The book cotais more material tha ormally would be taught i a oe-year course. This should give the teacher flexibility with respect to the selectio of the cotet ad level at which the book is to be used. It has arise from over 5 years of lectures i seior level calculus based courses i probability theory ad mathematical statistics at the Uiversity of Louisville. Probability theory ad mathematical statistics are difficult subjects both for studets to comprehed ad teachers to explai. A good set of examples makes these subjects easy to uderstad. For this reaso aloe we have icluded more tha 350 completely worked out examples ad over 65 illustratios. We give a rigorous treatmet of the fudametals of probability ad statistics usig mostly calculus. We have give great attetio to the clarity of the presetatio of the materials. I the text theoretical results are preseted as theorems, propositio or lemma, of which as a rule rigorous proofs are give. I the few exceptios to this rule refereces are give to idicate where details ca be foud. This book cotais over 450 problems of varyig degrees of difficulty to help studets master their problem solvig skill. To make this less wordy we have There are several good books o these subjects ad perhaps there is o eed to brig a ew oe to the market. So for several years, this was circulated as a series of typeset lecture otes amog my studets who were preparig for the examiatio 0 of the Actuarial Society of America. May of my studets ecouraged me to formally write it as a book. Actuarial studets will beefit greatly from this book. The book is writte i simple Eglish; this might be a advatage to studets whose ative laguage is ot Eglish.
7 I caot claim that all the materials I have writte i this book are mie. I have leared the subject from may excellet books, such as Itroductio to Mathematical Statistics by Hogg ad Craig, ad A Itroductio to Probability Theory ad Its Applicatios by Feller. I fact, these books have had a profoud impact o me, ad my explaatios are iflueced greatly by these textbooks. If there are some resemblaces, the it is perhaps due to the fact that I could ot improve the origial explaatios I have leared from these books. I am very thakful to the authors of these great textbooks. I am also thakful to the Actuarial Society of America for lettig me use their test problems. I thak all my studets i my probability theory ad mathematical statistics courses from 988 to 003 who helped me i may ways to make this book possible i the preset form. Lastly, if it was t for the ifiite patiece of my wife, Sadha, for last several years, this book would ever gotte out of the hard drive of my computer. The etire book was typeset by the author o a Macitosh computer usig TEX, the typesettig system desiged by Doald Kuth. The figures were geerated by the author usig MATHEMATICA, a system for doig mathematics desiged by Wolfram Research, ad MAPLE, a system for doig mathematics desiged by Maplesoft. The author is very thakful to the Uiversity of Louisville for providig may iteral fiacial grats while this book was uder preparatio. x Prasaa Sahoo, Louisville
8 xi
9 xii TABLE OF CONTENTS. Probability of Evets Itroductio.. Coutig Techiques.3. Probability Measure.4. Some Properties of the Probability Measure.5. Review Exercises. Coditioal Probability ad Bayes Theorem Coditioal Probability.. Bayes Theorem.3. Review Exercises 3. Radom Variables ad Distributio Fuctios Itroductio 3.. Distributio Fuctios of Discrete Variables 3.3. Distributio Fuctios of Cotiuous Variables 3.4. Percetile for Cotiuous Radom Variables 3.5. Review Exercises 4. Momets of Radom Variables ad Chebychev Iequality Momets of Radom Variables 4.. Expected Value of Radom Variables 4.3. Variace of Radom Variables 4.4. Chebychev Iequality 4.5. Momet Geeratig Fuctios 4.6. Review Exercises
10 xiii 5. Some Special Discrete Distributios Beroulli Distributio 5.. Biomial Distributio 5.3. Geometric Distributio 5.4. Negative Biomial Distributio 5.5. Hypergeometric Distributio 5.6. Poisso Distributio 5.7. Riema Zeta Distributio 5.8. Review Exercises 6. Some Special Cotiuous Distributios Uiform Distributio 6.. Gamma Distributio 6.3. Beta Distributio 6.4. Normal Distributio 6.5. Logormal Distributio 6.6. Iverse Gaussia Distributio 6.7. Logistic Distributio 6.8. Review Exercises 7. Two Radom Variables Bivariate Discrete Radom Variables 7.. Bivariate Cotiuous Radom Variables 7.3. Coditioal Distributios 7.4. Idepedece of Radom Variables 7.5. Review Exercises 8. Product Momets of Bivariate Radom Variables Covariace of Bivariate Radom Variables 8.. Idepedece of Radom Variables 8.3. Variace of the Liear Combiatio of Radom Variables 8.4. Correlatio ad Idepedece 8.5. Momet Geeratig Fuctios 8.6. Review Exercises
11 xiv 9. Coditioal Expectatios of Bivariate Radom Variables Coditioal Expected Values 9.. Coditioal Variace 9.3. Regressio Curve ad Scedastic Curves 9.4. Review Exercises 0. Fuctios of Radom Variables ad Their Distributio Distributio Fuctio Method 0.. Trasformatio Method for Uivariate Case 0.3. Trasformatio Method for Bivariate Case 0.4. Covolutio Method for Sums of Radom Variables 0.5. Momet Method for Sums of Radom Variables 0.6. Review Exercises. Some Special Discrete Bivariate Distributios Bivariate Beroulli Distributio.. Bivariate Biomial Distributio.3. Bivariate Geometric Distributio.4. Bivariate Negative Biomial Distributio.5. Bivariate Hypergeometric Distributio.6. Bivariate Poisso Distributio.7. Review Exercises. Some Special Cotiuous Bivariate Distributios Bivariate Uiform Distributio.. Bivariate Cauchy Distributio.3. Bivariate Gamma Distributio.4. Bivariate Beta Distributio.5. Bivariate Normal Distributio.6. Bivariate Logistic Distributio.7. Review Exercises
12 xv 3. Sequeces of Radom Variables ad Order Statistics Distributio of Sample Mea ad Variace 3.. Laws of Large Numbers 3.3. The Cetral Limit Theorem 3.4. Order Statistics 3.5. Sample Percetiles 3.6. Review Exercises 4. Samplig Distributios Associated with the Normal Populatio Chi-square distributio 4.. Studet s t-distributio 4.3. Sedecor s F -distributio 4.4. Review Exercises 5. Some Techiques for Fidig Poit Estimators of Parameters Momet Method 5.. Maximum Likelihood Method 5.3. Bayesia Method 5.3. Review Exercises 6. Criteria for Evaluatig the Goodess of Estimators The Ubiased Estimator 6.. The Relatively Efficiet Estimator 6.3. The Miimum Variace Ubiased Estimator 6.4. Sufficiet Estimator 6.5. Cosistet Estimator 6.6. Review Exercises
13 xvi 7. Some Techiques for Fidig Iterval Estimators of Parameters Iterval Estimators ad Cofidece Itervals for Parameters 7.. Pivotal Quatity Method 7.3. Cofidece Iterval for Populatio Mea 7.4. Cofidece Iterval for Populatio Variace 7.5. Cofidece Iterval for Parameter of some Distributios ot belogig to the Locatio-Scale Family 7.6. Approximate Cofidece Iterval for Parameter with MLE 7.7. The Statistical or Geeral Method 7.8. Criteria for Evaluatig Cofidece Itervals 7.9. Review Exercises 8. Test of Statistical Hypotheses Itroductio 8.. A Method of Fidig Tests 8.3. Methods of Evaluatig Tests 8.4. Some Examples of Likelihood Ratio Tests 8.5. Review Exercises 9. Simple Liear Regressio ad Correlatio Aalysis Least Squared Method 9.. Normal Regressio Aalysis 9.3. The Correlatio Aalysis 9.4. Review Exercises 0. Aalysis of Variace Oe-way Aalysis of Variace with Equal Sample Sizes 0.. Oe-way Aalysis of Variace with Uequal Sample Sizes 0.3. Pair wise Comparisos 0.4. Tests for the Homogeeity of Variaces 0.5. Review Exercises
14 xvii. Goodess of Fits Tests Chi-Squared test.. Kolmogorov-Smirov test.3. Review Exercises Refereces Aswers to Selected Review Exercises
15 Probability ad Mathematical Statistics 35 Chapter 3 SEQUENCES OF RANDOM VARIABLES AND ORDER STASTISTICS I this chapter, we geeralize some of the results we have studied i the previous chapters. We do these geeralizatios because the geeralizatios are eeded i the subsequet chapters relatig mathematical statistics. I this chapter, we also examie the weak law of large umbers, the Beroulli s law of large umbers, the strog law of large umbers, ad the cetral limit theorem. Further, i this chapter, we treat the order statistics ad percetiles. 3.. Distributio of sample mea ad variace Cosider a radom experimet. Let X be the radom variable associated with this experimet. Let f(x) be the probability desity fuctio of X. Let us repeat this experimet times. Let X k be the radom variable associated with the k th repetitio. The the collectio of the radom variables { X,X,..., X } is a radom sample of size. From here after, we simply deote X,X,..., X as a radom sample of size. The radom variables X,X,..., X are idepedet ad idetically distributed with the commo probability desity fuctio f(x). For a radom sample, fuctios such as the sample mea X, the sample variace S are called statistics. I a particular sample, say x,x,..., x,we
16 Sequeces of Radom Variables ad Order Statistics 35 observed x ad s. We may cosider ad X = S = X i ( Xi X ) as radom variables ad x ad s are the realizatios from a particular sample. I this sectio, we are maily iterested i fidig the probability distributios of the sample mea X ad sample variace S, that is the distributio of the statistics of samples. Example 3.. Let X ad X be a radom sample of size from a distributio with probability desity fuctio { f(x) = 6x( x) if 0 <x< 0 otherwise. What are the mea ad variace of sample sum Y = X + X? Aswer: The populatio mea µ X = E (X) = =6 0 0 x 6x( x) dx x ( x) dx = 6B(3, ) (here B deotes the beta fuctio) Γ(3) Γ() =6 Γ(5) ( ) =6 =. Sice X ad X have the same distributio, we obtai µ X = = µ X. Hece the mea of Y is give by E(Y )=E(X + X ) = E(X )+E(X ) = + =.
17 Probability ad Mathematical Statistics 353 Next, we compute the variace of the populatio X. The variace of X is give by Var(X) =E ( X ) E(X) ( ) = 6x 3 ( x) dx 0 =6 x 3 ( x) dx 0 ( ) =6B(4, ) 4 ( ) Γ(4) Γ() =6 Γ(6) 4 ( ) ( ) =6 0 4 ( 4 = = 0. Sice X ad X have the same distributio as the populatio X, we get ) Var(X )= 0 = Var(X ). Hece, the variace of the sample sum Y is give by Var(Y )=Var(X + X ) = Var(X )+Var(X )+Cov (X,X ) = Var(X )+Var(X ) = = 0. Example 3.. Let X ad X be a radom sample of size from a distributio with desity f(x) = { 4 for x =,, 3, 4 0 otherwise. What is the distributio of the sample sum Y = X + X?
18 Sequeces of Radom Variables ad Order Statistics 354 Aswer: Sice the rage space of X as well as X is {,, 3, 4}, the rage space of Y = X + X is R Y = {, 3, 4, 5, 6, 7, 8}. Let g(y) be the desity fuctio of Y. We wat to fid this desity fuctio. First, we fid g(), g(3) ad so o. g() = P (Y =) = P (X + X =) = P (X = ad X =) = P (X =)P (X = ) (by idepedece of X ad X ) = f() f() = ( )( ) = g(3) = P (Y =3) = P (X + X =3) = P (X = ad X =)+P (X = ad X =) = P (X =)P (X =) + P (X =)P (X = ) (by idepedece of X ad X ) = f() f() + f() f() = ( 4 )( ) + 4 ( 4 )( ) = 4 6.
19 Probability ad Mathematical Statistics 355 g(4) = P (Y =4) = P (X + X =4) = P (X = ad X =3)+P (X = 3 ad X =) + P (X = ad X =) = P (X =3)P (X =)+P (X =)P (X =3) + P (X =)P (X = ) (by idepedece of X ad X ) = f() f(3) + f(3) f() + f() f() ( )( ) ( )( ) ( )( ) = = 3 6. Similarly, we get g(5) = 4 6, g(6) = 3 6, g(7) = 6, g(8) = 6. Thus, puttig these ito oe expressio, we get g(y) =P (Y = y) y = f(k) f(y k) = k= 4 y 5, y =, 3, 4,..., 8. 6 y Remark 3.. Note that g(y) = f(k) f(y k) is the discrete covolutio k= of f with itself. The cocept of covolutio was itroduced i chapter 0. The above example ca also be doe usig the momet geeratig fuc-
20 Sequeces of Radom Variables ad Order Statistics 356 tio method as follows: M Y (t) =M X+X (t) = M X (t) M X (t) ( e t + e t + e 3t + e 4t )( e t + e t + e 3t + e 4t ) = 4 4 ( e t + e t + e 3t + e 4t ) = 4 = et +e 3t +3e 4t +4e 5t +3e 6t +e 7t + e 8t. 6 Hece, the desity of Y is give by g(y) = 4 y 5, y =, 3, 4,..., 8. 6 Theorem 3.. If X,X,..., X are mutually idepedet radom variables with desities f (x ),f (x ),..., f (x ) ad E[u i (X i )], i =,,..., exist, the [ ] E u i (X i ) = E[u i (X i )], where u i (i =,,..., ) are arbitrary fuctios. Proof: We prove the theorem assumig that the radom variables X,X,..., X are cotiuous. If the radom variables are ot cotiuous, the the proof follows exactly i the same maer if oe replaces the itegrals by summatios. Sice ( ) E u i (X i ) = E(u (X ) u (X )) = = = u (x ) u (x )f(x,..., x )dx dx u (x ) u (x )f (x ) f (x )dx dx u (x )f (x )dx = E (u (X )) E (u (X )) = E (u i (X i )), u (x )f (x )dx
21 Probability ad Mathematical Statistics 357 the proof of the theorem is ow complete. Example 3.3. Let X ad Y be two radom variables with the joit desity { e (x+y) for 0 <x,y< f(x, y) = 0 otherwise. What is the expected value of the cotiuous radom variable Z = X Y + XY + X + X? Aswer: Sice f(x, y) =e (x+y) = e x e y = f (x) f (y), the radom variables X ad Y are mutually idepedet. Hece, the expected value of X is E(X) = xf (x) dx = 0 0 xe x dx = Γ() =. Similarly, the expected value of X is give by E ( X ) = = 0 0 = Γ(3) =. x f (x) dx x e x dx Sice the margials of X ad Y are same, we also get E(Y ) = ad E(Y )=. Further, by Theorem 3., we get E [Z] =E [ X Y + XY + X + X ] = E [( X + X )( Y + )] = E [ X + X ] E [ Y + ] (by Theorem 3.) = ( E [ X ] + E [X] )( E [ Y ] + ) =(+)(+) =9.
22 Sequeces of Radom Variables ad Order Statistics 358 Theorem 3.. If X,X,..., X are mutually idepedet radom variables with respective meas µ,µ,..., µ ad variaces σ,σ,..., σ, the the mea ad variace of Y = a i X i, where a,a,..., a are real costats, are give by µ Y = a i µ i ad σy = Proof: First we show that µ Y = a i µ i. Sice µ Y = E(Y ) ( ) = E a i X i = = a i E(X i ) a i µ i a i σi. we have asserted result. Next we show σ Y = a i σ i. Cosider σy = Var(Y ) = Var(a i X i ) = a i Var(X i ) = a i σi. This completes the proof of the theorem. Example 3.4. Let the idepedet radom variables X ad X have meas µ = 4 ad µ = 3, respectively ad variaces σ = 4 ad σ =9. What are the mea ad variace of Y =3X X? Aswer: The mea of Y is µ Y =3µ µ =3( 4) (3) = 8.
23 Probability ad Mathematical Statistics 359 Similarly, the variace of Y is σy = (3) σ +( ) σ =9σ +4σ = 9(4) + 4(9) =7. Example 3.5. Let X,X,..., X 50 be a radom sample of size 50 from a distributio with desity { θ e x θ for 0 x< f(x) = 0 otherwise. What are the mea ad variace of the sample mea X? Aswer: Sice the distributio of the populatio X is expoetial, the mea ad variace of X are give by µ X = θ, ad σx = θ. Thus, the mea of the sample mea is E ( X ) ( ) X + X + + X 50 = E 50 = 50 = E (X i ) = 50 θ = θ. 50 The variace of the sample mea is give by θ Var ( X ) = Var = = =50 = θ 50. ( 50 ) 50 X i ( ) σx 50 i ( ) θ 50 ) θ ( 50
24 Sequeces of Radom Variables ad Order Statistics 360 Theorem 3.3. If X,X,..., X are idepedet radom variables with respective momet geeratig fuctios M Xi (t), i =,,...,, the the momet geeratig fuctio of Y = a ix i is give by Proof: Sice M Y (t) = M Xi (a i t). M Y (t) =M aixi(t) = M aix i (t) = M Xi (a i t) we have the asserted result ad the proof of the theorem is ow complete. Example 3.6. Let X,X,..., X 0 be the observatios from a radom sample of size 0 from a distributio with desity f(x) = π e x, <x<. What is the momet geeratig fuctio of the sample mea? Aswer: The desity of the populatio X is a stadard ormal. Hece, the momet geeratig fuctio of each X i is M Xi (t) =e t, i =,,..., 0. The momet geeratig fuctio of the sample mea is Hece X N ( 0, 0). M X (t) =M 0 = = = 0 (t) 0 Xi M Xi ( 0 t ) 0 e t 00 [ e t 00 ] 0 = e ( 0 t ).
25 Probability ad Mathematical Statistics 36 The last example tells us that if we take a sample of ay size from a ormal populatio, the the sample mea also has a ormal distributio. The followig theorem says that a liear combiatio of radom variables with ormal distributios is agai ormal. Theorem 3.4. If X,X,..., X are mutually idepedet radom variables such that X i N ( µ i,σi ), i =,,...,. The the radom variable Y = a i X i is a ormal radom variable with mea µ Y = a i µ i ad σy = a i σi, that is Y N ( a iµ i, ) a i σ i. Proof: Sice each X i N ( ) µ i,σi, the momet geeratig fuctio of each X i is give by M Xi (t) =e µit+ σ i t. Hece usig Theorem 3.3, we have M Y (t) = M Xi (a i t) = e µit+ σ i t µit+ = e ( Thus the radom variable Y N a i µ i, σ i t. ) a i σi. The proof of the theorem is ow complete. Example 3.7. Let X,X,..., X be the observatios from a radom sample of size from a ormal distributio with mea µ ad variace σ > 0. What are the mea ad variace of the sample mea X? Aswer: The expected value (or mea) of the sample mea is give by E ( X ) = E (X i ) = = µ. µ
26 Sequeces of Radom Variables ad Order Statistics 36 Similarly, the variace of the sample mea is Var ( X ) = ( ) Xi Var = ( ) σ = σ. This example alog with the previous theorem says that if we take a radom sample of size from a ormal populatio with mea µ ad variace σ, the the ( sample ) mea is also ormal with mea µ ad variace σ, that is X N µ, σ. Example 3.8. Let X,X,..., X 64 be a radom sample of size 64 from a ormal distributio with µ = 50 ad σ = 6. What are P (49 <X 8 < 5) ad P ( 49 < X<5 )? Aswer: Sice X 8 N(50, 6), we get P (49 <X 8 < 5) = P (49 50 <X 8 50 < 5 50) ( = P < X ) < ( = P 4 < X 8 50 < ) 4 4 ( = P ) 4 <Z< 4 ( =P Z< ) 4 = (from ormal table). By the previous theorem, we see that X N ( 50, 64) 6. Hece P ( 49 < X<5 ) = P ( < X 50 < 5 50 ) = P = P < X < X < < = P ( <Z<) =P (Z <) = (from ormal table).
27 Probability ad Mathematical Statistics 363 This example tells us that X has a greater probability of fallig i a iterval cotaiig µ, tha a sigle observatio, say X 8 (or i geeral ay X i ). Theorem 3.5. Let the distributios of the radom variables X,X,..., X be χ (r ),χ (r ),..., χ (r ), respectively. If X,X,..., X are mutually idepedet, the Y = X + X + + X χ ( r i). Proof: Sice each X i χ (r i ), the momet geeratig fuctio of each X i is give by M Xi (t) =( t) r i. By Theorem 3.3, we have M Y (t) = M Xi (t) = ( t) r i =( t) ri. Hece Y χ ( r i) ad the proof of the theorem is ow complete. The proof of the followig theorem is a easy cosequece of Theorem 3.5 ad we leave the proof to the reader. Theorem 3.6. If Z,Z,..., Z are mutually idepedet ad each oe is stadard ormal, the Z + Z + + Z χ (), that is the sum is chi-square with degrees of freedom. The followig theorem is very useful i mathematical statistics ad its proof is beyod the scope of this itroductory book. Theorem 3.7. If X,X,..., X are observatios of a radom sample of size from the ormal distributio N ( µ, σ ), the the sample mea X = X i ad the sample variace S = (X i X) have the followig properties: (A) X ad S are idepedet, ad (B) ( ) S σ χ ( ). Remark 3.. At first sight the statemet (A) might seem odd sice the sample mea X occurs explicitly i the defiitio of the sample variace S. This remarkable idepedece of X ad S is a uique property that distiguishes ormal distributio from all other probability distributios. Example 3.9. Let X,X,..., X deote a radom sample from a ormal distributio with variace σ > 0. If the first percetile of the statistics W = (X i X) σ is.4, where X deotes the sample mea, what is the sample size?
28 Sequeces of Radom Variables ad Order Statistics 364 Aswer: = P (W.4) 00 ( ) (X i X) = P σ.4 ) = P (( ) S σ.4 = P ( χ ( ).4 ). Thus from χ -table, we get =7 ad hece the sample size is 8. Example 3.0. Let X,X,..., X 4 be a radom sample from a ormal distributio with ukow mea ad variace equal to 9. Let S = 4 ( 3 Xi X ).IfP ( S k ) =0.05, the what is k? Aswer: 0.05 = P ( S k ) ( 3S = P 9 3 ) 9 k = P (χ (3) 39 ) k. From χ -table with 3 degrees of freedom, we get 3 9 k =0.35 ad thus the costat k is give by k = 3(0.35) = Laws of Large Numbers I this sectio, we maily examie the weak law of large umbers. The weak law of large umbers states that if X,X,..., X is a radom sample of size from a populatio X with mea µ, the the sample mea X rarely deviates from the populatio mea µ whe the sample size is very large. I other words, the sample mea X coverges i probability to the populatio mea µ. We begi this sectio with a result kow as Markov iequality which is eed to establish the weak law of large umbers.
29 Probability ad Mathematical Statistics 365 Theorem 3.8 (Markov Iequality). Suppose X is a oegative radom variable with mea E(X). The P (X t) E(X) t for all t>0. Proof: We assume the radom variable X is cotiuous. If X is ot cotiuous, the a proof ca be obtaied for this case by replacig the itegrals with summatios i the followig proof. Sice E(X) = = = t t t t t xf(x)dx xf(x)dx + xf(x)dx t xf(x)dx tf(x)dx because x [t, ) f(x)dx = tp(x t), we see that P (X t) E(X). t This completes the proof of the theorem. I Theorem 4.4 of the chapter 4, Chebychev iequality was treated. Let X be a radom variable with mea µ ad stadard deviatio σ. The Chebychev iequality says that P ( X µ <kσ) k for ay ozero positive costat k. This result ca be obtaied easily usig Theorem 3.8 as follows. By Markov iequality, we have P ((X µ) t ) E((X µ) ) t for all t>0. Sice the evets (X µ) t ad X µ t are same, we get P ((X µ) t )=P ( X µ t) E((X µ) ) t
30 Sequeces of Radom Variables ad Order Statistics 366 for all t>0. Hece P ( X µ t) σ t. Lettig k = σ t i the above equality, we see that P ( X µ kσ) k. Hece P ( X µ <kσ) k. The last iequality yields the Chebychev iequality P ( X µ <kσ) k. Now we are ready to treat the weak law of large umbers. Theorem 3.9. Let X,X,... be a sequece of idepedet ad idetically distributed radom variables with µ = E(X i ) ad σ = Var(X i ) < for i =,,...,. The lim P ( S µ ε) =0 for every ε. Here S deotes X+X+ +X. Proof: By Theorem 3. (or Example 3.7) we have E(S )=µ ad Var(S )= σ. By Chebychev s iequality P ( S E(S ) ε) Var(S ) ε for ε>0. Hece P ( S µ ε) σ ε. Takig the limit as teds to ifiity, we get lim P ( S σ µ ε) lim ε which yields lim P ( S µ ε) =0 ad the proof of the theorem is ow complete.
31 Probability ad Mathematical Statistics 367 It is possible to prove the weak law of large umbers assumig oly E(X) to exist ad fiite but the proof is more ivolved. The weak law of large umbers says that the sequece of sample meas { } S from a populatio X stays close to populatio mea E(X) most = of the times. Let us cosider a experimet that cosists of tossig a coi ifiitely may times. Let X i beifthei th toss results i a Head, ad 0 otherwise. The weak law of large umbers says that S = X + X + + X as (3.0) but it is easy to come up with sequeces of tosses for which (3.0) is false: H H H H H H H H H H H H H H T H H T H H T H H T The strog law of large umbers (Theorem 3.) states that the set of bad sequeces like the oes give above has probability zero. Note that the assertio of Theorem 3.9 for ay ε>0ca also be writte as lim P ( S µ <ε)=. The type of covergece we saw i the weak law of large umbers is ot the type of covergece discussed i calculus. This type of covergece is called covergece i probability ad defied as follows. Defiitio 3.. Suppose X,X,... is a sequece of radom variables defied o a sample space S. The sequece coverges i probability to the radom variable X if, for ay ε>0, lim P ( X X <ε)=. I view of the above defiitio, the weak law of large umbers states that the sample mea X coverges i probability to the populatio mea µ. The followig theorem is kow as the Beroulli law of large umbers ad is a special case of the weak law of large umbers. Theorem 3.0. Let X,X,... be a sequece of idepedet ad idetically distributed Beroulli radom variables with probability of success p. The, for ay ε>0, lim P ( S p <ε)=
32 Sequeces of Radom Variables ad Order Statistics 368 where S deotes X+X+ +X. The fact that the relative frequecy of occurrece of a evet E is very likely to be close to its probability P (E) for large ca be derived from the weak law of large umbers. Cosider a repeatable radom experimet repeated large umber of time idepedetly. Let X i =ife occurs o the i th repetitio ad X i =0ifE does ot occur o i th repetitio. The µ = E(X i )= P (E)+0 P (E) =P (E) for i =,, 3,... ad X + X + + X = N(E) where N(E) deotes the umber of times E occurs. Hece by the weak law of large umbers, we have ( ) ( ) lim P N(E) P (E) >ε = lim P X + X + + X µ >ε = lim P ( S µ ) >ε =0. Hece, for large, the relative frequecy of occurrece of the evet E is very likely to be close to its probability P (E). Now we preset the strog law of large umbers without a proof. Theorem 3.. Let X,X,... be a sequece of idepedet ad idetically distributed radom variables with µ = E(X i ) ad σ = Var(X i ) < for i =,,...,. The lim P ( S µ) ε) =0 for every ε. Here S deotes X+X+ +X. The type covergece i Theorem 3. is called almost sure covergece. The otio of almost sure covergece is defied as follows. Defiitio 3. Suppose the radom variable X ad the sequece X,X,..., of radom variables are defied o a sample space S. The sequece X (w) coverges almost surely to X(w) if P ({ w S lim X (w) =X(w) }) =. It ca be show that the covergece i probability implies the almost sure covergece but ot the coverse.
33 Probability ad Mathematical Statistics The Cetral Limit Theorem Cosider a radom sample of measuremet {X i }. The X i s are idetically distributed ad their commo distributio is the distributio of the populatio. We have see that if the populatio distributio is ormal, the the sample mea X is also ormal. More precisely, if X,X,..., X is a radom sample from a ormal distributio with desity f(x) = σ π e ( x µ σ ) the ) X N (µ, σ. The cetral limit theorem (also kow as Lideberg-Levy Theorem) states that eve though the populatio distributio may be far from beig ormal, still for large sample size, the distributio of the stadardized sample mea is approximately stadard ormal with better approximatios obtaied with the larger sample size. Mathematically this ca be stated as follows. Theorem 3. (Cetral Limit Theorem). Let X,X,..., X be a radom sample of size from a distributio with mea µ ad variace σ <, the the limitig distributio of Z = X µ σ is stadard ormal, that is Z coverges i distributio to Z where Z deotes a stadard ormal radom variable. The type of covergece used i the cetral limit theorem is called the covergece i distributio ad is defied as follows. Defiitio 3.3. Suppose X is a radom variable with cumulative desity fuctio F (x) ad the sequece X,X,... of radom variables with cumulative desity fuctios F (x),f (x),..., respectively. The sequece X coverges i distributio to X if lim F (x) =F (x) for all values x at which F (x) is cotiuous. The distributio of X is called the limitig distributio of X.
34 Sequeces of Radom Variables ad Order Statistics 370 Wheever a sequece of radom variables X,X,... coverges i distributio to the radom variable X, it will be deoted by X d X. Example 3.. Let Y = X + X + + X 5 be the sum of a radom sample of size 5 from the distributio whose desity fuctio is f(x) = { 3 x if <x< 0 otherwise. What is the approximate value of P ( 0.3 Y.5) whe oe uses the cetral limit theorem? Aswer: First, we fid the mea µ ad variace σ for the desity fuctio f(x). The mea for this distributio is give by µ = 3 x3 dx [ ] x 4 = 3 =0. 4 Hece the variace of this distributio is give by Var(X) =E(X ) [ E(X)] = 3 x4 dx [ ] x 5 = 3 = 3 5 = P ( 0.3 Y.5) = P ( Y 0.5 0) ( ) 0.3 = P Y 0.5 5(0.6) 5(0.6) 5(0.6) = P ( 0.0 Z 0.50) = P (Z 0.50) + P (Z 0.0) = =0.33.
35 Probability ad Mathematical Statistics 37 Example 3.. Let X,X,..., X be a radom sample of size = 5 from a populatio that has a mea µ =7.43 ad variace σ =56.5. Let X be the sample mea. What is the probability that the sample mea is betwee 68.9 ad 7.97? Aswer: The mea of X is give by E ( X ) =7.43. The variace of X is give by Var ( X ) = σ = 56.5 =.5. 5 I order to fid the probability that the sample mea is betwee 68.9 ad 7.97, we eed the distributio of the populatio. However, the populatio distributio is ukow. Therefore, we use the cetral limit theorem. The cetral limit theorem says that X µ σ N (0, ) as approaches ifiity. Therefore P ( 68.9 X 7.97 ) ( = X 7.43 ) = P ( 0.68 W 0.36) = P (W 0.36) + P (W 0.68) = Example 3.3. Light bulbs are istalled successively ito a socket. If we assume that each light bulb has a mea life of moths with a stadard deviatio of 0.5 moths, what is the probability that 40 bulbs last at least 7 years? Aswer: Let X i deote the life time of the i th bulb istalled. The 40 light bulbs last a total time of S 40 = X + X + + X 40. By the cetral limit theorem 40 X i µ N(0, ) σ as. Thus S 40 (40)() N(0, ). (40)(0.5)
36 Sequeces of Radom Variables ad Order Statistics 37 That is Therefore S N(0, ). P (S 40 7()) ( S40 80 = P.58 = P (Z.530) = ) Example 3.4. Light bulbs are istalled ito a socket. Assume that each has a mea life of moths with stadard deviatio of 0.5 moth. How may bulbs should be bought so that oe ca be 95% sure that the supply of bulbs will last 5 years? Aswer: Let X i deote the life time of the i th bulb istalled. The light bulbs last a total time of The total average life spa S has S = X + X + + X. E (S )= ad Var(S )= 6. By the cetral limit theorem, we get Thus, we seek such that S E (S ) 4 N(0, ) = P (S 60) ( ) S 60 = P 4 4 ( ) 40 8 = P Z ( ) 40 8 = P Z. From the stadard ormal table, we get 40 8 =.645
37 Probability ad Mathematical Statistics 373 which implies =0. Solvig this quadratic equatio for, we get = or Thus =3.5. So we should buy 3 bulbs. Example 3.5. America Airlies claims that the average umber of people who pay for i-flight movies, whe the plae is fully loaded, is 4 with a stadard deviatio of 8. A sample of 36 fully loaded plaes is take. What is the probability that fewer tha 38 people paid for the i-flight movies? Aswer: Here, we like to fid P (X < 38). Sice, we do ot kow the distributio of X, we will use the cetral limit theorem. We are give that the populatio mea is µ = 4 ad populatio stadard deviatio is σ =8. Moreover, we are dealig with sample of size = 36. Thus ( X 4 P (X <38) = P < 8 6 = P (Z < 3) = P (Z <3) = = ) Sice we have ot yet see the proof of the cetral limit theorem, first let us go through some examples to see the mai idea behid the proof of the cetral limit theorem. Later, at the ed of this sectio a proof of the cetral limit theorem will be give. We kow from the cetral limit theorem that if X,X,..., X is a radom sample of size from a distributio with mea µ ad variace σ, the X µ σ d Z N(0, ) as. However, the above expressio is ot equivalet to X d Z N ) (µ, σ as
38 Sequeces of Radom Variables ad Order Statistics 374 as the followig example shows. Example 3.6. Let X,X,..., X be a radom sample of size from a gamma distributio with parameters θ = ad α =. What is the distributio of the sample mea X? Also, what is the limitig distributio of X as? Aswer: Sice, each X i GAM(, ), the probability desity fuctio of each X i is give by { e x if x 0 f(x) = 0 otherwise ad hece the momet geeratig fuctio of each X i is M Xi (t) = t. First we determie the momet geeratig fuctio of the sample mea X, ad the examie this momet geeratig fuctio to fid the probability distributio of X. Sice M X (t) =M Xi(t) ( ) t = M Xi = ( ) t = ( ) t, therefore X GAM (,). Next, we fid the limitig distributio of X as. This ca be doe agai by fidig the limitig momet geeratig fuctio of X ad idetifyig the distributio of X. Cosider lim M X (t) = lim = ( ) t ( lim t = e t = e t. )
39 Probability ad Mathematical Statistics 375 Thus, the sample mea X has a degeerate distributio, that is all the probability mass is cocetrated at oe poit of the space of X. Example 3.7. Let X,X,..., X be a radom sample of size from a gamma distributio with parameters θ = ad α =. What is the distributio of X µ σ as where µ ad σ are the populatio mea ad variace, respectively? Aswer: From Example 3.7, we kow that M X (t) = ( ) t. Sice the populatio distributio is gamma with θ = ad α =, the populatio mea µ is ad populatio variace σ is also. Therefore M X (t) =M X (t) = e t M X ( t ) = e t ( ) t = ( ). t e t The limitig momet geeratig fuctio ca be obtaied by takig the limit of the above expressio as teds to ifiity. That is, lim M X (t) = lim e t ( ) t = e t (usig MAPLE) = X µ σ N(0, ). The followig theorem is used to prove the cetral limit theorem. Theorem 3.3 (Lévy Cotiuity Theorem). Let X,X,... be a sequece of radom variables with distributio fuctios F (x),f (x),... ad momet geeratig fuctios M X (t),m X (t),..., respectively. Let X be a
40 Sequeces of Radom Variables ad Order Statistics 376 radom variable with distributio fuctio F (x) ad momet geeratig fuctio M X (t). If for all t i the ope iterval ( h, h) for some h>0 lim M X (t) =M X (t), the at the poits of cotiuity of F (x) lim F (x) =F (x). The proof of this theorem is beyod the scope of this book. The followig limit lim [ + t + d() ] = e t, if lim d() =0, (3.) whose proof we leave it to the reader, ca be established usig advaced calculus. Here t is idepedet of. Now we proceed to prove the cetral limit theorem assumig that the momet geeratig fuctio of the populatio X exists. Let M X µ (t) be the momet geeratig fuctio of the radom variable X µ. We deote M X µ (t) asm(t) whe there is o dager of cofusio. The M(0) =, M (0) = E(X µ) =E(X) µ = µ µ =0, M (0) = E ( (3.) (X µ) ) = σ. By Taylor series expasio of M(t) about 0, we get M(t) =M(0) + M (0) t + M (η) t where η (0,t). Hece usig (3.), we have M(t) =+ M (η) t =+ σ t + M (η) t σ t =+ σ t + [ M (η) σ ] t. Now usig M(t) we compute the momet geeratig fuctio of Z. Note that Z = X µ σ = σ (X i µ).
41 Probability ad Mathematical Statistics 377 Hece ( ) t M Z (t) = M Xi µ σ ( ) t = M X µ σ [ ( )] t = M σ [ = + t + (M (η) σ ) t σ for 0 < η < σ t. Note that sice 0 < η < σ t, we have ] lim t σ =0, lim η =0, ad lim M (η) σ =0. (3.3) Lettig d() = (M (η) σ ) t σ ad usig (3.3), we see that lim d() = 0, ad Usig (3.) we have M Z (t) = [ + t + d() ]. (3.4) [ lim M Z (t) = lim + t + d() ] = e t. Hece by the Lévy cotiuity theorem, we obtai lim F (x) =Φ(x) where Φ(x) is the cumulative desity fuctio of the stadard ormal distributio. Thus Z Z ad the proof of the theorem is ow d complete. Remark 3.3. I cotrast to the momet geeratig fuctio, sice the characteristic fuctio of a radom variable always exists, the origial proof of the cetral limit theorem ivolved the characteristic fuctio (see for example A Itroductio to Probability Theory ad Its Applicatios, Volume II by Feller). I 988, Brow gave a elemetary proof usig very clever Taylor series expasios, where the use characteristic fuctio has bee avoided.
42 Sequeces of Radom Variables ad Order Statistics Order Statistics Ofte, sample values such as the smallest, largest, or middle observatio from a radom sample provide importat iformatio. For example, the highest flood water or lowest witer temperature recorded durig the last 50 years might be useful whe plaig for future emergecies. The media price of houses sold durig the previous moth might be useful for estimatig the cost of livig. The statistics highest, lowest or media are examples of order statistics. Defiitio 3.4. Let X,X,..., X be observatios from a radom sample of size from a distributio f(x). Let X () deote the smallest of {X,X,..., X }, X () deote the secod smallest of {X,X,..., X }, ad similarly X (r) deote the r th smallest of {X,X,..., X }. The the radom variables X (),X (),..., X () are called the order statistics of the sample X,X,..., X. I particular, X (r) is called the r th -order statistic of X,X,..., X. The sample rage, R, is the distace betwee the smallest ad the largest observatio. That is, R = X () X (). This is a importat statistic which is defied usig order statistics. The distributio of the order statistics are very importat whe oe uses these i ay statistical ivestigatio. The ext theorem gives the distributio of a order statistic. Theorem 3.4. Let X,X,..., X be a radom sample of size from a distributio with desity fuctio f(x). The the probability desity fuctio of the r th order statistic, X (r),is g(x) = where F (x) deotes the cdf of f(x).! (r )! ( r)! [F (x)]r f(x) [ F (x)] r, Proof: Let h be a positive real umber. Let us divide the real lie ito three segmets, amely IR =(, x) [x, x + h] (x + h, ).
43 Probability ad Mathematical Statistics 379 The probability, say p, of a sample value falls ito the first iterval (, x] ad is give by x p = f(t) dt. Similarly, the probability p of a sample value falls ito the secod iterval ad is x+h p = f(t) dt. x I the same toke, we ca compute the probability of a sample value which falls ito the third iterval p 3 = x+h f(t) dt. The the probability that (r ) sample values fall i the first iterval, oe falls i the secod iterval, ad ( r) fall i the third iterval is ( ) p r p p r! 3 = r,, r (r )! ( r)! pr p p r 3 =: P h (x). Sice P h (x) g(x) = lim h 0 h, the probability desity fuctio of the r th statistics is give by P h (x) g(x) = lim h 0 h [ ]! p = lim h 0 (r )! ( r)! pr h p r 3 [ (! x ) ] r = lim f(t) dt (r )! ( r)! h 0 [ [ x+h ( ) ] r lim f(t) dt] lim f(t) dt h 0 h h 0 x x+h! = [F (x)] r f(x) [ F (x)] r. (r )! ( r)! The secod limit is obtaied as f(x) due to the fact that lim h 0 h x+h x = lim h 0 = d dx = f(x) f(t) dt x+h a x a f(t) dt x a h f(t) dt f(t) dt, where x<a<x+ h
44 Sequeces of Radom Variables ad Order Statistics 380 Example 3.8. Let X,X be a radom sample from a distributio with desity fuctio f(x) = { e x for 0 x< 0 otherwise. What is the desity fuctio of Y = mi{x,x } where ozero? Aswer: The cumulative distributio fuctio of f(x) is F (x) = x 0 e t dt = e x I this example, = ad r =. Hece, the desity of Y is g(y) =! 0!! [F (y)]0 f(y) [ F (y)] =f(y) [ F (y)] =e y ( +e y) =e y. Example 3.9. Let Y <Y < <Y 6 be the order statistics from a radom sample of size 6 from a distributio with desity fuctio f(x) = What is the expected value of Y 6? Aswer: { x for 0 <x< 0 otherwise. f(x) =x F (x) = x = x. The desity fuctio of Y 6 is give by 0 tdt g(y) = 6! [F (y)] 5 f(y) 5! 0! =6 ( y ) 5 y =y.
45 Probability ad Mathematical Statistics 38 Hece, the expected value of Y 6 is E (Y 6 )= = 0 0 = 3 = 3. yg(y) dy y y dy [ y 3 ] 0 Example 3.0. Let X, Y ad Z be idepedet uiform radom variables o the iterval (0,a). Let W = mi{x, Y, Z}. What is the expected value of ( ) W? a Aswer: The probability distributio of X (or Y or Z) is f(x) = { a if 0 <x<a 0 otherwise. Thus the cumulative distributio of fuctio of f(x) is give by 0 if x 0 x F (x) = a if 0 <x<a if x a. Sice W = mi{x, Y, Z}, W is the first order statistic of the radom sample X, Y, Z. Thus, the desity fuctio of W is give by g(w) = 3! [F (w)] 0 f(w) [ F (w)] 0!!! =3f(w) [ F (w)] ( =3 w ) ( a a) = 3 ( w ). a a Thus, the pdf of W is give by ( ) 3 a w a if 0 <w<a g(w) = 0 otherwise.
46 Sequeces of Radom Variables ad Order Statistics 38 The expected value of W is [ ( E W ) ] a a ( = w ) g(w) dw a = = 0 a 0 a 0 = 3 5 = 3 5. ( w ) 3 a a 3 a ( w a ) 4 dx [ ( w a ) 5 ] a ( w a ) dw Example 3.. Let X,X,..., X be a radom sample from a populatio X with uiform distributio o the iterval [0, ]. What is the probability distributio of the sample rage W := X () X ()? Aswer: To fid the distributio of W, we eed the joit distributio of the radom variable ( X (),X () ). The joit distributio of ( X(),X () ) is give by h(x,x )=( )f(x )f(x )[F (x ) F (x )], where x x ad f(x) is the probability desity fuctio of X. To determie the probability distributio of the sample rage W, we cosider the trasformatio } U = X () which has a iverse 0 W = X () X () } X () = U X () = U + W. The Jacobia of this trasformatio is ( ) 0 J = det =. Hece the joit desity of (U, W ) is give by g(u, w) = J h(x,x ) = ( )f(u)f(u + w)[f (u + w) F (u)]
47 Probability ad Mathematical Statistics 383 where w 0. Sice f(u) ad f(u+w) are simultaeously ozero if 0 u ad 0 u + w. Hece f(u) ad f(u + w) are simultaeously ozero if 0 u w. Thus, the probability of W is give by j(w) = = = where 0 w. w 0 g(u, w) du ( )f(u)f(u + w)[f (u + w) F (u)] du w du = ( )( w) w 3.5. Sample Percetiles The sample media, M, is a umber such that approximately oe-half of the observatios are less tha M ad oe-half are greater tha M. Defiitio 3.5. Let X,X,..., X be a radom sample. The sample media M is defied as X ( + ) if is odd M = [ ] X ( ) + X ( + ) if is eve. The media is a measure of locatio like sample mea. Recall that for cotiuous distributio, 00p th percetile, π p,isaumber such that πp p = f(x) dx. Defiitio 3.6. The 00p th sample percetile is defied as π p = X ([p]) X (+ [( p)]) if p<0.5 if p>0.5. where [b] deote the umber b rouded to the earest iteger. Example 3.. Let X,X,..., X be a radom sample of size. What is the 65 th percetile of this sample?
48 Sequeces of Radom Variables ad Order Statistics 384 Aswer: 00p =65 p =0.65 ( p) = ()( 0.65) = 4. [( p)] = [4.] = 4 Hece by defiitio of 65 th percetile is π 0.65 = X (+ [( p)]) = X (3 4) = X (9). Thus, the 65 th percetile of the radom sample X,X,..., X is the 9 th - order statistic. For ay umber p betwee 0 ad, the 00p th sample percetile is a observatio such that approximately p observatios are less tha this observatio ad ( p) observatios are greater tha this. Defiitio 3.7. The 5 th percetile is called the lower quartile while the 75 th percetile is called the upper quartile. The distace betwee these two quartiles is called the iterquartile rage. Example 3.3. If a sample of size 3 from a uiform distributio over [0, ] is observed, what is the probability that the sample media is betwee 4 ad 3 4? Aswer: Whe a sample of ( + ) radom variables are observed, the ( +) th smallest radom variable is called the sample media. For our problem, the sample media is give by X () = d smallest {X,X,X 3 }. Let Y = X (). The desity fuctio of each X i is give by f(x) = { if 0 x 0 otherwise. Hece, the cumulative desity fuctio of f(x) is F (x) =x.
49 Probability ad Mathematical Statistics 385 Thus the desity fuctio of Y is give by g(y) = 3! [F (y)] f(y) [ F (y)] 3!! =6F (y) f(y) [ F (y)] =6y ( y). Therefore P ( ) 4 <Y <3 = 4 = [ y =6 = 6. g(y) dy 6 y ( y) dy y3 3 ] Review Exercises. Suppose we roll a die 000 times. What is the probability that the sum of the umbers obtaied lies betwee 3000 ad 4000?. Suppose Kathy flip a coi 000 times. What is the probability she will get at least 600 heads? 3. At a certai large uiversity the weight of the male studets ad female studets are approximately ormally distributed with meas ad stadard deviatios of 80, ad 0, ad 30 ad 5, respectively. If a male ad female are selected at radom, what is the probability that the sum of their weights is less tha 80? 4. Seve observatios are draw from a populatio with a ukow cotiuous distributio. What is the probability that the least ad the greatest observatios bracket the media? 5. If the radom variable X has the desity fuctio ( x) for 0 x f(x) = 0 otherwise, what is the probability that the larger of idepedet observatios of X will exceed?
50 Sequeces of Radom Variables ad Order Statistics Let X,X,X 3 be a radom sample from the uiform distributio o the iterval (0, ). What is the probability that the sample media is less tha 0.4? 7. Let X,X,X 3,X 4,X 5 be a radom sample from the uiform distributio o the iterval (0, θ), where θ is ukow, ad let X max deote the largest observatio. For what value of the costat k, the expected value of the radom variable kx max is equal to θ? 8. A radom sample of size 6 is to be take from a ormal populatio havig mea 00 ad variace 4. What is the 90 th percetile of the distributio of the sample mea? 9. If the desity fuctio of a radom variable X is give by x for e <x<e f(x) = 0 otherwise, what is the probability that oe of the two idepedet observatios of X is less tha ad the other is greater tha? 0. Five observatios have bee draw idepedetly ad at radom from a cotiuous distributio. What is the probability that the ext observatio will be less tha all of the first 5?. Let the radom variable X deote the legth of time it takes to complete a mathematics assigmet. Suppose the desity fuctio of X is give by e (x θ) for θ<x< f(x) = 0 otherwise, where θ is a positive costat that represets the miimum time to complete a mathematics assigmet. If X,X,..., X 5 is a radom sample from this distributio. What is the expected value of X ()?. Let X ad Y be two idepedet radom variables with idetical probability desity fuctio give by f(x) = { e x for x>0 0 elsewhere. What is the probability desity fuctio of W = max{x, Y }?
51 Probability ad Mathematical Statistics Let X ad Y be two idepedet radom variables with idetical probability desity fuctio give by 3 x θ for 0 x θ f(x) = 3 0 elsewhere, for some θ>0. What is the probability desity fuctio of W = mi{x, Y }? 4. Let X,X,..., X be a radom sample from a uiform distributio o the iterval from 0 to 5. What is the limitig momet geeratig fuctio of X µ σ as? 5. Let X,X,..., X be a radom sample of size from a ormal distributio with mea µ ad variace. If the 75 th percetile of the statistic W = ( Xi X ) is 8.4, what is the sample size? 6. Let X,X,..., X be a radom sample of size from a Beroulli distributio with probability of success p =. What is the limitig distributio the sample mea X? 7. Let X,X,..., X 995 be a radom sample of size 995 from a distributio with probability desity fuctio f(x) = e λ λ x x! x =0,,, 3,...,. What is the distributio of 995X? 8. Suppose X,X,..., X is a radom sample from the uiform distributio o (0, ) ad Z be the sample rage. What is the probability that Z is less tha or equal to 0.5? 9. Let X,X,..., X 9 be a radom sample from a uiform distributio o the iterval [, ]. Fid the probability that the ext to smallest is greater tha or equal to 4? 0. A machie eeds 4 out of its 6 idepedet compoets to operate. Let X,X,..., X 6 be the lifetime of the respective compoets. Suppose each is expoetially distributed with parameter θ. What is the probability desity fuctio of the machie lifetime?. Suppose X,X,..., X + is a radom sample from the uiform distributio o (0, ). What is the probability desity fuctio of the sample media X (+)?
52 Sequeces of Radom Variables ad Order Statistics 388. Let X ad Y be two radom variables with joit desity f(x, y) = { x if 0 <y<x < 0 otherwise. What is the expected value of the radom variable Z = X Y 3 +X X Y 3? 3. Let X,X,..., X 50 be a radom sample of size 50 from a distributio with desity f(x) = { Γ(α) θ α x α e x θ for 0 <x< 0 otherwise. What are the mea ad variace of the sample mea X? 4. Let X,X,..., X 00 be a radom sample of size 00 from a distributio with desity f(x) = { e λ λ x x! for x =0,,,..., 0 otherwise. What is the probability that X greater tha or equal to?
53 Probability ad Mathematical Statistics 389 Chapter 4 SAMPLING DISTRIBUTIONS ASSOCIATED WITH THE NORMAL POPULATIONS Give a radom sample X,X,..., X from a populatio X with probability distributio f(x; θ), where θ is a parameter, a statistic is a fuctio T of X,X,..., X, that is T = T (X,X,..., X ) which is free of the parameter θ. If the distributio of the populatio is kow, the sometimes it is possible to fid the probability distributio of the statistic T. The probability distributio of the statistic T is called the samplig distributio of T. The joit distributio of the radom variables X,X,..., X is called the distributio of the sample. The distributio of the sample is the joit desity f(x,x,..., x ; θ) =f(x ; θ)f(x ; θ) f(x ; θ) = f(x i ; θ) sice the radom variables X,X,..., X are idepedet ad idetically distributed. Sice the ormal populatio is very importat i statistics, the samplig distributios associated with the ormal populatio are very importat. The most importat samplig distributios which are associated with the ormal
54 Samplig Distributios Associated with the Normal Populatio 390 populatio are the followigs: the chi-square distributio, the studet s t- distributio, the F-distributio, ad the beta distributio. I this chapter, we oly cosider the first three distributios, sice the last distributio was cosidered earlier. 4.. Chi-square distributio I this sectio, we treat the Chi-square distributio, which is oe of the very useful samplig distributios. Defiitio 4.. A cotiuous radom variable X is said to have a chisquare distributio with r degrees of freedom if its probability desity fuctio is of the form f(x; r) = Γ( r ) x r r e x if 0 x< 0 otherwise, where r > 0. If X has chi-square distributio, the we deote it by writig X χ (r). Recall that a gamma distributio reduces to chi-square distributio if α = r ad θ =. The mea ad variace of X are r ad r, respectively. Thus, chi-square distributio is also a special case of gamma distributio. Further, if r, the chi-square distributio teds to ormal distributio. Example 4.. If X GAM(, ), the what is the probability desity fuctio of the radom variable X? Aswer: We will use the momet geeratig method to fid the distributio of X. The momet geeratig fuctio of a gamma radom variable is give by M(t) =( θt) α, if t< θ.
55 Probability ad Mathematical Statistics 39 Sice X GAM(, ), the momet geeratig fuctio of X is give by M X (t) = t, t <. Hece, the momet geeratig fuctio of X is M X (t) =M X (t) = t = ( t) = MGF of χ (). Hece, if X is GAM(, ) or is a expoetial with parameter, the X is chi-square with degrees of freedom. Example 4.. If X χ (5), the what is the probability that X is betwee.45 ad.83? Aswer: The probability of X betwee.45 ad.83 ca be calculated from the followig: P (.45 X.83) = P (X.83) P (X.45) = f(x) dx.45 0 f(x) dx = 0 Γ ( ) 5 5 = (from χ table) =0.95. x 5 e x dx.45 0 Γ ( ) 5 5 x 5 e x dx This above itegrals are hard to evaluate ad thus their values are take from the chi-square table. Example 4.3. If X χ (7), the what are values of the costats a ad b such that P (a <X<b)=0.95? Aswer: Sice 0.95 = P (a <X<b)=P (X <b) P (X <a), we get P (X <b)= P (X <a).
Distribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationIE 230 Seat # Name < KEY > Please read these directions. Closed book and notes. 60 minutes.
IE 230 Seat # Name < KEY > Please read these directios. Closed book ad otes. 60 miutes. Covers through the ormal distributio, Sectio 4.7 of Motgomery ad Ruger, fourth editio. Cover page ad four pages of
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
More informationSOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker
SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker CHAPTER 9. POINT ESTIMATION 9. Covergece i Probability. The bases of poit estimatio have already bee laid out i previous chapters. I chapter 5
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I
THE ROYAL STATISTICAL SOCIETY 5 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA STATISTICAL THEORY AND METHODS PAPER I The Society provides these solutios to assist cadidates preparig for the examiatios i future
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationBasis for simulation techniques
Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 19
CS 70 Discrete Mathematics ad Probability Theory Sprig 2016 Rao ad Walrad Note 19 Some Importat Distributios Recall our basic probabilistic experimet of tossig a biased coi times. This is a very simple
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationPRACTICE PROBLEMS FOR THE FINAL
PRACTICE PROBLEMS FOR THE FINAL Math 36Q Fall 25 Professor Hoh Below is a list of practice questios for the Fial Exam. I would suggest also goig over the practice problems ad exams for Exam ad Exam 2 to
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationTopic 10: Introduction to Estimation
Topic 0: Itroductio to Estimatio Jue, 0 Itroductio I the simplest possible terms, the goal of estimatio theory is to aswer the questio: What is that umber? What is the legth, the reactio rate, the fractio
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationt distribution [34] : used to test a mean against an hypothesized value (H 0 : µ = µ 0 ) or the difference
EXST30 Backgroud material Page From the textbook The Statistical Sleuth Mea [0]: I your text the word mea deotes a populatio mea (µ) while the work average deotes a sample average ( ). Variace [0]: The
More informationIE 230 Probability & Statistics in Engineering I. Closed book and notes. No calculators. 120 minutes.
Closed book ad otes. No calculators. 120 miutes. Cover page, five pages of exam, ad tables for discrete ad cotiuous distributios. Score X i =1 X i / S X 2 i =1 (X i X ) 2 / ( 1) = [i =1 X i 2 X 2 ] / (
More information(6) Fundamental Sampling Distribution and Data Discription
34 Stat Lecture Notes (6) Fudametal Samplig Distributio ad Data Discriptio ( Book*: Chapter 8,pg5) Probability& Statistics for Egieers & Scietists By Walpole, Myers, Myers, Ye 8.1 Radom Samplig: Populatio:
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More information