Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
|
|
- Sharon Horton
- 5 years ago
- Views:
Transcription
1 Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x g(x f X (xdx g(x f X (xdx E[g(X]. Markov s Iequality. For ay h > 0, P( X h E[ X ] h. Whe X oly takes o-egative values the for ay h > 0 P(X h E[X] h. Proof. Take g(x X h lemma. i the lemma. If X oly takes o-egative values take g(x X h i the 2. Chebyshev s Iequality. If E[X] µ ad Var(X σ 2, which are fiite, the for ay h > 0 P( X µ h σ2 h 2. Proof. Take g(x ( 2 X µ h i the lemma. Note: Chebyshev s iequality ca be used to derive the weak law of large umbers. This is specified i the theorem below. Theorem. Let X,X 2,... be a sequece of i.i.d. radom variables each with fiite mea µ ad fiite variace σ 2. The for ay ε > 0 ad δ > 0 there exists a N such that P( X µ ε δ for all N, where X j X j. Proof. Note that E[X ] µ ad Var(X σ2. Cosider ay ε > 0 ad δ > 0. Apply Chebyshev s iequality to X ad let h ε. The P( X µ ε σ2 δ provided σ2 ε 2 ε 2 δ. Therefore we eed oly choose N σ2 to obtai the result. ε 2 δ Note. Observe that lim P( X µ ε 0 for ay ε > 0. We say that X coverges i probability to µ as teds to ifiity. Some examples usig the iequalities.. From Markov s iequality with h Nµ, if X is a o-egative radom variable, P(X > Nµ for ay N > 0. µ Nµ N 2. If σ 2 0 the from Chebyshev s iequality for ay h > 0, P( X µ < h P( X µ h σ2. Hece P(X µ lim h 2 h 0 P( X µ < h. So variace zero implies the radom variable takes a sigle value with probability.
2 3. Whe σ 2 > 0 Chebyshev s iequality gives a lower boud o the probability that X lies withi k stadard deviatios from the mea. Take h kσ. The P( X µ < kσ P( X µ kσ σ2 (kσ 2 k 2 4. Whe σ, how large a sample is eeded if we wat to be at least 95% certai that the sample mea lies withi 0.5 of the true mea? We use Chebyshev s iequality for X with h 0.5. The P( X µ < 0.5 ( X µ 0.5 provided So we eed a miimum sample size of 80. The Cetral Limit Theorem. σ2 ( Let X,X 2,... be a sequece of i.i.d. radom variables each with fiite mea µ ad fiite variace σ 2 ad let X be the sample mea based o X,...,X. The we ca fid a approximatio for P(X A whe is large by writig the evet for X i terms of the stadardized variable Z (X µ/σ (i.e. P(X A P ( Z (A µ σ ad provig that lim P(Z z Φ(z z 2π e x2 /2 dx which is the c.d.f. of N(0,. The proof of this result uses the m.g.f. ad the followig lemma. Lemma. Let Z,Z 2,... be a sequece of radom variables. If lim M Z (t M(t, which is the m.g.f. of a distributio with c.d.f. F, the lim F Z (z F(z at all poits z for which F(z is cotiuous. Theorem (The Cetral Limit Theorem. Let X,X 2,... be a sequece of i.i.d. radom variables each with m.g.f. which exists for etries i a ope regio about zero so is differetiable with fiite mea, deoted by µ, ad fiite variace, deoted by σ 2. Let Z (X µ/σ, the lim P(Z z Φ(z. Proof. Let U j (X j µ/σ ad let M U (t be the commo m.g.f. The M U (t e µt/σ M X (t/σ exists i a ope iterval about t 0, M(0, M (0 E[U] 0 ad M (0 E[U 2 ] Var(U. So U,U 2,... are i.i.d. with mea zero ad variace oe. Now [ M Z (t E e t j U j/ ] j E[e tu j/ ] ( M U (t/ Takig logs to base e gives l(m Z (t (l(m U (t/. L Hopital s rule. The Now let x / ad use lim l(m U(t/ lim x 0 l(m U (xt x 2
3 lim tm x 0 U (xt/m U(xt lim 2x t 2 (M x 0 t2 (M U (0M U(0 (M U (02 2(M U (0 2 t2 2 U (xtm U(xt (M U (xt2 /(M U (xt 2 2 Hece lim t l(m Z (t t 2 /2 ad so lim t M Z (t e t2 /2. Sice this is the m.g.f. of a N(0, distributio, usig the lemma proves that lim P(Z z Φ(z z 2π e x2 /2 dx The bivariate ad multivariate ormal distributio. A idirect method was used o problem sheet 9 to get you to derive stadard results for a bivariate ormal distributio. The results are summarised below. The results may be proved directly, however it is messy uless you use matrix ad vector otatio. Oce you do this results ca just as easily be obtaied for the multivariate ormal, so we may just as well derive results immediately for the more geeral case. Summary of results for the bivariate ormal distributio.. If X ad X 2 have bivariate ormal distributio the the joit p.d.f. is f X,X 2 (x,x 2 2πσ σ 2 ( ρ 2 e ( ( x µ 2 2ρ ( ( ( x µ x2 µ 2 x2 µ 2 2( ρ 2 σ σ σ σ 2 for all x,y. The distributio has parameters µ,µ 2,σ 2,σ2 2,ρ. The parameter ρ is restricted so that < ρ <. 2. The joit m.g.f. is M X,X 2 (t,t 2 e (µ t +µ 2 t (σ2 t2 +2ρσ σ 2 t t 2 +σ 2 2 t2 2 This ca be used to idetify the parameters ad fid the margial distributios. M X (t M X,X 2 (t,0 e µ t + 2 σ2 t2. Hece X N(µ,σ 2. Similarly X 2 N(µ 2,σ 2 2. Differetiatig the joit m.g.f. i stadard maer shows that ρ(x,x 2 ρ. 3. X ad X 2 are idepedet iff ρ 0. This is easily see from either the joit p.d.f. or the joit m.g.f. 4. The coditioal distributio of X 2 X x is ormal with mea liear i x ad variace which does ot deped o x. A similar result holds for the coditioal distributio of X X 2 x 2. Usig vector ad matrix otatio. X ( X X 2 ( x ; x x 2 ( t ; t t 2 ( µ ; m µ 2 ( σ 2 ; V ρσ σ 2 ρσ σ 2 σ 2 2
4 The m is the vector of meas ad V is the variace-covariace matrix. Note that V σ 2 σ2 2 ( ρ2 ad V ( σ 2 ( ρ 2 ρ σ σ 2 ρ σ σ 2 σ 2 2 Hece f X (x (2π 2/2 V /2 e 2 (x mt V (x m for all x. Also M X (t e tt m+ 2 tt Vt. The Multivariate Normal Distributio. We agai use matrix ad vector otatio, but ow there are radom variables so that X, x, t ad m are ow -vectors with i th etries X i, x i, t i ad µ i ad V is the matrix with ii th etry σ 2 i ad i j th etry (for i j σ i j. Note that V is symmetric so that V T V. The joit p.d.f. is f X (x (2π /2 V /2 e 2 (x mt V (x m for all x. We say that X N(m,V. We ca fid the joit m.g.f. quite easily. ] M X (t E [e j t jx j E[e tt X ]... (2π /2 V /2 e 2((x m T V (x m 2t T x dx...dx We do the equivalet of completig the square, i.e. we write (x m T V (x m 2t T x (x m a T V (x m a + b for a suitable choice of the -vector a of costats ad a costat b. The M X (t e b/2... (2π /2 V /2 e 2 (x m at V (x m a dx...dx e b/2. We just eed to fid a ad b. Expadig we have ((x m a T V ((x m a + b (x m T V (x m 2a T V (x m + a T V a + b (x m T V (x m 2a T V x + [ 2a T V m + a T V a + b ] This has to equal (x m T V (x m 2t T x for all x. Hece we eed a T V t T ad b [ 2a T V m + a T V a ]. Hece a Vt ad b [ 2t T m + t T Vt ]. Therefore
5 M X (t e b/2 e tt m+ 2 tt Vt Results obtaied usig the m.g.f.. Ay (o-empty subset of multivariate ormals is multivariate ormal. Simply put t j 0 for all j for which X j is ot i the subset. For example M X (t M X,...,X (t,0,...,0 e t µ +t 2 σ2 /2. Hece X N(µ,σ 2. A similar result holds for X i. This idetifies the parameters µ i ad σ 2 i as the mea ad variace of X i. Also M X,X 2 (t,t 2 M X,...,X (t,t 2,0,...,0 e t µ +t 2 µ (t2 σ2 +2σ 2t t 2 +σ 2 2 t2 2 Hece X ad X 2 have bivariate ormal distributio with σ 2 Cov(X,X 2. A similar result holds for the joit distributio of X i ad X j for i j. This idetifies V as the variace-covariace matrix for X,...,X. 2. X is a vector of idepedet radom variables iff V is diagoal (i.e. all off-diagoal etries are zero so that σ i j 0 for i j. Proof. From (, if the X s are idepedet the σ i j Cov(X i,x j 0 for all i j, so that V is diagoal. If V is diagoal the t T Vt j σ2 j t2 j ad hece M X (t e tt m+ 2 tt Vt j ( e µ jt j + 2 σ2 j t2 j /2 j M Xj (t j By the uiqueess of the joit m.g.f., X,...,X are idepedet. 3. Liearly idepedet liear fuctios of multivariate ormal radom variables are multivariate ormal radom variables. If Y AX + b, where A is a o-sigular matrix ad b is a (colum -vector of costats, the Y N(Am + b,ava T. Proof. Use the joit m.g.f. M Y (t E[e tt Y ] E[e tt AX+b ] e tt b E[e (AT t T X ] e tt b M X (A T t e tt b e (AT t T m+ 2 (AT t T V(A T t e tt (Am+b+ 2 tt (AVA T t This is just the m.g.f. for the multivariate ormal distributio with vector of meas Am + b ad variace-covariace matrix AVA T. Hece, from the uiqueess of the joit m.g.f, Y N(Am + b,ava T. Note that from (2 a subset of the Y s is multivariate ormal.
6 NOTE. The results cocerig the vector of meas ad variace-covariace matrix for liear fuctios of radom variables hold regardless of the joit distributio of X,...,X. We defie the expectatio of a vector of radom variables X, E[X] to be the vector of the expectatios ad the expectatio of a matrix of radom variables Y, E[Y], to be the matrix of the expectatios. The the variace-covariace matrix of X is just E[(X E[X](X E[X] T ]. The followig results are easily obtaied: (i Let A be a m matrix of costats, B be a m k matrix of costats ad Y be a k matrix of radom variables. The E[AY + B] AE[Y] + B. Proof. The i j th etry of E[AY + B] is E[ r A iry r j + B i j ] r A ire[y r j ] + B i j, which is the i j th etry of AE[Y] + B. The result is the immediate. (ii Let C be a k m matrix of costats ad Y be a k matrix of radom variables. The E[YC] E[Y]C. Proof. Just traspose the equatio. The result the follows from (i. Hece if Z AX+b, where A is a m matrix of costats, b is a m-vector of costats ad X is a -vector of radom variables with E[X] m ad variace-covariace matrix V, the E[Z] E[AX + b] AE[X] + b Am + b Also the variace-covariace matrix for Y is just E[(Y E[Y](Y E[Y] T ] E[A(X m(x m T A T ] AE[(X m(x m T ]A T AVA T Example. Suppose that E[X ], E[X 2 ] 0, Var(X 2, Var(X 2 4 ad Cov(X,X 2. Let Y X + X 2 ad Y 2 X + ax 2. Fid the meas, variaces ad covariace ad hece fid a so that Y ad Y 2 are ucorrelated. Writig i vector ad matrix otatio we have E[Y] Am ad the variace-covariace matrix for Y is just AVA T where m ( 0 V ( 2 4 A ( a Therefore Am ( a ( 0 (
7 ( AVA T a ( 2 4 ( a ( a 3 + 5a 2 + 2a + 4a 2 Hece Y ad Y 2 have meas ad, variaces 8 ad 2 + 2a + 4a 2 ad covariace 3 + 5a. They are therefore ucorrelated if 3 + 5a 0, i.e. if a 3 5.
Lecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationSome Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationSTATISTICAL METHODS FOR BUSINESS
STATISTICAL METHODS FOR BUSINESS UNIT 5. Joit aalysis ad limit theorems. 5.1.- -dimesio distributios. Margial ad coditioal distributios 5.2.- Sequeces of idepedet radom variables. Properties 5.3.- Sums
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationEstimation of the Mean and the ACVF
Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationUnivariate Normal distribution. whereaandbareconstants. Theprobabilitydensityfunction(PDFfromnowon)ofZ andx is. ) 2π.
Uivariate Normal distributio I geeral, it has two parameters, µ ad σ mea ad stadard deviatio. A special case is stadardized Normal distributio, with the mea of 0 ad stadard deviatio equal to. Ay geeral
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationStatistical and Mathematical Methods DS-GA 1002 December 8, Sample Final Problems Solutions
Statistical ad Mathematical Methods DS-GA 00 December 8, 05. Short questios Sample Fial Problems Solutios a. Ax b has a solutio if b is i the rage of A. The dimesio of the rage of A is because A has liearly-idepedet
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationAMS 216 Stochastic Differential Equations Lecture 02 Copyright by Hongyun Wang, UCSC ( ( )) 2 = E X 2 ( ( )) 2
AMS 216 Stochastic Differetial Equatios Lecture 02 Copyright by Hogyu Wag, UCSC Review of probability theory (Cotiued) Variace: var X We obtai: = E X E( X ) 2 = E( X 2 ) 2E ( X )E X var( X ) = E X 2 Stadard
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMatrix Representation of Data in Experiment
Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationStatistical Theory; Why is the Gaussian Distribution so popular?
Statistical Theory; Why is the Gaussia Distributio so popular? Rob Nicholls MRC LMB Statistics Course 2014 Cotets Cotiuous Radom Variables Expectatio ad Variace Momets The Law of Large Numbers (LLN) The
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationSTAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationExercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
More information2. The volume of the solid of revolution generated by revolving the area bounded by the
IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationCH.25 Discrete Random Variables
CH.25 Discrete Radom Variables 25B PG.784-785 #1, 3, 4, 6 25C PG.788-789 #1, 3, 5, 8, 10, 11 25D PG.791-792 #1, 3, 6 25E PG.794-795 #1, 2, 3, 7, 10 25F.1 PG.798-799 #2, 3, 5 25F.2 PG. 800-802 #2, 4, 7,
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationJoint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }
UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig
More information