Probability and Statistics

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Probability and Statistics"

Transcription

1 ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, Basic robability Theory 11 robability Spaces A probability space is a triple (Ω, F, ), where Ω is a set of outcomes, F is a set of evets, ad : F [0, 1] is a fuctio that assigs probabilities to evets A σ-algebra (or σ-field) F is a collectio of subsets of Ω that satisfy 1, Ω F 2 if A F, the A c F 3 if A i F is a coutable sequece of sets, the i A i F A measurable space (Ω, F) is a space o which we ca put a measure A measure µ : F R is a oegative coutably additive set fuctio that satisfies 1 µ(a) µ( ) = 0 for all A F 2 if A i F is a coutable sequece of disjoit sets, the µ( i A i ) = i µ(a i ) If µ(ω) = 1, we call µ a probability measure Let µ be a measure o (Ω, F) 1 Mootoicity If A B, the µ(a) µ(b) 2 Subadditivity If A m=1a m, the µ(a) m=1 µ(a m) 3 Cotiuity from below If A i A (ie, A 1 A 2 ad i A i = A), the µ(a i ) µ(a) 4 Cotiuity from above If A i A (ie, A i A 2 ad i A i = A), with µ(a 1 ) <, the µ(a i ) µ(a) 12 istributios A radom variable X is a real-valued fuctio defied o Ω, such that for every Borel set B R, we have X 1 (B) = {ω Ω : X(ω) B} F A radom variable X is discrete if its possible values are fiite or coutably ifiite A radom variable X is cotiuous if its possible values form a ucoutable set ad the probability that X equals ay such value exactly is zero A trivial, but useful, type of example of a radom variable is the idicator fuctio of a set A F: { 1 ω A 1 A (ω) = 0 ω / A Luyag Che: 1

2 ICME Refresher Course: robability ad Statistics Staford Uiversity If X is a radom variable, the X iduces a probability measure o R called its distributio, by settig µ(a) = (X A) for Borel sets A The distributio of a radom variable X is described by givig its distributio fuctio F (x) = (X x) Ay distributio fuctio F has the followig properties: 1 F is odecreasig 2 lim x F (x) = 1, lim x F (x) = 0 3 F is right cotiuous, that is, lim y x F (y) = F (x) 4 lim y x F (y) = F (x ) = (X < x) Ay fuctio F satisfyig 1 3 above is the distributio fuctio of some radom variable Whe the distributio fuctio F (x) has the form we say that X has desity fuctio f 13 Itegratio & Expected Value F (x) = x Suppose f ad g are itegrable fuctios o (Ω, F, µ) 1 If f 0 ae, the fdµ 0 2 For all a R, afdµ = a fdµ 3 f + gdµ = fdµ + gdµ 4 If g f ae, the gdµ fdµ 5 If g = f ae, the gdµ = fdµ 6 fdµ f dµ f(y)dy If X is a radom variable o (Ω, F, ), the we defie its expected value to be E[X] = Xd E[X] does ot always exist Jese s iequality Suppose φ is covex, ad X ad φ(x) are both itegrable, the φ(e[x]) E[φ(X)] Hölder s iequality If p, q (1, ) with 1/p + 1/q = 1, the E[ XY ] (E[ X p ]) 1 p (E[ Y q ]) 1 q The special case p = q = 2 is called the Cauchy-Schwarz iequality Markov s iequality ( X a) a 1 E[ X ] Chebyshev s iequality ( X a) a 2 E[ X 2 ] If k is a positive iteger, the E[X k ] is called the kth momet of X The first momet E[X] is usually called the mea ad deoted by µ If E[X 2 ] <, the the variace of X is defied to be var(x) = E[(X µ) 2 ] = E[X 2 ] µ 2 The covariace of two radom variables X ad Y is defied as cov(x, Y ) = E[(X µ X )(Y µ Y )] = E[XY ] µ X µ Y Luyag Che: 2

3 ICME Refresher Course: robability ad Statistics Staford Uiversity 14 Itegratio to the Limit omiated Covergece Theorem If X X as, X Y for all ad E[Y ] <, the E[X ] E[X] Mootoe Covergece Theorem If 0 X X, the E[X ] E[X] Fatou s Lemma If X 0, the 15 Fubii s Theorem X Y lim if E[X ] E[lim if X ] Fubii s theorem If f 0 or f dµ <, the f(x, y)µ 2 (dy)µ 1 (dx) = fdµ = X Y Exercise Let X be a oegative radom variable Show that 2 Covergece 21 Covergece Cocepts E[X] = 0 Y (X t)dt X f(x, y)µ 1 (dx)µ 2 (dy) Coverge i probability We say that X X i probability, if for ay ε > 0, lim ( X X > ε) = 0 Coverge i L p We say that X X i L p, if lim E[ X X p ] = 0 Coverge almost surely We say that X X as, if (lim X = X) = 1 Coverge i distributio We say that X X i distributio, their CFs coverge, ie F (x) F (x) for ay cotiuous poit x of F Note The followig three statemets are equivalet: 1 lim E[g(X )] = E[g(X)] for all bouded ad cotiuous g(x) 2 lim E[e iαx ] = E[e iαx ] poitwise for all α R 3 lim F (x) = F (x) for ay cotiuous poit x of F 22 Relatioship betwee ifferet Covergeces If X as X, the X X roof ( ε>0 N>0 N { X X < ε}) = 1 = ( ε>0 N>0 N { X X ε}) = 0 = ( N>0 N { X N ε}) = 0 ε > 0 = lim ( X X ε) = 0 Luyag Che: 3

4 ICME Refresher Course: robability ad Statistics Staford Uiversity as X X does t imply X X Couterexample { i 1 t < i+1 f 2 +i(t) = 2 k 2 k k 0 otherwise i = 0, 1,, 2 k 1, k = 0, 1, X = f (U) where U is uiformly distributed o [0, 1] X coverges to 0 i probability, but ot as If X L p X, the X X roof ( X X ε) E[ X X p ] ε p 0 L X X does t imply p X X Couterexample f (t) = { 1/p 0 t < 1 0 otherwise X = f (U) where U is uiformly distributed o [0, 1] X coverges to 0 i probability, but ot i L p If X X, the X X If X a (costat), the X a 23 Cotiuous Mappig Theorem ad Slutsky s Theorem Cotiuous Mappig Theorem Suppose g : R R is a cotiuous fuctio 1 If X X, the g(x ) g(x) 2 If X X, the g(x ) g(x) 3 If X as X, the g(x ) as g(x) Slutsky s Theorem If X X ad Y a (costat), the X + Y X + a ad X Y ax 24 elta Method Theorem Let X 1, X 2, be a sequece of radom variables such that (X a) Z for some radom variable Z ad costat a Let g : R R be cotiuously differetiable at a The (g(x ) g(a)) g (a)z roof where X a X a (g(x ) g(a)) = g ( X ) (X a) (X a) Z X a X a g ( X ) g (a) The use Slutsky s Theorem Luyag Che: 4

5 ICME Refresher Course: robability ad Statistics Staford Uiversity 25 Weak Laws of Large Numbers (WLLN) Theorem Let X 1, X 2, be ucorrelated radom variables with E[X i ] = µ ad var(x i ) C < If S = X X the as, S / µ i L 2 ad also i probability roof E[S /] = µ E[ S / µ 2 ] = var(s /) = 1 2 var(s ) = 1 2 var(x i ) C 0 Theorem Let X 1, X 2, be iid radom variables with E[X i ] = µ ad E[ X i ] < If S = X X the as, S / µ i probability roof S / µ = 1 (X i 1 { Xi } + X i 1 { Xi >}) E[X 1 1 { X1 }] + E[X 1 1 { X1 }] E[X 1 ] ( 1 = ) (X i 1 { Xi } E[X 1 1 { X1 }]) + 1 = I + II + III ( ) X i 1 { Xi >} + E[X 1 1 { X1 }] E[X 1 ] E[ I 2 ] = 1 E[ X 11 { X1 } E[X 1 1 { X1 }] 2 ] 1 E[ X { X1 }] = 1 E[ X { X1 ε }] + 1 E[ X {ε < X1 }] ε 2 + E[ X 1 1 { X1 >ε }] [ 1 ] E[ II ] = E X i 1 { Xi >} 1 E[ X i 1 { Xi >}] = E[ X 1 1 { X1 >}] 0 III = E[X 1 1 { X1 }] E[X 1 ] E[ X 1 1 { X1 >}] 0 Note Neither idepedece of the X i or their fiite variace are eeded for the validity of WLLN 26 Strog Laws of Large Numbers (SLLN) Theorem Let X 1, X 2, be iid radom variables with E[X i ] = µ ad E[ X i ] < If S = X X the as, S / µ as If the iid radom variables {X i } have fiite forth order momets, E[ X i 4 ] < or E[ X i µ 4 ] <, the a applicatio of the Chebyshev iequality with p = 4 gives the eeded estimate ad we have the SLLN i this case Of course, this is oly a sufficiet coditio for its validity As with the WLLN, it is eough that E[ X i ] < 27 Cetral Limit Theorem Theorem Let X 1, x 2, be iid radom variables with E[X i ] = µ ad var(x i ) = σ 2 < If S = X X the (S / µ) N (0, σ 2 ) roof E[e iα (S / µ) ] = E[e i α j=1 (Xj µ) ] = φ ( α ) where φ(α) = E[e iα(x1 µ) ] The φ(0) = 1, φ (0) = 0, φ (0) = σ 2 By Taylor s theorem, we have where 0 < α < α φ( α ) = 1 φ (α ) α2 2 φ ( α ) e α2 σ 2 2 Luyag Che: 5

6 ICME Refresher Course: robability ad Statistics Staford Uiversity 3 Statistics 31 robability ad Statistics The basic problem of probability is: Give the distributio of the data, what are the properties (eg expectatio, variace, etc ) of the outcomes? The basic problem of statistics is: Give the outcomes, what ca we say about the distributio of the data? (Give X 1,, X F, what ca we say about F? ) 32 Fudametal Cocepts oit estimatio ivolves the use of sample data to calculate a sigle value (kow as a statistic) which is to serve as a best guess or best estimate of a ukow (fixed or radom) populatio parameter Let X 1,, X be iid data poits from some distributio F (x; θ ) A poit estimator ˆθ of parameter θ is some fuctio of X 1,, X : ˆθ = g(x 1,, X ) We itroduce the followig two methods: Method of Momets ad Maximum Likelihood I statistics, the bias of a estimator is the differece betwee this estimator s expected value ad the true value of the parameter beig estimated A estimator with zero bias is called ubiased Otherwise the estimator is said to be biased Let ˆθ be a estimate of a parameter θ based o a sample of size The ˆθ is said to be cosistet i probability if ˆθ coverges i probability to θ as approaches ifiity A 1 α cofidece iterval for a parameter θ is a iterval C = (a, b) where a = a(x 1,, X ) ad b = b(x 1,, X ) are fuctios of the data such that (θ C ) 1 α 33 The Methods of Momets The kth momet of a probability law is defied as µ k = E[X k ], where X is a radom variable followig that probability law If X 1,, X are iid radom variables from that distributio, the kth sample momet is defied as ˆµ k = 1 Xk i We ca view ˆµ k as a estimate of µ k The method of momets estimates parameters by fidig expressios for them i term of the lowest possible order momets ad the substitutig sample momets ito the expressios Example The first ad secod momets for the ormal distributio N (µ, σ 2 ) are µ 1 = E[X] = µ µ 2 = E[X 2 ] = µ 2 + σ 2 Therefore, µ = µ 1 ad σ 2 = µ 2 µ 2 1 The correspodig estimates of µ ad σ 2 from the sample momets are ˆσ 2 = 1 ˆµ = 1 ( 1 Xi 2 X i = X X i ) 2 = 1 (X i X) 2 Luyag Che: 6

7 ICME Refresher Course: robability ad Statistics Staford Uiversity Questio Are the two estimators above ubiased? Are the two estimators above cosistet? What are the cofidece itervals? E[ˆµ] = µ ˆσ 2 = 1 (X i µ) 2 ( X µ) 2 E[ˆσ 2 ] = σ 2 1 σ2 = 1 σ2 ˆµ is ubiased ˆσ 2 is biased Both ˆµ ad ˆσ 2 are cosistet estimators A 1 α cofidece iterval of ˆµ is [µ σ Φ 1 (1 α/2), µ + σ Φ 1 (1 α/2)] (ˆσ 2 /σ 2 χ 2 ( 1)) 34 The Method of Maximum Likelihood Suppose that radom variables X 1,, X have a joit desity f(x 1,, x θ) Give observed values X i = x i, i = 1,,, the likelihood of θ as a fuctio of x 1,, x is defied as L(θ) = f(x 1,, x θ) If X i are assumed to be iid, the likelihood is L(θ) = f(x i θ) The log likelihood is l(θ) = log L(θ) = log f(x i θ) The maximum likelihood estimate (MLE) of θ is that value of θ that maximizes the likelihood, that is, makes the observed data most probable or most likely The estimates obtaied by the method of maximum likelihood are ot always the same as those obtaied by the method of momets Example If X 1,, X are iid N (µ, σ 2 ), their joit desity is the product of their margial desities: 1 ( f(x 1,, x µ, σ) = exp 1 [ xi µ ] 2 ) 2πσ 2 2 σ The log likelihood is thus l(µ, σ) = log σ 2 The partials with respect to µ ad σ are l µ = 1 σ 2 log 2π 1 2σ 2 (X i µ) l σ = σ + 1 σ 3 The followig are the good properties of the MLE: (X i µ) 2 (X i µ) 2 ˆµ MLE = X ˆσ MLE = 1 (X i X) 2 1 Uder appropriate smoothess coditios o f, the MLE from a iid sample is cosistet 2 Uder appropriate smoothess coditios o f, (ˆθ θ ) N (0, 1/I(θ )) 3 The MLE achieves the Cramer-Rao lower boud Fisher Iformatio [ ] 2 [ 2 ] I(θ) = E θ log f(x θ) = E θ 2 log f(x θ) Luyag Che: 7

8 ICME Refresher Course: robability ad Statistics Staford Uiversity 35 Hypothesis Testig H 0 : the ull hypotheses H 1 (or H A ): the alterative hypothesis Rejectig H 0 whe it is true is called a type I error The probability of a type I error is called the sigificace level of the test ad is usually deoted by α Acceptig the ull hypothesis whe it is false is called a type II error Its probability is usually deoted by β The set of values of the test statistic that leads to rejectio of the ull hypothesis is called the rejectio regio, ad the set of values that leads to acceptace is called the acceptace regio The probability distributio of the test statistic whe the ull hypothesis is true is called the ull distributio The p-value is the probability of a result as or more extreme tha that actually observed if the ull hypothesis were true Some familiar hypothesis tests: z-test, Studet s t-test, Geeralized Likelihood Ratio Test Suppose that the observatios X = (X 1,, X ) have a joit desity fuctio f(x 1,, x θ) H 0 specifies that θ ω 0 ad H 1 specifies that θ ω 1, where ω 0 ω 1 = ad Ω = ω 0 ω 1 The test statistic Λ = max[l(θ)] θ ω 0 max [L(θ)] θ Ω Uder smoothess coditios o the probability desity, the ull distributio of 2 log Λ teds to a chi-square distributio with degrees of freedom equal to dim Ω dim ω 0 as the sample size teds to ifiity 36 Liear Regressio Cosider the followig regressio model: where Y = y 1 y The least square estimator β = β 1 β p Y = Xβ + ε ε = ε 1 ε X = ˆβ LS = arg mi Y Xβ 2 2 Cosider the model above ad we have the followig assumptios: 1 X is o-radom matrix with full colum rak 2 E[ε] = 0 3 cov(ε i, ε j ) = σ 2 δ ij 4 ε i iid N (0, σ 2 ) ˆβ LS = (X T X) 1 X T Y Uder assumptio 1-2, ˆβ LS is a ubiased estimator x 11 x 1p x 1 x p Luyag Che: 8

9 ICME Refresher Course: robability ad Statistics Staford Uiversity Uder assumptio 1-3, Cov( ˆβ LS ) = σ 2 (X T X) 1 A ubiased estimator of σ 2 is s 2 = 1 p RSS = 1 p (Y X ˆβ LS ) T (Y X ˆβ LS ) Uder assumptio 1 ad 4, ˆβ LS N (β, σ 2 (X T X) 1 ) RSS σ 2 χ 2 p ˆβ LS,j β j s t p c jj where c jj is the jth elemet o the diagoal of (X T X) 1 Luyag Che: 9

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Distribution of Random Samples & Limit theorems

Distribution of Random Samples & Limit theorems STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Mathematical Statistics - MS

Mathematical Statistics - MS Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Introductory statistics

Introductory statistics CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key

More information

LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Lecture 8: Convergence of transformations and law of large numbers

Lecture 8: Convergence of transformations and law of large numbers Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes. Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

IIT JAM Mathematical Statistics (MS) 2006 SECTION A

IIT JAM Mathematical Statistics (MS) 2006 SECTION A IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim

More information

Lecture 3 : Random variables and their distributions

Lecture 3 : Random variables and their distributions Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2. SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

More information

STA Object Data Analysis - A List of Projects. January 18, 2018

STA Object Data Analysis - A List of Projects. January 18, 2018 STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

ST5215: Advanced Statistical Theory

ST5215: Advanced Statistical Theory ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The

More information

TAMS24: Notations and Formulas

TAMS24: Notations and Formulas TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

More information

The Central Limit Theorem

The Central Limit Theorem Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx

More information

Notes 5 : More on the a.s. convergence of sums

Notes 5 : More on the a.s. convergence of sums Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series

More information

Summary. Recap ... Last Lecture. Summary. Theorem

Summary. Recap ... Last Lecture. Summary. Theorem Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

EE 4TM4: Digital Communications II Probability Theory

EE 4TM4: Digital Communications II Probability Theory 1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values

Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell

More information

STATISTICAL INFERENCE

STATISTICAL INFERENCE STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

Probability and Random Processes

Probability and Random Processes Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems

More information

Stat410 Probability and Statistics II (F16)

Stat410 Probability and Statistics II (F16) Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

This section is optional.

This section is optional. 4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

More information

Statistical Theory MT 2008 Problems 1: Solution sketches

Statistical Theory MT 2008 Problems 1: Solution sketches Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α

More information

Lecture 11 and 12: Basic estimation theory

Lecture 11 and 12: Basic estimation theory Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Lecture 12: September 27

Lecture 12: September 27 36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

[ ] ( ) ( ) [ ] ( ) 1 [ ] [ ] Sums of Random Variables Y = a 1 X 1 + a 2 X 2 + +a n X n The expected value of Y is:

[ ] ( ) ( ) [ ] ( ) 1 [ ] [ ] Sums of Random Variables Y = a 1 X 1 + a 2 X 2 + +a n X n The expected value of Y is: PROBABILITY FUNCTIONS A radom variable X has a probabilit associated with each of its possible values. The probabilit is termed a discrete probabilit if X ca assume ol discrete values, or X = x, x, x 3,,

More information

Singular Continuous Measures by Michael Pejic 5/14/10

Singular Continuous Measures by Michael Pejic 5/14/10 Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable

More information

An Introduction to Asymptotic Theory

An Introduction to Asymptotic Theory A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

More information

Statistical Theory MT 2009 Problems 1: Solution sketches

Statistical Theory MT 2009 Problems 1: Solution sketches Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where

More information

2. The volume of the solid of revolution generated by revolving the area bounded by the

2. The volume of the solid of revolution generated by revolving the area bounded by the IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M

More information

Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;

More information

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios

More information

Last Lecture. Unbiased Test

Last Lecture. Unbiased Test Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based

More information

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4 MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Introduction to Probability. Ariel Yadin

Introduction to Probability. Ariel Yadin Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Solution to Chapter 2 Analytical Exercises

Solution to Chapter 2 Analytical Exercises Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =

More information

Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett

Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets

More information

Mathematics 170B Selected HW Solutions.

Mathematics 170B Selected HW Solutions. Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli

More information

1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1

1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1 8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose

More information

Direction: This test is worth 150 points. You are required to complete this test within 55 minutes.

Direction: This test is worth 150 points. You are required to complete this test within 55 minutes. Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem

More information

Solutions to HW Assignment 1

Solutions to HW Assignment 1 Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.

More information

Stat 319 Theory of Statistics (2) Exercises

Stat 319 Theory of Statistics (2) Exercises Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.

More information

Lecture Notes 15 Hypothesis Testing (Chapter 10)

Lecture Notes 15 Hypothesis Testing (Chapter 10) 1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS

NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.

More information

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Final Examination Statistics 200C. T. Ferguson June 10, 2010

Final Examination Statistics 200C. T. Ferguson June 10, 2010 Fial Examiatio Statistics 00C T. Ferguso Jue 0, 00. (a State the Borel-Catelli Lemma ad its coverse. (b Let X,X,... be i.i.d. from a distributio with desity, f(x =θx (θ+ o the iterval (,. For what value

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

2.1. Convergence in distribution and characteristic functions.

2.1. Convergence in distribution and characteristic functions. 3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For

More information

Lecture 6 Simple alternatives and the Neyman-Pearson lemma

Lecture 6 Simple alternatives and the Neyman-Pearson lemma STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull

More information

Last Lecture. Wald Test

Last Lecture. Wald Test Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

TMA4245 Statistics. Corrected 30 May and 4 June Norwegian University of Science and Technology Department of Mathematical Sciences.

TMA4245 Statistics. Corrected 30 May and 4 June Norwegian University of Science and Technology Department of Mathematical Sciences. Norwegia Uiversity of Sciece ad Techology Departmet of Mathematical Scieces Corrected 3 May ad 4 Jue Solutios TMA445 Statistics Saturday 6 May 9: 3: Problem Sow desity a The probability is.9.5 6x x dx

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Probability for mathematicians INDEPENDENCE TAU

Probability for mathematicians INDEPENDENCE TAU Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet

More information

1.010 Uncertainty in Engineering Fall 2008

1.010 Uncertainty in Engineering Fall 2008 MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

More information

Summary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for

Summary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

More information

Point Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10.

Point Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10. Poit Estimatio: properties of estimators fiite-sample properties CB 7.3) large-sample properties CB 10.1) 1 FINITE-SAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:

More information

Lecture 3. Properties of Summary Statistics: Sampling Distribution

Lecture 3. Properties of Summary Statistics: Sampling Distribution Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

PAPER : IIT-JAM 2010

PAPER : IIT-JAM 2010 MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure

More information

of the matrix is =-85, so it is not positive definite. Thus, the first

of the matrix is =-85, so it is not positive definite. Thus, the first BOSTON COLLEGE Departmet of Ecoomics EC771: Ecoometrics Sprig 4 Prof. Baum, Ms. Uysal Solutio Key for Problem Set 1 1. Are the followig quadratic forms positive for all values of x? (a) y = x 1 8x 1 x

More information

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

Notes 27 : Brownian motion: path properties

Notes 27 : Brownian motion: path properties Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X

More information

LECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if

LECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio

More information

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo

More information