Summary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
|
|
- Sophie Bennett
- 5 years ago
- Views:
Transcription
1 Last Lecture Biostatistics Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square error loss? What is the Bayes rule Estimator miimizig absolte error loss? What are the tools for provig a poit estimator is cosistet? Ca a biase estimator be cosistet? Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Bayes Estimator base o absolute error loss Asymptotic Evaluatio of oit Estimators Suppose that Lθ, ˆθ = θ ˆθ The posterior expecte loss is E[Lθ, ˆθx] = θ ˆθx πθ xθ Ω = E[ θ ˆθ X = x] = ˆθ ˆθ E[Lθ, ˆθx] = Therefore, ˆθ is posterior meia θ ˆθπθ xθ + ˆθ πθ xθ ˆθ ˆθ θ ˆθπθ xθ πθ xθ = 0 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Whe the sample size approaches ifiity, the behaviors of a estimator are ukow as its asymptotic properties Defiitio - Cosistecy Let W = W X 1,, X = W X be a sequece of estimators for τθ We say W is cosistet for estimatig τθ if W τθ uer θ for every θ Ω W τθ coverges i probability to τθ meas that, give ay ϵ > 0 lim τθ ϵ = 0 lim τθ < ϵ = 1 Whe W τθ < ϵ ca also be represete that W is close to τθ Cosistecy implies that the probability of W close to τθ approaches to 1 as goes to Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
2 Tools for provig cosistecy Theorem for cosistecy Use efiitio complicate Chebychev s Iequality r W τθ ϵ = rw τθ 2 ϵ 2 E[W τθ] 2 ϵ 2 = MSEW ϵ 2 = Bias2 W + VarW ϵ 2 Nee to show that both BiasW a VarW coverges to zero Theorem 1013 If W is a sequece of estimators of τθ satisfyig lim > BiasW = 0 lim > VarW = 0 for all θ, the W is cosistet for τθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Weak Law of Large Numbers Cosistet sequece of estimators Theorem 552 Let X 1,, X be ii raom variables with EX = µ a VarX = σ 2 < The X coverges i probability to µ ie X µ Theorem 1015 Let W is a cosistet sequece of estimators of τθ Let a, b be sequeces of costats satisfyig 1 lim a = 1 2 lim b = 0 The U = a W + b is also a cosistet sequece of estimators of τθ Cotiuous Map Theorem If W is cosistet for θ a g is a cotiuous fuctio, the gw is cosistet for gθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
3 Example - Expoetial Family Cosistet estimator of rx c roblem ii Suppose X 1,, X Expoetialβ 1 ropose a cosistet estimator of the meia 2 ropose a cosistet estimator of rx c where c is costat rx c = c 0 1 β e x/β x = 1 e c/β As X is cosistet for β, 1 e c/β is cotiuous fuctio of β By cotiuous mappig Theorem, gx = 1 e c/x is cosistet for rx c = 1 e c/β = gβ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Cosistet estimator of rx c - Alterative Metho Cosistecy of MLEs Defie Y i = IX i c The Y i ii Beroullip where p = rx c Y = 1 Y i = 1 IX i c is cosistet for p by Law of Large Numbers Theorem Cosistecy of MLEs ii Suppose X i fx θ Let ˆθ be the MLE of θ, a τθ be a cotiuous fuctio of θ The uer regularity coitios o fx θ, the MLE of τθ ie τˆθ is cosistet for τθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
4 Cetral Limit Theorem Defiitio: A statistic or a estimator W X is asymptotically ormal if W τθ N 0, νθ for all θ where stas for coverge i istributio τθ : asymptotic mea νθ : asymptotic variace We eote W AN τθ, νθ Cetral Limit Theorem ii Assume X i fx θ with fiite mea µθ a variace σ 2 θ X AN µθ, σ2 θ X µθ Theorem Slutsky s Theorem If X X, Y 1 Y X ax 2 X + Y X + a N 0, σ 2 θ a, where a is a costat, Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Example - Estimator of rx c Example Defie Y i = IX i c The Y i ii Beroullip where p = rx c Y = 1 is cosistet for p Therefore, 1 Y i = 1 IX i c AN = = AN IX i c EY, VarY p, p1 p Let X 1,, X be ii samples with fiite mea µ a variace σ 2 Defie By Cetral Limit Theorem, S 2 = 1 1 X µ X µ σ X i X 2 X AN N 0, σ 2 N 0, 1 µ, σ2 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
5 Example cot Delta Metho X µ S = σ S X µ We showe previously S 2 σ 2 S σ σ/s 1 Therefore, By Slutsky s Theorem X µ N 0, 1 S σ Theorem Delta Metho Assume W AN θ, νθ If a fuctio g satisfies g θ 0, the gw AN gθ, [g θ] 2 νθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Delta Metho - Example ii X 1,, X Beroullip where p 1 2, we wat to kow the asymptotic istributio of X1 X By cetral limit Theorem, X p N 0, 1 p1 p X AN p, Defie gy = y1 y, the X1 X = gx g y = y y 2 = 1 2y p1 p By Delta Metho, gx = X1 X AN gp, [g 2 p1 p p] 2 p1 p = AN p1 p, 1 2p Give a statistic W X, for example X, s 2 X, e X W τθ N 0, νθ for all θ W AN τθ, νθ Tools to show asymptotic ormality 1 Cetral Limit Theorem 2 Slutsky Theorem 3 Delta Metho Theorem 5524 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
6 Usig Cetral Limit Theorem Usig Slutsky Theorem X AN µθ, σ2 θ where µθ = EX, a σ 2 θ = VarX For example, i orer to get the asymptotic istributio of 1 X2 i, efie Y i = X 2 i, the 1 X 2 i = 1 AN AN Y i = Y EY, VarY EX 2, VarX2 Whe X X, Y a, the 1 Y X ax 2 X + Y X + a Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Usig Delta Metho Theorem 5524 Example Assume W AN θ, νθ gw AN If a fuctio g satisfies g θ 0, the gθ, [g θ] 2 νθ roblem X 1,, X ii N µ, σ 2 µ 0 Fi the asymptotic istributio of MLE of µ 2 Solutio 1 It ca be easily show that MLE of µ is X 2 By the ivariace property of MLE, MLE of µ 2 is X 2 3 By cetral limit theorem, we kow that X AN µ, σ2 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
7 Solutio cot Asymptotic Relative Efficiecy ARE 4 Defie gy = y 2, a apply Delta Metho g y = 2y X 2 AN gµ, [g µ] 2 σ2 AN µ 2, 2µ 2 σ2 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 If both estimators are cosistet a asymptotic ormal, we ca compare their asymptotic variace Defiitio : Asymptotic Relatve Efficiecy If two estimators W a V satisfy [W τθ] [V τθ] N 0, σ 2 W N 0, σ 2 V The asymptotic relative efficiecy ARE of V with respect to W is AREV, W = σ2 W σ 2 V If AREV, W 1 for every θ Ω, the V is asymptotically more efficiet tha W Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Example Solutio - Asymptotic Distributio of V roblem ii Let X i oissoλ cosier estimatig rx = 0 = e λ Our estimators are W = 1 V = e X IX i = 0 V X = e X, by CLT, X AN EX, VarX/ AN λ, λ/ Defie gy = e y, the V = gx a g y = e y By Delta Metho V = e X AN gλ, [g λ] 2 λ AN e λ, e 2λ λ Determie which oe is more asymptotically efficiet estimator Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
8 Solutio - Asymptotic Distributio of W Solutio - Calculatig ARE Defie Z i = IX i = 0 By CLT, W = 1 IX i = 0 = Z Z i BeroulliEZ EZ = rx = 0 = e λ VarZ = e λ 1 e λ W = Z AN EZ, VarZ/ AN e λ, e λ 1 e λ AREW, V = = = = e 2λ λ/ e λ 1 e λ / λ e λ 1 e λ λ e λ 1 λ 1 + λ + λ2 2 + λ3 3! λ 0 Therefore W = 1 IXi = 0 is less efficiet tha V MLE, a ARE attais maximum at λ = 0 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Defiitio : Asympotic Efficiecy for ii samples A sequece of estimators W is asymptotically efficiet for τθ if for all θ Ω, W τθ N 0, [τ θ] 2 Iθ W AN τθ, [τ θ] 2 Iθ [ { } 2 Iθ = E log fx θ θ] θ [ ] 2 = E log fx θ θ if iterchageability hols θ2 Note: [τ θ] 2 Iθ is the C-R bou for ubiase estimators of τθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33 of MLEs Theorem Let X 1,, X be ii samples from fx θ Let ˆθ eote the MLE of θ Uer same regularity coitios, ˆθ is cosistet a asymptotically ormal for θ, ie ˆθ θ 1 N 0, for every θ Ω Iθ A if τθ is cotiuous a ifferetiable i θ, the ˆθ θ N 0, [τ θ] Iθ = τˆθ AN τθ, [τ θ] 2 Iθ Agai, ote that the asymptotic variace of τˆθ is Cramer-Rao lower bou for ubiase estimators of τθ Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
9 Toay Cetral Limit Theorem Slutsky Theorem Delta Metho Asymptotic Relative Efficiecy Next Lecture Hypothesis Testig Hyu Mi Kag Biostatistics Lecture 16 March 19th, / 33
Last Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator
Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationChapter 2 Transformations and Expectations
Chapter Trasformatios a Epectatios Chapter Distributios of Fuctios of a Raom Variable Problem: Let be a raom variable with cf F ( ) If we efie ay fuctio of, say g( ) g( ) is also a raom variable whose
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationLast Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationPoint Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10.
Poit Estimatio: properties of estimators fiite-sample properties CB 7.3) large-sample properties CB 10.1) 1 FINITE-SAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More informationLecture 6 Testing Nonlinear Restrictions 1. The previous lectures prepare us for the tests of nonlinear restrictions of the form:
Eco 75 Lecture 6 Testig Noliear Restrictios The previous lectures prepare us for the tests of oliear restrictios of the form: H 0 : h( 0 ) = 0 versus H : h( 0 ) 6= 0: () I this lecture, we cosier Wal,
More information11 THE GMM ESTIMATION
Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................
More informationSOLUTION FOR HOMEWORK 7, STAT np(1 p) (α + β + n) + ( np + α
SOLUTION FOR HOMEWORK 7, STAT 6331 1 Exerc733 Here we just recall that MSE(ˆp B ) = p(1 p) (α + β + ) + ( p + α 2 α + β + p) 2 The you plug i α = β = (/4) 1/2 After simplificatios MSE(ˆp B ) = 4( 1/2 +
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More information3. Calculus with distributions
6 RODICA D. COSTIN 3.1. Limits of istributios. 3. Calculus with istributios Defiitio 4. A sequece of istributios {u } coverges to the istributio u (all efie o the same space of test fuctios) if (φ, u )
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationSubsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf. (ˆθn θ (P ) under P. For real ˆθ n, (τ n
Subsamlig vs Bootstra Dimitris N. Politis, Joseh P. Romao, Michael Wolf R x, θ P = τ ˆθ θ P Examle: ˆθ = X, τ =, θ = EX = µ P orˆθ = mi X, τ =, θ P = su{x : F x 0} Defie: J P, the istributio of τ ˆθ θ
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationChapter 13: Tests of Hypothesis Section 13.1 Introduction
Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed
More informationCEU Department of Economics Econometrics 1, Problem Set 1 - Solutions
CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationNYU Center for Data Science: DS-GA 1003 Machine Learning and Computational Statistics (Spring 2018)
NYU Ceter for Data Sciece: DS-GA 003 Machie Learig ad Computatioal Statistics (Sprig 208) Brett Berstei, David Roseberg, Be Jakubowski Jauary 20, 208 Istructios: Followig most lab ad lecture sectios, we
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationx iu i E(x u) 0. In order to obtain a consistent estimator of β, we find the instrumental variable z which satisfies E(z u) = 0. z iu i E(z u) = 0.
27 However, β MM is icosistet whe E(x u) 0, i.e., β MM = (X X) X y = β + (X X) X u = β + ( X X ) ( X u ) \ β. Note as follows: X u = x iu i E(x u) 0. I order to obtai a cosistet estimator of β, we fid
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMathmatical Statisticals
Mathmatical Statisticals Che, L.-A. Chapter 4. Distributio of Fuctio of Radom variables Sample space S : set of possible outcome i a experimet. Probability set fuctio P: ()P (A) 0, A S. ()P (S) =. (3)P
More informationANSWERS TO MIDTERM EXAM # 2
MATH 03, FALL 003 ANSWERS TO MIDTERM EXAM # PENN STATE UNIVERSITY Problem 1 (18 pts). State ad prove the Itermediate Value Theorem. Solutio See class otes or Theorem 5.6.1 from our textbook. Problem (18
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationLecture 9: September 19
36-700: Probability ad Mathematical Statistics I Fall 206 Lecturer: Siva Balakrisha Lecture 9: September 9 9. Review ad Outlie Last class we discussed: Statistical estimatio broadly Pot estimatio Bias-Variace
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationLecture 27: Optimal Estimators and Functional Delta Method
Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationMBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS
MBACATÓLICA Quatitative Methods Miguel Gouveia Mauel Leite Moteiro Faculdade de Ciêcias Ecoómicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACatólica 006/07 Métodos Quatitativos
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationRecap! Good statistics, cont.! Sufficiency! What are good statistics?! 2/20/14
Recap Cramér-Rao iequality Best ubiased estimators What are good statistics? Parameter: ukow umber that we are tryig to get a idea about usig a sample X 1,,X Statistic: A fuctio of the sample. It is a
More informationLecture 13: Maximum Likelihood Estimation
ECE90 Sprig 007 Statistical Learig Theory Istructor: R. Nowak Lecture 3: Maximum Likelihood Estimatio Summary of Lecture I the last lecture we derived a risk (MSE) boud for regressio problems; i.e., select
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More information6.3.3 Parameter Estimation
130 CHAPTER 6. ARMA MODELS 6.3.3 Parameter Estimatio I this sectio we will iscuss methos of parameter estimatio for ARMAp,q assumig that the orers p a q are kow. Metho of Momets I this metho we equate
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More informationFinal Examination Statistics 200C. T. Ferguson June 10, 2010
Fial Examiatio Statistics 00C T. Ferguso Jue 0, 00. (a State the Borel-Catelli Lemma ad its coverse. (b Let X,X,... be i.i.d. from a distributio with desity, f(x =θx (θ+ o the iterval (,. For what value
More informationSequences. A Sequence is a list of numbers written in order.
Sequeces A Sequece is a list of umbers writte i order. {a, a 2, a 3,... } The sequece may be ifiite. The th term of the sequece is the th umber o the list. O the list above a = st term, a 2 = 2 d term,
More informationMaximum likelihood estimation from record-breaking data for the generalized Pareto distribution
METRON - Iteratioal Joural of Statistics 004, vol. LXII,. 3, pp. 377-389 NAGI S. ABD-EL-HAKIM KHALAF S. SULTAN Maximum likelihood estimatio from record-breakig data for the geeralized Pareto distributio
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationHomework for 2/3. 1. Determine the values of the following quantities: a. t 0.1,15 b. t 0.05,15 c. t 0.1,25 d. t 0.05,40 e. t 0.
Name: ID: Homework for /3. Determie the values of the followig quatities: a. t 0.5 b. t 0.055 c. t 0.5 d. t 0.0540 e. t 0.00540 f. χ 0.0 g. χ 0.0 h. χ 0.00 i. χ 0.0050 j. χ 0.990 a. t 0.5.34 b. t 0.055.753
More information