Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Size: px
Start display at page:

Download "Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n."

Transcription

1 Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator σ of σ, the stadard error is defied to be se = σ 2 / A cofidece iterval with approximate 95% coverage probability is [ θ ± 196 se Our strategy for estimatig σ 2 was based o the aalogue/plug-i priciple, ie, replace populatio momets/ukow quatities by their sample momets/estimates We eed kowledge of the expressio formula of σ 2 There are two computatio-itesive resamplig approaches that do the estimatio without requirig kowledge of the expressio of σ 2 Suppose we have some testig statistic W ad we eed to kow its distributio uder the ull hypothesis ad calculate its quatile The approach we took was to fid the asymptotic distributio of W, which was always stadard ormal or χ 2 The quatile of the asymptotic distributio ca be foud easily sice it does ot deped o ay ukow quatity/parameter We use it as approximatio to the true quatile of W Later we will see that there is aother approach to approximatig the true distributio of W Resamplig methods are ow core to moder ecoometrics behid the popularity of the resamplig methods There are least three motivatios Stadard errors are hard to get Suppose X 1,, X is a iid radom sample with mea µ ad variace σ 2 The the stadard error of the sample mea µ = 1 i=1 X i is se = σ 2 / where σ 2 = 1 i=1 X i µ 2 Suppose that X i is cotiuous with desity f X Assume for simplicity that its CDF F X is strictly icreasig The populatio media is m = F 1 X 1/2, ie, Pr X i m = 1/2 We order the data: X 1 X 2 X Defie the sample media: X 2 +X if is eve m = media {X 1,, X } = X +1 if is odd It is kow that m m d N 0, 4f X m 2 1 Costructig a plug-i estimator of the asymptotic variace 4f X m 2 1 requires kowledge of oparametric ecoometrics sice we eed to estimate the desity fuctio f X at a poit m There is also some subtle techical issue with this approach For this problem, resamplig methods come to rescue Almost othig else we ca do Suppose X 1,, X is a iid radom sample We wat to test H 0 : X is ormally distributed, ie, for some µ ad σ, X i N µ, σ 2 Remember that empirical 2 1

2 distributio fuctio F x = 1 i=1 1 X i x is cosistet for F X Ideed, we have a much stroger result: sup F x F X x p 0 Gliveko-Catelli theorem Let Φ µ,σ be the CDF x R of N µ, σ 2 The Kolmogorov Smirov test uses the statistic KS = sup F x Φ µ, σ x, x R where µ = 1 i=1 X i ad σ 2 = 1 i=1 X i µ 2 If H 0 is true, both F ad Φ µ, σ are cosistet for F X ad the statistic KS should be small So a large observed KS is regarded as evidece agaist H 0 We reject H 0 if KS > c We kow that KS d B, for some radom variable B with a very complicated distributio that depeds o ukow parameters So it is ot practically possible to choose c such that Pr B c = 1 α Agai for this problem, resamplig methods come to rescue For the traditioal cofidece iterval θ ±196 se, we kow that Pr θ [ θ ± 196 se 95% as Actually i may cases we ca show that Pr θ [ θ ± 196 se = 95% + O 1, ie, the error Pr θ [ θ ± 196 se 95% goes to zero at the rate 1 Some resamplig-based cofidece iterval [ θ + t 25% se, θ + t 975% se with some ew critical values t 25% ad t 975% has the property Pr θ [ θ + t 25% se, θ + t 975% se = 95% + O 3/2 So the error is smaller ad the coverage accuracy of the resamplig-based cofidece iterval is much better Jackkife Probably jackkife is the first-geeratio resamplig method Suppose X 1,, X is a iid radom sample For simplicity, assume X i is scalar A estimator θ ca be writte as θ = ϕ X 1,, X, eg, ϕ z 1,, z = 1 i=1 z i Suppose we kow θ θ d N 0, σ 2 ad we wat to estimate σ 2 Now deote θ j = ϕ 1 X 1,, X j 1, X j+1,, X, ie, θ j is a estimator obtaied by removig the j-th observatio from the etire sample { θ j } The variatio i : j = 1,, should be iformative about the populatio variace of θ Actually it is iformative about the populatio variace of θ 1 Note that θ a 1 N θ, σ 2 / 1 Deote θ = 1 j θ Now it seems reasoable to thik of 1 θ j 2 θ as a estimate of σ 2 / 1 ad 1 1 θ j 2 θ as a estimate of σ 2 Ideed i may cases oe ca show 1 The Jackkife stadard error is 1 se JK = A jackkife 95% cofidece iterval is θ j θ 2 p σ 2 1 θ j 2 θ [ θ ± 196 se JK If 1 is true, we say that jackkife is 2

3 cosistet Cosider the followig simple example: for iid radom sample X 1,, X, we use the sample average X as a estimator of µ = EX 1 It is kow that X µ d N 0, σ 2, where σ 2 = Var X 1 For this case, 1 θ j = θ j = 1 X X j, X X j = X, ad We have θ j θ = 1 X X j X = 1 X X j ˆθ j ˆθ 2 1 = 1 2 Xj X, which is the sample variace that is a cosistet ad ubiased estimator for σ 2 Note that ulike the plug-i approach, the jackkife approach does ot eve require kowledge of the expressio of σ 2 The limitatio of jackkife is that 1 is ot always true For the case of media, 1 fails ad jackkife is icosistet Bootstrap The secod-geeratio resamplig method is the bootstrap First, let us see how bootstrap gets the stadard error for estimatig the populatio media ad costructs the cofidece iterval for iid radom sample X 1,, X, let m = media {X 1,, X } First we idepedetly draw observatios with replacemet from X 1,, X ad get a set of ew observatios X 1 1,, X 1 The computer ca hadle this for us We repeat this resamplig procedure agai ad agai, B times B is a very large iteger Ideally how B is depeds solely o how powerful our computer is What we have is B bootstrap samples X 1 1 X 1 = m 1 { = media X 2 1 X 2 = m 2 = media X B 1 X B = m B = media } X 1 1,, X 1 { } X 2 1,, X 2 { X B 1,, X B ad for each bootstrap sample, we calculate its sample media We use the sample variace of m 1, m 2,, m B as a estimate of the true variace of m : Var BS m = 1 B { B b=1 The the bootstrap stadard error is se BS = m b 1 B B b=1 m b } 2 Var BS m ad a approximate 95% cofidece iterval usig the bootstrap stadard error is [ m ± 196 se BS I fact, there is aother seemigly } 3

4 simpler way to costruct the cofidece iterval We order the bootstrap sample medias: m 1 m 2 m B Suppose for simplicity B 25% ad B 975% are both itegers A bootstrap [ percetile cofidece iterval is simply m B 25%, m B 975% The bootstrap procedure we just described is called oparametric bootstrap or empirical bootstrap iveted by Professor Bradley Efro i 1979 The oparametric bootstrap takes the sample as the populatio A bootstrap sample is obtaied by idepedetly drawig observatios from the observed sample with replacemet The bootstrap sample has the same umber of observatios as the origial sample, however some observatios appear several times ad others ever Now we summarize the two procedures we itroduced Suppose we have a estimator which is asymptotically ormal: θ θ d N 0, σ 2 Bootstrap stadard errors Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ with each of the bootstrap samples, Step 3: Estimate the stadard error by where θ = B 1 B b b=1 θ se BS = 1 B B b=1 θ b θ 2 θ b for b = 1,, B Step 4: The bootstrap stadard errors ca be used to costruct approximate cofidece itervals, eg, if the coverage probability is 95%, a 95% cofidece iterval is [ θ ± 196 se BS Bootstrap percetile Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ with each of the bootstrap samples, Step 3: Order the bootstrap replicatios such that θ 1 θ B θ b for b = 1,, B Step 4: The lower ad upper cofidece bouds are B α /2-th ad B 1 α /2-th ordered elemets For B = 1000 ad α = 5%, these are the 25th ad 975th ordered elemets The estimated 1 α cofidece iterval is [ θ B α/2, θ B 1 α/2 What we did ot discuss is whether the bootstrap is correct We eed to show that for bootstrap stadard errors, ad for the bootstrap percetile cofidece iterval, se BS σ 2 / p 1 2 Pr θ [ θ B α/2, θ B 1 α/2 1 α 3 4

5 as This is a very difficult problem Below we provide some discussio about why bootstrap works Bootstrap percetile cofidece itervals ofte have more accurate coverage probabilities ie closer to the omial coverage probability 1 α tha the usual cofidece itervals based o stadard ormal quatiles ad estimated variace The bootstrap percetile method is simple but it should ot be abused Loosely, it works i the sese that 3 is true, oly if the estimator is asymptotically ormal Suppose we observe a radom sample X 1,, X from a uiform distributio o [0, θ, where θ > 0 is ukow θ = max {X 1,, X } is a cosistet estimator for θ ad θ θ coverges i distributio to the expoetial distributio For this case, 3 fails The bootstrap percetile method fails to give a asymptotically valid cofidece iterval How/Why Bootstrap Works? Suppose we have a iid radom sample X 1,, X with CDF F X Suppose S = ϕ X 1,, X is a statistic Its distributio should deped o F X : F S x = H x F X = Pr ϕ X 1,, X x We kow that the empirical CDF F X is a step fuctio that jumps at each of X 1,, X with size 1/ So F X is the CDF of a discrete radom variable Z with X 1,, X beig its possible realizatios ad 1/ beig the probability of ay of X 1,, X beig selected: Pr Z = X k = 1, for each k = 1, 2,, A radom observatio from X 1,, X is just a radom variable that has the same distributio as Z observatios radomly draw with replacemet from X 1,, X are just a radom sample from the distributio F X So each bootstrap sample is a iid radom sample from F X Note that the distributio here should be iterpreted as the coditioal distributio give X 1,, X Let X 1,, X be a iid radom sample from F X Let S = ϕ X 1,, X The coditioal CDF give X 1,, X of S is H x F X = Pr ϕ X1,, X x X 1,, X It seems reasoable to estimate H x F X by H x F X sice F X is a very good estimate of F X Similar the variace Var S should deped o F X as well ad it ca be estimated by Var S X 1,, X The true coditioal distributio of X 1,, X is kow We ca use computer simulatios kow as Mote Carlo simulatios to compute Var S X 1,, X The computer draws B very large iid radom samples from F X for us: X 1 1 X 1 iid F X X 2 1 X 2 iid F X X B 1 X B iid F X 5

6 These are just B idepedet bootstrap samples The, Var S X 1,, X 1 B B b=1 ϕ X b 2 1,, X b ϕ, 4 where ϕ = B 1 B b=1 ϕ X b 1,, X b is the bootstrap sample mea Sice B ca be arbitrarily large, by WLLN, the right had side of 4 should be very close to the left had side What we put forward is just the ituitio about how/why bootstrap works The theoretical proof ad also proof of the key results 2 ad 3 are very difficult Here is some further ituitio Let G x = Pr θ θ x be the distributio fuctio of θ θ If we kew G, we could easily costruct a cofidece iterval [ θ t 1 α/2, θ tα /2,where t α is the α-quatile of G: t α = G 1 α I reality, we do ot kow G ad we ca ofte show that G ca be approximated by the distributio fuctio of N 0, σ 2 The ormal approximatio with N 0, σ 2 requires that σ 2 ca be estimated cosistetly What bootstrap does is alterative approximatio It suggests that the coditioal distributio where θ is the bootstrap aalogue of θ Ĝ x = Pr θ θ x X 1,, X, θ is computed usig the bootstrap radom sample X 1,, X but the same formula as θ The bootstrap radom sample X 1,, X are iid with CDF F X We ca use the computer to geerate as may samples as we wat Ĝ is kow to us sice the distributio of the bootstrap sample is kow Ĝ ca be approximated by computer simulatios Ideed i may cases especially whe θ θ is asymptotically ormal, we have sup Ĝ x G x p 0 x R So the estimatio is cosistet But there are exceptios Bootstrap Refiemet If we have a plug-i estimator for σ ad the estimator σ is cosistet, we have T = θ θ σ d N 0, 1 Note that here σ ca be writte as a fuctio of the data ad we kow its fuctio form For each bootstrap sample b = 1,, B, we ca calculate σ usig the bootstrap sample For example, suppose X 1,, X is a iid radom sample with mea µ ad variace σ 2 Let µ = 1 i=1 X i ad σ 2 = 1 i=1 X i µ 2 We kow T = µ µ σ d N 0, 1 We ca compute σ as σ 2 Bootstrap-t = 1 i=1 X i µ 2, with µ = 1 i=1 X i 6

7 Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ ad σ with each of the bootstrap samples, ad the t-value for each bootstrap sample: θ b t b = θ σ b b θ, σ b Step 3: Order the bootstrap replicatios of t such that t 1 t B for b = 1,, B Step 4: The lower critical value t α/2 ad the upper critical value t 1 α/2 are the the B α /2- th ad B 1 α /2-th ordered elemets For B = 1000 ad α = 5%, these are the 25th ad 975th ordered elemets The bootstrap lower ad upper critical values geerally differ i absolute values The bootstrap-t cofidece iterval is [ θ + t 25% σ, θ + t 975% σ A strikig result is [ Pr θ θ + t 25% σ, θ + t 975% σ = 95% + O 3/2 compared with the cofidece iterval usig the stadard ormal critical values [ Pr θ θ 196 σ, θ σ = 95% + O 1 This is kow as asymptotic refiemet of bootstrap Residual Bootstrap ad Wild Bootstrap Cosider the cotext of liear regressio Our observed data is X 1, Y 1, X 2, Y 2,, X, Y ad we are iterested i the regressio coefficiets: Y i = α + βx i + e i I this case the oparametric/empirical bootstrap we itroduced works well, i the sese that the bootstrap stadard errors are cosistet ad the bootstrap percetile cofidece itervals have asymptotically correct coverage probabilities Empirical bootstrap treats the pair X, Y as oe object ad each bootstrap sample cosists of idepedet observatios draw with replacemet from the observatios X 1, Y 1, X 2, Y 2,, X, Y There are popular alteratives to the empirical bootstrap Bootstrap stadard errors, percetile cofidece itervals ad bootstrap-t are carried out by followig the same steps The oly thig that chages is how we resample to get the bootstrap samples Let ê i = Y i α βx i, where α, β is the LS estimator We draw fitted residuals idepedetly with replacemet from ê 1,, ê I other words, the bootstrap sample is a iid radom sample 7

8 ê 1,, ê, where for each i = 1,,, Pr ê i = ê k = 1, for each k = 1, 2,, Now for each i = 1, 2,,, let Xi = X i ad Yi = α + βx i + ê i Note that the idepedet variables are the same i all bootstrap samples This is kow as the residual bootstrap For wild bootstrap, let V 1,, V be computer-geerated idepedet radom variables with mea zero that are also idepedet of the data Now for each i = 1, 2,,, let ê i = V i ê i, X i = X i ad = α + βx i + ê i The most popular distributio for V s is the followig two-poit golde rule distributio: 5 1 /2 with probability / 2 5 V i = /2 with probability 5 1 / 2 5 Y i Its theoretical motivatio was provided by Professor Eo Mamme i 1993 Bootstrap Hypothesis Test We ow cosider testig H 0 : θ = θ 0 We ca use ay of the bootstrap-based cofidece itervals ad check if θ 0 is i the cofidece iterval We simply reject H 0 if θ 0 fails to be a elemet of the bootstrap percetile cofidece iterval Sice the t-statistic T = θ θ 0 σ d N 0, 1 uder H 0 We use the stadard ormal distributio as approximatio to the true distributio of T ad defie critical values based o stadard ormal quatile Alteratively, we ca do the followig bootstrap-t test Bootstrap-t test Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ ad σ with each of the bootstrap samples, ad the t-value for each bootstrap sample: θ b t b = θ σ b b θ, σ b Step 3: Order the bootstrap replicatios of t such that t 1 t B for b = 1,, B Step 4: The lower critical value t α/2 ad the upper critical value t 1 α/2 are the the B α /2- th ad B 1 α /2-th ordered elemets Reject H 0 if T < t α/2 or T > t 1 α/2 Cautio: a commo mistake is that i Step 2, oe mistakely computes θ b θ 0 σ b The test will have o power if we made this mistake The distributio of the t-statistic T = θ θ 0 σ 8

9 uder H 1 is differet from that uder H 0 Uder H 1, T is ot cetered: T = θ θ 0 σ = θ θ σ + θ θ0 σ A importat guidelie is that we should always approximate the distributio of T uder H 0, ie, the distributio of θ θ σ 9

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10 DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample. Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4 MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

More information

Stat 421-SP2012 Interval Estimation Section

Stat 421-SP2012 Interval Estimation Section Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible

More information

Simulation. Two Rule For Inverting A Distribution Function

Simulation. Two Rule For Inverting A Distribution Function Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump

More information

Parameter, Statistic and Random Samples

Parameter, Statistic and Random Samples Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,

More information

It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.

It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable. Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.

Direction: This test is worth 250 points. You are required to complete this test within 50 minutes. Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely

More information

Discrete Mathematics for CS Spring 2008 David Wagner Note 22

Discrete Mathematics for CS Spring 2008 David Wagner Note 22 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig

More information

Introductory statistics

Introductory statistics CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key

More information

Confidence Interval for Standard Deviation of Normal Distribution with Known Coefficients of Variation

Confidence Interval for Standard Deviation of Normal Distribution with Known Coefficients of Variation Cofidece Iterval for tadard Deviatio of Normal Distributio with Kow Coefficiets of Variatio uparat Niwitpog Departmet of Applied tatistics, Faculty of Applied ciece Kig Mogkut s Uiversity of Techology

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by

More information

1.010 Uncertainty in Engineering Fall 2008

1.010 Uncertainty in Engineering Fall 2008 MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

More information

( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2

( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2 82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

A quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population

A quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population A quick activity - Cetral Limit Theorem ad Proportios Lecture 21: Testig Proportios Statistics 10 Coli Rudel Flip a coi 30 times this is goig to get loud! Record the umber of heads you obtaied ad calculate

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Last Lecture. Wald Test

Last Lecture. Wald Test Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig

More information

Summary. Recap ... Last Lecture. Summary. Theorem

Summary. Recap ... Last Lecture. Summary. Theorem Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca

More information

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

More information

Output Analysis and Run-Length Control

Output Analysis and Run-Length Control IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

More information

Tests of Hypotheses Based on a Single Sample (Devore Chapter Eight)

Tests of Hypotheses Based on a Single Sample (Devore Chapter Eight) Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........

More information

Direction: This test is worth 150 points. You are required to complete this test within 55 minutes.

Direction: This test is worth 150 points. You are required to complete this test within 55 minutes. Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem

More information

1 Inferential Methods for Correlation and Regression Analysis

1 Inferential Methods for Correlation and Regression Analysis 1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

More information

Expectation and Variance of a random variable

Expectation and Variance of a random variable Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

Understanding Samples

Understanding Samples 1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

Module 1 Fundamentals in statistics

Module 1 Fundamentals in statistics Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly

More information

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determine sample size to estimate characteristic value of soil parameters A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig

More information

1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the

More information

5. Likelihood Ratio Tests

5. Likelihood Ratio Tests 1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,

More information

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

4.1 Non-parametric computational estimation

4.1 Non-parametric computational estimation Chapter 4 Resamplig Methods 4.1 No-parametric computatioal estimatio Let x 1,...,x be a realizatio of the i.i.d. r.vs X 1,...,X with a c.d.f. F. We are iterested i the precisio of estimatio of a populatio

More information

Chapter 6 Sampling Distributions

Chapter 6 Sampling Distributions Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

More information

POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES*

POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES* Kobe Uiversity Ecoomic Review 50(2004) 3 POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES* By HISASHI TANIZAKI There are various kids of oparametric

More information

Lecture 3. Properties of Summary Statistics: Sampling Distribution

Lecture 3. Properties of Summary Statistics: Sampling Distribution Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary

More information

Parameter, Statistic and Random Samples

Parameter, Statistic and Random Samples Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,

More information

Investigating the Significance of a Correlation Coefficient using Jackknife Estimates

Investigating the Significance of a Correlation Coefficient using Jackknife Estimates Iteratioal Joural of Scieces: Basic ad Applied Research (IJSBAR) ISSN 2307-4531 (Prit & Olie) http://gssrr.org/idex.php?joural=jouralofbasicadapplied ---------------------------------------------------------------------------------------------------------------------------

More information

Confidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M.

Confidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M. MATH1005 Statistics Lecture 24 M. Stewart School of Mathematics ad Statistics Uiversity of Sydey Outlie Cofidece itervals summary Coservative ad approximate cofidece itervals for a biomial p The aïve iterval

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Hypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance

Hypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Chi-Squared Tests Math 6070, Spring 2006

Chi-Squared Tests Math 6070, Spring 2006 Chi-Squared Tests Math 6070, Sprig 2006 Davar Khoshevisa Uiversity of Utah February XXX, 2006 Cotets MLE for Goodess-of Fit 2 2 The Multiomial Distributio 3 3 Applicatio to Goodess-of-Fit 6 3 Testig for

More information

Big Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates.

Big Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates. 5. Data, Estimates, ad Models: quatifyig the accuracy of estimates. 5. Estimatig a Normal Mea 5.2 The Distributio of the Normal Sample Mea 5.3 Normal data, cofidece iterval for, kow 5.4 Normal data, cofidece

More information

Recall the study where we estimated the difference between mean systolic blood pressure levels of users of oral contraceptives and non-users, x - y.

Recall the study where we estimated the difference between mean systolic blood pressure levels of users of oral contraceptives and non-users, x - y. Testig Statistical Hypotheses Recall the study where we estimated the differece betwee mea systolic blood pressure levels of users of oral cotraceptives ad o-users, x - y. Such studies are sometimes viewed

More information

Chapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.

Chapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc. Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo

More information

Bayesian Methods: Introduction to Multi-parameter Models

Bayesian Methods: Introduction to Multi-parameter Models Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested

More information

A new distribution-free quantile estimator

A new distribution-free quantile estimator Biometrika (1982), 69, 3, pp. 635-40 Prited i Great Britai 635 A ew distributio-free quatile estimator BY FRANK E. HARRELL Cliical Biostatistics, Duke Uiversity Medical Ceter, Durham, North Carolia, U.S.A.

More information

MATH/STAT 352: Lecture 15

MATH/STAT 352: Lecture 15 MATH/STAT 352: Lecture 15 Sectios 5.2 ad 5.3. Large sample CI for a proportio ad small sample CI for a mea. 1 5.2: Cofidece Iterval for a Proportio Estimatig proportio of successes i a biomial experimet

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Frequentist Inference

Frequentist Inference Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for

More information

KLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions

KLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give

More information

Statistical inference: example 1. Inferential Statistics

Statistical inference: example 1. Inferential Statistics Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

More information

Monte Carlo Integration

Monte Carlo Integration Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce

More information

Chapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.

Chapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc. Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more

More information

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber

More information

Binomial Distribution

Binomial Distribution 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible

More information

Statistics 511 Additional Materials

Statistics 511 Additional Materials Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability

More information

MA Advanced Econometrics: Properties of Least Squares Estimators

MA Advanced Econometrics: Properties of Least Squares Estimators MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample

More information

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen) Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................

More information

Elements of Statistical Methods Lots of Data or Large Samples (Ch 8)

Elements of Statistical Methods Lots of Data or Large Samples (Ch 8) Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x

More information

Element sampling: Part 2

Element sampling: Part 2 Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig

More information

Lecture Notes 15 Hypothesis Testing (Chapter 10)

Lecture Notes 15 Hypothesis Testing (Chapter 10) 1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we

More information

Posted-Price, Sealed-Bid Auctions

Posted-Price, Sealed-Bid Auctions Posted-Price, Sealed-Bid Auctios Professors Greewald ad Oyakawa 207-02-08 We itroduce the posted-price, sealed-bid auctio. This auctio format itroduces the idea of approximatios. We describe how well this

More information

TAMS24: Notations and Formulas

TAMS24: Notations and Formulas TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

More information

Department of Mathematics

Department of Mathematics Departmet of Mathematics Ma 3/103 KC Border Itroductio to Probability ad Statistics Witer 2017 Lecture 19: Estimatio II Relevat textbook passages: Larse Marx [1]: Sectios 5.2 5.7 19.1 The method of momets

More information

A goodness-of-fit test based on the empirical characteristic function and a comparison of tests for normality

A goodness-of-fit test based on the empirical characteristic function and a comparison of tests for normality A goodess-of-fit test based o the empirical characteristic fuctio ad a compariso of tests for ormality J. Marti va Zyl Departmet of Mathematical Statistics ad Actuarial Sciece, Uiversity of the Free State,

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

A NEW METHOD FOR CONSTRUCTING APPROXIMATE CONFIDENCE INTERVALS FOR M-ESTU1ATES. Dennis D. Boos

A NEW METHOD FOR CONSTRUCTING APPROXIMATE CONFIDENCE INTERVALS FOR M-ESTU1ATES. Dennis D. Boos .- A NEW METHOD FOR CONSTRUCTING APPROXIMATE CONFIDENCE INTERVALS FOR M-ESTU1ATES by Deis D. Boos Departmet of Statistics North Carolia State Uiversity Istitute of Statistics Mimeo Series #1198 September,

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Estimation of a population proportion March 23,

Estimation of a population proportion March 23, 1 Social Studies 201 Notes for March 23, 2005 Estimatio of a populatio proportio Sectio 8.5, p. 521. For the most part, we have dealt with meas ad stadard deviatios this semester. This sectio of the otes

More information

Stochastic Simulation

Stochastic Simulation Stochastic Simulatio 1 Itroductio Readig Assigmet: Read Chapter 1 of text. We shall itroduce may of the key issues to be discussed i this course via a couple of model problems. Model Problem 1 (Jackso

More information

ENGI 4421 Confidence Intervals (Two Samples) Page 12-01

ENGI 4421 Confidence Intervals (Two Samples) Page 12-01 ENGI 44 Cofidece Itervals (Two Samples) Page -0 Two Sample Cofidece Iterval for a Differece i Populatio Meas [Navidi sectios 5.4-5.7; Devore chapter 9] From the cetral limit theorem, we kow that, for sufficietly

More information

Stat 200 -Testing Summary Page 1

Stat 200 -Testing Summary Page 1 Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Stat 319 Theory of Statistics (2) Exercises

Stat 319 Theory of Statistics (2) Exercises Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.

More information

STA6938-Logistic Regression Model

STA6938-Logistic Regression Model Dr. Yig Zhag STA6938-Logistic Regressio Model Topic -Simple (Uivariate) Logistic Regressio Model Outlies:. Itroductio. A Example-Does the liear regressio model always work? 3. Maximum Likelihood Curve

More information

BIOSTATISTICS. Lecture 5 Interval Estimations for Mean and Proportion. dr. Petr Nazarov

BIOSTATISTICS. Lecture 5 Interval Estimations for Mean and Proportion. dr. Petr Nazarov Microarray Ceter BIOSTATISTICS Lecture 5 Iterval Estimatios for Mea ad Proportio dr. Petr Nazarov 15-03-013 petr.azarov@crp-sate.lu Lecture 5. Iterval estimatio for mea ad proportio OUTLINE Iterval estimatios

More information

STA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:

STA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to: STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio

More information

CSE 527, Additional notes on MLE & EM

CSE 527, Additional notes on MLE & EM CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be

More information

STATISTICAL INFERENCE

STATISTICAL INFERENCE STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample

More information

Introduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 3

Introduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 3 Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 3 (This versio August 17, 014) 015 Pearso Educatio, Ic. Stock/Watso

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

Questions and Answers on Maximum Likelihood

Questions and Answers on Maximum Likelihood Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,

More information