Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
|
|
- Sabina Nichols
- 5 years ago
- Views:
Transcription
1 Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator σ of σ, the stadard error is defied to be se = σ 2 / A cofidece iterval with approximate 95% coverage probability is [ θ ± 196 se Our strategy for estimatig σ 2 was based o the aalogue/plug-i priciple, ie, replace populatio momets/ukow quatities by their sample momets/estimates We eed kowledge of the expressio formula of σ 2 There are two computatio-itesive resamplig approaches that do the estimatio without requirig kowledge of the expressio of σ 2 Suppose we have some testig statistic W ad we eed to kow its distributio uder the ull hypothesis ad calculate its quatile The approach we took was to fid the asymptotic distributio of W, which was always stadard ormal or χ 2 The quatile of the asymptotic distributio ca be foud easily sice it does ot deped o ay ukow quatity/parameter We use it as approximatio to the true quatile of W Later we will see that there is aother approach to approximatig the true distributio of W Resamplig methods are ow core to moder ecoometrics behid the popularity of the resamplig methods There are least three motivatios Stadard errors are hard to get Suppose X 1,, X is a iid radom sample with mea µ ad variace σ 2 The the stadard error of the sample mea µ = 1 i=1 X i is se = σ 2 / where σ 2 = 1 i=1 X i µ 2 Suppose that X i is cotiuous with desity f X Assume for simplicity that its CDF F X is strictly icreasig The populatio media is m = F 1 X 1/2, ie, Pr X i m = 1/2 We order the data: X 1 X 2 X Defie the sample media: X 2 +X if is eve m = media {X 1,, X } = X +1 if is odd It is kow that m m d N 0, 4f X m 2 1 Costructig a plug-i estimator of the asymptotic variace 4f X m 2 1 requires kowledge of oparametric ecoometrics sice we eed to estimate the desity fuctio f X at a poit m There is also some subtle techical issue with this approach For this problem, resamplig methods come to rescue Almost othig else we ca do Suppose X 1,, X is a iid radom sample We wat to test H 0 : X is ormally distributed, ie, for some µ ad σ, X i N µ, σ 2 Remember that empirical 2 1
2 distributio fuctio F x = 1 i=1 1 X i x is cosistet for F X Ideed, we have a much stroger result: sup F x F X x p 0 Gliveko-Catelli theorem Let Φ µ,σ be the CDF x R of N µ, σ 2 The Kolmogorov Smirov test uses the statistic KS = sup F x Φ µ, σ x, x R where µ = 1 i=1 X i ad σ 2 = 1 i=1 X i µ 2 If H 0 is true, both F ad Φ µ, σ are cosistet for F X ad the statistic KS should be small So a large observed KS is regarded as evidece agaist H 0 We reject H 0 if KS > c We kow that KS d B, for some radom variable B with a very complicated distributio that depeds o ukow parameters So it is ot practically possible to choose c such that Pr B c = 1 α Agai for this problem, resamplig methods come to rescue For the traditioal cofidece iterval θ ±196 se, we kow that Pr θ [ θ ± 196 se 95% as Actually i may cases we ca show that Pr θ [ θ ± 196 se = 95% + O 1, ie, the error Pr θ [ θ ± 196 se 95% goes to zero at the rate 1 Some resamplig-based cofidece iterval [ θ + t 25% se, θ + t 975% se with some ew critical values t 25% ad t 975% has the property Pr θ [ θ + t 25% se, θ + t 975% se = 95% + O 3/2 So the error is smaller ad the coverage accuracy of the resamplig-based cofidece iterval is much better Jackkife Probably jackkife is the first-geeratio resamplig method Suppose X 1,, X is a iid radom sample For simplicity, assume X i is scalar A estimator θ ca be writte as θ = ϕ X 1,, X, eg, ϕ z 1,, z = 1 i=1 z i Suppose we kow θ θ d N 0, σ 2 ad we wat to estimate σ 2 Now deote θ j = ϕ 1 X 1,, X j 1, X j+1,, X, ie, θ j is a estimator obtaied by removig the j-th observatio from the etire sample { θ j } The variatio i : j = 1,, should be iformative about the populatio variace of θ Actually it is iformative about the populatio variace of θ 1 Note that θ a 1 N θ, σ 2 / 1 Deote θ = 1 j θ Now it seems reasoable to thik of 1 θ j 2 θ as a estimate of σ 2 / 1 ad 1 1 θ j 2 θ as a estimate of σ 2 Ideed i may cases oe ca show 1 The Jackkife stadard error is 1 se JK = A jackkife 95% cofidece iterval is θ j θ 2 p σ 2 1 θ j 2 θ [ θ ± 196 se JK If 1 is true, we say that jackkife is 2
3 cosistet Cosider the followig simple example: for iid radom sample X 1,, X, we use the sample average X as a estimator of µ = EX 1 It is kow that X µ d N 0, σ 2, where σ 2 = Var X 1 For this case, 1 θ j = θ j = 1 X X j, X X j = X, ad We have θ j θ = 1 X X j X = 1 X X j ˆθ j ˆθ 2 1 = 1 2 Xj X, which is the sample variace that is a cosistet ad ubiased estimator for σ 2 Note that ulike the plug-i approach, the jackkife approach does ot eve require kowledge of the expressio of σ 2 The limitatio of jackkife is that 1 is ot always true For the case of media, 1 fails ad jackkife is icosistet Bootstrap The secod-geeratio resamplig method is the bootstrap First, let us see how bootstrap gets the stadard error for estimatig the populatio media ad costructs the cofidece iterval for iid radom sample X 1,, X, let m = media {X 1,, X } First we idepedetly draw observatios with replacemet from X 1,, X ad get a set of ew observatios X 1 1,, X 1 The computer ca hadle this for us We repeat this resamplig procedure agai ad agai, B times B is a very large iteger Ideally how B is depeds solely o how powerful our computer is What we have is B bootstrap samples X 1 1 X 1 = m 1 { = media X 2 1 X 2 = m 2 = media X B 1 X B = m B = media } X 1 1,, X 1 { } X 2 1,, X 2 { X B 1,, X B ad for each bootstrap sample, we calculate its sample media We use the sample variace of m 1, m 2,, m B as a estimate of the true variace of m : Var BS m = 1 B { B b=1 The the bootstrap stadard error is se BS = m b 1 B B b=1 m b } 2 Var BS m ad a approximate 95% cofidece iterval usig the bootstrap stadard error is [ m ± 196 se BS I fact, there is aother seemigly } 3
4 simpler way to costruct the cofidece iterval We order the bootstrap sample medias: m 1 m 2 m B Suppose for simplicity B 25% ad B 975% are both itegers A bootstrap [ percetile cofidece iterval is simply m B 25%, m B 975% The bootstrap procedure we just described is called oparametric bootstrap or empirical bootstrap iveted by Professor Bradley Efro i 1979 The oparametric bootstrap takes the sample as the populatio A bootstrap sample is obtaied by idepedetly drawig observatios from the observed sample with replacemet The bootstrap sample has the same umber of observatios as the origial sample, however some observatios appear several times ad others ever Now we summarize the two procedures we itroduced Suppose we have a estimator which is asymptotically ormal: θ θ d N 0, σ 2 Bootstrap stadard errors Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ with each of the bootstrap samples, Step 3: Estimate the stadard error by where θ = B 1 B b b=1 θ se BS = 1 B B b=1 θ b θ 2 θ b for b = 1,, B Step 4: The bootstrap stadard errors ca be used to costruct approximate cofidece itervals, eg, if the coverage probability is 95%, a 95% cofidece iterval is [ θ ± 196 se BS Bootstrap percetile Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ with each of the bootstrap samples, Step 3: Order the bootstrap replicatios such that θ 1 θ B θ b for b = 1,, B Step 4: The lower ad upper cofidece bouds are B α /2-th ad B 1 α /2-th ordered elemets For B = 1000 ad α = 5%, these are the 25th ad 975th ordered elemets The estimated 1 α cofidece iterval is [ θ B α/2, θ B 1 α/2 What we did ot discuss is whether the bootstrap is correct We eed to show that for bootstrap stadard errors, ad for the bootstrap percetile cofidece iterval, se BS σ 2 / p 1 2 Pr θ [ θ B α/2, θ B 1 α/2 1 α 3 4
5 as This is a very difficult problem Below we provide some discussio about why bootstrap works Bootstrap percetile cofidece itervals ofte have more accurate coverage probabilities ie closer to the omial coverage probability 1 α tha the usual cofidece itervals based o stadard ormal quatiles ad estimated variace The bootstrap percetile method is simple but it should ot be abused Loosely, it works i the sese that 3 is true, oly if the estimator is asymptotically ormal Suppose we observe a radom sample X 1,, X from a uiform distributio o [0, θ, where θ > 0 is ukow θ = max {X 1,, X } is a cosistet estimator for θ ad θ θ coverges i distributio to the expoetial distributio For this case, 3 fails The bootstrap percetile method fails to give a asymptotically valid cofidece iterval How/Why Bootstrap Works? Suppose we have a iid radom sample X 1,, X with CDF F X Suppose S = ϕ X 1,, X is a statistic Its distributio should deped o F X : F S x = H x F X = Pr ϕ X 1,, X x We kow that the empirical CDF F X is a step fuctio that jumps at each of X 1,, X with size 1/ So F X is the CDF of a discrete radom variable Z with X 1,, X beig its possible realizatios ad 1/ beig the probability of ay of X 1,, X beig selected: Pr Z = X k = 1, for each k = 1, 2,, A radom observatio from X 1,, X is just a radom variable that has the same distributio as Z observatios radomly draw with replacemet from X 1,, X are just a radom sample from the distributio F X So each bootstrap sample is a iid radom sample from F X Note that the distributio here should be iterpreted as the coditioal distributio give X 1,, X Let X 1,, X be a iid radom sample from F X Let S = ϕ X 1,, X The coditioal CDF give X 1,, X of S is H x F X = Pr ϕ X1,, X x X 1,, X It seems reasoable to estimate H x F X by H x F X sice F X is a very good estimate of F X Similar the variace Var S should deped o F X as well ad it ca be estimated by Var S X 1,, X The true coditioal distributio of X 1,, X is kow We ca use computer simulatios kow as Mote Carlo simulatios to compute Var S X 1,, X The computer draws B very large iid radom samples from F X for us: X 1 1 X 1 iid F X X 2 1 X 2 iid F X X B 1 X B iid F X 5
6 These are just B idepedet bootstrap samples The, Var S X 1,, X 1 B B b=1 ϕ X b 2 1,, X b ϕ, 4 where ϕ = B 1 B b=1 ϕ X b 1,, X b is the bootstrap sample mea Sice B ca be arbitrarily large, by WLLN, the right had side of 4 should be very close to the left had side What we put forward is just the ituitio about how/why bootstrap works The theoretical proof ad also proof of the key results 2 ad 3 are very difficult Here is some further ituitio Let G x = Pr θ θ x be the distributio fuctio of θ θ If we kew G, we could easily costruct a cofidece iterval [ θ t 1 α/2, θ tα /2,where t α is the α-quatile of G: t α = G 1 α I reality, we do ot kow G ad we ca ofte show that G ca be approximated by the distributio fuctio of N 0, σ 2 The ormal approximatio with N 0, σ 2 requires that σ 2 ca be estimated cosistetly What bootstrap does is alterative approximatio It suggests that the coditioal distributio where θ is the bootstrap aalogue of θ Ĝ x = Pr θ θ x X 1,, X, θ is computed usig the bootstrap radom sample X 1,, X but the same formula as θ The bootstrap radom sample X 1,, X are iid with CDF F X We ca use the computer to geerate as may samples as we wat Ĝ is kow to us sice the distributio of the bootstrap sample is kow Ĝ ca be approximated by computer simulatios Ideed i may cases especially whe θ θ is asymptotically ormal, we have sup Ĝ x G x p 0 x R So the estimatio is cosistet But there are exceptios Bootstrap Refiemet If we have a plug-i estimator for σ ad the estimator σ is cosistet, we have T = θ θ σ d N 0, 1 Note that here σ ca be writte as a fuctio of the data ad we kow its fuctio form For each bootstrap sample b = 1,, B, we ca calculate σ usig the bootstrap sample For example, suppose X 1,, X is a iid radom sample with mea µ ad variace σ 2 Let µ = 1 i=1 X i ad σ 2 = 1 i=1 X i µ 2 We kow T = µ µ σ d N 0, 1 We ca compute σ as σ 2 Bootstrap-t = 1 i=1 X i µ 2, with µ = 1 i=1 X i 6
7 Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ ad σ with each of the bootstrap samples, ad the t-value for each bootstrap sample: θ b t b = θ σ b b θ, σ b Step 3: Order the bootstrap replicatios of t such that t 1 t B for b = 1,, B Step 4: The lower critical value t α/2 ad the upper critical value t 1 α/2 are the the B α /2- th ad B 1 α /2-th ordered elemets For B = 1000 ad α = 5%, these are the 25th ad 975th ordered elemets The bootstrap lower ad upper critical values geerally differ i absolute values The bootstrap-t cofidece iterval is [ θ + t 25% σ, θ + t 975% σ A strikig result is [ Pr θ θ + t 25% σ, θ + t 975% σ = 95% + O 3/2 compared with the cofidece iterval usig the stadard ormal critical values [ Pr θ θ 196 σ, θ σ = 95% + O 1 This is kow as asymptotic refiemet of bootstrap Residual Bootstrap ad Wild Bootstrap Cosider the cotext of liear regressio Our observed data is X 1, Y 1, X 2, Y 2,, X, Y ad we are iterested i the regressio coefficiets: Y i = α + βx i + e i I this case the oparametric/empirical bootstrap we itroduced works well, i the sese that the bootstrap stadard errors are cosistet ad the bootstrap percetile cofidece itervals have asymptotically correct coverage probabilities Empirical bootstrap treats the pair X, Y as oe object ad each bootstrap sample cosists of idepedet observatios draw with replacemet from the observatios X 1, Y 1, X 2, Y 2,, X, Y There are popular alteratives to the empirical bootstrap Bootstrap stadard errors, percetile cofidece itervals ad bootstrap-t are carried out by followig the same steps The oly thig that chages is how we resample to get the bootstrap samples Let ê i = Y i α βx i, where α, β is the LS estimator We draw fitted residuals idepedetly with replacemet from ê 1,, ê I other words, the bootstrap sample is a iid radom sample 7
8 ê 1,, ê, where for each i = 1,,, Pr ê i = ê k = 1, for each k = 1, 2,, Now for each i = 1, 2,,, let Xi = X i ad Yi = α + βx i + ê i Note that the idepedet variables are the same i all bootstrap samples This is kow as the residual bootstrap For wild bootstrap, let V 1,, V be computer-geerated idepedet radom variables with mea zero that are also idepedet of the data Now for each i = 1, 2,,, let ê i = V i ê i, X i = X i ad = α + βx i + ê i The most popular distributio for V s is the followig two-poit golde rule distributio: 5 1 /2 with probability / 2 5 V i = /2 with probability 5 1 / 2 5 Y i Its theoretical motivatio was provided by Professor Eo Mamme i 1993 Bootstrap Hypothesis Test We ow cosider testig H 0 : θ = θ 0 We ca use ay of the bootstrap-based cofidece itervals ad check if θ 0 is i the cofidece iterval We simply reject H 0 if θ 0 fails to be a elemet of the bootstrap percetile cofidece iterval Sice the t-statistic T = θ θ 0 σ d N 0, 1 uder H 0 We use the stadard ormal distributio as approximatio to the true distributio of T ad defie critical values based o stadard ormal quatile Alteratively, we ca do the followig bootstrap-t test Bootstrap-t test Step 1: Draw B idepedet bootstrap samples B ca be as large as possible We ca take B = 1000 Step 2: Estimate θ ad σ with each of the bootstrap samples, ad the t-value for each bootstrap sample: θ b t b = θ σ b b θ, σ b Step 3: Order the bootstrap replicatios of t such that t 1 t B for b = 1,, B Step 4: The lower critical value t α/2 ad the upper critical value t 1 α/2 are the the B α /2- th ad B 1 α /2-th ordered elemets Reject H 0 if T < t α/2 or T > t 1 α/2 Cautio: a commo mistake is that i Step 2, oe mistakely computes θ b θ 0 σ b The test will have o power if we made this mistake The distributio of the t-statistic T = θ θ 0 σ 8
9 uder H 1 is differet from that uder H 0 Uder H 1, T is ot cetered: T = θ θ 0 σ = θ θ σ + θ θ0 σ A importat guidelie is that we should always approximate the distributio of T uder H 0, ie, the distributio of θ θ σ 9
Lecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationIt should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.
Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationConfidence Interval for Standard Deviation of Normal Distribution with Known Coefficients of Variation
Cofidece Iterval for tadard Deviatio of Normal Distributio with Kow Coefficiets of Variatio uparat Niwitpog Departmet of Applied tatistics, Faculty of Applied ciece Kig Mogkut s Uiversity of Techology
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationA quick activity - Central Limit Theorem and Proportions. Lecture 21: Testing Proportions. Results from the GSS. Statistics and the General Population
A quick activity - Cetral Limit Theorem ad Proportios Lecture 21: Testig Proportios Statistics 10 Coli Rudel Flip a coi 30 times this is goig to get loud! Record the umber of heads you obtaied ad calculate
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationUnderstanding Samples
1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationA RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS
J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More information4.1 Non-parametric computational estimation
Chapter 4 Resamplig Methods 4.1 No-parametric computatioal estimatio Let x 1,...,x be a realizatio of the i.i.d. r.vs X 1,...,X with a c.d.f. F. We are iterested i the precisio of estimatio of a populatio
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationPOWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES*
Kobe Uiversity Ecoomic Review 50(2004) 3 POWER COMPARISON OF EMPIRICAL LIKELIHOOD RATIO TESTS: SMALL SAMPLE PROPERTIES THROUGH MONTE CARLO STUDIES* By HISASHI TANIZAKI There are various kids of oparametric
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationInvestigating the Significance of a Correlation Coefficient using Jackknife Estimates
Iteratioal Joural of Scieces: Basic ad Applied Research (IJSBAR) ISSN 2307-4531 (Prit & Olie) http://gssrr.org/idex.php?joural=jouralofbasicadapplied ---------------------------------------------------------------------------------------------------------------------------
More informationConfidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M.
MATH1005 Statistics Lecture 24 M. Stewart School of Mathematics ad Statistics Uiversity of Sydey Outlie Cofidece itervals summary Coservative ad approximate cofidece itervals for a biomial p The aïve iterval
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationHypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance
Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationChi-Squared Tests Math 6070, Spring 2006
Chi-Squared Tests Math 6070, Sprig 2006 Davar Khoshevisa Uiversity of Utah February XXX, 2006 Cotets MLE for Goodess-of Fit 2 2 The Multiomial Distributio 3 3 Applicatio to Goodess-of-Fit 6 3 Testig for
More informationBig Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates.
5. Data, Estimates, ad Models: quatifyig the accuracy of estimates. 5. Estimatig a Normal Mea 5.2 The Distributio of the Normal Sample Mea 5.3 Normal data, cofidece iterval for, kow 5.4 Normal data, cofidece
More informationRecall the study where we estimated the difference between mean systolic blood pressure levels of users of oral contraceptives and non-users, x - y.
Testig Statistical Hypotheses Recall the study where we estimated the differece betwee mea systolic blood pressure levels of users of oral cotraceptives ad o-users, x - y. Such studies are sometimes viewed
More informationChapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationA new distribution-free quantile estimator
Biometrika (1982), 69, 3, pp. 635-40 Prited i Great Britai 635 A ew distributio-free quatile estimator BY FRANK E. HARRELL Cliical Biostatistics, Duke Uiversity Medical Ceter, Durham, North Carolia, U.S.A.
More informationMATH/STAT 352: Lecture 15
MATH/STAT 352: Lecture 15 Sectios 5.2 ad 5.3. Large sample CI for a proportio ad small sample CI for a mea. 1 5.2: Cofidece Iterval for a Proportio Estimatig proportio of successes i a biomial experimet
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationKLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions
We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give
More informationStatistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
More informationMonte Carlo Integration
Mote Carlo Itegratio I these otes we first review basic umerical itegratio methods (usig Riema approximatio ad the trapezoidal rule) ad their limitatios for evaluatig multidimesioal itegrals. Next we itroduce
More informationChapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.
Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationElement sampling: Part 2
Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig
More informationLecture Notes 15 Hypothesis Testing (Chapter 10)
1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we
More informationPosted-Price, Sealed-Bid Auctions
Posted-Price, Sealed-Bid Auctios Professors Greewald ad Oyakawa 207-02-08 We itroduce the posted-price, sealed-bid auctio. This auctio format itroduces the idea of approximatios. We describe how well this
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationDepartment of Mathematics
Departmet of Mathematics Ma 3/103 KC Border Itroductio to Probability ad Statistics Witer 2017 Lecture 19: Estimatio II Relevat textbook passages: Larse Marx [1]: Sectios 5.2 5.7 19.1 The method of momets
More informationA goodness-of-fit test based on the empirical characteristic function and a comparison of tests for normality
A goodess-of-fit test based o the empirical characteristic fuctio ad a compariso of tests for ormality J. Marti va Zyl Departmet of Mathematical Statistics ad Actuarial Sciece, Uiversity of the Free State,
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationA NEW METHOD FOR CONSTRUCTING APPROXIMATE CONFIDENCE INTERVALS FOR M-ESTU1ATES. Dennis D. Boos
.- A NEW METHOD FOR CONSTRUCTING APPROXIMATE CONFIDENCE INTERVALS FOR M-ESTU1ATES by Deis D. Boos Departmet of Statistics North Carolia State Uiversity Istitute of Statistics Mimeo Series #1198 September,
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationEstimation of a population proportion March 23,
1 Social Studies 201 Notes for March 23, 2005 Estimatio of a populatio proportio Sectio 8.5, p. 521. For the most part, we have dealt with meas ad stadard deviatios this semester. This sectio of the otes
More informationStochastic Simulation
Stochastic Simulatio 1 Itroductio Readig Assigmet: Read Chapter 1 of text. We shall itroduce may of the key issues to be discussed i this course via a couple of model problems. Model Problem 1 (Jackso
More informationENGI 4421 Confidence Intervals (Two Samples) Page 12-01
ENGI 44 Cofidece Itervals (Two Samples) Page -0 Two Sample Cofidece Iterval for a Differece i Populatio Meas [Navidi sectios 5.4-5.7; Devore chapter 9] From the cetral limit theorem, we kow that, for sufficietly
More informationStat 200 -Testing Summary Page 1
Stat 00 -Testig Summary Page 1 Mathematicias are like Frechme; whatever you say to them, they traslate it ito their ow laguage ad forthwith it is somethig etirely differet Goethe 1 Large Sample Cofidece
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationSTA6938-Logistic Regression Model
Dr. Yig Zhag STA6938-Logistic Regressio Model Topic -Simple (Uivariate) Logistic Regressio Model Outlies:. Itroductio. A Example-Does the liear regressio model always work? 3. Maximum Likelihood Curve
More informationBIOSTATISTICS. Lecture 5 Interval Estimations for Mean and Proportion. dr. Petr Nazarov
Microarray Ceter BIOSTATISTICS Lecture 5 Iterval Estimatios for Mea ad Proportio dr. Petr Nazarov 15-03-013 petr.azarov@crp-sate.lu Lecture 5. Iterval estimatio for mea ad proportio OUTLINE Iterval estimatios
More informationSTA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:
STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio
More informationCSE 527, Additional notes on MLE & EM
CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationIntroduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 3
Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 3 (This versio August 17, 014) 015 Pearso Educatio, Ic. Stock/Watso
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More information