Lecture 13: Subsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf
|
|
- Jasmin Perry
- 5 years ago
- Views:
Transcription
1 Lecture 13: 2011
2 Bootstrap ) R n x n, θ P)) = τ n ˆθn θ P) Example: ˆθn = X n, τ n = n, θ = EX = µ P) ˆθ = min X n, τ n = n, θ P) = sup{x : F x) 0} ) Define: J n P), the distribution of τ n ˆθ n θ P) under P. For real ˆθ n, ) ) J n x, P) Prob P τ n ˆθn θ P) x Since P is unknown, θ P) is unknown, and J n x, P) is also unknown.
3 The bootstrap estimate J n x, P) by J n x, ˆP n ), where ˆP n is a consistent estimate of P in some sense. For example, take ˆP n x) = 1 n n i=1 1 X i x) sup x ˆPn x) P x) 0 a.s. Similarly estimate 1 α)th quantile of J n x, P) by J n x, ˆP n ): i.e. Estimate Jn 1 x, P) by Jn 1 x, ˆP n ). Usually J n x, ˆP n ) can t be explicitly calculated, use MC: J n x, ˆP n ) 1 B for ˆθ n,i = ˆθ X 1,i,..., X n,i). B i=1 1 τ n ˆθn,i ˆθ ) ) n x
4 When bootstrap works, for each x, J n x, ˆP n ) J n x, P) p 0 = Jn 1 1 α, ˆP n ) Jn 1 1 α, P) p 0 When should Bootstrap work? Need local uniformity in weak convergence: Usually J n x, P) J x, P). Usually ˆP n a.s. P in some sense, say sup x ˆP n x) P x) a.s. 0 Suppose for each sequence P n s.t. P n P, say sup x P n P 0, it is also true that J n x, P n ) J x, P), then it must be true that a.s. J n x, ˆP n ) J x, P) So it ends up having to show for P n P, J n x, P n ) J x, P), use triangular array formulation.
5 Case When Bootstrap Works Sample mean with finite variance. sup x ˆF n x) F x) a.s. 0. θˆf n ) = 1 n n i=1 X a.s. i θ F ) = EX ). σ 2 ˆF n ) = 1 n n i=1 Xi X ) 2 a.s. n σ 2 F ) = VarX ). Use Linderberg-Feller for the triangular array, applied to the deterministic sequence of P n such that: 1) sup x P n x) P n x) 0; 2) θ P n ) θ P); 3) σ 2 P n ) σ 2 P), it can be shown that n Xn θ P n ) ) d N 0, σ 2 ) under P n. Since ˆP n satisfies 1,2,3 a.s., therefore J n x, ˆP n ) a.s. J x, P) So local uniformity of weak convergence is satisfied here.
6 Cases When Bootstrap Fails Order Statistics: F U 0, θ), and X 1),..., X n) is the order statistics of the sample, so X n) is the maximum: P n θ X ) n) > x = P X n) < θ θx ) = P X i < θ θx ) n θ n n 1 = θ θx )) n = 1 x ) n n e x θ n n The bootstrap version: P n X n) Xn) ) = 0 = ) n ) X n) n n 1 e 1) 0.63
7 Degenerate U-statistics: Take w x, y) = xy, θ F ) = w x, y) df x) df y) = µ F ) 2. If µ F ) 0 it is known that ) 1 ˆθ n = θ ˆFn = X i X j n n 1) i j S x) = xydf y) = xµ F ) n ˆθn θ) d N 0, 4Var S X ))) = N 0, 4 µ 2 EX 2 µ 4)) The bootstrap works.
8 But if µ F ) = 0 = θ F ) = 0: 1 θˆf n ) = X i X j = X n 2 1 n n 1) n i j ) ) n θ ˆF n θ F ) = n X n 2 Sn 2 1 n 1 i Xi X n ) 2 = X 2 n S 2 n n d N 0, σ 2) σ 2 [ ) )] However the bootstrap version of n θ ˆF n θ ˆF n : [ n X n 2 1 ] [ n S n 2 X n 2 1 ]) n S n 2 2 = n X n Sn 2 n X n 2 + Sn 2 n 2 X n X n 2 ) = [ n X n X n )] n X n X n ) n Xn d N 0, σ 2) 2 + 2N 0, σ 2 ) n X n
9 Subsampling iid case: Y i block of size b from X 1,..., X n ), i = 1,..., q, for q = n b). Let ˆθn,b,i = ˆθ Y i ) calculated with ith block of data. Use the empirical distribution of τ b ˆθ n,b,i ˆθ) over the q pseudo-estimates to approximate the distribution of τ n ˆθ θ): ) ) Approximate J n x, P) =P τ n ˆθ n θ x by L n,b x) =q 1 q i=1 1 τ b ˆθn,b,i ˆθ ) ) n x d Claim: If b, b/n 0, τ b /τ n 0, as long as τ n ˆθ θ) something, J n x, P) L n,b x) p 0
10 Different Motivation for Subsampling vs. Bootstrap Subsampling: Each subset of size b comes from the TRUE model. Since d τ n ˆθ n θ) J x, P), so as long as b : τ b ˆθ b θ) d J x, P) The distributions of τ n ˆθ n θ) and τ b ˆθ b θ) should be close. But τ b ˆθ b θ) = τ b ˆθ b ˆθ n ) + τ b ˆθ n θ). Since ) ) τb τ b ˆθn θ = O p = o p 1) τ n The distributions of τ b ˆθ b θ) and τ b ˆθ b ˆθ n ) should be close. The distribution of τ b ˆθ b ˆθ n ) is estimated by the empirical distribution over q = n b) pseudo-estimates.
11 Bootstrap: Recalculate the statistics from the ESTIMATED model ˆP n. Given that ˆP n is close to P, hopefully J n x, ˆP n ) is close to J n x, P) Or to J x, P), the limit distribution). But when bootstrap fails ˆP n P J n x, ˆP n ) J x, P)
12 Formal Proof of Consistency of Subsampling d Assumptions: τ n ˆθ n θ) J x, P), b, b n 0, τ b τ n 0. Need to show: L n,b x) J x, P) p 0. Since τ θ n θ) U n,b x) = q 1 p 0, it is enough to show q i=1 ) ) p 1 τ b ˆθn,b,i θ x J x, P). U n,b x) is a bth order U-statistics with kernel function bounded by 1, 1). U n,b x) J x, P) = U n,b x) EU n,b x) + EU n,b x) J x, P), it is enough to show U n,b x) EU n,b x) p 0 and EU n,b x) J x, P) 0
13 But EU n,b x) J x, P) = J b x, P) 0 Use Hoeffding exponential-type inequality Serfling1980), Thm A. p201): P U n,b x) J b x, P) ɛ) exp 2 n ) b ɛ2 / [1 1)] = exp n b t2) 0 as n. b So Q.E.D. L n,b x) J x, P) = L n,b x) U n,b x) + U n,b x) J b x, P) + J b x, P) J x, P) p 0.
14 Time Series Respect the ordering of the data to preserve correlation. ˆθ n,b,t = ˆθ b X t,..., X t+b 1 ), q = T b + 1. L n,b x) = 1 q ) ) 1 τ b ˆθ n,b,t ˆθ n x q i=1 d Assumption: τ n ˆθ n θ) J x, P), b, b n 0, τ b τ n 0, α m) 0. Result: L n,b x) J x, P) p 0. Most difficult part: To show τ n ˆθ n θ) d J x, P).
15 Can treat iid data as time series, or even using non-overlapping blocks k = [ n b ], but using n b) more efficient. For example, if then Ū n x) = k 1 k 1 τ b [R n,b,j θ P)] x) j=1 U n,b x) = E [ Ū n x) X n ] = E [1 τb [R n,b,j θ P)] x) X n ] for X n = X 1),..., X n) ). U n,b x) is better than Ū n x) since X n is sufficient statistics for iid data.
16 Hypothesis Testing: T n = τ n t n X 1,..., X n ), G n x, P) = Prob p τ n x) P P0 J x, P) q q Ĝ n,b x) = q 1 1 T n,b,i x) = q 1 1 τ b t n,b,i x) i=1 i=1 As long as b, b n 0, then under P P 0: Ĝ n,b x) G x, P) If under P P 1, T n, then x, Ĝ n,b x) 0. Key difference with confidence interval: don t need τ b τ n 0, because don t need to estimate θ 0 but assumed known under the null hypothesis.
17 Estimating the unknown rate of convergence Assume that τ n = n β, for some unknown β > 0. Estimate β using different size of subsampling distribution. Key idea: Compare the shape of the empirical distributions of ˆθ b ˆθ n for different values of b to infer the value of β. Let q = n b) for iid data, or q = T b + 1 for time series data: This implies L n,b x τ b ) q 1 L n,b x 1) q 1 q a=1 q a=1 1 τ b ˆθn,b,a ˆθ ) ) n x ) 1 ˆθ n,b,a ˆθ n x L n,b x τ b ) = L n,b τ 1 b x 1 ) t
18 x = L 1 n,b t τ b) = τ b τ 1 b x) = τ b L 1 n,b t 1) p Since L n,b x τ b ) J x, P), if J x, P) is continuous and increasing, it can be infered that Same as So L 1 n,b t τ b) = J 1 t, P) + o p 1) τ b L 1 n,b t 1) = J 1 t, P) + o p 1) b β L 1 n,b t 1) = J 1 t, P) + o p 1) Assuming J 1 t, P) > 0, or t > J 0, P), take log.
19 For different b 1 and b 2, then this becomes ) β log b 1 + log L 1 n,b 1 t 1) = log J 1 t, P) + o p 1) ) β log b 2 + log L 1 n,b 2 t 1) = log J 1 t, P) + o p 1) Different out the fixed effect ) β log b 1 log b 2 ) = log L 1 n,b 2 t 1) ) log L 1 n,b 1 t 1) + o p 1) So estimate β by ˆβ ) )) = log b 1 log b 2 ) 1 log L 1 n,b 2 t 1) log L 1 n,b 1 t 1) = β + log b 1 log b 2 ) 1 o p 1) Take b 1 = n γ1, b 2 = n γ2, 1 γ 1 > γ 2 > 0) ˆβ β = γ 1 γ 2 ) log n) 1 o p 1) = o p log n) 1)
20 How to know t > J 0, P), L n,b 0 τ b ) = L n,b 0 1) = J 0, P) + o p 1) So estimating J 0, P) not a problem. Alternatively, take t 2 0.5, 1), take t 1 0, 0.5) ) b β L 1 n,b t 2 1) L 1 n,b t 1 1) = J 1 t 2 P) J 1 t 1 P) + o p 1) ) β log b + log L 1 n,b t 2 1) L 1 n,b t 1 1) = log J 1 t 2 P) J 1 t 1 P) ) + o p 1) [ ˆβ = log b 1 log b 2 ) 1 ) log L 1 n,b 2 t 2 1) L 1 n,b 2 t 1 1) )] log L 1 n,b 1 t 2 1) L 1 n,b 1 t 1 1) Take b 1 = n γ1, b 2 = n γ2 1 > γ 1 > γ 2 > 0), ˆβ β = o p log n) 1 ).
21 Two Step Subsampling ˆτ n = n ˆβ L n,b x ˆτ b ) = q 1 q a=1 1 ˆτ b ˆθn,b,a ˆθ ) ) n x Can show that sup x L n,b x ˆτ b ) J x, P) p 0. Problem: imprecise in small samples. In variation estimation, best choice of b gives On 1/3 ) error rate. Parameter estimates, if model is true, gives On 1/2 ) error rate. Bootstrap pivotal statistics, when applicable, gives even better than On 1/2 ) error rate.
large number of i.i.d. observations from P. For concreteness, suppose
1 Subsampling Suppose X i, i = 1,..., n is an i.i.d. sequence of random variables with distribution P. Let θ(p ) be some real-valued parameter of interest, and let ˆθ n = ˆθ n (X 1,..., X n ) be some estimate
More informationSubsampling vs Bootstrap. Dimitris N. Politis, Joseph P. Romano, Michael Wolf. (ˆθn θ (P ) under P. For real ˆθ n, (τ n
Subsamlig vs Bootstra Dimitris N. Politis, Joseh P. Romao, Michael Wolf R x, θ P = τ ˆθ θ P Examle: ˆθ = X, τ =, θ = EX = µ P orˆθ = mi X, τ =, θ P = su{x : F x 0} Defie: J P, the istributio of τ ˆθ θ
More informationUniversity of California San Diego and Stanford University and
First International Workshop on Functional and Operatorial Statistics. Toulouse, June 19-21, 2008 K-sample Subsampling Dimitris N. olitis andjoseph.romano University of California San Diego and Stanford
More informationOn the Uniform Asymptotic Validity of Subsampling and the Bootstrap
On the Uniform Asymptotic Validity of Subsampling and the Bootstrap Joseph P. Romano Departments of Economics and Statistics Stanford University romano@stanford.edu Azeem M. Shaikh Department of Economics
More informationLecture 8 Inequality Testing and Moment Inequality Models
Lecture 8 Inequality Testing and Moment Inequality Models Inequality Testing In the previous lecture, we discussed how to test the nonlinear hypothesis H 0 : h(θ 0 ) 0 when the sample information comes
More informationSTAT 135 Lab 5 Bootstrapping and Hypothesis Testing
STAT 135 Lab 5 Bootstrapping and Hypothesis Testing Rebecca Barter March 2, 2015 The Bootstrap Bootstrap Suppose that we are interested in estimating a parameter θ from some population with members x 1,...,
More information1 Glivenko-Cantelli type theorems
STA79 Lecture Spring Semester Glivenko-Cantelli type theorems Given i.i.d. observations X,..., X n with unknown distribution function F (t, consider the empirical (sample CDF ˆF n (t = I [Xi t]. n Then
More informationON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP. Joseph P. Romano Azeem M. Shaikh
ON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP By Joseph P. Romano Azeem M. Shaikh Technical Report No. 2010-03 April 2010 Department of Statistics STANFORD UNIVERSITY Stanford, California
More information1 General problem. 2 Terminalogy. Estimation. Estimate θ. (Pick a plausible distribution from family. ) Or estimate τ = τ(θ).
Estimation February 3, 206 Debdeep Pati General problem Model: {P θ : θ Θ}. Observe X P θ, θ Θ unknown. Estimate θ. (Pick a plausible distribution from family. ) Or estimate τ = τ(θ). Examples: θ = (µ,
More informationLecture 8. October 22, Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University.
Lecture 8 Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University October 22, 2007 1 2 3 4 5 6 1 Define convergent series 2 Define the Law of Large Numbers
More informationIntroduction to Empirical Processes and Semiparametric Inference Lecture 02: Overview Continued
Introduction to Empirical Processes and Semiparametric Inference Lecture 02: Overview Continued Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations Research
More informationStatistics Ph.D. Qualifying Exam
Department of Statistics Carnegie Mellon University May 7 2008 Statistics Ph.D. Qualifying Exam You are not expected to solve all five problems. Complete solutions to few problems will be preferred to
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationIntroductory Econometrics
Session 4 - Testing hypotheses Roland Sciences Po July 2011 Motivation After estimation, delivering information involves testing hypotheses Did this drug had any effect on the survival rate? Is this drug
More informationDA Freedman Notes on the MLE Fall 2003
DA Freedman Notes on the MLE Fall 2003 The object here is to provide a sketch of the theory of the MLE. Rigorous presentations can be found in the references cited below. Calculus. Let f be a smooth, scalar
More informationThis does not cover everything on the final. Look at the posted practice problems for other topics.
Class 7: Review Problems for Final Exam 8.5 Spring 7 This does not cover everything on the final. Look at the posted practice problems for other topics. To save time in class: set up, but do not carry
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationInference for Identifiable Parameters in Partially Identified Econometric Models
Inference for Identifiable Parameters in Partially Identified Econometric Models Joseph P. Romano Department of Statistics Stanford University romano@stat.stanford.edu Azeem M. Shaikh Department of Economics
More information36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1
36. Multisample U-statistics jointly distributed U-statistics Lehmann 6.1 In this topic, we generalize the idea of U-statistics in two different directions. First, we consider single U-statistics for situations
More informationChapter 10. U-statistics Statistical Functionals and V-Statistics
Chapter 10 U-statistics When one is willing to assume the existence of a simple random sample X 1,..., X n, U- statistics generalize common notions of unbiased estimation such as the sample mean and the
More informationBootstrap Confidence Intervals
Bootstrap Confidence Intervals Patrick Breheny September 18 Patrick Breheny STA 621: Nonparametric Statistics 1/22 Introduction Bootstrap confidence intervals So far, we have discussed the idea behind
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationMathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
More information7 Influence Functions
7 Influence Functions The influence function is used to approximate the standard error of a plug-in estimator. The formal definition is as follows. 7.1 Definition. The Gâteaux derivative of T at F in the
More informationCentral Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
More informationIf g is also continuous and strictly increasing on J, we may apply the strictly increasing inverse function g 1 to this inequality to get
18:2 1/24/2 TOPIC. Inequalities; measures of spread. This lecture explores the implications of Jensen s inequality for g-means in general, and for harmonic, geometric, arithmetic, and related means in
More informationThe Slow Convergence of OLS Estimators of α, β and Portfolio. β and Portfolio Weights under Long Memory Stochastic Volatility
The Slow Convergence of OLS Estimators of α, β and Portfolio Weights under Long Memory Stochastic Volatility New York University Stern School of Business June 21, 2018 Introduction Bivariate long memory
More informationSome Assorted Formulae. Some confidence intervals: σ n. x ± z α/2. x ± t n 1;α/2 n. ˆp(1 ˆp) ˆp ± z α/2 n. χ 2 n 1;1 α/2. n 1;α/2
STA 248 H1S MIDTERM TEST February 26, 2008 SURNAME: SOLUTIONS GIVEN NAME: STUDENT NUMBER: INSTRUCTIONS: Time: 1 hour and 50 minutes Aids allowed: calculator Tables of the standard normal, t and chi-square
More informationFinal Examination Statistics 200C. T. Ferguson June 11, 2009
Final Examination Statistics 00C T. Ferguson June, 009. (a) Define: X n converges in probability to X. (b) Define: X m converges in quadratic mean to X. (c) Show that if X n converges in quadratic mean
More informationStatistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationStat 710: Mathematical Statistics Lecture 31
Stat 710: Mathematical Statistics Lecture 31 Jun Shao Department of Statistics University of Wisconsin Madison, WI 53706, USA Jun Shao (UW-Madison) Stat 710, Lecture 31 April 13, 2009 1 / 13 Lecture 31:
More informationSTAT 830 Non-parametric Inference Basics
STAT 830 Non-parametric Inference Basics Richard Lockhart Simon Fraser University STAT 801=830 Fall 2012 Richard Lockhart (Simon Fraser University)STAT 830 Non-parametric Inference Basics STAT 801=830
More informationLecture 32: Asymptotic confidence sets and likelihoods
Lecture 32: Asymptotic confidence sets and likelihoods Asymptotic criterion In some problems, especially in nonparametric problems, it is difficult to find a reasonable confidence set with a given confidence
More informationInterval estimation. October 3, Basic ideas CLT and CI CI for a population mean CI for a population proportion CI for a Normal mean
Interval estimation October 3, 2018 STAT 151 Class 7 Slide 1 Pandemic data Treatment outcome, X, from n = 100 patients in a pandemic: 1 = recovered and 0 = not recovered 1 1 1 0 0 0 1 1 1 0 0 1 0 1 0 0
More informationSpatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood
Spatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood Kuangyu Wen & Ximing Wu Texas A&M University Info-Metrics Institute Conference: Recent Innovations in Info-Metrics October
More informationLecture Notes 3 Convergence (Chapter 5)
Lecture Notes 3 Convergence (Chapter 5) 1 Convergence of Random Variables Let X 1, X 2,... be a sequence of random variables and let X be another random variable. Let F n denote the cdf of X n and let
More informationTail bound inequalities and empirical likelihood for the mean
Tail bound inequalities and empirical likelihood for the mean Sandra Vucane 1 1 University of Latvia, Riga 29 th of September, 2011 Sandra Vucane (LU) Tail bound inequalities and EL for the mean 29.09.2011
More informationMTMS Mathematical Statistics
MTMS.01.099 Mathematical Statistics Lecture 12. Hypothesis testing. Power function. Approximation of Normal distribution and application to Binomial distribution Tõnu Kollo Fall 2016 Hypothesis Testing
More informationMath 141. Lecture 16: More than one group. Albyn Jones 1. jones/courses/ Library 304. Albyn Jones Math 141
Math 141 Lecture 16: More than one group Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Comparing two population means If two distributions have the same shape and spread,
More informationBootstrap. Director of Center for Astrostatistics. G. Jogesh Babu. Penn State University babu.
Bootstrap G. Jogesh Babu Penn State University http://www.stat.psu.edu/ babu Director of Center for Astrostatistics http://astrostatistics.psu.edu Outline 1 Motivation 2 Simple statistical problem 3 Resampling
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationCOMPSCI 240: Reasoning Under Uncertainty
COMPSCI 240: Reasoning Under Uncertainty Andrew Lan and Nic Herndon University of Massachusetts at Amherst Spring 2019 Lecture 20: Central limit theorem & The strong law of large numbers Markov and Chebyshev
More informationIntroduction to Self-normalized Limit Theory
Introduction to Self-normalized Limit Theory Qi-Man Shao The Chinese University of Hong Kong E-mail: qmshao@cuhk.edu.hk Outline What is the self-normalization? Why? Classical limit theorems Self-normalized
More informationBootstrap - theoretical problems
Date: January 23th 2006 Bootstrap - theoretical problems This is a new version of the problems. There is an added subproblem in problem 4, problem 6 is completely rewritten, the assumptions in problem
More informationi=1 X i/n i=1 (X i X) 2 /(n 1). Find the constant c so that the statistic c(x X n+1 )/S has a t-distribution. If n = 8, determine k such that
Math 47 Homework Assignment 4 Problem 411 Let X 1, X,, X n, X n+1 be a random sample of size n + 1, n > 1, from a distribution that is N(µ, σ ) Let X = n i=1 X i/n and S = n i=1 (X i X) /(n 1) Find the
More informationIEOR E4703: Monte-Carlo Simulation
IEOR E4703: Monte-Carlo Simulation Output Analysis for Monte-Carlo Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com Output Analysis
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
More informationProving the central limit theorem
SOR3012: Stochastic Processes Proving the central limit theorem Gareth Tribello March 3, 2019 1 Purpose In the lectures and exercises we have learnt about the law of large numbers and the central limit
More informationComprehensive Examination Quantitative Methods Spring, 2018
Comprehensive Examination Quantitative Methods Spring, 2018 Instruction: This exam consists of three parts. You are required to answer all the questions in all the parts. 1 Grading policy: 1. Each part
More informationInverse Statistical Learning
Inverse Statistical Learning Minimax theory, adaptation and algorithm avec (par ordre d apparition) C. Marteau, M. Chichignoud, C. Brunet and S. Souchet Dijon, le 15 janvier 2014 Inverse Statistical Learning
More informationSTAT 461/561- Assignments, Year 2015
STAT 461/561- Assignments, Year 2015 This is the second set of assignment problems. When you hand in any problem, include the problem itself and its number. pdf are welcome. If so, use large fonts and
More informationLecture 28: Asymptotic confidence sets
Lecture 28: Asymptotic confidence sets 1 α asymptotic confidence sets Similar to testing hypotheses, in many situations it is difficult to find a confidence set with a given confidence coefficient or level
More informationMathematical statistics
October 18 th, 2018 Lecture 16: Midterm review Countdown to mid-term exam: 7 days Week 1 Chapter 1: Probability review Week 2 Week 4 Week 7 Chapter 6: Statistics Chapter 7: Point Estimation Chapter 8:
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationLecture 4: September Reminder: convergence of sequences
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 4: September 6 In this lecture we discuss the convergence of random variables. At a high-level, our first few lectures focused
More informationStat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS
Stat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS 1a. Under the null hypothesis X has the binomial (100,.5) distribution with E(X) = 50 and SE(X) = 5. So P ( X 50 > 10) is (approximately) two tails
More informationHypothesis testing: theory and methods
Statistical Methods Warsaw School of Economics November 3, 2017 Statistical hypothesis is the name of any conjecture about unknown parameters of a population distribution. The hypothesis should be verifiable
More informationSDS : Theoretical Statistics
SDS 384 11: Theoretical Statistics Lecture 1: Introduction Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin https://psarkar.github.io/teaching Manegerial Stuff
More informationThe Bootstrap: Theory and Applications. Biing-Shen Kuo National Chengchi University
The Bootstrap: Theory and Applications Biing-Shen Kuo National Chengchi University Motivation: Poor Asymptotic Approximation Most of statistical inference relies on asymptotic theory. Motivation: Poor
More informationBootstrapping high dimensional vector: interplay between dependence and dimensionality
Bootstrapping high dimensional vector: interplay between dependence and dimensionality Xianyang Zhang Joint work with Guang Cheng University of Missouri-Columbia LDHD: Transition Workshop, 2014 Xianyang
More informationGeneralization Bounds and Stability
Generalization Bounds and Stability Lorenzo Rosasco Tomaso Poggio 9.520 Class 9 2009 About this class Goal To recall the notion of generalization bounds and show how they can be derived from a stability
More informationInference For High Dimensional M-estimates. Fixed Design Results
: Fixed Design Results Lihua Lei Advisors: Peter J. Bickel, Michael I. Jordan joint work with Peter J. Bickel and Noureddine El Karoui Dec. 8, 2016 1/57 Table of Contents 1 Background 2 Main Results and
More informationThe regression model with one fixed regressor cont d
The regression model with one fixed regressor cont d 3150/4150 Lecture 4 Ragnar Nymoen 27 January 2012 The model with transformed variables Regression with transformed variables I References HGL Ch 2.8
More informationRecitation 5. Inference and Power Calculations. Yiqing Xu. March 7, 2014 MIT
17.802 Recitation 5 Inference and Power Calculations Yiqing Xu MIT March 7, 2014 1 Inference of Frequentists 2 Power Calculations Inference (mostly MHE Ch8) Inference in Asymptopia (and with Weak Null)
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More informationQuantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing
Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October
More informationStat 5102 Final Exam May 14, 2015
Stat 5102 Final Exam May 14, 2015 Name Student ID The exam is closed book and closed notes. You may use three 8 1 11 2 sheets of paper with formulas, etc. You may also use the handouts on brand name distributions
More informationStatistics 3858 : Maximum Likelihood Estimators
Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,
More informationK-sample subsampling in general spaces: the case of independent time series
K-sample subsampling in general spaces: the case of independent time series Dimitris N. Politis Joseph P. Romano Abstract The problem of subsampling in two-sample and K-sample settings is addressed where
More informationDistributions of Functions of Random Variables. 5.1 Functions of One Random Variable
Distributions of Functions of Random Variables 5.1 Functions of One Random Variable 5.2 Transformations of Two Random Variables 5.3 Several Random Variables 5.4 The Moment-Generating Function Technique
More informationM(t) = 1 t. (1 t), 6 M (0) = 20 P (95. X i 110) i=1
Math 66/566 - Midterm Solutions NOTE: These solutions are for both the 66 and 566 exam. The problems are the same until questions and 5. 1. The moment generating function of a random variable X is M(t)
More informationReliable Inference in Conditions of Extreme Events. Adriana Cornea
Reliable Inference in Conditions of Extreme Events by Adriana Cornea University of Exeter Business School Department of Economics ExISta Early Career Event October 17, 2012 Outline of the talk Extreme
More informationNon-parametric bootstrap mean squared error estimation for M-quantile estimates of small area means, quantiles and poverty indicators
Non-parametric bootstrap mean squared error estimation for M-quantile estimates of small area means, quantiles and poverty indicators Stefano Marchetti 1 Nikos Tzavidis 2 Monica Pratesi 3 1,3 Department
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationSupplement to Quantile-Based Nonparametric Inference for First-Price Auctions
Supplement to Quantile-Based Nonparametric Inference for First-Price Auctions Vadim Marmer University of British Columbia Artyom Shneyerov CIRANO, CIREQ, and Concordia University August 30, 2010 Abstract
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationLecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2
Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2 Fall, 2013 Page 1 Random Variable and Probability Distribution Discrete random variable Y : Finite possible values {y
More informationSTAT Homework X - Solutions
STAT-36700 Homework X - Solutions Fall 201 November 12, 201 Tis contains solutions for Homework 4. Please note tat we ave included several additional comments and approaces to te problems to give you better
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationEconomics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
More informationSpring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationInference for identifiable parameters in partially identified econometric models
Journal of Statistical Planning and Inference 138 (2008) 2786 2807 www.elsevier.com/locate/jspi Inference for identifiable parameters in partially identified econometric models Joseph P. Romano a,b,, Azeem
More informationOne-Sample Numerical Data
One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html
More informationMAS113 Introduction to Probability and Statistics
MAS113 Introduction to Probability and Statistics School of Mathematics and Statistics, University of Sheffield 2018 19 Identically distributed Suppose we have n random variables X 1, X 2,..., X n. Identically
More informationUQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables
UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables To be provided to students with STAT2201 or CIVIL-2530 (Probability and Statistics) Exam Main exam date: Tuesday, 20 June 1
More informationAdvanced Statistics II: Non Parametric Tests
Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney
More informationEstimation of the quantile function using Bernstein-Durrmeyer polynomials
Estimation of the quantile function using Bernstein-Durrmeyer polynomials Andrey Pepelyshev Ansgar Steland Ewaryst Rafaj lowic The work is supported by the German Federal Ministry of the Environment, Nature
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationSummary of Chapters 7-9
Summary of Chapters 7-9 Chapter 7. Interval Estimation 7.2. Confidence Intervals for Difference of Two Means Let X 1,, X n and Y 1, Y 2,, Y m be two independent random samples of sizes n and m from two
More informationStatistical Inference of Covariate-Adjusted Randomized Experiments
1 Statistical Inference of Covariate-Adjusted Randomized Experiments Feifang Hu Department of Statistics George Washington University Joint research with Wei Ma, Yichen Qin and Yang Li Email: feifang@gwu.edu
More informationBootstrap & Confidence/Prediction intervals
Bootstrap & Confidence/Prediction intervals Olivier Roustant Mines Saint-Étienne 2017/11 Olivier Roustant (EMSE) Bootstrap & Confidence/Prediction intervals 2017/11 1 / 9 Framework Consider a model with
More informationStatistics. Statistics
The main aims of statistics 1 1 Choosing a model 2 Estimating its parameter(s) 1 point estimates 2 interval estimates 3 Testing hypotheses Distributions used in statistics: χ 2 n-distribution 2 Let X 1,
More informationCross-Validation with Confidence
Cross-Validation with Confidence Jing Lei Department of Statistics, Carnegie Mellon University UMN Statistics Seminar, Mar 30, 2017 Overview Parameter est. Model selection Point est. MLE, M-est.,... Cross-validation
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationThe Numerical Delta Method and Bootstrap
The Numerical Delta Method and Bootstrap Han Hong and Jessie Li Stanford University and UCSC 1 / 41 Motivation Recent developments in econometrics have given empirical researchers access to estimators
More informationLarge Sample Theory. Consider a sequence of random variables Z 1, Z 2,..., Z n. Convergence in probability: Z n
Large Sample Theory In statistics, we are interested in the properties of particular random variables (or estimators ), which are functions of our data. In ymptotic analysis, we focus on describing the
More informationPreliminary Statistics Lecture 5: Hypothesis Testing (Outline)
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 5: Hypothesis Testing (Outline) Gujarati D. Basic Econometrics, Appendix A.8 Barrow M. Statistics
More informationHT Introduction. P(X i = x i ) = e λ λ x i
MODS STATISTICS Introduction. HT 2012 Simon Myers, Department of Statistics (and The Wellcome Trust Centre for Human Genetics) myers@stats.ox.ac.uk We will be concerned with the mathematical framework
More information