The Nonparametric Bootstrap

Size: px
Start display at page:

Download "The Nonparametric Bootstrap"

Transcription

1 The Nonparametric Bootstrap The nonparametric bootstrap may involve inferences about a parameter, but we use a nonparametric procedure in approximating the parametric distribution using the ECDF. We use statistical functions (plug-ins) of the ECDF that correspond to statistical functions of the underlying distribution. In the process, we may also use some parametric approximations (such as an approximate t distribution).

2 Bootstrap Estimate of the Variance of an Estimator For the variance of T the bootstrap estimator is V(T ); that is, it is the variance of T based on samples of size n taken from P n. If T is the sample mean, for example, the bootstrap estimate of the variance ˆσ/n, where ˆσ is an estimate of the variance of the underlying sample. There s really no way the bootstrap procedure helps in this situation.

3 Monte Carlo Bootstrap Usually, however, we cannot simply write the variance of T in terms of some other simple estimator. We usually have to resort to bootstrap sampling. Example: The Correlation of LSAT Scores and GPA The Law School Data Generate m samples of size 15 with replacement from the original data (ind <- sample(15, replace=t) each time); compute correlation (cor(law[c(ind),1],law[c(ind),2])); and compute variance or standard deviation of these correlation coefficients.

4 Bootstrap Confidence Intervals A method of forming a confidence interval for a parameter θ is to find a pivotal quantity that involves θ and a statistic T, f(t, θ), and then to rearrange the terms in a probability statement of the form Pr(f (α/2) f(t, θ) f (1 α/2) ) = 1 α. When distributions are difficult to work out, we may use bootstrap methods for estimating and/or approximating the percentiles, f (α/2) and f (1 α/2).

5 Basic Intervals For computing confidence intervals for a mean, the pivotal quantity is likely to be of the form T θ. The simplest application of the bootstrap to forming a confidence interval is to use the sampling distribution of T T 0 as an approximation to the sampling distribution of T θ; that is, instead of using f(t, θ), we use f(t, T 0 ), where T 0 is the value of T in the given sample. The percentiles of the sampling distribution determine f (α/2) and f (1 α/2) in the expressions above. If we cannot determine the sampling distribution of T T 0, we can easily estimate it by Monte Carlo methods. For the case f(t, θ) = T θ, the probability statement above is equivalent to Pr(T f (1 α/2) θ T f (α/2) ) = 1 α. The f (π) may be estimated from the percentiles of a Monte Carlo sample of T T 0.

6 Bootstrap-t Intervals Methods of inference based on a normal distribution often work well even when the underlying distribution is not normal. A useful approximate confidence interval for a location parameter can often be constructed using as a template the familiar confidence interval for the mean of a normal distribution, (Y t (1 α/2) s/ n, Y t (α/2) s/ n), where t (π) is a percentile from the Student s t distribution, and s 2 is the usual sample variance. A confidence interval for any parameter constructed in this pattern is called a bootstrap-t interval.

7 A bootstrap-t interval has the form (T t (1 α/2) V(T), T t (α/2) V(T)), where t (π) is the estimated percentile from the studentized statistic, T T 0 V(T ). For many estimators T, no simple expression is available for V(T). The variance could be estimated using a bootstrap. This bootstrap nested in the bootstrap to determine t (π) increases the computational burden multiplicatively. If the underlying distribution is normal and T is a sample mean, the interval above is an exact (1 α)100% confidence interval of shortest length.

8 If the underlying distribution is not normal, however, this confidence interval may not have good properties. In particular, it may not even be of size (1 α)100%. An asymmetric underlying distribution can have particularly deleterious effects on one-sided confidence intervals. If the estimators T and V(T) are based on sums of squares of deviations, the bootstrap-t interval performs very poorly when the underlying distribution has heavy tails. This is to be expected, of course. Bootstrap procedures can be no better than the statistics used.

9 Bootstrap Percentile Confidence Intervals Given a random sample (y 1,..., y n ) from an unknown distribution with CDF P, we want an interval estimate of a parameter, θ = Θ(P), for which we have a point estimator, T. A bootstrap estimator for θ is T, based on the bootstrap sample (y 1,..., y n). Now, if G T (t) is the distribution function for T, then the exact upper 1 α confidence limit for θ is the value t (1 α), such that G T (t (1 α) ) = 1 α. This is called the percentile upper confidence limit. A lower limit is obtained similarly, and an interval is based on the lower and upper limits.

10 In practice, we generally use Monte Carlo and m bootstrap samples to estimate these quantities. The probability-symmetric bootstrap percentile confidence interval of size (1 α)100% is thus (t (α/2), t (1 α/2) ), where t (π) is the [πm]th order statistic of a sample of size m of T. Note that we are using T and t, and hence T and t, to represent estimators and estimates in general; that is, t (π) here does not refer to a percentile of the Student s t distribution. This percentile interval is based on the ideal bootstrap and may be estimated by Monte Carlo simulation.

11 Confidence Intervals Based on Transformations Suppose that there is a monotonically increasing transformation g and a constant c such that the random variable W = c(g(t ) g(θ)) has a symmetric distribution about zero. Here g(θ) is in the role of a mean and c is a scale or standard deviation. Let H be the distribution function of W, so G T (t) = H(c(g(t) g(θ))) and t (1 α/2) = g 1 (g(t ) + w (1 α/2) /c), where w (1 α/2) is the (1 α/2) quantile of W. The other quantile t (α/2) would be determined analogously.

12 Instead of approximating the ideal interval with a Monte Carlo sample, we could use a transformation to a known W and compute the interval that way. Use of an exact transformation g to a known random variable W, of course, is just as difficult as evaluation of the ideal bootstrap interval. Nevertheless, we see that forming the ideal bootstrap confidence interval is equivalent to using the transformation g and the distribution function H. Because transformations to approximate normality are well-understood and widely used, in practice, we generally choose g as a transformation to normality.

13 The random variable W on the previous slide is a standard normal random variable, Z. The relevant distribution function is Φ, the normal CDF. The normal approximations have a basis in the central limit property. Central limit approximations often have a bias of order O(n 1 ), however, so in small samples, the percentile intervals may not be very good.

14 Better Bootstrap Confidence Intervals It is likely that the transformed statistic g(t ) is biased for the transformed θ, even if the untransformed statistic is unbiased for θ. We can account for the possible bias by using the transformation Z = c(g(t ) g(θ)) + z 0, and we have G T (t) = Φ(c(g(t) g(θ)) + z 0 ). The bias correction z 0 is Φ 1 (G T (t)). Even when we are estimating θ directly with T (that is, g is the identity), another possible problem in determining percentiles for the confidence interval is the lack of symmetry of the distribution about z 0.

15 These problems indicate the need to make some adjustments in the quantiles. Rather than correcting the quantiles directly, we may adjust their levels. For an interval of confidence (1 α), instead of (t (α/2), t (1 α/2) ), we take (t (α 1 ), t (α 2 ) ), where the adjusted probabilities α 1 and α 2 are determined so as to reduce the bias and to allow for the lack of symmetry. As we often do, even for a nonnormal underlying distribution, we relate α 1 and α 2 to percentiles of the normal distribution.

16 To allow for the lack of symmetry that is, for a scale difference below and above z 0 we use quantiles about that point. Efron (1987), who developed this method, introduced an acceleration, a, and used the distance a(z 0 + z (π) ). Using values for the bias correction and the acceleration determined from the data, Efron suggested the quantile adjustments α 1 = Φ ẑ 0 + ẑ 0 + z (α/2) 1 â(ẑ 0 + z (α/2) ) and α 2 = Φ ẑ 0 + ẑ 0 + z (1 α/2) 1 â(ẑ 0 + z (1 α/2) ).

17 Use of these adjustments to the level of the quantiles for confidence intervals is called the accelerated bias-corrected, or BC a, method. This method automatically takes care of the problems of bias or asymmetry resulting from transformations that we discussed above. Note that if â = ẑ 0 = 0, then α 1 = Φ(z (α) ) and α 2 = Φ(z (1 α) ). In this case, the BC a is the same as the ordinary percentile method.

18 The problem now is to estimate the acceleration a and the bias correction z 0 from the data. The bias-correction term z 0 is estimated by correcting the percentile near the median of the m bootstrap samples: ẑ 0 = Φ 1 1 m j I (,T] (T j ). The idea is that we approximate the bias of the median (that is, the bias of a central quantile) and then adjust the other quantiles accordingly.

19 Estimating a is a little more difficult. The way we proceed depends on the form the bias may take and how we choose to represent it. Because one cause of bias may be skewness, Efron (1987) adjusted for the skewness of the distribution of the estimator in the neighborhood of θ. The skewness is measured by a function of the second and third moments of T. We can use the jackknife to estimate those moments. The expression is (J(T) T(i) ) 3 â = 6( (J(T) T (i) ) 2 ) 3/2.

20 There may be a bias that results from other departures from normality, such as heavy tails. This adjustment does nothing for this kind of bias. Bootstrap-t and BC a confidence intervals may be used for inference concerning the difference in the means of two groups. For moderate and approximately equal sample sizes, the coverage of BC a intervals is often closer to the nominal confidence level, but for samples with very different sizes, the bootstrap-t intervals are often better in the sense of coverage frequency. Because of the variance of the components in the BC a method, it generally requires relatively large numbers of bootstrap samples.

21 Examples in R There are several R packages for various bootstrap procedures. One is called bootstrap. One is called boot. Also, Maria Rizzo has written some ad hoc functions. Available at Let s look at a couple of examples.

22 How Large Should the Monte Carlo Bootstrap Sample Be In the ideal bootstrap, there is no sampling, or equivalently, the sample size is. In most applications, of course, we do bootstrap sampling. Our bootstrap estimate is V m (T ) instead of the ideal V(T ), or V (T ). How many bootstrap observations should we take? This depends on the underlying distribution. The variance of the bootstrap estimator is decreasing, but it is not approaching 0! This also depends on what kind of inference is being made. Setting confidence intervals, for example, usually would suggest larger bootstrap sample sizes than just estimating the variance. This is similar to the situation in ordinary inference; it takes larger samples of the underlying distribution to estimate a probability density than it does to estimate a mean.

23 Monte Carlo Bootstrap Sample Size It is a good idea to consider the sample of bootstrap observations as a set of data, rather than just simply computing its variance. (By the way, how do you compute its variance? Use m or m 1? (It doesn t really matter.)) This means plotting a histogram of the bootstrap observations. Is it skewed? How spread out is it? etc. In practice, bootstrap samples of size a few hunderd (200 or so) are usually sufficient for point estimates. For confidence intervals, a few thousand (2000 or so) may be more appropriate. For selecting optimal bandwidth in density estimation, 100 or so may be OK. (Notice I did not say estimating the variance of the density estimate. The bootstrap is often used in selecting parameters in adaptive inference schemes. In such cases, we may get by with smaller bootstrap samples.)

24 An aside on sample. In statistics, this word is equivalent to set (or set of indices ). A sample is a set of observations. Each observation is unique, even if it has the same value as another observation. In a bootstrap replication, two pseudo-observations corresponding to the same original observation are considered distinct (so the mathematical object is still a set). The bootstrap sample is a collection of replications. We usually denote the number of replications (the bootstrap sample size) by m. In some disciplines, especially engineering, it is common to equate sample with observation. This can lead to lack of precision in the language; whereas using sample to mean a collection of observations rarely leads to confusion.

25 Parametric Bootstrap In the parametric bootstrap, the parameters of the underlying distribution are estimated using the given sample, then m random pseudo samples of size n are generated from that parametric distribution. These m samples are then used for the bootstrap inference. An aside: How to generate multivariate normals. If x MVN(0, I) and y = Ax, the y MVN(0, AA T ). In the other direction, suppose we want MVN(0,Σ). Find Σ C such that Σ = Σ T C Σ C, then take y = Σ T C x. The matrix Σ C is an upper-triangular matrix called the Cholesky factor of Σ. This can be obtained in S-Plus using chol for the Cholesky decomposition. (Works for a symmetric positive definite or positive semi-definite matrix.)

26 Bias Reduction Find f t (i.e., t) so that E(f t (P, P n ) P) = 0 or E(T(P n ) θ(p) + t P) = 0. Change the problem to the sample: whose solution is so the bias-reduced estimate is E(T(P (1) n ) T(P n ) + t 1 P n ) = 0, t 1 = T(P n ) E(T(P (1) n ) P n ), T 1 = 2T(P n ) E(T(P (1) n ) P n ).

27 Bias Reduction We may be able to compute E(T(P n ) P n ). If so, do so. If not, estimate by Monte Carlo.

28 Resampling to Estimate Bias Problem: given a random sample (x 1, x 2,..., x n ) from an unknown distribution with df P, we want to estimate a parameter, θ = θ(p). One of the most important properties of an estimator, T = t(x 1, x 2,..., x n ), is its bias, E P (T) θ. A good estimator, of course, has a small bias, probably even zero, i.e., the estimator is unbiased. The plug-in estimator, T = θ(p n ), may or may not be unbiased, but its bias is often small (e.g., the plug-in estimator of σ 2 ; the bias correction is n/(n 1)). Of course, without fairly strong assumptions on the underlying distribution, it is unlikely that we would know the bias of an estimator. Resampling methods can be used to estimate the bias. In particular, the bootstrap estimate of the bias is E Pn (T) θ(p n )

29 The Bootstrap Estimate of the Bias The bootstrap estimate of the bias is the plug-in estimate of E P (T) θ. The plug-in step occurs in two places, for estimating E P (T) and then for estimating θ. Consider the simple estimators (both plug-in estimators): Sample mean: E Pn (T) θ(p n ) = 0. The sample mean is unbiased for the population mean (if the population mean exists). Sample second central moment: E Pn (T) θ(p n ) = (x i x) 2 /n. This is also what we would want. This ideal bootstrap estimate must generally be approximated by Monte Carlo simulation.

30 The Monte Carlo Bootstrap Estimate of the Bias The ideal estimate is an expected value of a funtion: E Pn (T) θ(p n ); the Monte Carlo estimate of an expected value of a function is the Monte Carlo sample mean of that function. For the Monte Carlo simulation, we generate a number of bootstrap samples, (x 1, x 2,..., x n ), drawn from the empirical ditribution resulting from (x 1, x 2,..., x n ). Letting (x j 1, x j 2,..., x j n ) represent the j th bootstrap sample, the Monte Carlo estimate of the bootstrap estimate of the bias is j t(x 1, x j 2,..., x j n )/m θ(p n ) For the Monte Carlo estimate we can define a resampling vector, P, corresponding to each bootstrap sample as the vector of proportions of the elements of the original sample in the given bootstrap sample.

31 The Resampling Vector If the bootstrap sample (x 1, x 2, x 3, x 4 ) is really the sample (x 2, x 2, x 4, x 3 ), the resampling vector P is (0, 1/2, 1/4, 1/4) The resampling vector has random components that sum to 1. The bootstrap replication of the estimator T is a function of P. The Monte Carlo estimate of the bootstrap estimate of the bias can be improved if the estimator whose bias is being estimated is a plug-in estimator.

32 The Bootstrap Estimate of the Bias of a Plug-In Estimator Consider the resampling vector, P 0 = (1/n, 1/n,..., 1/n). Such a resampling vector corresponds to a permutation of the original sample. If the estimator is a plug-in estimator, then its value is invariant to permutations of the sample; and, in fact, θ(p 0 ) = θ(p n ), so the Monte Carlo estimate of the bootstrap estimate of the bias, j t(x 1, x j 2,..., x j n )/m θ(p n), can be written as j t(x 1, x j 2,..., x j n )/m θ(p 0 ). Instead of using θ(p 0 ), we can increase the precision of the Monte Carlo estimate by using the individual P s actually obtained: t(x j 1, x j 2,..., x j n )/m θ( P j /m), that is, by using the mean of the resampling vectors. Notice that for an unbiased plug-in estimator, e.g., the sample mean, this quantity is 0.

33 Variance Reduction in Monte Carlo The use of θ( P ) is a type of variance reduction in a Monte Carlo procedure. Remember that a Monte Carlo procedure is estimating an integral. Suppose the integral is (f(x) + g(x)) dx = f(x)dx + g(x))dx, and suppose we know g(x))dx. What is the best way to do the Monte Carlo, to do the integral of the sum, or just to do the integral of f(x) and add on the known value of the integral of g(x)? What makes one estimator better than another? Is the variance of f(x) + g(x) smaller than the variance of f(x)? Remember that the variance of the Monte Carlo estimator, Î, of an integral, h(x) dx, is generally proportional to the variance of h(x), where X is a random variable. (There are, of course, different ways of using random variables in the Monte Carlo estimation of the integral.)

34 Variance Reduction in Monte Carlo If the objective in Monte Carlo experimentation is to estimate some quantity, just as in any estimation procedure, we want to reduce the variance of our estimator (while preserving its other good qualities). The basic idea is usually to reduce the problem analytically as far as possible, and then to Monte Carlo what is left. Beyond that general reduction principle, in Monte Carlo experimentation, there are several possibilities for reducing the variance.

35 Variance Reduction in Monte Carlo judicious use of an auxilliary variable control variates (any correlated variable, either positively or negatively correlated) antithetic variates (in the basic uniform generator) regression methods use of probability sampling discrete: stratified sampling continuous: importance sampling

36 Balanced Resampling Another way of reducing the variance in Monte Carlo experimentation is to constrain the sampling so that some aspects of the samples reflect precisely some aspects of the population. What about constraining P so as to equal P 0? This makes θ( P ) = θ(p 0 ), and hopefully makes t(x j 1, x j 2,..., x j n )/m closer to its expected value, while preserving its correlation with θ( P ). This is called balanced resampling. Hall (1990) has shown that the balanced resampling Monte Carlo estimator of the bootstrap estimator has a bias O(m 1 ), but that the reduced variance generally more than makes up for it.

Better Bootstrap Confidence Intervals

Better Bootstrap Confidence Intervals by Bradley Efron University of Washington, Department of Statistics April 12, 2012 An example Suppose we wish to make inference on some parameter θ T (F ) (e.g. θ = E F X ), based on data We might suppose

More information

4 Resampling Methods: The Bootstrap

4 Resampling Methods: The Bootstrap 4 Resampling Methods: The Bootstrap Situation: Let x 1, x 2,..., x n be a SRS of size n taken from a distribution that is unknown. Let θ be a parameter of interest associated with this distribution and

More information

Bootstrap Confidence Intervals

Bootstrap Confidence Intervals Bootstrap Confidence Intervals Patrick Breheny September 18 Patrick Breheny STA 621: Nonparametric Statistics 1/22 Introduction Bootstrap confidence intervals So far, we have discussed the idea behind

More information

Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods

Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods Chapter 4 Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods 4.1 Introduction It is now explicable that ridge regression estimator (here we take ordinary ridge estimator (ORE)

More information

The bootstrap. Patrick Breheny. December 6. The empirical distribution function The bootstrap

The bootstrap. Patrick Breheny. December 6. The empirical distribution function The bootstrap Patrick Breheny December 6 Patrick Breheny BST 764: Applied Statistical Modeling 1/21 The empirical distribution function Suppose X F, where F (x) = Pr(X x) is a distribution function, and we wish to estimate

More information

Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One. Outline. Charlotte Wickham  1. Basic ideas about estimation Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

More information

Monte Carlo Studies. The response in a Monte Carlo study is a random variable.

Monte Carlo Studies. The response in a Monte Carlo study is a random variable. Monte Carlo Studies The response in a Monte Carlo study is a random variable. The response in a Monte Carlo study has a variance that comes from the variance of the stochastic elements in the data-generating

More information

11. Bootstrap Methods

11. Bootstrap Methods 11. Bootstrap Methods c A. Colin Cameron & Pravin K. Trivedi 2006 These transparencies were prepared in 20043. They can be used as an adjunct to Chapter 11 of our subsequent book Microeconometrics: Methods

More information

The Bootstrap Suppose we draw aniid sample Y 1 ;:::;Y B from a distribution G. Bythe law of large numbers, Y n = 1 B BX j=1 Y j P! Z ydg(y) =E

The Bootstrap Suppose we draw aniid sample Y 1 ;:::;Y B from a distribution G. Bythe law of large numbers, Y n = 1 B BX j=1 Y j P! Z ydg(y) =E 9 The Bootstrap The bootstrap is a nonparametric method for estimating standard errors and computing confidence intervals. Let T n = g(x 1 ;:::;X n ) be a statistic, that is, any function of the data.

More information

Chapter 2: Resampling Maarten Jansen

Chapter 2: Resampling Maarten Jansen Chapter 2: Resampling Maarten Jansen Randomization tests Randomized experiment random assignment of sample subjects to groups Example: medical experiment with control group n 1 subjects for true medicine,

More information

Bootstrap (Part 3) Christof Seiler. Stanford University, Spring 2016, Stats 205

Bootstrap (Part 3) Christof Seiler. Stanford University, Spring 2016, Stats 205 Bootstrap (Part 3) Christof Seiler Stanford University, Spring 2016, Stats 205 Overview So far we used three different bootstraps: Nonparametric bootstrap on the rows (e.g. regression, PCA with random

More information

Simulating Uniform- and Triangular- Based Double Power Method Distributions

Simulating Uniform- and Triangular- Based Double Power Method Distributions Journal of Statistical and Econometric Methods, vol.6, no.1, 2017, 1-44 ISSN: 1792-6602 (print), 1792-6939 (online) Scienpress Ltd, 2017 Simulating Uniform- and Triangular- Based Double Power Method Distributions

More information

One-Sample Numerical Data

One-Sample Numerical Data One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html

More information

Bootstrap, Jackknife and other resampling methods

Bootstrap, Jackknife and other resampling methods Bootstrap, Jackknife and other resampling methods Part III: Parametric Bootstrap Rozenn Dahyot Room 128, Department of Statistics Trinity College Dublin, Ireland dahyot@mee.tcd.ie 2005 R. Dahyot (TCD)

More information

The assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and p-values

The assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and p-values Statistical Consulting Topics The Bootstrap... The bootstrap is a computer-based method for assigning measures of accuracy to statistical estimates. (Efron and Tibshrani, 1998.) What do we do when our

More information

Hypothesis Testing with the Bootstrap. Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods

Hypothesis Testing with the Bootstrap. Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods Hypothesis Testing with the Bootstrap Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods Bootstrap Hypothesis Testing A bootstrap hypothesis test starts with a test statistic

More information

STAT440/840: Statistical Computing

STAT440/840: Statistical Computing First Prev Next Last STAT440/840: Statistical Computing Paul Marriott pmarriott@math.uwaterloo.ca MC 6096 February 2, 2005 Page 1 of 41 First Prev Next Last Page 2 of 41 Chapter 3: Data resampling: the

More information

Bootstrapping the Confidence Intervals of R 2 MAD for Samples from Contaminated Standard Logistic Distribution

Bootstrapping the Confidence Intervals of R 2 MAD for Samples from Contaminated Standard Logistic Distribution Pertanika J. Sci. & Technol. 18 (1): 209 221 (2010) ISSN: 0128-7680 Universiti Putra Malaysia Press Bootstrapping the Confidence Intervals of R 2 MAD for Samples from Contaminated Standard Logistic Distribution

More information

Finite Population Correction Methods

Finite Population Correction Methods Finite Population Correction Methods Moses Obiri May 5, 2017 Contents 1 Introduction 1 2 Normal-based Confidence Interval 2 3 Bootstrap Confidence Interval 3 4 Finite Population Bootstrap Sampling 5 4.1

More information

The exact bootstrap method shown on the example of the mean and variance estimation

The exact bootstrap method shown on the example of the mean and variance estimation Comput Stat (2013) 28:1061 1077 DOI 10.1007/s00180-012-0350-0 ORIGINAL PAPER The exact bootstrap method shown on the example of the mean and variance estimation Joanna Kisielinska Received: 21 May 2011

More information

STAT Section 2.1: Basic Inference. Basic Definitions

STAT Section 2.1: Basic Inference. Basic Definitions STAT 518 --- Section 2.1: Basic Inference Basic Definitions Population: The collection of all the individuals of interest. This collection may be or even. Sample: A collection of elements of the population.

More information

On robust and efficient estimation of the center of. Symmetry.

On robust and efficient estimation of the center of. Symmetry. On robust and efficient estimation of the center of symmetry Howard D. Bondell Department of Statistics, North Carolina State University Raleigh, NC 27695-8203, U.S.A (email: bondell@stat.ncsu.edu) Abstract

More information

TESTING FOR NORMALITY IN THE LINEAR REGRESSION MODEL: AN EMPIRICAL LIKELIHOOD RATIO TEST

TESTING FOR NORMALITY IN THE LINEAR REGRESSION MODEL: AN EMPIRICAL LIKELIHOOD RATIO TEST Econometrics Working Paper EWP0402 ISSN 1485-6441 Department of Economics TESTING FOR NORMALITY IN THE LINEAR REGRESSION MODEL: AN EMPIRICAL LIKELIHOOD RATIO TEST Lauren Bin Dong & David E. A. Giles Department

More information

ST745: Survival Analysis: Nonparametric methods

ST745: Survival Analysis: Nonparametric methods ST745: Survival Analysis: Nonparametric methods Eric B. Laber Department of Statistics, North Carolina State University February 5, 2015 The KM estimator is used ubiquitously in medical studies to estimate

More information

Bootstrapping, Randomization, 2B-PLS

Bootstrapping, Randomization, 2B-PLS Bootstrapping, Randomization, 2B-PLS Statistics, Tests, and Bootstrapping Statistic a measure that summarizes some feature of a set of data (e.g., mean, standard deviation, skew, coefficient of variation,

More information

The automatic construction of bootstrap confidence intervals

The automatic construction of bootstrap confidence intervals The automatic construction of bootstrap confidence intervals Bradley Efron Balasubramanian Narasimhan Abstract The standard intervals, e.g., ˆθ ± 1.96ˆσ for nominal 95% two-sided coverage, are familiar

More information

STAT 830 Non-parametric Inference Basics

STAT 830 Non-parametric Inference Basics STAT 830 Non-parametric Inference Basics Richard Lockhart Simon Fraser University STAT 801=830 Fall 2012 Richard Lockhart (Simon Fraser University)STAT 830 Non-parametric Inference Basics STAT 801=830

More information

MA 575 Linear Models: Cedric E. Ginestet, Boston University Bootstrap for Regression Week 9, Lecture 1

MA 575 Linear Models: Cedric E. Ginestet, Boston University Bootstrap for Regression Week 9, Lecture 1 MA 575 Linear Models: Cedric E. Ginestet, Boston University Bootstrap for Regression Week 9, Lecture 1 1 The General Bootstrap This is a computer-intensive resampling algorithm for estimating the empirical

More information

Supporting Information for Estimating restricted mean. treatment effects with stacked survival models

Supporting Information for Estimating restricted mean. treatment effects with stacked survival models Supporting Information for Estimating restricted mean treatment effects with stacked survival models Andrew Wey, David Vock, John Connett, and Kyle Rudser Section 1 presents several extensions to the simulation

More information

A Simulation Comparison Study for Estimating the Process Capability Index C pm with Asymmetric Tolerances

A Simulation Comparison Study for Estimating the Process Capability Index C pm with Asymmetric Tolerances Available online at ijims.ms.tku.edu.tw/list.asp International Journal of Information and Management Sciences 20 (2009), 243-253 A Simulation Comparison Study for Estimating the Process Capability Index

More information

Multivariate Distributions

Multivariate Distributions IEOR E4602: Quantitative Risk Management Spring 2016 c 2016 by Martin Haugh Multivariate Distributions We will study multivariate distributions in these notes, focusing 1 in particular on multivariate

More information

12 - Nonparametric Density Estimation

12 - Nonparametric Density Estimation ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6

More information

Nonparametric Methods II

Nonparametric Methods II Nonparametric Methods II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm 1 PART 3: Statistical Inference by

More information

Post-exam 2 practice questions 18.05, Spring 2014

Post-exam 2 practice questions 18.05, Spring 2014 Post-exam 2 practice questions 18.05, Spring 2014 Note: This is a set of practice problems for the material that came after exam 2. In preparing for the final you should use the previous review materials,

More information

Bayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University

Bayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University Bayesian Inference and the Parametric Bootstrap Bradley Efron Stanford University Importance Sampling for Bayes Posterior Distribution Newton and Raftery (1994 JRSS-B) Nonparametric Bootstrap: good choice

More information

A better way to bootstrap pairs

A better way to bootstrap pairs A better way to bootstrap pairs Emmanuel Flachaire GREQAM - Université de la Méditerranée CORE - Université Catholique de Louvain April 999 Abstract In this paper we are interested in heteroskedastic regression

More information

Lecture 30. DATA 8 Summer Regression Inference

Lecture 30. DATA 8 Summer Regression Inference DATA 8 Summer 2018 Lecture 30 Regression Inference Slides created by John DeNero (denero@berkeley.edu) and Ani Adhikari (adhikari@berkeley.edu) Contributions by Fahad Kamran (fhdkmrn@berkeley.edu) and

More information

Answers and expectations

Answers and expectations Answers and expectations For a function f(x) and distribution P(x), the expectation of f with respect to P is The expectation is the average of f, when x is drawn from the probability distribution P E

More information

Permutation Tests. Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods

Permutation Tests. Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods Permutation Tests Noa Haas Statistics M.Sc. Seminar, Spring 2017 Bootstrap and Resampling Methods The Two-Sample Problem We observe two independent random samples: F z = z 1, z 2,, z n independently of

More information

MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2

MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 1 Bootstrapped Bias and CIs Given a multiple regression model with mean and

More information

AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY

AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY Econometrics Working Paper EWP0401 ISSN 1485-6441 Department of Economics AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY Lauren Bin Dong & David E. A. Giles Department of Economics, University of Victoria

More information

Statistical Inference: Estimation and Confidence Intervals Hypothesis Testing

Statistical Inference: Estimation and Confidence Intervals Hypothesis Testing Statistical Inference: Estimation and Confidence Intervals Hypothesis Testing 1 In most statistics problems, we assume that the data have been generated from some unknown probability distribution. We desire

More information

Fin285a:Computer Simulations and Risk Assessment Section 2.3.2:Hypothesis testing, and Confidence Intervals

Fin285a:Computer Simulations and Risk Assessment Section 2.3.2:Hypothesis testing, and Confidence Intervals Fin285a:Computer Simulations and Risk Assessment Section 2.3.2:Hypothesis testing, and Confidence Intervals Overview Hypothesis testing terms Testing a die Testing issues Estimating means Confidence intervals

More information

Lecture 1: Random number generation, permutation test, and the bootstrap. August 25, 2016

Lecture 1: Random number generation, permutation test, and the bootstrap. August 25, 2016 Lecture 1: Random number generation, permutation test, and the bootstrap August 25, 2016 Statistical simulation 1/21 Statistical simulation (Monte Carlo) is an important part of statistical method research.

More information

Confidence intervals for kernel density estimation

Confidence intervals for kernel density estimation Stata User Group - 9th UK meeting - 19/20 May 2003 Confidence intervals for kernel density estimation Carlo Fiorio c.fiorio@lse.ac.uk London School of Economics and STICERD Stata User Group - 9th UK meeting

More information

Political Science 236 Hypothesis Testing: Review and Bootstrapping

Political Science 236 Hypothesis Testing: Review and Bootstrapping Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall 2007 1 Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The

More information

Investigation of Possible Biases in Tau Neutrino Mass Limits

Investigation of Possible Biases in Tau Neutrino Mass Limits Investigation of Possible Biases in Tau Neutrino Mass Limits Kyle Armour Departments of Physics and Mathematics, University of California, San Diego, La Jolla, CA 92093 (Dated: August 8, 2003) We study

More information

Bootstrap tests. Patrick Breheny. October 11. Bootstrap vs. permutation tests Testing for equality of location

Bootstrap tests. Patrick Breheny. October 11. Bootstrap vs. permutation tests Testing for equality of location Bootstrap tests Patrick Breheny October 11 Patrick Breheny STA 621: Nonparametric Statistics 1/14 Introduction Conditioning on the observed data to obtain permutation tests is certainly an important idea

More information

Estimation of AUC from 0 to Infinity in Serial Sacrifice Designs

Estimation of AUC from 0 to Infinity in Serial Sacrifice Designs Estimation of AUC from 0 to Infinity in Serial Sacrifice Designs Martin J. Wolfsegger Department of Biostatistics, Baxter AG, Vienna, Austria Thomas Jaki Department of Statistics, University of South Carolina,

More information

THE EFFECTS OF NONNORMAL DISTRIBUTIONS ON CONFIDENCE INTERVALS AROUND THE STANDARDIZED MEAN DIFFERENCE: BOOTSTRAP AND PARAMETRIC CONFIDENCE INTERVALS

THE EFFECTS OF NONNORMAL DISTRIBUTIONS ON CONFIDENCE INTERVALS AROUND THE STANDARDIZED MEAN DIFFERENCE: BOOTSTRAP AND PARAMETRIC CONFIDENCE INTERVALS EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 10.1177/0013164404264850 KELLEY THE EFFECTS OF NONNORMAL DISTRIBUTIONS ON CONFIDENCE INTERVALS AROUND THE STANDARDIZED MEAN DIFFERENCE: BOOTSTRAP AND PARAMETRIC

More information

Resampling and the Bootstrap

Resampling and the Bootstrap Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D-69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing

More information

36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1

36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1 36. Multisample U-statistics jointly distributed U-statistics Lehmann 6.1 In this topic, we generalize the idea of U-statistics in two different directions. First, we consider single U-statistics for situations

More information

A note on multivariate Gauss-Hermite quadrature

A note on multivariate Gauss-Hermite quadrature A note on multivariate Gauss-Hermite quadrature Peter Jäckel 1th May 5 1 Introduction Gaussian quadratures are an ingenious way to approximate the integral of an unknown function f(x) over a specified

More information

On the Accuracy of Bootstrap Confidence Intervals for Efficiency Levels in Stochastic Frontier Models with Panel Data

On the Accuracy of Bootstrap Confidence Intervals for Efficiency Levels in Stochastic Frontier Models with Panel Data On the Accuracy of Bootstrap Confidence Intervals for Efficiency Levels in Stochastic Frontier Models with Panel Data Myungsup Kim University of North Texas Yangseon Kim East-West Center Peter Schmidt

More information

The Union and Intersection for Different Configurations of Two Events Mutually Exclusive vs Independency of Events

The Union and Intersection for Different Configurations of Two Events Mutually Exclusive vs Independency of Events Section 1: Introductory Probability Basic Probability Facts Probabilities of Simple Events Overview of Set Language Venn Diagrams Probabilities of Compound Events Choices of Events The Addition Rule Combinations

More information

Introduction to statistics. Laurent Eyer Maria Suveges, Marie Heim-Vögtlin SNSF grant

Introduction to statistics. Laurent Eyer Maria Suveges, Marie Heim-Vögtlin SNSF grant Introduction to statistics Laurent Eyer Maria Suveges, Marie Heim-Vögtlin SNSF grant Recent history at the Observatory Request of something on statistics from PhD students, because of an impression of

More information

Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, What about continuous variables?

Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, What about continuous variables? Linear Regression Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2014 1 What about continuous variables? n Billionaire says: If I am measuring a continuous variable, what

More information

Chapter 6. Estimation of Confidence Intervals for Nodal Maximum Power Consumption per Customer

Chapter 6. Estimation of Confidence Intervals for Nodal Maximum Power Consumption per Customer Chapter 6 Estimation of Confidence Intervals for Nodal Maximum Power Consumption per Customer The aim of this chapter is to calculate confidence intervals for the maximum power consumption per customer

More information

A Non-parametric bootstrap for multilevel models

A Non-parametric bootstrap for multilevel models A Non-parametric bootstrap for multilevel models By James Carpenter London School of Hygiene and ropical Medicine Harvey Goldstein and Jon asbash Institute of Education 1. Introduction Bootstrapping is

More information

inferences on stress-strength reliability from lindley distributions

inferences on stress-strength reliability from lindley distributions inferences on stress-strength reliability from lindley distributions D.K. Al-Mutairi, M.E. Ghitany & Debasis Kundu Abstract This paper deals with the estimation of the stress-strength parameter R = P (Y

More information

Machine Learning CSE546 Sham Kakade University of Washington. Oct 4, What about continuous variables?

Machine Learning CSE546 Sham Kakade University of Washington. Oct 4, What about continuous variables? Linear Regression Machine Learning CSE546 Sham Kakade University of Washington Oct 4, 2016 1 What about continuous variables? Billionaire says: If I am measuring a continuous variable, what can you do

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

A Bootstrap Test for Conditional Symmetry

A Bootstrap Test for Conditional Symmetry ANNALS OF ECONOMICS AND FINANCE 6, 51 61 005) A Bootstrap Test for Conditional Symmetry Liangjun Su Guanghua School of Management, Peking University E-mail: lsu@gsm.pku.edu.cn and Sainan Jin Guanghua School

More information

Nonparametric confidence intervals. for receiver operating characteristic curves

Nonparametric confidence intervals. for receiver operating characteristic curves Nonparametric confidence intervals for receiver operating characteristic curves Peter G. Hall 1, Rob J. Hyndman 2, and Yanan Fan 3 5 December 2003 Abstract: We study methods for constructing confidence

More information

UNIVERSITÄT POTSDAM Institut für Mathematik

UNIVERSITÄT POTSDAM Institut für Mathematik UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam

More information

Bootstrap confidence levels for phylogenetic trees B. Efron, E. Halloran, and S. Holmes, 1996

Bootstrap confidence levels for phylogenetic trees B. Efron, E. Halloran, and S. Holmes, 1996 Bootstrap confidence levels for phylogenetic trees B. Efron, E. Halloran, and S. Holmes, 1996 Following Confidence limits on phylogenies: an approach using the bootstrap, J. Felsenstein, 1985 1 I. Short

More information

BOOTSTRAP CONFIDENCE INTERVALS FOR VARIANCE COMPONENTS IN THE UNBALANCED RANDOM ONE WAY MODEL M.A.T. ElshahaT 1 and E.H.

BOOTSTRAP CONFIDENCE INTERVALS FOR VARIANCE COMPONENTS IN THE UNBALANCED RANDOM ONE WAY MODEL M.A.T. ElshahaT 1 and E.H. BOOTSTRAP CONFIDENCE INTERVALS FOR VARIANCE COMPONENTS IN THE UNBALANCED RANDOM ONE WAY MODEL MAT ElshahaT 1 and EH Abd El-Sattar 2 1 Faculty of Business, University of Jeddah, Kingdom of Saudi Arabia

More information

5.6 The Normal Distributions

5.6 The Normal Distributions STAT 41 Lecture Notes 13 5.6 The Normal Distributions Definition 5.6.1. A (continuous) random variable X has a normal distribution with mean µ R and variance < R if the p.d.f. of X is f(x µ, ) ( π ) 1/

More information

Robust Outcome Analysis for Observational Studies Designed Using Propensity Score Matching

Robust Outcome Analysis for Observational Studies Designed Using Propensity Score Matching The work of Kosten and McKean was partially supported by NIAAA Grant 1R21AA017906-01A1 Robust Outcome Analysis for Observational Studies Designed Using Propensity Score Matching Bradley E. Huitema Western

More information

Resampling and the Bootstrap

Resampling and the Bootstrap Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D-69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing

More information

Asymptotic distribution of the sample average value-at-risk

Asymptotic distribution of the sample average value-at-risk Asymptotic distribution of the sample average value-at-risk Stoyan V. Stoyanov Svetlozar T. Rachev September 3, 7 Abstract In this paper, we prove a result for the asymptotic distribution of the sample

More information

Contents 1. Contents

Contents 1. Contents Contents 1 Contents 1 One-Sample Methods 3 1.1 Parametric Methods.................... 4 1.1.1 One-sample Z-test (see Chapter 0.3.1)...... 4 1.1.2 One-sample t-test................. 6 1.1.3 Large sample

More information

ON THE NUMBER OF BOOTSTRAP REPETITIONS FOR BC a CONFIDENCE INTERVALS. DONALD W. K. ANDREWS and MOSHE BUCHINSKY COWLES FOUNDATION PAPER NO.

ON THE NUMBER OF BOOTSTRAP REPETITIONS FOR BC a CONFIDENCE INTERVALS. DONALD W. K. ANDREWS and MOSHE BUCHINSKY COWLES FOUNDATION PAPER NO. ON THE NUMBER OF BOOTSTRAP REPETITIONS FOR BC a CONFIDENCE INTERVALS BY DONALD W. K. ANDREWS and MOSHE BUCHINSKY COWLES FOUNDATION PAPER NO. 1069 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY

More information

Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference

Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference 1 / 171 Bootstrap inference Francisco Cribari-Neto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil email: cribari@gmail.com October 2013 2 / 171 Unpaid advertisement

More information

Power Calculations for Preclinical Studies Using a K-Sample Rank Test and the Lehmann Alternative Hypothesis

Power Calculations for Preclinical Studies Using a K-Sample Rank Test and the Lehmann Alternative Hypothesis Power Calculations for Preclinical Studies Using a K-Sample Rank Test and the Lehmann Alternative Hypothesis Glenn Heller Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center,

More information

Model-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego

Model-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships

More information

Terminology Suppose we have N observations {x(n)} N 1. Estimators as Random Variables. {x(n)} N 1

Terminology Suppose we have N observations {x(n)} N 1. Estimators as Random Variables. {x(n)} N 1 Estimation Theory Overview Properties Bias, Variance, and Mean Square Error Cramér-Rao lower bound Maximum likelihood Consistency Confidence intervals Properties of the mean estimator Properties of the

More information

(NRH: Sections 2.6, 2.7, 2.11, 2.12 (at this point in the course the sections will be difficult to follow))

(NRH: Sections 2.6, 2.7, 2.11, 2.12 (at this point in the course the sections will be difficult to follow)) Curriculum, second lecture: Niels Richard Hansen November 23, 2011 NRH: Handout pages 1-13 PD: Pages 55-75 (NRH: Sections 2.6, 2.7, 2.11, 2.12 (at this point in the course the sections will be difficult

More information

Basic Computations in Statistical Inference

Basic Computations in Statistical Inference 1 Basic Computations in Statistical Inference The purpose of an exploration of data may be rather limited and ad hoc, or the purpose may be more general, perhaps to gain understanding of some natural phenomenon.

More information

EXPLICIT NONPARAMETRIC CONFIDENCE INTERVALS FOR THE VARIANCE WITH GUARANTEED COVERAGE

EXPLICIT NONPARAMETRIC CONFIDENCE INTERVALS FOR THE VARIANCE WITH GUARANTEED COVERAGE EXPLICIT NONPARAMETRIC CONFIDENCE INTERVALS FOR THE VARIANCE WITH GUARANTEED COVERAGE Joseph P. Romano Department of Statistics Stanford University Stanford, California 94305 romano@stat.stanford.edu Michael

More information

Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data

Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data Journal of Modern Applied Statistical Methods Volume 13 Issue 2 Article 22 11-2014 Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data Jayanthi Arasan University Putra Malaysia,

More information

Generating the Sample

Generating the Sample STAT 80: Mathematical Statistics Monte Carlo Suppose you are given random variables X,..., X n whose joint density f (or distribution) is specified and a statistic T (X,..., X n ) whose distribution you

More information

Random Numbers and Simulation

Random Numbers and Simulation Random Numbers and Simulation Generating random numbers: Typically impossible/unfeasible to obtain truly random numbers Programs have been developed to generate pseudo-random numbers: Values generated

More information

Gaussians Linear Regression Bias-Variance Tradeoff

Gaussians Linear Regression Bias-Variance Tradeoff Readings listed in class website Gaussians Linear Regression Bias-Variance Tradeoff Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University January 22 nd, 2007 Maximum Likelihood Estimation

More information

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).

More information

Ø Set of mutually exclusive categories. Ø Classify or categorize subject. Ø No meaningful order to categorization.

Ø Set of mutually exclusive categories. Ø Classify or categorize subject. Ø No meaningful order to categorization. Statistical Tools in Evaluation HPS 41 Dr. Joe G. Schmalfeldt Types of Scores Continuous Scores scores with a potentially infinite number of values. Discrete Scores scores limited to a specific number

More information

Some Assorted Formulae. Some confidence intervals: σ n. x ± z α/2. x ± t n 1;α/2 n. ˆp(1 ˆp) ˆp ± z α/2 n. χ 2 n 1;1 α/2. n 1;α/2

Some Assorted Formulae. Some confidence intervals: σ n. x ± z α/2. x ± t n 1;α/2 n. ˆp(1 ˆp) ˆp ± z α/2 n. χ 2 n 1;1 α/2. n 1;α/2 STA 248 H1S MIDTERM TEST February 26, 2008 SURNAME: SOLUTIONS GIVEN NAME: STUDENT NUMBER: INSTRUCTIONS: Time: 1 hour and 50 minutes Aids allowed: calculator Tables of the standard normal, t and chi-square

More information

Eco517 Fall 2004 C. Sims MIDTERM EXAM

Eco517 Fall 2004 C. Sims MIDTERM EXAM Eco517 Fall 2004 C. Sims MIDTERM EXAM Answer all four questions. Each is worth 23 points. Do not devote disproportionate time to any one question unless you have answered all the others. (1) We are considering

More information

Does k-th Moment Exist?

Does k-th Moment Exist? Does k-th Moment Exist? Hitomi, K. 1 and Y. Nishiyama 2 1 Kyoto Institute of Technology, Japan 2 Institute of Economic Research, Kyoto University, Japan Email: hitomi@kit.ac.jp Keywords: Existence of moments,

More information

Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring

Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring A. Ganguly, S. Mitra, D. Samanta, D. Kundu,2 Abstract Epstein [9] introduced the Type-I hybrid censoring scheme

More information

Bayes: All uncertainty is described using probability.

Bayes: All uncertainty is described using probability. Bayes: All uncertainty is described using probability. Let w be the data and θ be any unknown quantities. Likelihood. The probability model π(w θ) has θ fixed and w varying. The likelihood L(θ; w) is π(w

More information

Frank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c /9/9 page 78 le-tex

Frank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c /9/9 page 78 le-tex Frank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c04 203/9/9 page 78 le-tex 78 4 Resampling Techniques.2 bootstrap interval bounds 0.8 0.6 0.4 0.2 0 0 200 400 600

More information

Chapter 7. Data Partitioning. 7.1 Introduction

Chapter 7. Data Partitioning. 7.1 Introduction Chapter 7 Data Partitioning 7.1 Introduction In this book, data partitioning refers to procedures where some observations from the sample are removed as part of the analysis. These techniques are used

More information

POLI 8501 Introduction to Maximum Likelihood Estimation

POLI 8501 Introduction to Maximum Likelihood Estimation POLI 8501 Introduction to Maximum Likelihood Estimation Maximum Likelihood Intuition Consider a model that looks like this: Y i N(µ, σ 2 ) So: E(Y ) = µ V ar(y ) = σ 2 Suppose you have some data on Y,

More information

Statistics 3858 : Maximum Likelihood Estimators

Statistics 3858 : Maximum Likelihood Estimators Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,

More information

Bootstrap, Jackknife and other resampling methods

Bootstrap, Jackknife and other resampling methods Bootstrap, Jackknife and other resampling methods Part VI: Cross-validation Rozenn Dahyot Room 128, Department of Statistics Trinity College Dublin, Ireland dahyot@mee.tcd.ie 2005 R. Dahyot (TCD) 453 Modern

More information

A Monte-Carlo study of asymptotically robust tests for correlation coefficients

A Monte-Carlo study of asymptotically robust tests for correlation coefficients Biometrika (1973), 6, 3, p. 661 551 Printed in Great Britain A Monte-Carlo study of asymptotically robust tests for correlation coefficients BY G. T. DUNCAN AND M. W. J. LAYAKD University of California,

More information

Model Selection, Estimation, and Bootstrap Smoothing. Bradley Efron Stanford University

Model Selection, Estimation, and Bootstrap Smoothing. Bradley Efron Stanford University Model Selection, Estimation, and Bootstrap Smoothing Bradley Efron Stanford University Estimation After Model Selection Usually: (a) look at data (b) choose model (linear, quad, cubic...?) (c) fit estimates

More information

5 Introduction to the Theory of Order Statistics and Rank Statistics

5 Introduction to the Theory of Order Statistics and Rank Statistics 5 Introduction to the Theory of Order Statistics and Rank Statistics This section will contain a summary of important definitions and theorems that will be useful for understanding the theory of order

More information