Political Science 236 Hypothesis Testing: Review and Bootstrapping

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Political Science 236 Hypothesis Testing: Review and Bootstrapping"

Transcription

1 Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The goal of hypothesis testing is to decide, using a sample from the population, which of two complimentary hypotheses is true. In general, the two complimentary hypotheses are called the null hypothesis and the alternative hypothesis. If we let θ be a population parameter and Θ be the parameter space, we can define these complementary hypotheses as follows: Definition 1.2 Let Θ 0 and Θ 1 Θ c 0 alternative hypothesis are defined as follows be a partition of the parameter space Θ. Then the null and 1. Null Hypothesis: H 0 : θ Θ 0 2. Alternative Hypothesis: H 1 : θ Θ 1 Definition 1.3 Testing Procedure. A testing procedure is a rule, based on the outcome of a random sample from the population under study, used to decide whether to reject H 0. 1

2 The subset of the sample space for which H 0 will be rejected is called the critical region ( or the rejection region), and its complement is called the acceptance region. In general, a hypothesis test will be specified in terms of a test statistic T (X 1, X 2,..., X N ) T (X), which is a function of the sample. We can define the critical region formally as follows. Definition 1.4 Critical Region. The subset C R N of the sample space for which H 0 is rejected is called the critical region and is defined by C c = { x R N : T (x) > c } for some c R. The value c is called the critical value. The complement of C c, C a C c c, is called the acceptance region. If we let C T c be the critical region of the test statistic T (X) (i.e. C T c is defined by C c = { x R N : T (x) C T } c ), a statistical test of H0 against H 1 will generally be defined as: 1. T (x) C T c = Reject H 0 T (x) / C T c = Accept H 0 A hypothesis test of H 0 : θ Θ 0 against H 1 : θ Θ 1 can make one of two types of errors. Definition 1.5 Type I and Type II Errors. Let H 0 be a null hypothesis being tested for acceptance or rejection. The two types of errors that can be made are 1. Type I Error: rejecting H 0 when θ Θ 0 (i.e, when H 0 is true) 2. Type II Error: accepting H 0 when θ Θ 1 (i.e, when H 0 is false) So a type I error is committed when the statistical test mistakenly rejects the null hypothesis, and a type II error is committed when the test mistakenly accepts the null hypothesis. The ideal 2

3 test is one where the hypothesis would always be correctly identified as being either true or false. For such an ideal test to exist, we must partition the range of potential sample outcomes in such a way that outcomes in the critical region C c would occur if and only if H 0 were true and outcomes in the acceptance region C a would occur if and only if H 0 were false. In general, ideal tests cannot be constructed. For θ Θ 0, the test will make a mistake if x C c and therefore the probability of a type I error is P θ (X C c ) and for θ Θ 1, the test will make a mistake if x C a and therefore the probability of a type II error is P θ (X C a ). Note that P θ (X C c ) = 1 P θ (X C a ). We will now define the power function of a test. The power function completely summarizes all of the operating characteristics of a statistical test with respect to probabilities of making correct and incorrect decisions about H 0. The power function is defined below. Definition 1.6 Let H 0 be defined as H 0 : θ Θ 0 and H 1 be defined as H 1 : θ Θ 1. Let the critical region C c define a test of H 0. Then the power function of the statistical test is the function of θ defined by β (θ) P θ (X C c ) = probability of Type I error if θ Θ 0 one minus probability of Type II error if θ Θ 1 In words, the power function indicates the probability of rejecting H 0 for every value of θ Θ. The value of the power function at a particular value of the parameter space θ p Θ is called the power of the test at θ p and represents the probability of rejecting H 0 if θ p were the true value of the parameter vector. The ideal power function is 0 for all θ Θ 0 and 1 for all θ Θ 1. In general, this ideal cannot be attained and we say that a good test has power function near 0 for all θ Θ 0 and near 1 for all θ Θ 1. When comparing two tests for a given H 0, a test is better if it has lower power for θ Θ 0 and higher power for θ Θ 1 which implies that the test has lower probabilities of both type I and type II error. We now define the size and level of a test: 3

4 Definition 1.7 Size. For 0 α 1, a test with power function β (θ) is a size-α test if sup θ Θ0 β (θ) = α Definition 1.8 Level. sup θ Θ0 β (θ) α For 0 α 1, a test with power function β (θ) is a level-α test if In words, the size of the test is the maximum probability of Type I error associated with a given test rule. The lower the size of the test, the lower the maximum probability of mistakenly rejecting H 0. The level of a test is an upper bound to the type I error probability of a statistical test. The key difference between these two concepts is that the size represents the maximum value of β (θ) for θ Θ 0 (i.e. the maximum type I error) while the level is only a bound that might not equal β (θ) for any θ Θ 0 nor equal the supremum of β (θ) for θ Θ 0. Thus, the set of level-α tests contains the set of size-α tests. In other words, a test of H 0 having size γ is a α-level test for any α γ. In applications, when we say that H 0 is (not) rejected at the α-significance level, we often mean that α was the bound on the level of protection against type I error that was used when constructing the test. A more accurate statement is regarding the level of protection against type I error is that H 0 is (not) rejected using a size-α test. 2 Bootstrapping Hypothesis Tests The simplest situation involves a simple null hypothesis H 0 that completely specifies the probability distribution of the data. Thus, if we have a sample x 1, x 2,..., x n from a population with CDF F, then H 0 specifies that F = F 0 where F 0 contains no unknown parameters. A statistical test is based on a test statistic T which measures the discrepancy between the data and the null hypothesis. We will follow the convention that large values of T are evidence against H 0. If the null hypothesis is simple and the observed value of the test statistics is denoted by t, then the level of evidence 4

5 against H 0 is measured by the significance probability p = P (T t H 0 ) which is referred to as the p-value. The p-value is effectively the marginal size test at which a given hypothesis would be rejected based on the observed outcome of X. A corresponding notion is that of a critical value t p for t, which is associated with testing at level p: if t t p then H 0 is rejected at level p or 100p%. It follows that t p is defined as P (T t p H 0 ) = p Note that p is what we defined earlier as the size of the test and the set {(x 1, x 2,..., x n ) : t t p H 0 } is the level p critical region of the test. distribution of T. The distribution of the T under H 0 is called the null 2.1 How to choose the test-statistic In a parametric setting, there is an explicit form of the sampling distribution of the data with a finite number of unknown parameters. In these cases the alternative hypothesis guides the choice of the test statistic (usually through use of the likelihood function of the data). In non-parametric settings, no particular forms are specified for the distributions and hence the appropriate choice of T is less clear. However, the choice of T should be always based on some notion of what is of concern in the case that H 0 turns out to be false. In all non-parametric problems, the null hypothesis H 0 leaves some parameters unknown and therefore does not completely specify F. In this case, the p-value is not well defined because P (T t F ) may depend upon which F satisfying H 0 is taken Pivot Tests When H 0 concerns a particular parameter value, we can use the equivalence between hypothesis tests and confidence intervals. This equivalence implies that if the value of θ 0 is outside a 1 α 5

6 confidence interval for θ, then θ differs from θ 0 with p-value less than α. A specific form of test based on this equivalence is a pivot test. Suppose that T is an estimator for a scalar θ, with estimated variance V. Suppose also that the studentized version of T, Z = T θ, is a pivot (i.e. V 1/2 its distribution is the same for all relevant F, and in particular for all θ). For a one-sided test of H 0 : θ = θ 0 versus H 1 : θ > θ 0, the p-value that corresponds to the observed studentized test statistic z 0 = t θ 0 v 1/2 is p = P However, since Z is a pivot we have P { T θ0 V 1/2 t θ } 0 v 1/2 H 0 and therefore the p-value can be written as { T θ0 V 1/2 t θ } 0 v 1/2 H 0 { = P Z t θ } 0 v 1/2 H 0 { = P Z t θ } 0 v 1/2 F p = P {Z z 0 F } Note that this has a big advantage in the context of bootstrapping, because we do not have to construct a special null-hypothesis sampling distribution. 2.2 Non-Parametric Bootstrap Tests Testing hypothesis requires that probability calculations be done under the null hypothesis model. This means that the usual bootstrap setting must be modified, since resampling from the empirical CDF F and applying the plug-in principle to obtain θ ( ) = t F won t give us an estimator of θ under the null hypothesis H 0. In the hypothesis testing context, instead of resampling from the empirical CDF F, we must resample from an empirical CDF F 0 which satisfies the relevant null hypothesis H 0. (Unless, as we mentioned above, we can construct a pivot test-statistic). 6

7 Once we have decided on the null resampling distribution F 0, the basic bootstrap test will compute the p-value as or will approximate it by using the results t 1, t 2,..., t B p boot = P { T t F 0 } p boot = # {t b t} B from B bootstrap samples. Example 2.1 Difference in means. Suppose we want to compare two population means µ 1 and µ 2 using the test statistic t = x 1 x 2. We will use the following sample data: sample sample If the shapes of the underlying distributions are identical, then under H 0 : µ 1 = µ 2 the two distributions are the same. In this case, it is sensible to choose for F 0 the pooled empirical CDF of the two samples. Applying this procedure with 1, 000 bootstrap samples yielded 52 values of t greater than the observed value t = = 2.84, which implies a p-value of cannot reject the null at 5% (but we can at 5.2%!!) = So we Studentized Bootstrap Method For some problems, it is possible to obtain more stable significance tests by studentizing comparisons. Remember that because of the relationship between confidence sets and hypothesis tests, such a test can be obtained calculating a 1 p confidence set by the studentized bootstrap method and concluding that the p-value is less than p is the null hypothesis parameter falls outside the confidence set. We can also implement this idea by bootstrapping the test statistic directly rather than constructing confidence intervals. In this case, the p-value can be obtained directly. Suppose that θ is 7

8 a scalar with estimator T and that we want to test H 0 : θ = θ 0 against H 1 : θ > θ 0. The method we mentioned in the section Pivot Tests applies when Z = T θ V 1/2 is approximately a pivot (i.e. its distribution is approximately independent of unknown parameters). Then, with z 0 = t θ 0 of v 1/2 being the observed studentized test statistic the bootstrap analog p = P {Z z 0 F } is p = P { Z z 0 F } which we can approximate by bootstrapping without having to decide on a null empirical distribution F 0. Example 2.2 Let s continue the example of the difference in means. We were comparing compare two population means µ 1 and µ 2 using the test statistic t = x 1 x 2. Now, it would reasonable to suppose that the usual two-sample t-statistic Z = X 2 X 1 (µ 2 µ 1 ) ( S 2 2 /n 2 + S 2 1 /n 1) 1/2 is approximately pivotal. We take F to be the empirical CDF of the two samples taken together, provided that no assumptions are made connecting the two distributions. The observed value of the test statistic under the null is We also calculate B values of z 0 = x 2 x 1 ( s 2 2 /n 2 + s 2 1 /n 1) 1/2 z = x 2 x 1 (x 2 x 1 ) ( s 2 2 /n 2 + s 2 1 /n 1 ) 1/2 8

9 3 Testing Linear Restrictions in OLS Consider the problem of testing the following null hypothesis H 0 : Rβ = r where the d K matrix R is matrix of restrictions (where d is the number of restrictions) and r is a p 1 vector of constants. The alternative hypothesis is H 1 : Rβ r. Using standard results from multivariate normal distributions, we now that T 1 ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r T 2 and hence we have pivotal statistic given by σ 2 ( ) T ( ) y X β y X β σ 2 T 1 T 2 χ 2 N K χ 2 d F = = (Rβ r) b T R(X T X) 1 1 R T (Rβ r) b 1 σ 2 d (y Xβ) b T (y Xβ) b 1 σ 2 N r ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r 1 d ( ) T ( ) 1 y X β y X β N K ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r ds 2 F d,n K References Davidson, A. C. and D.V. Hinkley, Bootstrap Methods and their Application. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press. 9

Statistical Inference

Statistical Inference Statistical Inference Classical and Bayesian Methods Class 6 AMS-UCSC Thu 26, 2012 Winter 2012. Session 1 (Class 6) AMS-132/206 Thu 26, 2012 1 / 15 Topics Topics We will talk about... 1 Hypothesis testing

More information

INTERVAL ESTIMATION AND HYPOTHESES TESTING

INTERVAL ESTIMATION AND HYPOTHESES TESTING INTERVAL ESTIMATION AND HYPOTHESES TESTING 1. IDEA An interval rather than a point estimate is often of interest. Confidence intervals are thus important in empirical work. To construct interval estimates,

More information

Ch. 5 Hypothesis Testing

Ch. 5 Hypothesis Testing Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,

More information

CH.9 Tests of Hypotheses for a Single Sample

CH.9 Tests of Hypotheses for a Single Sample CH.9 Tests of Hypotheses for a Single Sample Hypotheses testing Tests on the mean of a normal distributionvariance known Tests on the mean of a normal distributionvariance unknown Tests on the variance

More information

LECTURE 5. Introduction to Econometrics. Hypothesis testing

LECTURE 5. Introduction to Econometrics. Hypothesis testing LECTURE 5 Introduction to Econometrics Hypothesis testing October 18, 2016 1 / 26 ON TODAY S LECTURE We are going to discuss how hypotheses about coefficients can be tested in regression models We will

More information

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3 Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest

More information

hypothesis a claim about the value of some parameter (like p)

hypothesis a claim about the value of some parameter (like p) Testing hypotheses hypothesis a claim about the value of some parameter (like p) significance test procedure to assess the strength of evidence provided by a sample of data against the claim of a hypothesized

More information

Quantitative Methods for Economics, Finance and Management (A86050 F86050)

Quantitative Methods for Economics, Finance and Management (A86050 F86050) Quantitative Methods for Economics, Finance and Management (A86050 F86050) Matteo Manera matteo.manera@unimib.it Marzio Galeotti marzio.galeotti@unimi.it 1 This material is taken and adapted from Guy Judge

More information

Sampling Distributions: Central Limit Theorem

Sampling Distributions: Central Limit Theorem Review for Exam 2 Sampling Distributions: Central Limit Theorem Conceptually, we can break up the theorem into three parts: 1. The mean (µ M ) of a population of sample means (M) is equal to the mean (µ)

More information

Bootstrap tests. Patrick Breheny. October 11. Bootstrap vs. permutation tests Testing for equality of location

Bootstrap tests. Patrick Breheny. October 11. Bootstrap vs. permutation tests Testing for equality of location Bootstrap tests Patrick Breheny October 11 Patrick Breheny STA 621: Nonparametric Statistics 1/14 Introduction Conditioning on the observed data to obtain permutation tests is certainly an important idea

More information

4 Hypothesis testing. 4.1 Types of hypothesis and types of error 4 HYPOTHESIS TESTING 49

4 Hypothesis testing. 4.1 Types of hypothesis and types of error 4 HYPOTHESIS TESTING 49 4 HYPOTHESIS TESTING 49 4 Hypothesis testing In sections 2 and 3 we considered the problem of estimating a single parameter of interest, θ. In this section we consider the related problem of testing whether

More information

18.05 Final Exam. Good luck! Name. No calculators. Number of problems 16 concept questions, 16 problems, 21 pages

18.05 Final Exam. Good luck! Name. No calculators. Number of problems 16 concept questions, 16 problems, 21 pages Name No calculators. 18.05 Final Exam Number of problems 16 concept questions, 16 problems, 21 pages Extra paper If you need more space we will provide some blank paper. Indicate clearly that your solution

More information

MTMS Mathematical Statistics

MTMS Mathematical Statistics MTMS.01.099 Mathematical Statistics Lecture 12. Hypothesis testing. Power function. Approximation of Normal distribution and application to Binomial distribution Tõnu Kollo Fall 2016 Hypothesis Testing

More information

14.30 Introduction to Statistical Methods in Economics Spring 2009

14.30 Introduction to Statistical Methods in Economics Spring 2009 MIT OpenCourseWare http://ocw.mit.edu 4.0 Introduction to Statistical Methods in Economics Spring 009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing. Road Map Sampling Distributions, Confidence Intervals & Hypothesis Testing

Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing. Road Map Sampling Distributions, Confidence Intervals & Hypothesis Testing Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing ECO22Y5Y: Quantitative Methods in Economics Dr. Nick Zammit University of Toronto Department of Economics Room KN3272 n.zammit

More information

Hypothesis Testing. We normally talk about two types of hypothesis: the null hypothesis and the research or alternative hypothesis.

Hypothesis Testing. We normally talk about two types of hypothesis: the null hypothesis and the research or alternative hypothesis. Hypothesis Testing Today, we are going to begin talking about the idea of hypothesis testing how we can use statistics to show that our causal models are valid or invalid. We normally talk about two types

More information

Partitioning the Parameter Space. Topic 18 Composite Hypotheses

Partitioning the Parameter Space. Topic 18 Composite Hypotheses Topic 18 Composite Hypotheses Partitioning the Parameter Space 1 / 10 Outline Partitioning the Parameter Space 2 / 10 Partitioning the Parameter Space Simple hypotheses limit us to a decision between one

More information

HYPOTHESIS TESTING: FREQUENTIST APPROACH.

HYPOTHESIS TESTING: FREQUENTIST APPROACH. HYPOTHESIS TESTING: FREQUENTIST APPROACH. These notes summarize the lectures on (the frequentist approach to) hypothesis testing. You should be familiar with the standard hypothesis testing from previous

More information

One-Sample Numerical Data

One-Sample Numerical Data One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html

More information

Estimating the accuracy of a hypothesis Setting. Assume a binary classification setting

Estimating the accuracy of a hypothesis Setting. Assume a binary classification setting Estimating the accuracy of a hypothesis Setting Assume a binary classification setting Assume input/output pairs (x, y) are sampled from an unknown probability distribution D = p(x, y) Train a binary classifier

More information

ECO220Y Review and Introduction to Hypothesis Testing Readings: Chapter 12

ECO220Y Review and Introduction to Hypothesis Testing Readings: Chapter 12 ECO220Y Review and Introduction to Hypothesis Testing Readings: Chapter 12 Winter 2012 Lecture 13 (Winter 2011) Estimation Lecture 13 1 / 33 Review of Main Concepts Sampling Distribution of Sample Mean

More information

EC2001 Econometrics 1 Dr. Jose Olmo Room D309

EC2001 Econometrics 1 Dr. Jose Olmo Room D309 EC2001 Econometrics 1 Dr. Jose Olmo Room D309 J.Olmo@City.ac.uk 1 Revision of Statistical Inference 1.1 Sample, observations, population A sample is a number of observations drawn from a population. Population:

More information

Probability and Statistics

Probability and Statistics Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be CHAPTER 4: IT IS ALL ABOUT DATA 4a - 1 CHAPTER 4: IT

More information

Hypothesis Testing. ECE 3530 Spring Antonio Paiva

Hypothesis Testing. ECE 3530 Spring Antonio Paiva Hypothesis Testing ECE 3530 Spring 2010 Antonio Paiva What is hypothesis testing? A statistical hypothesis is an assertion or conjecture concerning one or more populations. To prove that a hypothesis is

More information

Confidence Distribution

Confidence Distribution Confidence Distribution Xie and Singh (2013): Confidence distribution, the frequentist distribution estimator of a parameter: A Review Céline Cunen, 15/09/2014 Outline of Article Introduction The concept

More information

Evaluation requires to define performance measures to be optimized

Evaluation requires to define performance measures to be optimized Evaluation Basic concepts Evaluation requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain (generalization error) approximation

More information

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy Department of Mathematics Imperial College London a.gandy@imperial.ac.uk user! 2009, Rennes July 8-10, 2009

More information

Big Data Analysis with Apache Spark UC#BERKELEY

Big Data Analysis with Apache Spark UC#BERKELEY Big Data Analysis with Apache Spark UC#BERKELEY This Lecture: Relation between Variables An association A trend» Positive association or Negative association A pattern» Could be any discernible shape»

More information

Inferences about a Mean Vector

Inferences about a Mean Vector Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University

More information

UNIVERSITÄT POTSDAM Institut für Mathematik

UNIVERSITÄT POTSDAM Institut für Mathematik UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam

More information

Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference

Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference 1 / 171 Bootstrap inference Francisco Cribari-Neto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil email: cribari@gmail.com October 2013 2 / 171 Unpaid advertisement

More information

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015 STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Single Sample Means. SOCY601 Alan Neustadtl

Single Sample Means. SOCY601 Alan Neustadtl Single Sample Means SOCY601 Alan Neustadtl The Central Limit Theorem If we have a population measured by a variable with a mean µ and a standard deviation σ, and if all possible random samples of size

More information

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).

More information

TUTORIAL 8 SOLUTIONS #

TUTORIAL 8 SOLUTIONS # TUTORIAL 8 SOLUTIONS #9.11.21 Suppose that a single observation X is taken from a uniform density on [0,θ], and consider testing H 0 : θ = 1 versus H 1 : θ =2. (a) Find a test that has significance level

More information

Null Hypothesis Significance Testing p-values, significance level, power, t-tests Spring 2017

Null Hypothesis Significance Testing p-values, significance level, power, t-tests Spring 2017 Null Hypothesis Significance Testing p-values, significance level, power, t-tests 18.05 Spring 2017 Understand this figure f(x H 0 ) x reject H 0 don t reject H 0 reject H 0 x = test statistic f (x H 0

More information

Interpreting Regression Results -Part II

Interpreting Regression Results -Part II Interpreting Regression Results -Part II Carlo Favero Favero () Interpreting Regression Results -Part II / 9 The Partitioned Regression Model Given the linear model: y = Xβ + ɛ, Partition X in two blocks

More information

EXAM 3 Math 1342 Elementary Statistics 6-7

EXAM 3 Math 1342 Elementary Statistics 6-7 EXAM 3 Math 1342 Elementary Statistics 6-7 Name Date ********************************************************************************************************************************************** MULTIPLE

More information

Lecture 7: Hypothesis Testing and ANOVA

Lecture 7: Hypothesis Testing and ANOVA Lecture 7: Hypothesis Testing and ANOVA Goals Overview of key elements of hypothesis testing Review of common one and two sample tests Introduction to ANOVA Hypothesis Testing The intent of hypothesis

More information

Advanced Statistics II: Non Parametric Tests

Advanced Statistics II: Non Parametric Tests Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney

More information

Confidence Intervals and Hypothesis Tests

Confidence Intervals and Hypothesis Tests Confidence Intervals and Hypothesis Tests STA 281 Fall 2011 1 Background The central limit theorem provides a very powerful tool for determining the distribution of sample means for large sample sizes.

More information

Null Hypothesis Significance Testing p-values, significance level, power, t-tests

Null Hypothesis Significance Testing p-values, significance level, power, t-tests Null Hypothesis Significance Testing p-values, significance level, power, t-tests 18.05 Spring 2014 January 1, 2017 1 /22 Understand this figure f(x H 0 ) x reject H 0 don t reject H 0 reject H 0 x = test

More information

Resampling and the Bootstrap

Resampling and the Bootstrap Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D-69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing

More information

IMPROVING TWO RESULTS IN MULTIPLE TESTING

IMPROVING TWO RESULTS IN MULTIPLE TESTING IMPROVING TWO RESULTS IN MULTIPLE TESTING By Sanat K. Sarkar 1, Pranab K. Sen and Helmut Finner Temple University, University of North Carolina at Chapel Hill and University of Duesseldorf October 11,

More information

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December

More information

PSY 216. Assignment 9 Answers. Under what circumstances is a t statistic used instead of a z-score for a hypothesis test

PSY 216. Assignment 9 Answers. Under what circumstances is a t statistic used instead of a z-score for a hypothesis test PSY 216 Assignment 9 Answers 1. Problem 1 from the text Under what circumstances is a t statistic used instead of a z-score for a hypothesis test The t statistic should be used when the population standard

More information

CIVL /8904 T R A F F I C F L O W T H E O R Y L E C T U R E - 8

CIVL /8904 T R A F F I C F L O W T H E O R Y L E C T U R E - 8 CIVL - 7904/8904 T R A F F I C F L O W T H E O R Y L E C T U R E - 8 Chi-square Test How to determine the interval from a continuous distribution I = Range 1 + 3.322(logN) I-> Range of the class interval

More information

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)? ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we

More information

PRINCIPLES OF STATISTICAL INFERENCE

PRINCIPLES OF STATISTICAL INFERENCE Advanced Series on Statistical Science & Applied Probability PRINCIPLES OF STATISTICAL INFERENCE from a Neo-Fisherian Perspective Luigi Pace Department of Statistics University ofudine, Italy Alessandra

More information

P Values and Nuisance Parameters

P Values and Nuisance Parameters P Values and Nuisance Parameters Luc Demortier The Rockefeller University PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics CERN, Geneva, June 27 29, 2007 Definition and interpretation of p values;

More information

STAT T&E COE-Report Reliability Test Planning for Mean Time Between Failures. Best Practice. Authored by: Jennifer Kensler, PhD STAT T&E COE

STAT T&E COE-Report Reliability Test Planning for Mean Time Between Failures. Best Practice. Authored by: Jennifer Kensler, PhD STAT T&E COE Reliability est Planning for Mean ime Between Failures Best Practice Authored by: Jennifer Kensler, PhD SA &E COE March 21, 2014 he goal of the SA &E COE is to assist in developing rigorous, defensible

More information

Finite Mixture Model Diagnostics Using Resampling Methods

Finite Mixture Model Diagnostics Using Resampling Methods Finite Mixture Model Diagnostics Using Resampling Methods Bettina Grün Johannes Kepler Universität Linz Friedrich Leisch Universität für Bodenkultur Wien Abstract This paper illustrates the implementation

More information

STA Module 10 Comparing Two Proportions

STA Module 10 Comparing Two Proportions STA 2023 Module 10 Comparing Two Proportions Learning Objectives Upon completing this module, you should be able to: 1. Perform large-sample inferences (hypothesis test and confidence intervals) to compare

More information

Tutorial 2: Power and Sample Size for the Paired Sample t-test

Tutorial 2: Power and Sample Size for the Paired Sample t-test Tutorial 2: Power and Sample Size for the Paired Sample t-test Preface Power is the probability that a study will reject the null hypothesis. The estimated probability is a function of sample size, variability,

More information

Purposes of Data Analysis. Variables and Samples. Parameters and Statistics. Part 1: Probability Distributions

Purposes of Data Analysis. Variables and Samples. Parameters and Statistics. Part 1: Probability Distributions Part 1: Probability Distributions Purposes of Data Analysis True Distributions or Relationships in the Earths System Probability Distribution Normal Distribution Student-t Distribution Chi Square Distribution

More information

Probability theory and inference statistics! Dr. Paola Grosso! SNE research group!! (preferred!)!!

Probability theory and inference statistics! Dr. Paola Grosso! SNE research group!!  (preferred!)!! Probability theory and inference statistics Dr. Paola Grosso SNE research group p.grosso@uva.nl paola.grosso@os3.nl (preferred) Roadmap Lecture 1: Monday Sep. 22nd Collecting data Presenting data Descriptive

More information

PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007

PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007 Cohort study s formulations PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007 Srine Dudoit Division of Biostatistics Department of Statistics University of California, Berkeley www.stat.berkeley.edu/~srine

More information

CHAPTER 8. Test Procedures is a rule, based on sample data, for deciding whether to reject H 0 and contains:

CHAPTER 8. Test Procedures is a rule, based on sample data, for deciding whether to reject H 0 and contains: CHAPTER 8 Test of Hypotheses Based on a Single Sample Hypothesis testing is the method that decide which of two contradictory claims about the parameter is correct. Here the parameters of interest are

More information

Principles of Statistical Inference

Principles of Statistical Inference Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical

More information

Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process

Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University

More information

Hypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test.

Hypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test. Hypothesis Testing Hypothesis testing is a statistical problem where you must choose, on the basis of data X, between two alternatives. We formalize this as the problem of choosing between two hypotheses:

More information

Lecture 5: ANOVA and Correlation

Lecture 5: ANOVA and Correlation Lecture 5: ANOVA and Correlation Ani Manichaikul amanicha@jhsph.edu 23 April 2007 1 / 62 Comparing Multiple Groups Continous data: comparing means Analysis of variance Binary data: comparing proportions

More information

Econometrics Review questions for exam

Econometrics Review questions for exam Econometrics Review questions for exam Nathaniel Higgins nhiggins@jhu.edu, 1. Suppose you have a model: y = β 0 x 1 + u You propose the model above and then estimate the model using OLS to obtain: ŷ =

More information

L2: Review of probability and statistics

L2: Review of probability and statistics Probability L2: Review of probability and statistics Definition of probability Axioms and properties Conditional probability Bayes theorem Random variables Definition of a random variable Cumulative distribution

More information

6.4 Type I and Type II Errors

6.4 Type I and Type II Errors 6.4 Type I and Type II Errors Ulrich Hoensch Friday, March 22, 2013 Null and Alternative Hypothesis Neyman-Pearson Approach to Statistical Inference: A statistical test (also known as a hypothesis test)

More information

University of California San Diego and Stanford University and

University of California San Diego and Stanford University and First International Workshop on Functional and Operatorial Statistics. Toulouse, June 19-21, 2008 K-sample Subsampling Dimitris N. olitis andjoseph.romano University of California San Diego and Stanford

More information

Confidence Intervals for Normal Data Spring 2014

Confidence Intervals for Normal Data Spring 2014 Confidence Intervals for Normal Data 18.05 Spring 2014 Agenda Today Review of critical values and quantiles. Computing z, t, χ 2 confidence intervals for normal data. Conceptual view of confidence intervals.

More information

Hypothesis Tests and Estimation for Population Variances. Copyright 2014 Pearson Education, Inc.

Hypothesis Tests and Estimation for Population Variances. Copyright 2014 Pearson Education, Inc. Hypothesis Tests and Estimation for Population Variances 11-1 Learning Outcomes Outcome 1. Formulate and carry out hypothesis tests for a single population variance. Outcome 2. Develop and interpret confidence

More information

The outline for Unit 3

The outline for Unit 3 The outline for Unit 3 Unit 1. Introduction: The regression model. Unit 2. Estimation principles. Unit 3: Hypothesis testing principles. 3.1 Wald test. 3.2 Lagrange Multiplier. 3.3 Likelihood Ratio Test.

More information

Variable Selection and Model Building

Variable Selection and Model Building LINEAR REGRESSION ANALYSIS MODULE XIII Lecture - 37 Variable Selection and Model Building Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur The complete regression

More information

Institute for the Advancement of University Learning & Department of Statistics

Institute for the Advancement of University Learning & Department of Statistics Institute for the Advancement of University Learning & Department of Statistics Descriptive Statistics for Research (Hilary Term, 00) Lecture 7: Hypothesis Testing (I.) Introduction An important area of

More information

CHAPTER 17 CHI-SQUARE AND OTHER NONPARAMETRIC TESTS FROM: PAGANO, R. R. (2007)

CHAPTER 17 CHI-SQUARE AND OTHER NONPARAMETRIC TESTS FROM: PAGANO, R. R. (2007) FROM: PAGANO, R. R. (007) I. INTRODUCTION: DISTINCTION BETWEEN PARAMETRIC AND NON-PARAMETRIC TESTS Statistical inference tests are often classified as to whether they are parametric or nonparametric Parameter

More information

Statistics for Managers Using Microsoft Excel/SPSS Chapter 8 Fundamentals of Hypothesis Testing: One-Sample Tests

Statistics for Managers Using Microsoft Excel/SPSS Chapter 8 Fundamentals of Hypothesis Testing: One-Sample Tests Statistics for Managers Using Microsoft Excel/SPSS Chapter 8 Fundamentals of Hypothesis Testing: One-Sample Tests 1999 Prentice-Hall, Inc. Chap. 8-1 Chapter Topics Hypothesis Testing Methodology Z Test

More information

Introduction to Statistical Inference

Introduction to Statistical Inference Introduction to Statistical Inference Dr. Fatima Sanchez-Cabo f.sanchezcabo@tugraz.at http://www.genome.tugraz.at Institute for Genomics and Bioinformatics, Graz University of Technology, Austria Introduction

More information

Statistical Analysis of Engineering Data The Bare Bones Edition. Precision, Bias, Accuracy, Measures of Precision, Propagation of Error

Statistical Analysis of Engineering Data The Bare Bones Edition. Precision, Bias, Accuracy, Measures of Precision, Propagation of Error Statistical Analysis of Engineering Data The Bare Bones Edition (I) Precision, Bias, Accuracy, Measures of Precision, Propagation of Error PRIOR TO DATA ACQUISITION ONE SHOULD CONSIDER: 1. The accuracy

More information

1; (f) H 0 : = 55 db, H 1 : < 55.

1; (f) H 0 : = 55 db, H 1 : < 55. Reference: Chapter 8 of J. L. Devore s 8 th Edition By S. Maghsoodloo TESTING a STATISTICAL HYPOTHESIS A statistical hypothesis is an assumption about the frequency function(s) (i.e., pmf or pdf) of one

More information

Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions

Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions Journal of Modern Applied Statistical Methods Volume 8 Issue 1 Article 13 5-1-2009 Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error

More information

Analysis of Variance (ANOVA)

Analysis of Variance (ANOVA) Analysis of Variance (ANOVA) Used for comparing or more means an extension of the t test Independent Variable (factor) = categorical (qualita5ve) predictor should have at least levels, but can have many

More information

Multiple Dependent Hypothesis Tests in Geographically Weighted Regression

Multiple Dependent Hypothesis Tests in Geographically Weighted Regression Multiple Dependent Hypothesis Tests in Geographically Weighted Regression Graeme Byrne 1, Martin Charlton 2, and Stewart Fotheringham 3 1 La Trobe University, Bendigo, Victoria Austrlaia Telephone: +61

More information

What p values really mean (and why I should care) Francis C. Dane, PhD

What p values really mean (and why I should care) Francis C. Dane, PhD What p values really mean (and why I should care) Francis C. Dane, PhD Session Objectives Understand the statistical decision process Appreciate the limitations of interpreting p values Value the use of

More information

Background to Statistics

Background to Statistics FACT SHEET Background to Statistics Introduction Statistics include a broad range of methods for manipulating, presenting and interpreting data. Professional scientists of all kinds need to be proficient

More information

HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007

HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007 MIT OpenCourseWare http://ocw.mit.edu HST.582J / 6.555J / 16.456J Biomedical Signal and Image Processing Spring 2007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

Statistical Applications in Genetics and Molecular Biology

Statistical Applications in Genetics and Molecular Biology Statistical Applications in Genetics and Molecular Biology Volume 5, Issue 1 2006 Article 28 A Two-Step Multiple Comparison Procedure for a Large Number of Tests and Multiple Treatments Hongmei Jiang Rebecca

More information

Reports of the Institute of Biostatistics

Reports of the Institute of Biostatistics Reports of the Institute of Biostatistics No 01 / 2010 Leibniz University of Hannover Natural Sciences Faculty Titel: Multiple contrast tests for multiple endpoints Author: Mario Hasler 1 1 Lehrfach Variationsstatistik,

More information

Lectures on Statistics. William G. Faris

Lectures on Statistics. William G. Faris Lectures on Statistics William G. Faris December 1, 2003 ii Contents 1 Expectation 1 1.1 Random variables and expectation................. 1 1.2 The sample mean........................... 3 1.3 The sample

More information

Chapter 7: Hypothesis Testing - Solutions

Chapter 7: Hypothesis Testing - Solutions Chapter 7: Hypothesis Testing - Solutions 7.1 Introduction to Hypothesis Testing The problem with applying the techniques learned in Chapter 5 is that typically, the population mean (µ) and standard deviation

More information

Statistics 135: Fall 2004 Final Exam

Statistics 135: Fall 2004 Final Exam Name: SID#: Statistics 135: Fall 2004 Final Exam There are 10 problems and the number of points for each is shown in parentheses. There is a normal table at the end. Show your work. 1. The designer of

More information

Bootstrapping, Randomization, 2B-PLS

Bootstrapping, Randomization, 2B-PLS Bootstrapping, Randomization, 2B-PLS Statistics, Tests, and Bootstrapping Statistic a measure that summarizes some feature of a set of data (e.g., mean, standard deviation, skew, coefficient of variation,

More information

Tutorial 5: Power and Sample Size for One-way Analysis of Variance (ANOVA) with Equal Variances Across Groups. Acknowledgements:

Tutorial 5: Power and Sample Size for One-way Analysis of Variance (ANOVA) with Equal Variances Across Groups. Acknowledgements: Tutorial 5: Power and Sample Size for One-way Analysis of Variance (ANOVA) with Equal Variances Across Groups Anna E. Barón, Keith E. Muller, Sarah M. Kreidler, and Deborah H. Glueck Acknowledgements:

More information

Testing Statistical Hypotheses

Testing Statistical Hypotheses E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

More information

The One-Way Independent-Samples ANOVA. (For Between-Subjects Designs)

The One-Way Independent-Samples ANOVA. (For Between-Subjects Designs) The One-Way Independent-Samples ANOVA (For Between-Subjects Designs) Computations for the ANOVA In computing the terms required for the F-statistic, we won t explicitly compute any sample variances or

More information

Generalized Linear Models (1/29/13)

Generalized Linear Models (1/29/13) STA613/CBB540: Statistical methods in computational biology Generalized Linear Models (1/29/13) Lecturer: Barbara Engelhardt Scribe: Yangxiaolu Cao When processing discrete data, two commonly used probability

More information

Lecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 9.1-1

Lecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 9.1-1 Lecture Slides Elementary Statistics Eleventh Edition and the Triola Statistics Series by Mario F. Triola Copyright 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. 9.1-1 Chapter 9 Inferences

More information

Chapter 22. Comparing Two Proportions 1 /29

Chapter 22. Comparing Two Proportions 1 /29 Chapter 22 Comparing Two Proportions 1 /29 Homework p519 2, 4, 12, 13, 15, 17, 18, 19, 24 2 /29 Objective Students test null and alternate hypothesis about two population proportions. 3 /29 Comparing Two

More information

Testing Research and Statistical Hypotheses

Testing Research and Statistical Hypotheses Testing Research and Statistical Hypotheses Introduction In the last lab we analyzed metric artifact attributes such as thickness or width/thickness ratio. Those were continuous variables, which as you

More information

Chapter 1 Statistical Inference

Chapter 1 Statistical Inference Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

More information

Direction: This test is worth 250 points and each problem worth points. DO ANY SIX

Direction: This test is worth 250 points and each problem worth points. DO ANY SIX Term Test 3 December 5, 2003 Name Math 52 Student Number Direction: This test is worth 250 points and each problem worth 4 points DO ANY SIX PROBLEMS You are required to complete this test within 50 minutes

More information

Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets

Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets Confidence sets X: a sample from a population P P. θ = θ(p): a functional from P to Θ R k for a fixed integer k. C(X): a confidence

More information

A Very Brief Summary of Statistical Inference, and Examples

A Very Brief Summary of Statistical Inference, and Examples A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random

More information