Political Science 236 Hypothesis Testing: Review and Bootstrapping

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The goal of hypothesis testing is to decide, using a sample from the population, which of two complimentary hypotheses is true. In general, the two complimentary hypotheses are called the null hypothesis and the alternative hypothesis. If we let θ be a population parameter and Θ be the parameter space, we can define these complementary hypotheses as follows: Definition 1.2 Let Θ 0 and Θ 1 Θ c 0 alternative hypothesis are defined as follows be a partition of the parameter space Θ. Then the null and 1. Null Hypothesis: H 0 : θ Θ 0 2. Alternative Hypothesis: H 1 : θ Θ 1 Definition 1.3 Testing Procedure. A testing procedure is a rule, based on the outcome of a random sample from the population under study, used to decide whether to reject H 0. 1

2 The subset of the sample space for which H 0 will be rejected is called the critical region ( or the rejection region), and its complement is called the acceptance region. In general, a hypothesis test will be specified in terms of a test statistic T (X 1, X 2,..., X N ) T (X), which is a function of the sample. We can define the critical region formally as follows. Definition 1.4 Critical Region. The subset C R N of the sample space for which H 0 is rejected is called the critical region and is defined by C c = { x R N : T (x) > c } for some c R. The value c is called the critical value. The complement of C c, C a C c c, is called the acceptance region. If we let C T c be the critical region of the test statistic T (X) (i.e. C T c is defined by C c = { x R N : T (x) C T } c ), a statistical test of H0 against H 1 will generally be defined as: 1. T (x) C T c = Reject H 0 T (x) / C T c = Accept H 0 A hypothesis test of H 0 : θ Θ 0 against H 1 : θ Θ 1 can make one of two types of errors. Definition 1.5 Type I and Type II Errors. Let H 0 be a null hypothesis being tested for acceptance or rejection. The two types of errors that can be made are 1. Type I Error: rejecting H 0 when θ Θ 0 (i.e, when H 0 is true) 2. Type II Error: accepting H 0 when θ Θ 1 (i.e, when H 0 is false) So a type I error is committed when the statistical test mistakenly rejects the null hypothesis, and a type II error is committed when the test mistakenly accepts the null hypothesis. The ideal 2

3 test is one where the hypothesis would always be correctly identified as being either true or false. For such an ideal test to exist, we must partition the range of potential sample outcomes in such a way that outcomes in the critical region C c would occur if and only if H 0 were true and outcomes in the acceptance region C a would occur if and only if H 0 were false. In general, ideal tests cannot be constructed. For θ Θ 0, the test will make a mistake if x C c and therefore the probability of a type I error is P θ (X C c ) and for θ Θ 1, the test will make a mistake if x C a and therefore the probability of a type II error is P θ (X C a ). Note that P θ (X C c ) = 1 P θ (X C a ). We will now define the power function of a test. The power function completely summarizes all of the operating characteristics of a statistical test with respect to probabilities of making correct and incorrect decisions about H 0. The power function is defined below. Definition 1.6 Let H 0 be defined as H 0 : θ Θ 0 and H 1 be defined as H 1 : θ Θ 1. Let the critical region C c define a test of H 0. Then the power function of the statistical test is the function of θ defined by β (θ) P θ (X C c ) = probability of Type I error if θ Θ 0 one minus probability of Type II error if θ Θ 1 In words, the power function indicates the probability of rejecting H 0 for every value of θ Θ. The value of the power function at a particular value of the parameter space θ p Θ is called the power of the test at θ p and represents the probability of rejecting H 0 if θ p were the true value of the parameter vector. The ideal power function is 0 for all θ Θ 0 and 1 for all θ Θ 1. In general, this ideal cannot be attained and we say that a good test has power function near 0 for all θ Θ 0 and near 1 for all θ Θ 1. When comparing two tests for a given H 0, a test is better if it has lower power for θ Θ 0 and higher power for θ Θ 1 which implies that the test has lower probabilities of both type I and type II error. We now define the size and level of a test: 3

4 Definition 1.7 Size. For 0 α 1, a test with power function β (θ) is a size-α test if sup θ Θ0 β (θ) = α Definition 1.8 Level. sup θ Θ0 β (θ) α For 0 α 1, a test with power function β (θ) is a level-α test if In words, the size of the test is the maximum probability of Type I error associated with a given test rule. The lower the size of the test, the lower the maximum probability of mistakenly rejecting H 0. The level of a test is an upper bound to the type I error probability of a statistical test. The key difference between these two concepts is that the size represents the maximum value of β (θ) for θ Θ 0 (i.e. the maximum type I error) while the level is only a bound that might not equal β (θ) for any θ Θ 0 nor equal the supremum of β (θ) for θ Θ 0. Thus, the set of level-α tests contains the set of size-α tests. In other words, a test of H 0 having size γ is a α-level test for any α γ. In applications, when we say that H 0 is (not) rejected at the α-significance level, we often mean that α was the bound on the level of protection against type I error that was used when constructing the test. A more accurate statement is regarding the level of protection against type I error is that H 0 is (not) rejected using a size-α test. 2 Bootstrapping Hypothesis Tests The simplest situation involves a simple null hypothesis H 0 that completely specifies the probability distribution of the data. Thus, if we have a sample x 1, x 2,..., x n from a population with CDF F, then H 0 specifies that F = F 0 where F 0 contains no unknown parameters. A statistical test is based on a test statistic T which measures the discrepancy between the data and the null hypothesis. We will follow the convention that large values of T are evidence against H 0. If the null hypothesis is simple and the observed value of the test statistics is denoted by t, then the level of evidence 4

5 against H 0 is measured by the significance probability p = P (T t H 0 ) which is referred to as the p-value. The p-value is effectively the marginal size test at which a given hypothesis would be rejected based on the observed outcome of X. A corresponding notion is that of a critical value t p for t, which is associated with testing at level p: if t t p then H 0 is rejected at level p or 100p%. It follows that t p is defined as P (T t p H 0 ) = p Note that p is what we defined earlier as the size of the test and the set {(x 1, x 2,..., x n ) : t t p H 0 } is the level p critical region of the test. distribution of T. The distribution of the T under H 0 is called the null 2.1 How to choose the test-statistic In a parametric setting, there is an explicit form of the sampling distribution of the data with a finite number of unknown parameters. In these cases the alternative hypothesis guides the choice of the test statistic (usually through use of the likelihood function of the data). In non-parametric settings, no particular forms are specified for the distributions and hence the appropriate choice of T is less clear. However, the choice of T should be always based on some notion of what is of concern in the case that H 0 turns out to be false. In all non-parametric problems, the null hypothesis H 0 leaves some parameters unknown and therefore does not completely specify F. In this case, the p-value is not well defined because P (T t F ) may depend upon which F satisfying H 0 is taken Pivot Tests When H 0 concerns a particular parameter value, we can use the equivalence between hypothesis tests and confidence intervals. This equivalence implies that if the value of θ 0 is outside a 1 α 5

6 confidence interval for θ, then θ differs from θ 0 with p-value less than α. A specific form of test based on this equivalence is a pivot test. Suppose that T is an estimator for a scalar θ, with estimated variance V. Suppose also that the studentized version of T, Z = T θ, is a pivot (i.e. V 1/2 its distribution is the same for all relevant F, and in particular for all θ). For a one-sided test of H 0 : θ = θ 0 versus H 1 : θ > θ 0, the p-value that corresponds to the observed studentized test statistic z 0 = t θ 0 v 1/2 is p = P However, since Z is a pivot we have P { T θ0 V 1/2 t θ } 0 v 1/2 H 0 and therefore the p-value can be written as { T θ0 V 1/2 t θ } 0 v 1/2 H 0 { = P Z t θ } 0 v 1/2 H 0 { = P Z t θ } 0 v 1/2 F p = P {Z z 0 F } Note that this has a big advantage in the context of bootstrapping, because we do not have to construct a special null-hypothesis sampling distribution. 2.2 Non-Parametric Bootstrap Tests Testing hypothesis requires that probability calculations be done under the null hypothesis model. This means that the usual bootstrap setting must be modified, since resampling from the empirical CDF F and applying the plug-in principle to obtain θ ( ) = t F won t give us an estimator of θ under the null hypothesis H 0. In the hypothesis testing context, instead of resampling from the empirical CDF F, we must resample from an empirical CDF F 0 which satisfies the relevant null hypothesis H 0. (Unless, as we mentioned above, we can construct a pivot test-statistic). 6

7 Once we have decided on the null resampling distribution F 0, the basic bootstrap test will compute the p-value as or will approximate it by using the results t 1, t 2,..., t B p boot = P { T t F 0 } p boot = # {t b t} B from B bootstrap samples. Example 2.1 Difference in means. Suppose we want to compare two population means µ 1 and µ 2 using the test statistic t = x 1 x 2. We will use the following sample data: sample sample If the shapes of the underlying distributions are identical, then under H 0 : µ 1 = µ 2 the two distributions are the same. In this case, it is sensible to choose for F 0 the pooled empirical CDF of the two samples. Applying this procedure with 1, 000 bootstrap samples yielded 52 values of t greater than the observed value t = = 2.84, which implies a p-value of cannot reject the null at 5% (but we can at 5.2%!!) = So we Studentized Bootstrap Method For some problems, it is possible to obtain more stable significance tests by studentizing comparisons. Remember that because of the relationship between confidence sets and hypothesis tests, such a test can be obtained calculating a 1 p confidence set by the studentized bootstrap method and concluding that the p-value is less than p is the null hypothesis parameter falls outside the confidence set. We can also implement this idea by bootstrapping the test statistic directly rather than constructing confidence intervals. In this case, the p-value can be obtained directly. Suppose that θ is 7

8 a scalar with estimator T and that we want to test H 0 : θ = θ 0 against H 1 : θ > θ 0. The method we mentioned in the section Pivot Tests applies when Z = T θ V 1/2 is approximately a pivot (i.e. its distribution is approximately independent of unknown parameters). Then, with z 0 = t θ 0 of v 1/2 being the observed studentized test statistic the bootstrap analog p = P {Z z 0 F } is p = P { Z z 0 F } which we can approximate by bootstrapping without having to decide on a null empirical distribution F 0. Example 2.2 Let s continue the example of the difference in means. We were comparing compare two population means µ 1 and µ 2 using the test statistic t = x 1 x 2. Now, it would reasonable to suppose that the usual two-sample t-statistic Z = X 2 X 1 (µ 2 µ 1 ) ( S 2 2 /n 2 + S 2 1 /n 1) 1/2 is approximately pivotal. We take F to be the empirical CDF of the two samples taken together, provided that no assumptions are made connecting the two distributions. The observed value of the test statistic under the null is We also calculate B values of z 0 = x 2 x 1 ( s 2 2 /n 2 + s 2 1 /n 1) 1/2 z = x 2 x 1 (x 2 x 1 ) ( s 2 2 /n 2 + s 2 1 /n 1 ) 1/2 8

9 3 Testing Linear Restrictions in OLS Consider the problem of testing the following null hypothesis H 0 : Rβ = r where the d K matrix R is matrix of restrictions (where d is the number of restrictions) and r is a p 1 vector of constants. The alternative hypothesis is H 1 : Rβ r. Using standard results from multivariate normal distributions, we now that T 1 ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r T 2 and hence we have pivotal statistic given by σ 2 ( ) T ( ) y X β y X β σ 2 T 1 T 2 χ 2 N K χ 2 d F = = (Rβ r) b T R(X T X) 1 1 R T (Rβ r) b 1 σ 2 d (y Xβ) b T (y Xβ) b 1 σ 2 N r ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r 1 d ( ) T ( ) 1 y X β y X β N K ( ) T ( R β r R ( X T X ) ) 1 1 ( ) R T R β r ds 2 F d,n K References Davidson, A. C. and D.V. Hinkley, Bootstrap Methods and their Application. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press. 9

Statistical Inference

Statistical Inference Classical and Bayesian Methods Class 6 AMS-UCSC Thu 26, 2012 Winter 2012. Session 1 (Class 6) AMS-132/206 Thu 26, 2012 1 / 15 Topics Topics We will talk about... 1 Hypothesis testing

INTERVAL ESTIMATION AND HYPOTHESES TESTING

INTERVAL ESTIMATION AND HYPOTHESES TESTING 1. IDEA An interval rather than a point estimate is often of interest. Confidence intervals are thus important in empirical work. To construct interval estimates,

Ch. 5 Hypothesis Testing

Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,

CH.9 Tests of Hypotheses for a Single Sample

CH.9 Tests of Hypotheses for a Single Sample Hypotheses testing Tests on the mean of a normal distributionvariance known Tests on the mean of a normal distributionvariance unknown Tests on the variance

LECTURE 5. Introduction to Econometrics. Hypothesis testing

LECTURE 5 Introduction to Econometrics Hypothesis testing October 18, 2016 1 / 26 ON TODAY S LECTURE We are going to discuss how hypotheses about coefficients can be tested in regression models We will

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest

hypothesis a claim about the value of some parameter (like p)

Testing hypotheses hypothesis a claim about the value of some parameter (like p) significance test procedure to assess the strength of evidence provided by a sample of data against the claim of a hypothesized

Quantitative Methods for Economics, Finance and Management (A86050 F86050)

Quantitative Methods for Economics, Finance and Management (A86050 F86050) Matteo Manera matteo.manera@unimib.it Marzio Galeotti marzio.galeotti@unimi.it 1 This material is taken and adapted from Guy Judge

Sampling Distributions: Central Limit Theorem

Review for Exam 2 Sampling Distributions: Central Limit Theorem Conceptually, we can break up the theorem into three parts: 1. The mean (µ M ) of a population of sample means (M) is equal to the mean (µ)

Bootstrap tests. Patrick Breheny. October 11. Bootstrap vs. permutation tests Testing for equality of location

Bootstrap tests Patrick Breheny October 11 Patrick Breheny STA 621: Nonparametric Statistics 1/14 Introduction Conditioning on the observed data to obtain permutation tests is certainly an important idea

4 Hypothesis testing. 4.1 Types of hypothesis and types of error 4 HYPOTHESIS TESTING 49

4 HYPOTHESIS TESTING 49 4 Hypothesis testing In sections 2 and 3 we considered the problem of estimating a single parameter of interest, θ. In this section we consider the related problem of testing whether

18.05 Final Exam. Good luck! Name. No calculators. Number of problems 16 concept questions, 16 problems, 21 pages

Name No calculators. 18.05 Final Exam Number of problems 16 concept questions, 16 problems, 21 pages Extra paper If you need more space we will provide some blank paper. Indicate clearly that your solution

MTMS Mathematical Statistics

MTMS.01.099 Mathematical Statistics Lecture 12. Hypothesis testing. Power function. Approximation of Normal distribution and application to Binomial distribution Tõnu Kollo Fall 2016 Hypothesis Testing

14.30 Introduction to Statistical Methods in Economics Spring 2009

MIT OpenCourseWare http://ocw.mit.edu 4.0 Introduction to Statistical Methods in Economics Spring 009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing. Road Map Sampling Distributions, Confidence Intervals & Hypothesis Testing

Topic 3: Sampling Distributions, Confidence Intervals & Hypothesis Testing ECO22Y5Y: Quantitative Methods in Economics Dr. Nick Zammit University of Toronto Department of Economics Room KN3272 n.zammit

Hypothesis Testing. We normally talk about two types of hypothesis: the null hypothesis and the research or alternative hypothesis.

Hypothesis Testing Today, we are going to begin talking about the idea of hypothesis testing how we can use statistics to show that our causal models are valid or invalid. We normally talk about two types

Partitioning the Parameter Space. Topic 18 Composite Hypotheses

Topic 18 Composite Hypotheses Partitioning the Parameter Space 1 / 10 Outline Partitioning the Parameter Space 2 / 10 Partitioning the Parameter Space Simple hypotheses limit us to a decision between one

HYPOTHESIS TESTING: FREQUENTIST APPROACH.

HYPOTHESIS TESTING: FREQUENTIST APPROACH. These notes summarize the lectures on (the frequentist approach to) hypothesis testing. You should be familiar with the standard hypothesis testing from previous

One-Sample Numerical Data

One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html

Estimating the accuracy of a hypothesis Setting. Assume a binary classification setting

Estimating the accuracy of a hypothesis Setting Assume a binary classification setting Assume input/output pairs (x, y) are sampled from an unknown probability distribution D = p(x, y) Train a binary classifier

ECO220Y Review and Introduction to Hypothesis Testing Readings: Chapter 12

ECO220Y Review and Introduction to Hypothesis Testing Readings: Chapter 12 Winter 2012 Lecture 13 (Winter 2011) Estimation Lecture 13 1 / 33 Review of Main Concepts Sampling Distribution of Sample Mean

EC2001 Econometrics 1 Dr. Jose Olmo Room D309

EC2001 Econometrics 1 Dr. Jose Olmo Room D309 J.Olmo@City.ac.uk 1 Revision of Statistical Inference 1.1 Sample, observations, population A sample is a number of observations drawn from a population. Population:

Probability and Statistics

Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be CHAPTER 4: IT IS ALL ABOUT DATA 4a - 1 CHAPTER 4: IT

Hypothesis Testing. ECE 3530 Spring Antonio Paiva

Hypothesis Testing ECE 3530 Spring 2010 Antonio Paiva What is hypothesis testing? A statistical hypothesis is an assertion or conjecture concerning one or more populations. To prove that a hypothesis is

Confidence Distribution

Confidence Distribution Xie and Singh (2013): Confidence distribution, the frequentist distribution estimator of a parameter: A Review Céline Cunen, 15/09/2014 Outline of Article Introduction The concept

Evaluation requires to define performance measures to be optimized

Evaluation Basic concepts Evaluation requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain (generalization error) approximation

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy Department of Mathematics Imperial College London a.gandy@imperial.ac.uk user! 2009, Rennes July 8-10, 2009

Big Data Analysis with Apache Spark UC#BERKELEY

Big Data Analysis with Apache Spark UC#BERKELEY This Lecture: Relation between Variables An association A trend» Positive association or Negative association A pattern» Could be any discernible shape»

Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University

UNIVERSITÄT POTSDAM Institut für Mathematik

UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam

Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference

1 / 171 Bootstrap inference Francisco Cribari-Neto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil email: cribari@gmail.com October 2013 2 / 171 Unpaid advertisement

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

Single Sample Means. SOCY601 Alan Neustadtl

Single Sample Means SOCY601 Alan Neustadtl The Central Limit Theorem If we have a population measured by a variable with a mean µ and a standard deviation σ, and if all possible random samples of size

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).

TUTORIAL 8 SOLUTIONS #

TUTORIAL 8 SOLUTIONS #9.11.21 Suppose that a single observation X is taken from a uniform density on [0,θ], and consider testing H 0 : θ = 1 versus H 1 : θ =2. (a) Find a test that has significance level

Null Hypothesis Significance Testing p-values, significance level, power, t-tests Spring 2017

Null Hypothesis Significance Testing p-values, significance level, power, t-tests 18.05 Spring 2017 Understand this figure f(x H 0 ) x reject H 0 don t reject H 0 reject H 0 x = test statistic f (x H 0

Interpreting Regression Results -Part II

Interpreting Regression Results -Part II Carlo Favero Favero () Interpreting Regression Results -Part II / 9 The Partitioned Regression Model Given the linear model: y = Xβ + ɛ, Partition X in two blocks

EXAM 3 Math 1342 Elementary Statistics 6-7

EXAM 3 Math 1342 Elementary Statistics 6-7 Name Date ********************************************************************************************************************************************** MULTIPLE

Lecture 7: Hypothesis Testing and ANOVA

Lecture 7: Hypothesis Testing and ANOVA Goals Overview of key elements of hypothesis testing Review of common one and two sample tests Introduction to ANOVA Hypothesis Testing The intent of hypothesis

Advanced Statistics II: Non Parametric Tests

Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney

Confidence Intervals and Hypothesis Tests

Confidence Intervals and Hypothesis Tests STA 281 Fall 2011 1 Background The central limit theorem provides a very powerful tool for determining the distribution of sample means for large sample sizes.

Null Hypothesis Significance Testing p-values, significance level, power, t-tests

Null Hypothesis Significance Testing p-values, significance level, power, t-tests 18.05 Spring 2014 January 1, 2017 1 /22 Understand this figure f(x H 0 ) x reject H 0 don t reject H 0 reject H 0 x = test

Resampling and the Bootstrap

Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D-69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing

IMPROVING TWO RESULTS IN MULTIPLE TESTING

IMPROVING TWO RESULTS IN MULTIPLE TESTING By Sanat K. Sarkar 1, Pranab K. Sen and Helmut Finner Temple University, University of North Carolina at Chapel Hill and University of Duesseldorf October 11,

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December

PSY 216. Assignment 9 Answers. Under what circumstances is a t statistic used instead of a z-score for a hypothesis test

PSY 216 Assignment 9 Answers 1. Problem 1 from the text Under what circumstances is a t statistic used instead of a z-score for a hypothesis test The t statistic should be used when the population standard

CIVL /8904 T R A F F I C F L O W T H E O R Y L E C T U R E - 8

CIVL - 7904/8904 T R A F F I C F L O W T H E O R Y L E C T U R E - 8 Chi-square Test How to determine the interval from a continuous distribution I = Range 1 + 3.322(logN) I-> Range of the class interval

2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?

ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we

PRINCIPLES OF STATISTICAL INFERENCE

Advanced Series on Statistical Science & Applied Probability PRINCIPLES OF STATISTICAL INFERENCE from a Neo-Fisherian Perspective Luigi Pace Department of Statistics University ofudine, Italy Alessandra

P Values and Nuisance Parameters

P Values and Nuisance Parameters Luc Demortier The Rockefeller University PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics CERN, Geneva, June 27 29, 2007 Definition and interpretation of p values;

STAT T&E COE-Report Reliability Test Planning for Mean Time Between Failures. Best Practice. Authored by: Jennifer Kensler, PhD STAT T&E COE

Reliability est Planning for Mean ime Between Failures Best Practice Authored by: Jennifer Kensler, PhD SA &E COE March 21, 2014 he goal of the SA &E COE is to assist in developing rigorous, defensible

Finite Mixture Model Diagnostics Using Resampling Methods

Finite Mixture Model Diagnostics Using Resampling Methods Bettina Grün Johannes Kepler Universität Linz Friedrich Leisch Universität für Bodenkultur Wien Abstract This paper illustrates the implementation

STA Module 10 Comparing Two Proportions

STA 2023 Module 10 Comparing Two Proportions Learning Objectives Upon completing this module, you should be able to: 1. Perform large-sample inferences (hypothesis test and confidence intervals) to compare

Tutorial 2: Power and Sample Size for the Paired Sample t-test

Tutorial 2: Power and Sample Size for the Paired Sample t-test Preface Power is the probability that a study will reject the null hypothesis. The estimated probability is a function of sample size, variability,

Purposes of Data Analysis. Variables and Samples. Parameters and Statistics. Part 1: Probability Distributions

Part 1: Probability Distributions Purposes of Data Analysis True Distributions or Relationships in the Earths System Probability Distribution Normal Distribution Student-t Distribution Chi Square Distribution

Probability theory and inference statistics! Dr. Paola Grosso! SNE research group!! (preferred!)!!

Probability theory and inference statistics Dr. Paola Grosso SNE research group p.grosso@uva.nl paola.grosso@os3.nl (preferred) Roadmap Lecture 1: Monday Sep. 22nd Collecting data Presenting data Descriptive

PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007

Cohort study s formulations PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007 Srine Dudoit Division of Biostatistics Department of Statistics University of California, Berkeley www.stat.berkeley.edu/~srine

CHAPTER 8. Test Procedures is a rule, based on sample data, for deciding whether to reject H 0 and contains:

CHAPTER 8 Test of Hypotheses Based on a Single Sample Hypothesis testing is the method that decide which of two contradictory claims about the parameter is correct. Here the parameters of interest are

Principles of Statistical Inference

Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical

Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process

Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University

Hypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test.

Hypothesis Testing Hypothesis testing is a statistical problem where you must choose, on the basis of data X, between two alternatives. We formalize this as the problem of choosing between two hypotheses:

Lecture 5: ANOVA and Correlation

Lecture 5: ANOVA and Correlation Ani Manichaikul amanicha@jhsph.edu 23 April 2007 1 / 62 Comparing Multiple Groups Continous data: comparing means Analysis of variance Binary data: comparing proportions

Econometrics Review questions for exam

Econometrics Review questions for exam Nathaniel Higgins nhiggins@jhu.edu, 1. Suppose you have a model: y = β 0 x 1 + u You propose the model above and then estimate the model using OLS to obtain: ŷ =

L2: Review of probability and statistics

Probability L2: Review of probability and statistics Definition of probability Axioms and properties Conditional probability Bayes theorem Random variables Definition of a random variable Cumulative distribution

6.4 Type I and Type II Errors

6.4 Type I and Type II Errors Ulrich Hoensch Friday, March 22, 2013 Null and Alternative Hypothesis Neyman-Pearson Approach to Statistical Inference: A statistical test (also known as a hypothesis test)

University of California San Diego and Stanford University and

First International Workshop on Functional and Operatorial Statistics. Toulouse, June 19-21, 2008 K-sample Subsampling Dimitris N. olitis andjoseph.romano University of California San Diego and Stanford

Confidence Intervals for Normal Data Spring 2014

Confidence Intervals for Normal Data 18.05 Spring 2014 Agenda Today Review of critical values and quantiles. Computing z, t, χ 2 confidence intervals for normal data. Conceptual view of confidence intervals.

Hypothesis Tests and Estimation for Population Variances. Copyright 2014 Pearson Education, Inc.

Hypothesis Tests and Estimation for Population Variances 11-1 Learning Outcomes Outcome 1. Formulate and carry out hypothesis tests for a single population variance. Outcome 2. Develop and interpret confidence

The outline for Unit 3

The outline for Unit 3 Unit 1. Introduction: The regression model. Unit 2. Estimation principles. Unit 3: Hypothesis testing principles. 3.1 Wald test. 3.2 Lagrange Multiplier. 3.3 Likelihood Ratio Test.

Variable Selection and Model Building

LINEAR REGRESSION ANALYSIS MODULE XIII Lecture - 37 Variable Selection and Model Building Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur The complete regression

Institute for the Advancement of University Learning & Department of Statistics

Institute for the Advancement of University Learning & Department of Statistics Descriptive Statistics for Research (Hilary Term, 00) Lecture 7: Hypothesis Testing (I.) Introduction An important area of

CHAPTER 17 CHI-SQUARE AND OTHER NONPARAMETRIC TESTS FROM: PAGANO, R. R. (2007)

FROM: PAGANO, R. R. (007) I. INTRODUCTION: DISTINCTION BETWEEN PARAMETRIC AND NON-PARAMETRIC TESTS Statistical inference tests are often classified as to whether they are parametric or nonparametric Parameter

Statistics for Managers Using Microsoft Excel/SPSS Chapter 8 Fundamentals of Hypothesis Testing: One-Sample Tests

Statistics for Managers Using Microsoft Excel/SPSS Chapter 8 Fundamentals of Hypothesis Testing: One-Sample Tests 1999 Prentice-Hall, Inc. Chap. 8-1 Chapter Topics Hypothesis Testing Methodology Z Test

Introduction to Statistical Inference

Introduction to Statistical Inference Dr. Fatima Sanchez-Cabo f.sanchezcabo@tugraz.at http://www.genome.tugraz.at Institute for Genomics and Bioinformatics, Graz University of Technology, Austria Introduction

Statistical Analysis of Engineering Data The Bare Bones Edition. Precision, Bias, Accuracy, Measures of Precision, Propagation of Error

Statistical Analysis of Engineering Data The Bare Bones Edition (I) Precision, Bias, Accuracy, Measures of Precision, Propagation of Error PRIOR TO DATA ACQUISITION ONE SHOULD CONSIDER: 1. The accuracy

1; (f) H 0 : = 55 db, H 1 : < 55.

Reference: Chapter 8 of J. L. Devore s 8 th Edition By S. Maghsoodloo TESTING a STATISTICAL HYPOTHESIS A statistical hypothesis is an assumption about the frequency function(s) (i.e., pmf or pdf) of one

Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions

Journal of Modern Applied Statistical Methods Volume 8 Issue 1 Article 13 5-1-2009 Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error

Analysis of Variance (ANOVA)

Analysis of Variance (ANOVA) Used for comparing or more means an extension of the t test Independent Variable (factor) = categorical (qualita5ve) predictor should have at least levels, but can have many

Multiple Dependent Hypothesis Tests in Geographically Weighted Regression

Multiple Dependent Hypothesis Tests in Geographically Weighted Regression Graeme Byrne 1, Martin Charlton 2, and Stewart Fotheringham 3 1 La Trobe University, Bendigo, Victoria Austrlaia Telephone: +61

What p values really mean (and why I should care) Francis C. Dane, PhD

What p values really mean (and why I should care) Francis C. Dane, PhD Session Objectives Understand the statistical decision process Appreciate the limitations of interpreting p values Value the use of

Background to Statistics

FACT SHEET Background to Statistics Introduction Statistics include a broad range of methods for manipulating, presenting and interpreting data. Professional scientists of all kinds need to be proficient

HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007

MIT OpenCourseWare http://ocw.mit.edu HST.582J / 6.555J / 16.456J Biomedical Signal and Image Processing Spring 2007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Statistical Applications in Genetics and Molecular Biology

Statistical Applications in Genetics and Molecular Biology Volume 5, Issue 1 2006 Article 28 A Two-Step Multiple Comparison Procedure for a Large Number of Tests and Multiple Treatments Hongmei Jiang Rebecca

Reports of the Institute of Biostatistics

Reports of the Institute of Biostatistics No 01 / 2010 Leibniz University of Hannover Natural Sciences Faculty Titel: Multiple contrast tests for multiple endpoints Author: Mario Hasler 1 1 Lehrfach Variationsstatistik,

Lectures on Statistics. William G. Faris

Lectures on Statistics William G. Faris December 1, 2003 ii Contents 1 Expectation 1 1.1 Random variables and expectation................. 1 1.2 The sample mean........................... 3 1.3 The sample

Chapter 7: Hypothesis Testing - Solutions

Chapter 7: Hypothesis Testing - Solutions 7.1 Introduction to Hypothesis Testing The problem with applying the techniques learned in Chapter 5 is that typically, the population mean (µ) and standard deviation

Statistics 135: Fall 2004 Final Exam

Name: SID#: Statistics 135: Fall 2004 Final Exam There are 10 problems and the number of points for each is shown in parentheses. There is a normal table at the end. Show your work. 1. The designer of

Bootstrapping, Randomization, 2B-PLS

Bootstrapping, Randomization, 2B-PLS Statistics, Tests, and Bootstrapping Statistic a measure that summarizes some feature of a set of data (e.g., mean, standard deviation, skew, coefficient of variation,

Tutorial 5: Power and Sample Size for One-way Analysis of Variance (ANOVA) with Equal Variances Across Groups. Acknowledgements:

Tutorial 5: Power and Sample Size for One-way Analysis of Variance (ANOVA) with Equal Variances Across Groups Anna E. Barón, Keith E. Muller, Sarah M. Kreidler, and Deborah H. Glueck Acknowledgements:

Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

The One-Way Independent-Samples ANOVA. (For Between-Subjects Designs)

The One-Way Independent-Samples ANOVA (For Between-Subjects Designs) Computations for the ANOVA In computing the terms required for the F-statistic, we won t explicitly compute any sample variances or

Generalized Linear Models (1/29/13)

STA613/CBB540: Statistical methods in computational biology Generalized Linear Models (1/29/13) Lecturer: Barbara Engelhardt Scribe: Yangxiaolu Cao When processing discrete data, two commonly used probability

Lecture Slides. Elementary Statistics Eleventh Edition. by Mario F. Triola. and the Triola Statistics Series 9.1-1

Lecture Slides Elementary Statistics Eleventh Edition and the Triola Statistics Series by Mario F. Triola Copyright 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. 9.1-1 Chapter 9 Inferences

Chapter 22. Comparing Two Proportions 1 /29

Chapter 22 Comparing Two Proportions 1 /29 Homework p519 2, 4, 12, 13, 15, 17, 18, 19, 24 2 /29 Objective Students test null and alternate hypothesis about two population proportions. 3 /29 Comparing Two

Testing Research and Statistical Hypotheses

Testing Research and Statistical Hypotheses Introduction In the last lab we analyzed metric artifact attributes such as thickness or width/thickness ratio. Those were continuous variables, which as you

Chapter 1 Statistical Inference

Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

Direction: This test is worth 250 points and each problem worth points. DO ANY SIX

Term Test 3 December 5, 2003 Name Math 52 Student Number Direction: This test is worth 250 points and each problem worth 4 points DO ANY SIX PROBLEMS You are required to complete this test within 50 minutes