# Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk

Save this PDF as:

Size: px
Start display at page:

Download "Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk"

## Transcription

1 Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy Department of Mathematics Imperial College London user! 2009, Rennes July 8-10, 2009

2 Introduction Test statistic T, reject for large values. Observation: t. p-value: p = P(T t) Often not available in closed form. Monte Carlo Test: ˆp naive = 1 n where T, T 1,... T n i.i.d. Examples: Bootstrap, Permutation tests. Goal: Estimate p using few X i n I(T i t), i=1 Mainly interested in deciding if p α for some α. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 2

3 Introduction Test statistic T, reject for large values. Observation: t. p-value: p = P(T t) Often not available in closed form. Monte Carlo Test: ˆp naive = 1 n where T, T 1,... T n i.i.d. Examples: Bootstrap, Permutation tests. Goal: Estimate p using few X i n I(T i t), }{{} =:X i B(1,p) i=1 Mainly interested in deciding if p α for some α. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 2

4 Introduction Test statistic T, reject for large values. Observation: t. p-value: p = P(T t) Often not available in closed form. Monte Carlo Test: ˆp naive = 1 n where T, T 1,... T n i.i.d. Examples: Bootstrap, Permutation tests. Goal: Estimate p using few X i n I(T i t), }{{} =:X i B(1,p) i=1 Mainly interested in deciding if p α for some α. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 2

5 Introduction Test statistic T, reject for large values. Observation: t. p-value: p = P(T t) Often not available in closed form. Monte Carlo Test: ˆp naive = 1 n where T, T 1,... T n i.i.d. Examples: Bootstrap, Permutation tests. Goal: Estimate p using few X i n I(T i t), }{{} =:X i B(1,p) i=1 Mainly interested in deciding if p α for some α. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 2

6 Sequential approaches based on S n = n i=1 X i S n Stop once S n U n or S n L n τ: hitting time Compute ˆp based on S τ and τ. Hit B U : decide p > α, Hit B L : decide p α, n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 3

7 Sequential approaches based on S n = n i=1 X i S n Stop once S n U n or S n L n τ: hitting time Compute ˆp based on S τ and τ. Hit B U : decide p > α, Hit B L : decide p α, n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 3

8 Sequential approaches based on S n = n i=1 X i S n Stop once S n U n or S n L n τ: hitting time Compute ˆp based on S τ and τ. Hit B U : decide p > α, Hit B L : decide p α, n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 3

9 Previous Approaches Besag & Clifford (1991): h S n 0 n 0 m (Truncated) Sequential Probability Ratio Test, Fay et al. (2007) h S n 0 0 m R-package MChtest. n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 4

10 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

11 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

12 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

13 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

14 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

15 What do we really want? Is p α? Two individuals using the same statistical method on the same data should arrive at the same conclusion. First law of applied statistics, Gleser (1996) Consider the resampling risk Want: RR p (ˆp) { P p (ˆp > α) if p α, P p (ˆp α) if p > α. sup RR p (ˆp) ɛ p [0,1] for some (small) ɛ > 0. For Besag & Clifford (1991), SPRT: sup p RR P 0.5 Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 5

16 Recursive Definition of the Boundaries Want: sup RR p (ˆp) ɛ p Suffices to ensure P α (hit B U ) ɛ P α (hit B L ) ɛ Recursive definition: Given U 1,..., U n 1 and L 1,..., L n 1, define U n as the minimal value such that P α (hit B U until n) ɛ n and L n as the maximal value such that P α (hit B L until n) ɛ n where ɛ n 0 with ɛ n ɛ (spending sequence). Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 6

17 Recursive Definition of the Boundaries Want: sup RR p (ˆp) ɛ p Suffices to ensure P α (hit B U ) ɛ P α (hit B L ) ɛ Recursive definition: Given U 1,..., U n 1 and L 1,..., L n 1, define U n as the minimal value such that P α (hit B U until n) ɛ n and L n as the maximal value such that P α (hit B L until n) ɛ n where ɛ n 0 with ɛ n ɛ (spending sequence). Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 6

18 Recursive Definition of the Boundaries Want: sup RR p (ˆp) ɛ p Suffices to ensure P α (hit B U ) ɛ P α (hit B L ) ɛ Recursive definition: Given U 1,..., U n 1 and L 1,..., L n 1, define U n as the minimal value such that P α (hit B U until n) ɛ n and L n as the maximal value such that P α (hit B L until n) ɛ n where ɛ n 0 with ɛ n ɛ (spending sequence). Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 6

19 Recursive Definition of the Boundaries Want: sup RR p (ˆp) ɛ p Suffices to ensure P α (hit B U ) ɛ P α (hit B L ) ɛ Recursive definition: Given U 1,..., U n 1 and L 1,..., L n 1, define U n as the minimal value such that P α (hit B U until n) ɛ n and L n as the maximal value such that P α (hit B L until n) ɛ n where ɛ n 0 with ɛ n ɛ (spending sequence). Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 6

20 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n P α (S n =k, τ n) 0 k= 3 k= 2 k= 1 k= 0 1 ɛ n 0 U n 1 L n -1 n = Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

21 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n P α (S n =k, τ n) 0 1 k= 3 k= 2 k= 1.2 k= ɛ n 0.07 U n 1 2 L n -1-1 n = Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

22 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n P α (S n =k, τ n) k= 3 k= 2.04 k= k= ɛ n U n L n n = Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

23 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n P α (S n =k, τ n) k= 3 k= k= k= ɛ n U n L n n = Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

24 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n n = P α (S n =k, τ n) k= 3 k= k= k= ɛ n U n L n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

25 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n n = P α (S n =k, τ n) k= 3.02 k= k= k= ɛ n U n L n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

26 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n n = P α (S n =k, τ n) k= k= k= k= ɛ n U n L n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

27 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n n = P α (S n =k, τ n) k= k= k= k= ɛ n U n L n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

28 Recursive Definition - Example α = 0.2, ɛ n = 0.4 n 5+n. U n =the minimal value such that L n = maximal value such that P α (hit B U until n) ɛ n P α (hit B L until n) ɛ n n = P α (S n =k, τ n) k= k= k= k= ɛ n U n L n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 7

29 Sequential Decision Procedure - Example α = 0.2, ɛ n = 0.4 n 5+n n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 8

30 Influence of ɛ on the stopping rule ɛ = 0.1, 0.001, 10 5, 10 7 ; ɛ n = ɛ n 1000+n n Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 9

31 Sequential Estimation based on the MLE One can show: S τ ˆp = τ, τ < α, τ =, hitting the upper boundary implies ˆp > α, hitting the lower boundary implies ˆp < α. Hence, sup RR p (ˆp) ɛ p Furthermore, random interval I n s.t. In only depends on X 1,..., X n, ˆp In. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 10

32 Example - Two-way sparse contingency table H 0 : variables are independent. Reject for large values of the likelihood ratio test statistic T T d χ 2 (7 1)(5 1) under H 0. Based on this: p = Matrix sparse - approximation poor? Use parametric bootstrap based on row and column sums. Naive test statistic ˆp naive with n = 1,000 replicates: p = < Probability of reporting p > 0.05: roughly Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 11

33 Example - Two-way sparse contingency table H 0 : variables are independent. Reject for large values of the likelihood ratio test statistic T T d χ 2 (7 1)(5 1) under H 0. Based on this: p = Matrix sparse - approximation poor? Use parametric bootstrap based on row and column sums. Naive test statistic ˆp naive with n = 1,000 replicates: p = < Probability of reporting p > 0.05: roughly Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 11

34 Example - Two-way sparse contingency table H 0 : variables are independent. Reject for large values of the likelihood ratio test statistic T T d χ 2 (7 1)(5 1) under H 0. Based on this: p = Matrix sparse - approximation poor? Use parametric bootstrap based on row and column sums. Naive test statistic ˆp naive with n = 1,000 replicates: p = < Probability of reporting p > 0.05: roughly Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 11

35 Example - Bootstrap and Sequential Algorithm > dat <- matrix(c(1,2,2,1,1,0,1, 2,0,0,2,3,0,0, 0,1,1,1,2,7,3, 1,1,2,0,0,0,1, + 0,1,1,1,1,0,0), nrow=5,ncol=7,byrow=true) > loglikrat <- function(data){ + cs <- colsums(data);rs <- rowsums(data); mu <- outer(rs,cs)/sum(rs) + 2*sum(ifelse(data<=0.5, 0,data*log(data/mu))) + } > resample <- function(data){ + cs <- colsums(data);rs <- rowsums(data); n <- sum(rs) + mu <- outer(rs,cs)/n/n + matrix(rmultinom(1,n,c(mu)),nrow=dim(data)[1],ncol=dim(data)[2]) + } > t <- loglikrat(dat); > library(simctest) > res <- simctest(function(){loglikrat(resample(dat))>=t},maxsteps=1000) > res No decision reached. Final estimate will be in [ , ] Current estimate of the p.value: Number of samples: 1000 > cont(res, steps=10000) p.value: Number of samples: 8574

36 Further Uses of the Algorithm Simulation study to evaluate whether a test is liberal/conservative. Determining the sample size to achieve a certain power. Iterated Use: Determining the power of a bootstrap test. Simulation study to evaluate whether a bootstrap test is liberal/conservative. Double bootstrap test. Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 13

37 Expected Hitting Time Result: E p (τ) < p α Example with α = 0.05, ɛ n = ɛ n 1000+n : E p (τ) ε = ε = 1e 05 ε = 1e 07 E p (τ) µ p p µ p = theoretical lower bound on E p (τ). Note: 1 0 µ pdp = ; for iterated use: Need to limit the number of steps.

38 Expected Hitting Time Result: E p (τ) < p α Example with α = 0.05, ɛ n = ɛ n 1000+n : E p (τ) ε = ε = 1e 05 ε = 1e 07 E p (τ) µ p p µ p = theoretical lower bound on E p (τ). Note: 1 0 µ pdp = ; for iterated use: Need to limit the number of steps.

39 Summary Sequential implementation of Monte Carlo Tests and computation of p-values. Useful when implementing tests in packages. After a finite number of steps: ˆp or interval [ˆp L n, ˆp U n ] in which ˆp will lie. Guarantee (up to a very small error probability): ˆp is on the correct side of α. R-package simctest available on CRAN. (efficient implementation with C-code) For details see Gandy (2009). Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 15

40 References Besag, J. & Clifford, P. (1991). Sequential Monte Carlo p-values. Biometrika 78, Davison, A. & Hinkley, D. (1997). Bootstrap methods and their application. Cambridge University Press. Fay, M. P., Kim, H.-J. & Hachey, M. (2007). On using truncated sequential probability ratio test boundaries for Monte Carlo implementation of hypothesis tests. Journal of Computational & Graphical Statistics 16, Gandy, A. (2009). Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk. Accepted for publication in JASA. Gleser, L. J. (1996). Comment on Bootstrap Confidence Intervals by T. J. DiCiccio and B. Efron. Statistical Science 11, Axel Gandy Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk 16

### Political Science 236 Hypothesis Testing: Review and Bootstrapping

Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall 2007 1 Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The

### Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference

1 / 171 Bootstrap inference Francisco Cribari-Neto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil email: cribari@gmail.com October 2013 2 / 171 Unpaid advertisement

### UNIVERSITÄT POTSDAM Institut für Mathematik

UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam

### Resampling and the Bootstrap

Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D-69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing

### STAT 730 Chapter 5: Hypothesis Testing

STAT 730 Chapter 5: Hypothesis Testing Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 28 Likelihood ratio test def n: Data X depend on θ. The

### Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process

Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 3083-3093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko Helmut-Schmidt University Hamburg University

### Post-exam 2 practice questions 18.05, Spring 2014

Post-exam 2 practice questions 18.05, Spring 2014 Note: This is a set of practice problems for the material that came after exam 2. In preparing for the final you should use the previous review materials,

### Package dhsic. R topics documented: July 27, 2017

Package dhsic July 27, 2017 Type Package Title Independence Testing via Hilbert Schmidt Independence Criterion Version 2.0 Date 2017-07-24 Author Niklas Pfister and Jonas Peters Maintainer Niklas Pfister

### Lecture 1: Random number generation, permutation test, and the bootstrap. August 25, 2016

Lecture 1: Random number generation, permutation test, and the bootstrap August 25, 2016 Statistical simulation 1/21 Statistical simulation (Monte Carlo) is an important part of statistical method research.

### large number of i.i.d. observations from P. For concreteness, suppose

1 Subsampling Suppose X i, i = 1,..., n is an i.i.d. sequence of random variables with distribution P. Let θ(p ) be some real-valued parameter of interest, and let ˆθ n = ˆθ n (X 1,..., X n ) be some estimate

### Bootstrapping high dimensional vector: interplay between dependence and dimensionality

Bootstrapping high dimensional vector: interplay between dependence and dimensionality Xianyang Zhang Joint work with Guang Cheng University of Missouri-Columbia LDHD: Transition Workshop, 2014 Xianyang

### One-Sample Numerical Data

One-Sample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodness-of-fit tests University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html

### 8 Error analysis: jackknife & bootstrap

8 Error analysis: jackknife & bootstrap As discussed before, it is no problem to calculate the expectation values and statistical error estimates of normal observables from Monte Carlo. However, often

### Direct Simulation Methods #2

Direct Simulation Methods #2 Econ 690 Purdue University Outline 1 A Generalized Rejection Sampling Algorithm 2 The Weighted Bootstrap 3 Importance Sampling Rejection Sampling Algorithm #2 Suppose we wish

### Multivariate Time Series: VAR(p) Processes and Models

Multivariate Time Series: VAR(p) Processes and Models A VAR(p) model, for p > 0 is X t = φ 0 + Φ 1 X t 1 + + Φ p X t p + A t, where X t, φ 0, and X t i are k-vectors, Φ 1,..., Φ p are k k matrices, with

### The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study

MATEMATIKA, 2012, Volume 28, Number 1, 35 48 c Department of Mathematics, UTM. The Goodness-of-fit Test for Gumbel Distribution: A Comparative Study 1 Nahdiya Zainal Abidin, 2 Mohd Bakri Adam and 3 Habshah

### Introduction to Survey Analysis!

Introduction to Survey Analysis! Professor Ron Fricker! Naval Postgraduate School! Monterey, California! Reading Assignment:! 2/22/13 None! 1 Goals for this Lecture! Introduction to analysis for surveys!

### Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering

Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Axel Gandy Department of Mathematics Imperial College London http://www2.imperial.ac.uk/~agandy London

### Bootstrapping, Randomization, 2B-PLS

Bootstrapping, Randomization, 2B-PLS Statistics, Tests, and Bootstrapping Statistic a measure that summarizes some feature of a set of data (e.g., mean, standard deviation, skew, coefficient of variation,

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

### Double Bootstrap Confidence Intervals in the Two Stage DEA approach. Essex Business School University of Essex

Double Bootstrap Confidence Intervals in the Two Stage DEA approach D.K. Chronopoulos, C. Girardone and J.C. Nankervis Essex Business School University of Essex 1 Determinants of efficiency DEA can be

### Bootstrap Methods and their Application

Bootstrap Methods and their Application Anthony Davison c 2006 A short course based on the book Bootstrap Methods and their Application, by A C Davison and D V Hinkley c Cambridge University Press, 1997

### Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

### STATISTICS-STAT (STAT)

Statistics-STAT (STAT) 1 STATISTICS-STAT (STAT) Courses STAT 158 Introduction to R Programming Credit: 1 (1-0-0) Programming using the R Project for the Statistical Computing. Data objects, for loops,

### P Values and Nuisance Parameters

P Values and Nuisance Parameters Luc Demortier The Rockefeller University PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics CERN, Geneva, June 27 29, 2007 Definition and interpretation of p values;

### Particle Filters. Outline

Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical

### where x i and u i are iid N (0; 1) random variates and are mutually independent, ff =0; and fi =1. ff(x i )=fl0 + fl1x i with fl0 =1. We examine the e

Inference on the Quantile Regression Process Electronic Appendix Roger Koenker and Zhijie Xiao 1 Asymptotic Critical Values Like many other Kolmogorov-Smirnov type tests (see, e.g. Andrews (1993)), the

### Markov Chains. Arnoldo Frigessi Bernd Heidergott November 4, 2015

Markov Chains Arnoldo Frigessi Bernd Heidergott November 4, 2015 1 Introduction Markov chains are stochastic models which play an important role in many applications in areas as diverse as biology, finance,

### ECE521 Lecture7. Logistic Regression

ECE521 Lecture7 Logistic Regression Outline Review of decision theory Logistic regression A single neuron Multi-class classification 2 Outline Decision theory is conceptually easy and computationally hard

### DEPARTMENT OF STATISTICS

Tests for causaly between integrated variables using asymptotic and bootstrap distributions R Scott Hacker and Abdulnasser Hatemi-J October 2003 2003:2 DEPARTMENT OF STATISTICS S-220 07 LUND SWEDEN Tests

### Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

### Figure 1: The fitted line using the shipment route-number of ampules data. STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim

0.0 1.0 1.5 2.0 2.5 3.0 8 10 12 14 16 18 20 22 y x Figure 1: The fitted line using the shipment route-number of ampules data STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim Problem#

### Computer Intensive Methods in Mathematical Statistics

Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of

### Outline. Mixed models in R using the lme4 package Part 3: Longitudinal data. Sleep deprivation data. Simple longitudinal data

Outline Mixed models in R using the lme4 package Part 3: Longitudinal data Douglas Bates Longitudinal data: sleepstudy A model with random effects for intercept and slope University of Wisconsin - Madison

### Characterizing Forecast Uncertainty Prediction Intervals. The estimated AR (and VAR) models generate point forecasts of y t+s, y ˆ

Characterizing Forecast Uncertainty Prediction Intervals The estimated AR (and VAR) models generate point forecasts of y t+s, y ˆ t + s, t. Under our assumptions the point forecasts are asymtotically unbiased

### PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007

Cohort study s formulations PB HLTH 240A: Advanced Categorical Data Analysis Fall 2007 Srine Dudoit Division of Biostatistics Department of Statistics University of California, Berkeley www.stat.berkeley.edu/~srine

### Package mcmcse. July 4, 2017

Version 1.3-2 Date 2017-07-03 Title Monte Carlo Standard Errors for MCMC Package mcmcse July 4, 2017 Author James M. Flegal , John Hughes , Dootika Vats ,

### ROBUSTNESS OF TWO-PHASE REGRESSION TESTS

REVSTAT Statistical Journal Volume 3, Number 1, June 2005, 1 18 ROBUSTNESS OF TWO-PHASE REGRESSION TESTS Authors: Carlos A.R. Diniz Departamento de Estatística, Universidade Federal de São Carlos, São

### Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data

Journal of Modern Applied Statistical Methods Volume 13 Issue 2 Article 22 11-2014 Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data Jayanthi Arasan University Putra Malaysia,

### Uniform Random Number Generators

JHU 553.633/433: Monte Carlo Methods J. C. Spall 25 September 2017 CHAPTER 2 RANDOM NUMBER GENERATION Motivation and criteria for generators Linear generators (e.g., linear congruential generators) Multiple

### A Bootstrap Test for Causality with Endogenous Lag Length Choice. - theory and application in finance

CESIS Electronic Working Paper Series Paper No. 223 A Bootstrap Test for Causality with Endogenous Lag Length Choice - theory and application in finance R. Scott Hacker and Abdulnasser Hatemi-J April 200

### Theory and Methods of Statistical Inference. PART I Frequentist theory and methods

PhD School in Statistics cycle XXVI, 2011 Theory and Methods of Statistical Inference PART I Frequentist theory and methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution

Self Adaptive Particle Filter Alvaro Soto Pontificia Universidad Catolica de Chile Department of Computer Science Vicuna Mackenna 4860 (143), Santiago 22, Chile asoto@ing.puc.cl Abstract The particle filter

### Final Overview. Introduction to ML. Marek Petrik 4/25/2017

Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,

### VAR Models and Cointegration 1

VAR Models and Cointegration 1 Sebastian Fossati University of Alberta 1 These slides are based on Eric Zivot s time series notes available at: http://faculty.washington.edu/ezivot The Cointegrated VAR

### The assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and p-values

Statistical Consulting Topics The Bootstrap... The bootstrap is a computer-based method for assigning measures of accuracy to statistical estimates. (Efron and Tibshrani, 1998.) What do we do when our

### Advanced Statistics II: Non Parametric Tests

Advanced Statistics II: Non Parametric Tests Aurélien Garivier ParisTech February 27, 2011 Outline Fitting a distribution Rank Tests for the comparison of two samples Two unrelated samples: Mann-Whitney

### arxiv: v2 [math.st] 20 Jul 2016

arxiv:1603.02791v2 [math.st] 20 Jul 2016 Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals Y. Song and G. Fellouris Department of Statistics,

### Theory and Methods of Statistical Inference

PhD School in Statistics cycle XXIX, 2014 Theory and Methods of Statistical Inference Instructors: B. Liseo, L. Pace, A. Salvan (course coordinator), N. Sartori, A. Tancredi, L. Ventura Syllabus Some prerequisites:

### An introduction to Sequential Monte Carlo

An introduction to Sequential Monte Carlo Thang Bui Jes Frellsen Department of Engineering University of Cambridge Research and Communication Club 6 February 2014 1 Sequential Monte Carlo (SMC) methods

### Modern Likelihood-Frequentist Inference. Donald A Pierce, OHSU and Ruggero Bellio, Univ of Udine

Modern Likelihood-Frequentist Inference Donald A Pierce, OHSU and Ruggero Bellio, Univ of Udine Shortly before 1980, important developments in frequency theory of inference were in the air. Strictly, this

### Bivariate Data: Graphical Display The scatterplot is the basic tool for graphically displaying bivariate quantitative data.

Bivariate Data: Graphical Display The scatterplot is the basic tool for graphically displaying bivariate quantitative data. Example: Some investors think that the performance of the stock market in January

### An introduction to particle filters

An introduction to particle filters Andreas Svensson Department of Information Technology Uppsala University June 10, 2014 June 10, 2014, 1 / 16 Andreas Svensson - An introduction to particle filters Outline

### Ch. 5 Hypothesis Testing

Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,

### Previous lecture. Single variant association. Use genome-wide SNPs to account for confounding (population substructure)

Previous lecture Single variant association Use genome-wide SNPs to account for confounding (population substructure) Estimation of effect size and winner s curse Meta-Analysis Today s outline P-value

### Chris Fraley and Daniel Percival. August 22, 2008, revised May 14, 2010

Model-Averaged l 1 Regularization using Markov Chain Monte Carlo Model Composition Technical Report No. 541 Department of Statistics, University of Washington Chris Fraley and Daniel Percival August 22,

### Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions

Journal of Modern Applied Statistical Methods Volume 8 Issue 1 Article 13 5-1-2009 Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error

### Alpha-Investing. Sequential Control of Expected False Discoveries

Alpha-Investing Sequential Control of Expected False Discoveries Dean Foster Bob Stine Department of Statistics Wharton School of the University of Pennsylvania www-stat.wharton.upenn.edu/ stine Joint

### EXACT AND ASYMPTOTICALLY ROBUST PERMUTATION TESTS. Eun Yi Chung Joseph P. Romano

EXACT AND ASYMPTOTICALLY ROBUST PERMUTATION TESTS By Eun Yi Chung Joseph P. Romano Technical Report No. 20-05 May 20 Department of Statistics STANFORD UNIVERSITY Stanford, California 94305-4065 EXACT AND

### Theory and Methods of Statistical Inference. PART I Frequentist likelihood methods

PhD School in Statistics XXV cycle, 2010 Theory and Methods of Statistical Inference PART I Frequentist likelihood methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution

### Statistical Applications in Genetics and Molecular Biology

Statistical Applications in Genetics and Molecular Biology Volume 5, Issue 1 2006 Article 28 A Two-Step Multiple Comparison Procedure for a Large Number of Tests and Multiple Treatments Hongmei Jiang Rebecca

### Two examples of the use of fuzzy set theory in statistics. Glen Meeden University of Minnesota.

Two examples of the use of fuzzy set theory in statistics Glen Meeden University of Minnesota http://www.stat.umn.edu/~glen/talks 1 Fuzzy set theory Fuzzy set theory was introduced by Zadeh in (1965) as

### Sequential Monte Carlo Methods

University of Pennsylvania Bradley Visitor Lectures October 23, 2017 Introduction Unfortunately, standard MCMC can be inaccurate, especially in medium and large-scale DSGE models: disentangling importance

### AVOIDING BOUNDARY ESTIMATES IN LATENT CLASS ANALYSIS BY BAYESIAN POSTERIOR MODE ESTIMATION

Behaviormetrika Vol33, No1, 2006, 43 59 AVOIDING BOUNDARY ESTIMATES IN LATENT CLASS ANALYSIS BY BAYESIAN POSTERIOR MODE ESTIMATION Francisca Galindo Garre andjeroenkvermunt In maximum likelihood estimation

### Lecture 22: Error exponents in hypothesis testing, GLRT

10-704: Information Processing and Learning Spring 2012 Lecture 22: Error exponents in hypothesis testing, GLRT Lecturer: Aarti Singh Scribe: Aarti Singh Disclaimer: These notes have not been subjected

### Bias-Correction in Vector Autoregressive Models: A Simulation Study

Econometrics 2014, 2, 45-71; doi:10.3390/econometrics2010045 OPEN ACCESS econometrics ISSN 2225-1146 www.mdpi.com/journal/econometrics Article Bias-Correction in Vector Autoregressive Models: A Simulation

### Probabilistic Graphical Models

2016 Robert Nowak Probabilistic Graphical Models 1 Introduction We have focused mainly on linear models for signals, in particular the subspace model x = Uθ, where U is a n k matrix and θ R k is a vector

### Statistics 135: Fall 2004 Final Exam

Name: SID#: Statistics 135: Fall 2004 Final Exam There are 10 problems and the number of points for each is shown in parentheses. There is a normal table at the end. Show your work. 1. The designer of

### Greene, Econometric Analysis (6th ed, 2008)

EC771: Econometrics, Spring 2010 Greene, Econometric Analysis (6th ed, 2008) Chapter 17: Maximum Likelihood Estimation The preferred estimator in a wide variety of econometric settings is that derived

### Moment Condition Selection for Dynamic Panel Data Models

Moment Condition Selection for Dynamic Panel Data Models Dylan Small Department of Statistics The Wharton School, University of Pennsylvania Joint work with Tze Leung Lai and Jia Liu Dynamic Panel Data

### Introduction to Statistical Inference

Introduction to Statistical Inference Dr. Fatima Sanchez-Cabo f.sanchezcabo@tugraz.at http://www.genome.tugraz.at Institute for Genomics and Bioinformatics, Graz University of Technology, Austria Introduction

### A Simulation Comparison Study for Estimating the Process Capability Index C pm with Asymmetric Tolerances

Available online at ijims.ms.tku.edu.tw/list.asp International Journal of Information and Management Sciences 20 (2009), 243-253 A Simulation Comparison Study for Estimating the Process Capability Index

### Statistics 135 Fall 2008 Final Exam

Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations

### The New Stats Engine

The New Stats Engine Leo Pekelis 1,2, David Walsh 1,2, and Ramesh Johari 1,3 1 Optimizely 2 Department of Statistics, Stanford University 3 Department of Management Science & Engineering, Stanford University

### [y i α βx i ] 2 (2) Q = i=1

Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation

### Analysis of Variance

Analysis of Variance Blood coagulation time T avg A 62 60 63 59 61 B 63 67 71 64 65 66 66 C 68 66 71 67 68 68 68 D 56 62 60 61 63 64 63 59 61 64 Blood coagulation time A B C D Combined 56 57 58 59 60 61

### Probabilistic Graphical Models

Parameter Estimation December 14, 2015 Overview 1 Motivation 2 3 4 What did we have so far? 1 Representations: how do we model the problem? (directed/undirected). 2 Inference: given a model and partially

### Hotelling s One- Sample T2

Chapter 405 Hotelling s One- Sample T2 Introduction The one-sample Hotelling s T2 is the multivariate extension of the common one-sample or paired Student s t-test. In a one-sample t-test, the mean response

### Finding Relationships Among Variables

Finding Relationships Among Variables BUS 230: Business and Economic Research and Communication 1 Goals Specific goals: Re-familiarize ourselves with basic statistics ideas: sampling distributions, hypothesis

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........

### Inference via Kernel Smoothing of Bootstrap P Values

Queen s Economics Department Working Paper No. 1054 Inference via Kernel Smoothing of Bootstrap P Values Jeff Racine McMaster University James G. MacKinnon Queen s University Department of Economics Queen

### EMPIRICAL BAYES IN THE PRESENCE OF EXPLANATORY VARIABLES

Statistica Sinica 23 (2013), 333-357 doi:http://dx.doi.org/10.5705/ss.2011.071 EMPIRICAL BAYES IN THE PRESENCE OF EXPLANATORY VARIABLES Noam Cohen 1, Eitan Greenshtein 1 and Ya acov Ritov 2 1 Israeli CBS

### 4 Hypothesis testing. 4.1 Types of hypothesis and types of error 4 HYPOTHESIS TESTING 49

4 HYPOTHESIS TESTING 49 4 Hypothesis testing In sections 2 and 3 we considered the problem of estimating a single parameter of interest, θ. In this section we consider the related problem of testing whether

### Package multivariance

Package multivariance January 10, 2018 Title Measuring Multivariate Dependence Using Distance Multivariance Version 1.1.0 Date 2018-01-09 Distance multivariance is a measure of dependence which can be

### Problem Set 2. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 30

Problem Set MAS 6J/1.16J: Pattern Recognition and Analysis Due: 5:00 p.m. on September 30 [Note: All instructions to plot data or write a program should be carried out using Matlab. In order to maintain

### AFT Models and Empirical Likelihood

AFT Models and Empirical Likelihood Mai Zhou Department of Statistics, University of Kentucky Collaborators: Gang Li (UCLA); A. Bathke; M. Kim (Kentucky) Accelerated Failure Time (AFT) models: Y = log(t

### A Maximally Controllable Indifference-Zone Policy

A Maimally Controllable Indifference-Zone Policy Peter I. Frazier Operations Research and Information Engineering Cornell University Ithaca, NY 14853, USA February 21, 2011 Abstract We consider the indifference-zone

### Quantifying Weather Risk Analysis

Quantifying Weather Risk Analysis Now that an index has been selected and calibrated, it can be used to conduct a more thorough risk analysis. The objective of such a risk analysis is to gain a better

### Efficient Importance Sampling for Rare Event Simulation with Applications

Efficient Importance Sampling for Rare Event Simulation with Applications Cheng-Der Fuh, Huei-Wen Teng and Ren-Her Wang February 11, 2013 arxiv:1302.0583v1 [stat.me] 4 Feb 2013 Abstract Importance sampling

### Topic 5: Generalized Likelihood Ratio Test

CS 885: Advanced Machine Learning Fall 7 Topic 5: Generalized Likelihood Ratio Test Instructor: Daniel L. Pimentel-Alarcón Copyright 7 5. Introduction We already started studying composite hypothesis problems

### Estimation of a Two-component Mixture Model

Estimation of a Two-component Mixture Model Bodhisattva Sen 1,2 University of Cambridge, Cambridge, UK Columbia University, New York, USA Indian Statistical Institute, Kolkata, India 6 August, 2012 1 Joint

### Introduction to SparseGrid

Introduction to SparseGrid Jelmer Ypma July 31, 2013 Abstract This vignette describes how to use SparseGrid, which is an R translation 1 of the Matlab code on http://www.sparse-grids.de (Heiss and Winschel,

### Probabilistic Graphical Models

Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft

### HYPOTHESIS TESTING: FREQUENTIST APPROACH.

HYPOTHESIS TESTING: FREQUENTIST APPROACH. These notes summarize the lectures on (the frequentist approach to) hypothesis testing. You should be familiar with the standard hypothesis testing from previous

### Chapter 1. Bayesian Inference for D-vines: Estimation and Model Selection

Chapter 1 Bayesian Inference for D-vines: Estimation and Model Selection Claudia Czado and Aleksey Min Technische Universität München, Zentrum Mathematik, Boltzmannstr. 3, 85747 Garching, Germany cczado@ma.tum.de

### Testing an Autoregressive Structure in Binary Time Series Models

ömmföäflsäafaäsflassflassflas ffffffffffffffffffffffffffffffffffff Discussion Papers Testing an Autoregressive Structure in Binary Time Series Models Henri Nyberg University of Helsinki and HECER Discussion

### Are Declustered Earthquake Catalogs Poisson?

Are Declustered Earthquake Catalogs Poisson? Philip B. Stark Department of Statistics, UC Berkeley Brad Luen Department of Mathematics, Reed College 14 October 2010 Department of Statistics, Penn State

### Reports of the Institute of Biostatistics

Reports of the Institute of Biostatistics No 01 / 2010 Leibniz University of Hannover Natural Sciences Faculty Titel: Multiple contrast tests for multiple endpoints Author: Mario Hasler 1 1 Lehrfach Variationsstatistik,