Stochastic Simulation

Similar documents
H 2 : otherwise. that is simply the proportion of the sample points below level x. For any fixed point x the law of large numbers gives that

Random Number Generation. CS1538: Introduction to simulations

Chapte The McGraw-Hill Companies, Inc. All rights reserved.

Systems Simulation Chapter 7: Random-Number Generation

What to do today (Nov 22, 2018)?

Stat 710: Mathematical Statistics Lecture 31

CPSC 531: Random Numbers. Jonathan Hudson Department of Computer Science University of Calgary

Declarative Statistics

Slides 3: Random Numbers

Qualifying Exam CS 661: System Simulation Summer 2013 Prof. Marvin K. Nakayama

MA : Introductory Probability

Recall the Basics of Hypothesis Testing

Dr. Maddah ENMG 617 EM Statistics 10/15/12. Nonparametric Statistics (2) (Goodness of fit tests)

ST4241 Design and Analysis of Clinical Trials Lecture 7: N. Lecture 7: Non-parametric tests for PDG data

Advanced Statistics II: Non Parametric Tests

Asymptotic Statistics-VI. Changliang Zou

Goodness-of-fit Tests for the Normal Distribution Project 1

Part 1.) We know that the probability of any specific x only given p ij = p i p j is just multinomial(n, p) where p k1 k 2

B.N.Bandodkar College of Science, Thane. Random-Number Generation. Mrs M.J.Gholba

Rank-Based Methods. Lukas Meier

Bivariate Paired Numerical Data

Section 3 : Permutation Inference

Contingency Tables. Safety equipment in use Fatal Non-fatal Total. None 1, , ,128 Seat belt , ,878

Investigation of goodness-of-fit test statistic distributions by random censored samples

Transformations and A Universal First Order Taylor Expansion

So far we discussed random number generators that need to have the maximum length period.

Has the crisis changed the monetary transmission mechanism in Albania? An application of kernel density estimation technique.

Lecture 2: CDF and EDF

Chapter 7: Hypothesis testing

Chapter 9: Hypothesis Testing Sections

An Analysis of College Algebra Exam Scores December 14, James D Jones Math Section 01

Categorical Data Analysis. The data are often just counts of how many things each category has.

THE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE

Convergence Concepts of Random Variables and Functions

Section 3: Permutation Inference

Robustness and Distribution Assumptions

Lehmer Random Number Generators: Introduction

Chapter 4. Theory of Tests. 4.1 Introduction

Violating the normal distribution assumption. So what do you do if the data are not normal and you still need to perform a test?

STAT5044: Regression and Anova

One-Sample Numerical Data

TESTS BASED ON EMPIRICAL DISTRIBUTION FUNCTION. Submitted in partial fulfillment of the requirements for the award of the degree of

Signal Time of Arrival based on the Kolmogorov-Smirnov Test

The Chi-Square Distributions

ISyE 6644 Fall 2014 Test 3 Solutions

Independent Events. Two events are independent if knowing that one occurs does not change the probability of the other occurring

14.30 Introduction to Statistical Methods in Economics Spring 2009

errors every 1 hour unless he falls asleep, in which case he just reports the total errors

d(x n, x) d(x n, x nk ) + d(x nk, x) where we chose any fixed k > N

Adv. App. Stat. Presentation On a paradoxical property of the Kolmogorov Smirnov two-sample test

Asymptotic results for empirical measures of weighted sums of independent random variables

INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY. Lecture -27 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

Some Statistical Inferences For Two Frequency Distributions Arising In Bioinformatics

UNIT 5:Random number generation And Variation Generation

Hochdimensionale Integration

Lecture 21. Hypothesis Testing II

10.4 Hypothesis Testing: Two Independent Samples Proportion

BEST TESTS. Abstract. We will discuss the Neymann-Pearson theorem and certain best test where the power function is optimized.

Lecture 6: The Normal distribution

Contingency Tables. Contingency tables are used when we want to looking at two (or more) factors. Each factor might have two more or levels.

1 The Glivenko-Cantelli Theorem

Hypothesis testing I. - In particular, we are talking about statistical hypotheses. [get everyone s finger length!] n =

Correlation. We don't consider one variable independent and the other dependent. Does x go up as y goes up? Does x go down as y goes up?

Supplementary Materials for Residuals and Diagnostics for Ordinal Regression Models: A Surrogate Approach

Sources of randomness

Random numbers and generators

EE/CpE 345. Modeling and Simulation. Fall Class 10 November 18, 2002

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede

Financial Econometrics and Quantitative Risk Managenent Return Properties

Modeling and Performance Analysis with Discrete-Event Simulation

Probability Methods in Civil Engineering Prof. Dr. Rajib Maity Department of Civil Engineering Indian Institute of Technology, Kharagpur

Computer Science, Informatik 4 Communication and Distributed Systems. Simulation. Discrete-Event System Simulation. Dr.

Finding Outliers in Monte Carlo Computations

2008 Winton. Statistical Testing of RNGs

S6880 #6. Random Number Generation #2: Testing RNGs

Inferential Statistics

Uniform random numbers generators

Probably Approximately Correct (PAC) Learning

HYPOTHESIS TESTING: THE CHI-SQUARE STATISTIC

Hypothesis testing:power, test statistic CMS:

2.1.3 The Testing Problem and Neave s Step Method

MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015

14 : Approximate Inference Monte Carlo Methods

Statistics 100A Homework 5 Solutions

A new method of nonparametric density estimation

Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing

Answers and expectations

Joseph O. Marker Marker Actuarial a Services, LLC and University of Michigan CLRS 2010 Meeting. J. Marker, LSMWP, CLRS 1

EE/CpE 345. Modeling and Simulation. Fall Class 9

Asymptotic results for empirical measures of weighted sums of independent random variables

MGMT 69000: Topics in High-dimensional Data Analysis Falll 2016

An invariant of metric spaces under bornologous equivalences

Goodness-of-Fit Considerations and Comparisons Example: Testing Consistency of Two Histograms

Stochastic Convergence, Delta Method & Moment Estimators

Kolmogorov-Smirnov Test for Goodness of Fit in an ordered sequence

Statistic Distribution Models for Some Nonparametric Goodness-of-Fit Tests in Testing Composite Hypotheses

AB.Q103.NOTES: Chapter 2.4, 3.1, 3.2 LESSON 1. Discovering the derivative at x = a: Slopes of secants and tangents to a curve

Uniform Random Binary Floating Point Number Generation

DUBLIN CITY UNIVERSITY

Simulation model input analysis

Transcription:

Stochastic Simulation APPM 7400 Lesson 3: Testing Random Number Generators Part II: Uniformity September 5, 2018 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 1/ 24

RNGs: Tests for Uniformity χ 2 test serial test Kolmogorov-Smirnov test Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 2/ 24

RNGs: Tests for Uniformity χ 2 Test Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 3/ 24

RNGs: Tests for Uniformity χ 2 Test Break up the unit interval into k bins. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 3/ 24

RNGs: Tests for Uniformity χ 2 Test Break up the unit interval into k bins. If data are uniform, expect n/k in each bin. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 3/ 24

RNGs: Tests for Uniformity χ 2 Test Break up the unit interval into k bins. If data are uniform, expect n/k in each bin. Do a χ 2 test to compare observed and expected values in each bin. (Make sure that you expect at least 5 in each bin.) Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 3/ 24

χ 2 Test In our sample with n = 100,000 and k = 20: Bin Observed Expected 1 4962 5000 2 4989 5000 3 5038 5000 4 5072 5000 (partial table). Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 4/ 24

χ 2 Test In our sample with n = 100,000 and k = 20: W = k (O i E i ) 2 5.776 E i i=1 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 5/ 24

χ 2 Test In our sample with n = 100,000 and k = 20: W = k (O i E i ) 2 5.776 E i i=1 Compare to the critical value χ 2 0.05 (19) = 30.14351 WE PASSED! Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 5/ 24

RNGs: Tests for Uniformity Serial Test (For Uniformity and Independence) Idea: Bunch up the data into m-dimensional vectors. If the individual uniform values are independent, the vectors should be uniformly distributed in the m-dimensional unit cube. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 6/ 24

Serial Test (For Uniformity and Independence) Example: 2 dimensions Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 7/ 24

Serial Test (For Uniformity and Independence) k by k grid Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 8/ 24

Serial Test (For Uniformity and Independence) k by k grid let O ij be the number of observations in the ijth bin Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 8/ 24

Serial Test (For Uniformity and Independence) k by k grid let O ij be the number of observations in the ijth bin if uniform and members of pairs independent, expect (n/2)/k 2 in each bin Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 8/ 24

Serial Test (For Uniformity and Independence) k by k grid let O ij be the number of observations in the ijth bin if uniform and members of pairs independent, expect (n/2)/k 2 in each bin now a standard χ 2 test Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 8/ 24

Serial Test (For Uniformity and Independence) k by k grid let O ij be the number of observations in the ijth bin if uniform and members of pairs independent, expect (n/2)/k 2 in each bin now a standard χ 2 test (Really we are testing for a local independence.) Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 8/ 24

Serial Test (For Uniformity and Local Independence) For our sample, I used a 20 by 20 grid. Of the 50,000 pairs... We expect 125 in each cell... W = 400 i=1 (O i E i ) 2 E i 422.4 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 9/ 24

Serial Test (For Uniformity and Local Independence) For our sample, I used a 20 by 20 grid. Of the 50,000 pairs... We expect 125 in each cell... W = 400 i=1 (O i E i ) 2 E i 422.4 χ 2 0.05 (399) = 446.5742 WE PASSED! Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 9/ 24

Serial Test (For Uniformity and Local Independence) Incidentally, k W χ 2 0.05 (k 1) result 2 10.873 7.815 failed 6 42.255 49.802 passed 10 122.512 123.225 passed Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 10/ 24

RNGs: Tests for Uniformity χ 2 test serial test Kolmogorov-Smirnov test Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 11/ 24

RNGs: Tests for Uniformity Kolmogorov-Smirnov Test Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 12/ 24

RNGs: Tests for Uniformity Kolmogorov-Smirnov Test Let F(x) = P(X x) be the cdf for the distribution. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 12/ 24

RNGs: Tests for Uniformity Kolmogorov-Smirnov Test Let F(x) = P(X x) be the cdf for the distribution. In the uniform(0,1) case: F(x) = x,0 x 1. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 12/ 24

RNGs: Tests for Uniformity Kolmogorov-Smirnov Test Let F(x) = P(X x) be the cdf for the distribution. In the uniform(0,1) case: F(x) = x,0 x 1. Compare this to the empirical distribution function : ˆF n (x) = #X i in the sample x n Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 12/ 24

RNGs: Tests for Uniformity Empirical and Hypothesized uniform CDFs 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 solid line is the empirical d.f. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 13/ 24

RNGs: Tests for Uniformity Empirical and Hypothesized uniform CDFs 0.0 0.2 0.4 0.6 0.8 1.0 (dashed line is uniform cdf) 0.0 0.2 0.4 0.6 0.8 1.0 solid line is the empirical d.f. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 13/ 24

RNGs: Tests for Uniformity Empirical and Hypothesized uniform CDFs 0.0 0.2 0.4 0.6 0.8 1.0 (dashed line is uniform cdf) (We can use the KS test for any distribution.) 0.0 0.2 0.4 0.6 0.8 1.0 solid line is the empirical d.f. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 13/ 24

K-S: Test for a Distribution If X 1,X 2,...,X n really come from the distribution with cdf F, the distance should be small. D = D n = max ˆF n (x) F(x) x Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 14/ 24

K-S: Test for a Distribution Computing the test statistic: Suppose we simulate 7 uniform(0,1) s and get: 0.6 0.2 0.5 0.9 0.1 0.4 0.2 (obviously simplified) Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 15/ 24

K-S: Test for a Distribution Put them in order: 0.6 0.2 0.5 0.9 0.1 0.4 0.2 0.1 0.2 0.2 0.4 0.5 0.6 0.9 Now the empirical cdf is: ˆF 7 (x) = 0 for x < 0.1 1/7 for 0.1 x < 0.2 3/7 for 0.2 x < 0.4 4/7 for 0.4 x < 0.5 5/7 for 0.5 x < 0.6 6/7 for 0.6 x < 0.9 1 for x 0.9 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 16/ 24

K-S: Test for a Distribution Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 17/ 24

K-S: Test for a Distribution 0.6 0.2 0.5 0.9 0.1 0.4 0.2 D 7 = 9 35 0.2571429 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 18/ 24

K-S: Test for a Distribution Let X (1),X (2),...,X (n) be the ordered sample. Then D n can be computed as D n = max{d + n,d n } where D + n D n = max 1 i n { i n F(X (i)) } = max 1 i n { F(X(i) ) i 1 n } (assuming non-repeating values) Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 19/ 24

K-S: Test for a Distribution We reject that this sample came from the proposed distribution if the empirical cdf is too far away from the true cdf of the proposed distribution. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 20/ 24

K-S: Test for a Distribution We reject that this sample came from the proposed distribution if the empirical cdf is too far away from the true cdf of the proposed distribution. ie: We reject if D n is too large. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 20/ 24

K-S: Test for a Distribution We reject that this sample came from the proposed distribution if the empirical cdf is too far away from the true cdf of the proposed distribution. ie: We reject if D n is too large. How large is large? Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 20/ 24

K-S: Test for a Distribution In the 1930 s, Kolmogorov and Smirnov showed that lim P( nd n t) = 1 2 n ( 1) i 1 e 2i2 t 2. i=1 So, for large sample sizes, you could assume that P( nd n t) 1 2 ( 1) i 1 e 2i2 t 2. and find the value of t that makes the right hand side 1 α for an α level test. i=1 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 21/ 24

K-S: Test for a Distribution For small samples, people have worked out and tabulated critical values, but there is no nice closed form solution. J. Pomerantz (1973) J. Durbin (1968) Good approximations for n > 40: α 0.20 0.10 0.05 0.02 0.01 c.v. 1.0730 n 1.2239 n 1.3581 n 1.5174 n 1.6276 n Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 22/ 24

K-S: Test for a Distribution For our small sample of size 7, D 7 = 9 35 0.2571429. From a table, the critical value for a 0.05 level test for n = 7 is 0.483. Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 23/ 24

K-S: Test for a Distribution For our small sample of size 7, D 7 = 9 35 0.2571429. From a table, the critical value for a 0.05 level test for n = 7 is 0.483. WE PASSED! Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 23/ 24

K-S: Test for a Distribution For our large sample of size 100,000, D 100000 = 0.001523922. The approximate critical value for a 0.05 level test for n = 100,000 is 1.3518 100000 0.004294689 Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 24/ 24

K-S: Test for a Distribution For our large sample of size 100,000, D 100000 = 0.001523922. The approximate critical value for a 0.05 level test for n = 100,000 is 1.3518 100000 0.004294689 WE PASSED! Lesson 3: Testing Random Number GeneratorsPart II: Uniformity Stochastic Simulation September 5, 2018 24/ 24

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback