Lecture 09: Sep 19, Randomness. Random Variables Sampling Probability Distributions Caching. James Balamuta STAT UIUC

Transcription

1 Lecture 09: Sep 19, 2018 Randomness Random Variables Sampling Probability Distributions Caching James Balamuta STAT UIUC

2 Announcements hw03 is due on Friday, Sep 21st, 6:00 PM Quiz 04 covers Week 3 CBTF. Window: Sep 18th - 20th EC Opportunity: Attend the Reflections and Projections Conference Write a 2-3 paragraph reflection about a talk and take a picture with the speaker for 1% extra credit. Fine Print Got caught using GitHub's web interface in hw01 or hw02? Let's chat.

3 Last Time SLR and MLR Estimating a linear regression with 2 parameters vs. p parameters Underlying design matrix construction Factors Provide a mapping of levels to indicator variables inside the design matrix.

4 Topic Outline Tools for Reproducibility Working with Data Simulations Visual Exploratory Data Analysis Unstructured Data Data Wrangling Functions Modeling Data Interactive Interfaces High Performance Computing

5 Topic Outline

6 Lecture Objectives Describe what a random variable is. Create and use a pseudo-random number generator. Generate random variables from probability distributions in R Explain the differences between sampling with and without replacement.

7 Random Variables

8 Definition: Sample Space contains all possible outcomes that can be obtained. They can be either: 1. discrete / finite in size, e.g. heads / tails, or 2. continuous / infinite in possibilities, e.g.weight / height Discrete Continuous "Heads" "Tails" 1 Coin 2 Coins US Height Distribution

9 Definition: Random Variables are variables that take on an unknown value in a sample space. P( X = Heads) = 0.5 P( X = Tails) = 0.5 Assumes a "fair" coin

10 Kinds of Randomness Source Source Pseudo-Random Number Generators (PRNGs) Deterministic Efficient Linear Congruential Method True Random Number Generators (TRNGs) Based on randomness from physical phenomena

11 Deterministic or Nondeterministic Source Same Input, Same Output Same Input, Different Outputs

12 LCM PRNG overview of LCM Algorithm Seed X 0, X 1, X 2, X 3,!, X n (0 X0 < m) Generated ( ) mod m X i+1 = ax i + c Generated Multiplier (0 a < m) Increment (0 c < m) Modulus (0 < m) Integer Range X 0,m 1 i! R = X i i m 0,1 ) Numeric Does not include 1

13 Prior generated value LCM Computations computation dependency X = ( ax + c)mod m i+1 i X 0 = 20, a = 13, c = 3, m = 64 ( X = 7 = 263mod64 = 13 ( 20) + 3)mod64 1 Simulated Integers ( X = 30 = 94mod64 = 13 ( 7) + 3)mod64 2 ( X = 9 = 393mod64 = 13 ( 30) + 3)mod64 3 ( X = 56 = 120mod64 = 13 ( 9) + 3)mod64 4 ( X = 27 = 731mod64 = 13 ( 56) + 3)mod64 5

14 Previously Modulus Operator a mod q 12 %% 7 # a = n*q + r => 12 = 1*7 + 5 # [1] 5 outer(9:1, 2:9, `%%`) # Compute the cross between X & Y mod 2 <int> mod 3 <int> mod 4 <int> mod 5 <int> mod 6 <int> mod 7 <int> mod 8 <int> mod 9 <int> x = x = x = x = x = x = x = x = x =

15 How could we generate Xi so that Ri includes 1?

16 # Setup values a = c = m = 2^32 # Initial Seed seed = as.numeric(sys.time()) * 1000 # Pre-allocation of numeric vector x = numeric(length = 2) Sample Implementation LCM Algorithm # Set initial value x[1] = seed x[2] = (a * x[1] + c) %% m # Real number (not including 1) r = x[2]/m

17 PRNG Periodicity a = 13, c = 0, m = 2 6 X i i X 0 = 1 X 0 = 2 X 0 = 3 X 0 =

18 Quality of R.V. uniqueness of numbers Independent Dependent a = , c = 12581, m = 2 32 a = 1229, c = 1, m = 2 11

19 Source XKCD 221: Random Number

20 Sampling

21 Definition: Sampling without Replacement involves picking values from a sample space and not adding them back for subsequent picks. In essence, a value may only be selected once. Available Available Picked

22 Definition: Sampling with Replacement involves picking values from a sample space, adding the picked value back for subsequent picks. In essence, a value may be selected multiple times.

23 Sampling picking r.v. values Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). Probabilities Weighting for each observation. Optional. sample(x = <sample-space>, size = <nobs>, replace = FALSE, prob = NULL)

24 Sampling without Replacement picking r.v. values once Set a seed to control reproducibility set.seed(90210) Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). sample(x = 8, size = 4, replace = FALSE) # [1] NB: Sample creates a vector of 10 from a single number. seq(1, 10) # [1]

25 Sampling without Replacement picking character r.v. values Set a seed to control reproducibility set.seed(42) Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). sample(x = c("heads", "tails"), size = 1, replace = FALSE) # [1] "tails"

26 Your Turn Modify the sample code such that it can roll a six-sided die once. Set a seed to control reproducibility set.seed(385) Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). Probabilities Weighting for each observation. Optional. sample(x = <sample-space>, size = <nobs>, replace = FALSE, prob = NULL)

27 Sampling with Replacement picking r.v. values Set a seed to control reproducibility set.seed( ) Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). sample(x = 10, size = 20, replace = TRUE) # [1]

28 Sampling with Replacement biased coin sample Set a seed to control reproducibility set.seed(376006) Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). Probabilities Weighting for each observation. Optional. y = sample(x = c("heads", "tails"), size = 10, replace = TRUE, prob = c(0.4, 0.6)) y # [1] "tails" "heads" "tails" "tails" "tails" "tails" "tails" "heads" "tails" "tails"

29 Frequencies and Proportions counting sampled data # Previously obtained sample under probability of 0.4/0.6 y # [1] "tails" "heads" "tails" "tails" "tails" "tails" "tails" "heads" "tails" "tails" # Count observations and/or obtain the proportion table(y) # heads tails # 2 8 prop.table(table(y)) # heads tails # # Why doesn't this match the requested probability of 0.4 / 0.6?

30 Your Turn Create a call to sample such that it can roll a "loaded" six-sided die twice with probabilities: 1 := 0.1, 2 := 0.3, 3 = 0.1, 4 := 0.15, 5 := 0.15, 6 := 0.2 Sample Space Either a vector or number to sample from. Sample Size Number of values to pick from sample space. Replacement Sample without/with replacement (FALSE/TRUE). Probabilities Weighting for each observation. Optional. sample(x = <sample-space>, size = <nobs>, replace = FALSE, prob = NULL) Source

31 Probability Distributions

32 Definition: Probability Distribution provides the outcome of a statistical event with the probability it occurs. Source

33 Supported Distributions possible distributions Distribution Functions Distribution Functions Beta pbeta qbeta dbeta rbeta Log Normal plnorm qlnorm dlnorm rlnorm Binomial pbinom qbinom dbinom rbinom Negative Binomial pnbinom qnbinom dnbinom rnbinom Cauchy pcauchy qcauchy dcauchy rcauchy Normal pnorm qnorm dnorm rnorm Chi-Squared pchisq qchisq dchisq rchisq Poison ppois qpois dpois rpois Exponential pexp qexp dexp rexp Student's t pt qt dt rt F pf qf df rf Studentized Range ptukey qtukey dtukey rtukey Gamma pgamma qgamma dgamma rgamma Uniform punif qunif dunif runif Geometric pgeom qgeom dgeom rgeom Weibull pweibull qweibull dweibull rweibull Hypergeometric phyper qhyper dhyper rhyper Wilcoxon Rank Sum pwilcox qwilcox dwilcox rwilcox Logistic plogis qlogis dlogis rlogis Wilcoxon Signed Rank psignrank qsignrank dsignrank rsignrank See for more Distributions the CRAN Task View

34 Normal Distribution an example of a probability distribution f ( x µ,σ ) = ( 1 exp x µ ) 2 σ 2π 2σ 2 µ is the mean σ is the standard deviation σ 2 is the variance

35 Distribution Functions d: density or the probability density function (PDF) p: probability or the cumulative distribution function (CDF) q: quantile or the inverse CDF r: random variable generation. # Normal PDF dnorm(c(-1, 0, 1)) # [1] # Normal CDF pnorm(c(-1, 0, 1)) # [1] # Normal icdf qnorm(c(0.15, 0.5, 0.84)) # [1] # Normal RNG set.seed(15) rnorm(2) # [1]

36 Cumulative and Density # Compute the probability pnorm(1.96) # [1] F X ( x) = P( X x) # Compute the density # at a specific point (uh oh) dnorm(1.96) # [1] # Only works for discrete # distributions.. Why? f X ( x) = F X ( x) # Integration required to # match output of pnorm integrate(dnorm(x), lower = -10, upper = 1.96 ) # with # absolute error < 2e-07

37 Quantiles finding the p-th quantile # Obtain the x value for # a probability of qnorm(0.975) # [1] # Note, this is taking the # inverse of pnorm, c.f. F X ( 1 F ( X p) ) = p x = F X 1 ( p) qnorm( pnorm( ) ) # [1] pnorm( qnorm(0.975) ) # [1] F X ( x) = P( X x)

38 Random Values generating lots of numbers # Set a seed set.seed(1) # Generate 1000 values # from a standard normal rnorm(1000) # [1] # [2] # [3] Box-Muller Transform

39 # Random Number Generation r*(n, param1 =, param2 = ) Generic Distribution Form pattern in statistical distribution functions # * represents a supported distribution (e.g. norm, t, f, chisq) # Density d*(x, param1 =, param2 =, log = FALSE) # Probability p*(q, param1 =, param2 =, lower.tail = TRUE, log.p = FALSE) # Quantile (inverse probability) q*(p, param1 =, param2 =, lower.tail = TRUE, log.p = FALSE)

40 Your Turn Using the qnorm() function, determine what the quantiles are at probabilities 0.5, 0.95, and Have you seen these values before?

41 Caching

42 Definition: Caches refers to storing data locally in order to speed up subsequent retrievals. Source

43 Cache in Action large value calculation * ```{r cache-large-computation, cache = TRUE} x = rnorm(10000*10000) system.time({ sorted_x = sort(x) }) ``` # user system elapsed # * The code chunk will be run once, the resulting objects are then stored within a file in the caches directory, and the stored data is then displayed on subsequent runs.

44 There are only two hard things in Computer Science: cache invalidation and naming things Phil Karlton

45 Warning Do not try the next example in an RMarkdown document with set.seed

46 Caution Required randomness and storage ```{r bad-sampling} x = rnorm(3) x ``` # [1] ```{r cached-calc, cache = TRUE} x y = x + 2 y ``` # [1] # [1]

47 Redux Caution Required re-running the aforementioned code ```{r bad-sampling} x = rnorm(3) x ``` # [1] ```{r cached-calc, cache = TRUE} x y = x + 2 y ``` # [1] # [1]

48 Depends On specifying dependency ```{r sampling, cache = TRUE} x = rnorm(3) x ``` # [1] ```{r cached-calc, cache = TRUE, dependson = "sampling"} x y = x + 2 y ``` # [1] # [1]

49 Recap Random Variables Variables that take on an unknown value LCM is one way to generate random numbers Distributions Probability Density/Mass Functions (PD/MF) is d* Cumulative Distribution Function (CDF) is p* Inverse CDF is q* Random Variables from Distribution is r* Sampling Sample without replacement does not add the picked object back. Sample with replacement does add the picked object back. Caches Speed up computationally intensive reports by storing results and re-using them.

50 This work is licensed under the Creative Commons Attribution-NonCommercial- ShareAlike 4.0 International License