Multivariate Simulations

Transcription

1 Multivariate Simulations Katarína Starinská Charles University Faculty of Mathematics and Physics Prague, Czech Republic Katarína Starinská Multivariate Simulations 1 / 23

2 Content 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 2 / 23

3 Content History 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 3 / 23

4 History Some History Earliest methods were manual (throwing dice, dealing cards, drawing balls from an urn) Mechanized devices in early 1900 s Later, methods based on electric circuits Many other schemes based on phone books, π etc. Advent of computing led to interest in numerical and arithmetic methods. Such methods generate numbers sequentially and are easily programmed. The most commonly used methods today Katarína Starinská Multivariate Simulations 4 / 23

5 Content Univariate Generation Techniques 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 5 / 23

6 Univariate Generation Techniques Inverse method X random variable with distribution function F(x) = P(X x). Inverse distribution function F 1 (u) = sup{x : F(x) u}. Let U Uni(0,1), then F 1 (U) = X. Cannot be generalized for multivariate generations. Acceptance-Rejection Method (next slide) Transformation Method The Box-Muller Method Marsaglia s Polar Method Normal Variance Mixture Composition Method Katarína Starinská Multivariate Simulations 6 / 23

7 Univariate Generation Techniques Acceptance-Rejection Method Simulate r.v. X (the target) from distribution with density f( ) and distribution function F( ). We start with a random variable Y (the proposal) with the density g( ) such that f(x) Cg(x), C < for all x. Given Y = x, one accepts Y and lets X = Y with probability f(x)/cg(x). Otherwise, a new Y is generated and one continues until eventual acceptance f(x) Cg(x) Katarína Starinská Multivariate x Simulations 7 / 23

8 Univariate Generation Techniques Algorithm C = max f(x) g(x) Generate Y from g( ) and U from Uni(0,1) If U > C f(y) g(y), reject and generate another Y and U Return Y Easy to prove that P ( Y x U < 1 C Number 1/C is called rate of acceptance. ) f(y) = F(x). g(y) Katarína Starinská Multivariate Simulations 8 / 23

9 Content Multivariate Generation Techniques 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 9 / 23

10 Multivariate Generation Techniques Independent Components Independent Components Apply the inverse-transform method or other generation method of our choice to each component individually. Example: Generate X = (X 1,...,X n ) for n-dimensional rectangle D = {(x 1,...,x n ) : a i x i b i,i = 1,...,n}. Components of X are independent and uniformly distributed X i U[a i,b i ], for i = 1,...,n. Inverse-transform method: X i = a i +(b i a i )U i, where U 1,...,U n are i.i.d. from U(0, 1). Katarína Starinská Multivariate Simulations 10 / 23

11 Multivariate Generation Techniques Independent Components Dependent Components No equivalent for inverse-transform method in multivariate generations On principle Conditional distributions Acceptance-Rejection method Composition method Transformation Katarína Starinská Multivariate Simulations 11 / 23

12 Multivariate Generation Techniques Conditional Distribution Approach Conditional Distribution Approach Need to know the conditional distributions of X i X i 1,...,X Generate X 1 = x 1 from the marginal distribution of X Generate X 2 = x 2 from the conditional distribution of X 2 given X 1 = x Generate X 3 = x 3 from the conditional distribution of X 3 given X 2 = x 2 and X 1 = x p. Generate X p = x p from the conditional distribution of X p given X p 1 = x p 1, X p 2 = x p 2,...,X 1 = x 1. Reduces the problem of generating a p-dimensional random vector into a series of p univariate generation problems. Requires the determination of p univariate distributions and their appropriate generation scheme. Katarína Starinská Multivariate Simulations 12 / 23

13 Multivariate Generation Techniques Conditional Distribution Approach Generation from Cauchy distribution: ( ) p +1 f(x) = π (p+1)/2 Γ (1+x x) (p+1)/2, x R p 2 Each component X i of X has a univariate Cauchy distribution and the conditional distribution of X m given X 1 = x 1,...,X m 1 = x m 1 is 1+ m 1 m i=1 x i t m where t m is a univariate Student s t distribution with m degrees of freedom. X 1 = tan(π(u 1/2)), where U is uniform (0,1). t variates: Y/ Z/m, where Y is standard normal and Z is independent Γ(m/2, 2) variate. Katarína Starinská Multivariate Simulations 13 / 23

14 Multivariate Generation Techniques Rejection Approach Rejection Approach Algorithm for univariate case can be generalized for more dimensions. Difficulty in finding a dominating function Cg for f if the dependence among components of X is strong. A logical choice for g is the density corresponding to independent components with the same marginal distributions as X. In most cases, however, as the dependencies in X increase, the extent to which g approximates f decreases, and thus the efficiency approaches zero. More complicated choices of g have the disadvantage of making the search for C = supf(x)/g(x) where x R p more difficult. Katarína Starinská Multivariate Simulations 14 / 23

15 Multivariate Generation Techniques Rejection Approach Example: 1. Generate X uniformly distributed in W, where W is a regular region. 2. If X G, accept Z = X as the random vector uniformly distributed over G. Otherwise, return to first step. Special case: G - n-dimensional unit ball, W - n-dimensional hypercube. Efficiency goes to zero as n goes to. Katarína Starinská Multivariate Simulations 15 / 23

16 Multivariate Generation Techniques Composition Method Composition Method Probability densities f i ( ), p i > 0,i = 1,...,r, r i=1 p i = 1 f(x) = r p i f i (x) i=1 Algorithm: 1. Let F j = j i=1 p i for j = 1,...,r 2. Let U Uni(0,1) 3. Set i = 1 4. While F i < U {i = i +1} 5. Return X = X i, a variate drawn from the density f i For queueing simulation where customer arrivals are composed of a mixture of different customers types, each with its own arrival pattern Katarína Starinská Multivariate Simulations 16 / 23

17 Multivariate Generation Techniques Transformation Approach Transformation Approach Represent X as a function of (usually independent) univariate random variables, each of which can be easily generated. Box-Muller transformation: Yields a pair of independent standard normal variates X 1 = ( 2lnU 1 ) 1/2 cos(2πu 2 ) X 2 = ( 2lnU 1 ) 1/2 sin(2πu 2 ) where U 1,U 2 are independent uniform (0,1) variates. Multivariate Cauchy distribution: X i = Z i /W, where Z i s and W are independent, the Z i s are standard normal and W is the square root of a Γ(1/2,1) variate. Katarína Starinská Multivariate Simulations 17 / 23

18 Multivariate Generation Techniques Transformation Approach How to find a particular transformation method to generate a multivariate distribution X specified by its density function f? 1. Apply invertible transformations to the components of X. Or try the probability integral transformation and compare to known multivariate distributions with uniform marginal distributions. 2. Consider transformations of X that simplify arguments of transcendental functions in the density f. 3. Attempt to decompose f as the (probabilistic) mixture f(x) = pf 1 (x)+(1 p)f 2 (x), where f 1 and f 2 are recognizable and easy to generate. 4. Track down earlier references to the distribution. Possibly an earlier author/inventor of the distribution was motivated by a physical process that could be modeled via mixtures, convolutions, or products of random variables. Katarína Starinská Multivariate Simulations 18 / 23

19 Content Multivariate Normal Distribution 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 19 / 23

20 Multivariate Normal Distribution Multivariate Normal Distribution The most popular multivariate distribution. X N(µ, Σ), Σ is symmetric, positive-definite matrix Many simulation methods. Cholesky decomposition is typically used Σ = LL, where L is a lower triangular matrix. Simulate Z N(0,I), then X = µ+lz R: package mvtnorm (Multivariate Normal and t Distributions) - Computes multivariate normal and t probabilities, quantiles, random deviates and densities. Katarína Starinská Multivariate Simulations 20 / 23

21 Content NORTA 1 History 2 Univariate Generation Techniques 3 Multivariate Generation Techniques Independent Components Conditional Distribution Approach Rejection Approach Composition Method Transformation Approach 4 Multivariate Normal Distribution 5 NORTA Katarína Starinská Multivariate Simulations 21 / 23

22 NORTA Normal To Anything The goal is to generate a r.v. X with the following properties X i F Xi, i = 1,...,k Corr(X) = Σ X, where Σ X is given Represent X as a transformation of a k-dimensional, standard multivariate normal vector Z with correlation matrix Σ Z. F 1 X 1 (Φ(Z 1 )) X = F 1 X 2 (Φ(Z 2 ))... F 1 X k (Φ(Z k )) Φ is a univariate standard normal distribution function and F 1 X denotes the inverse function. This transformation ensures, that X i has a desired marginal distribution. How to set Σ Z? Can be calculated for some cases (see Cario & Nelson, (1997)). Katarína Starinská Multivariate Simulations 22 / 23

23 NORTA Johnson M.E.: Multivariate Statistical Simulation, Wiley Series in Probability and Statistics, Ripley B.D.: Stochastic Simulation, Wiley Series in Probability and Statistics, Asmussen S., Glynn P.W.: Stochastic simulation: Algorithm and Analysis, Springer-Verlag, Cario M.C., Nelson B.L.: Modeling and generating random vectors with arbitrary marginal distributions and correlation matrix,technical Report, Download link Katarína Starinská Multivariate Simulations 23 / 23