Stat410 Probability and Statistics II (F16)

Transcription

1 Stat4 Probability and Statistics II (F6 Exponential, Poisson and Gamma Suppose on average every /λ hours, a Stochastic train arrives at the Random station. Further we assume the waiting time between two adjacent train follows independent Ex(λ. Next we show that N(t the number of trains arrived during the time interval [, t] (from time to t hours Po(λt. X k waiting time (in hours for the kth train Ga(k, β /λ. In particular, X Ga(, β /λ Ex(λ. Let s set our arrival time at the station as time, and let X waiting time for the first train afterwards. Does it make sense to model X by Ex(λ, since the assumption that the waiting time between two adjacent trains follows Ex(λ seems to only imply that X + Y Ex(λ, where Y denotes the time between the arrival time of the previous train and our arrival time? The answer is Yes, and it s related to the memoryless property of the Exponential distribution. Find the distribution of X 2 X + W where X and W are independent and follow Ex(λ. Using the convolution formula, we have f X2 (x f X (x f W (x x dx λe λx λe λ(x x dx λ 2 e λx dx λ 2 e λx dx λ 2 xe λx, where the integration range for x is (, x because x > (the original constraint on x and 2 x x > (w x x and w >. Find the distribution of X 3 X 2 + W where f X2 (x is given above and W follows an independent Ex(λ. Using the convolution formula, we have f X3 (x f X2 (x 2 f W (x x 2 dx 2 λ 2 x 2 e λx 2 λe λ(x x 2 dx 2 λ 3 x 2 e λx dx 2 λ 3 λx x2 e 2. We can prove by induction that the pdf for X k is f Xk (x λk x k (k! e λx. (

2 Stat4 Probability and Statistics II (F6 Suppose ( holds for k and then use the convolution formula to show that ( also holds for X k X k + W where W follows an independent Ex(λ. The distribution ( is also known as the Erlang distribution, a special case of the Gamma distribution. Show that N(t Po(λt, that is, for any integer k,. When k, 2. For any k >, P ( N(t k (λtk e λt. (2 k! P ( N(t P(X > t e λt (λt e λt.! P ( N(t k P(X k t AND X k+ > t P(X k t AND X k + W > t. Using the joint pdf for (X k, W and the appropriate integration region, we have P ( N(t k The Gamma Distribution λ k x k (k! e λx( t x λ k x k (k! e λx e λ(t x dx λ k (k! e λt λe λw dw dx x k dx λk t k (k! e λt. X Ga(α, β /λ α > : shape parameter ; β > : scale parameter. pdf OR f(x f(x Γ(αβ α xα e x/β, x > ; λα Γ(α xα e λx, x >. where Γ( is called the Gamma function. Recall some results on Gamma functions: Γ(α (α Γ(α, Γ(n (n!, Γ(, Γ( 2 π. mean and variance E(X αβ, Var(X αβ 2. 2

3 Stat4 Probability and Statistics II (F6 mgf M(t ( βt α, t < β. Ex(λ Ga(, /λ. χ 2 (r Ga(r/2, 2. Recall the mgf of χ 2 (r is given by ( 2t r/2. X Ga(α, β where α is an integer, then 2Y/β χ 2 (2α. This can be proved easily by computing the mgf of 2Y/β. Scaling sga(α, β Ga(α, sβ Additivity of two independent Gammas that have the same scale parameter: Ga(α, β + Ga(α 2, β Ga(α + α 2, β. Especially, X Ex(λ and Y Ex(λ, then X + Y Ga(2, /λ; X χ 2 (r and Y χ 2 (r 2, then X + Y χ 2 (r + r 2. Jensen s Inequality g is a convex function if tg(t + ( tg(t 2 g ( tx + ( tx 2, for any t [, ], and x, x 2, tx + ( tx 2 are all in the support of g. If g is a convex on an open interval I and X is a random variable whose support is contained in I, then E [ g(x ] g(ex, provided that both expectations exist. If g is strictly convex, then the equality holds only when X is a constant random variable. Recall that the equality holds if g is a linear function. Use Jensen s inequality to show the following results. E(X 2 (EX 2 (of course, this result can also be obtained from Var(X. Ee tx e te(x, therefore, M X (t e tµ. E ( X E(X for a positive random variable X. E[ln X] ln E(X for a positive random variable X. 3

4 Stat4 Probability and Statistics II (F6 (Feel free the skip the remaining material on KL divergence. How to measure the distance between two distributions/random variables? The Kullback-Leibler divergence between two distributions is defined to be x p (x log p (x p 2 (x, for two discrete distributions; f (x log f (x f 2 (x dx. for two continuous distributions, For example, the KL divergence between Bern(p and Bern(p 2 is p log p p 2 + ( p log p p 2, and the one between N(µ, σ 2 and N(µ 2, σ 2 is { exp (x µ 2 2πσ 2 2σ 2 { exp (x µ 2 2πσ 2 2σ 2 } log e (x µ 2 /(2σ 2 e (x µ 2 2 /(2σ 2 dx } [ (x µ 2 2σ 2 + (x µ 2 2 2σ 2 E X 2σ 2 [ (X µ2 2 (X µ 2] X N(µ, σ 2 2σ 2 [ (µ µ σ 2 σ 2] (µ µ 2 2 2σ 2 ] dx Note that the KL divergence is not necessarily symmetric, so it is not a distance metric in the strict sense, but like any other distance metric, the KL divergence is always nonnegative and equals zero if and only the two arguments (two distributions are the same, which can be proved using Jensen s inequality. For the continuous case, f (x log f (x f 2 (x dx E X f log f (X f 2 (X [ E X f log f 2(X ( log log ] f (X f 2 (X E X f f (X ( f (x f 2(x f (x dx ( log f 2 (xdx log, where E X f means averaging out X with respect to the pdf f (x and we have used the Jensen s inequality on g(x log(x. 4

5 Stat4 Probability and Statistics II (F6 Example..4 (on p.7: Let a,..., a n be a set of positive numbers and let a + i a i. Show that ( a a 2 a n /n a + n, (3 i.e., geometric mean arithmetic mean. This can be proved by the non-negativity of the KL divergence. Consider two discrete distributions over, 2,..., n with P(X j n, P(Y j a j a + j,..., n. The KL divergence between the two random variables (i.e., two distributions X and Y is i i /n log n a i /a + which is always, therefore leads to (3. i n log a + n n log a i i n log a +/n a i log a + n log ( a a 2 a n /n, 5