Order Statistics. The order statistics of a set of random variables X 1, X 2,, X n are the same random variables arranged in increasing order.

Transcription

1 Order Statistics The order statistics of a set of random variables 1, 2,, n are the same random variables arranged in increasing order. Denote by (1) = smallest of 1, 2,, n (2) = 2 nd smallest of 1, 2,, n M (n) = largest of 1, 2,, n Note, even if i s are independent, (i) s can not be independent since (1) (2) (n) Distribution of i s and (i) s are NOT the same. week 10 1

2 Distribution of the Largest order statistic (n) Suppose 1, 2,, n are i.i.d random variables with common distribution function F (x) and common density function f (x). The CDF of the largest order statistic, (n), is given by F n x ( ) ( ) ( n) ( ) = = P x The density function of (n) is then f d dx ( x ) = F ( x)= ( n ) ( n ) week 10 2

3 Example Suppose 1, 2,, n are i.i.d Uniform(0,1) random variables. Find the density function of (n). week 10 3

4 Distribution of the Smallest order statistic (1) Suppose 1, 2,, n are i.i.d random variables with common distribution function F (x) and common density function f (x). The CDF of the smallest order statistic (1) is given by ( ) ( 1) () ( 1 x) = 1 P( () > ) = F x = P 1 x The density function of (1) is then f d dx ( x ) = F ( x)= ( 1) ( ) 1 week 10 4

5 Example Suppose 1, 2,, n are i.i.d Uniform(0,1) random variables. Find the density function of (1). week 10 5

6 Distribution of the kth order statistic (k) Suppose 1, 2,, n are i.i.d random variables with common distribution function F (x) and common density function f (x). The density function of (k) is f n x = n! 1 1! ( ) ( ) ( k 1! )( n k) k n k ( F ( x) ) ( F ( x) ) f ( x) week 10 6

7 Example Suppose 1, 2,, n are i.i.d Uniform(0,1) random variables. Find the density function of (k). week 10 7

8 Some facts about Power Series Consider the power series a t k k with non-negative coefficients a k. =0 k=0 If a t k k converges for any positive value of t, say for t = r, then it k converges for all t in the interval [-r, r] and thus defines a function of t on that interval. For any t in (-r, r), this function is differentiable at t and the series converges to the derivatives. k= 0 ka t k k 1 Example: For k = 0, 1, 2, and -1< x < 1 we have that k 1 ( ) 1 x = m= (differentiating geometric series). 0 k + m x k m week 10 8

9 Generating Functions For a sequence of real numbers {a j } = a 0, a 1, a 2,, the generating function of {a j } is A t j = a t j () = j if this converges for t < t 0 for some t 0 > 0. 0 week 10 9

10 Probability Generating Functions Suppose is a random variable taking the values 0, 1, 2, (or a subset of the non-negative integers). Let p j = P( = j), j = 0, 1, 2,. This is in fact a sequence p 0, p 1, p 2, Definition: The probability generating function of is π 2 () t = p0 + p1t + p2t + L = = j Since p jt p j if t < 1 and p j = 1 the pgf converges absolutely at least j= 0 for t < 1. In general, π (1) = p 0 + p 1 + p 2 + = 1. The pgf of is expressible as an expectation: π j () t p t = E( t ) = =0 j j j 0 p j t j week 10 10

11 ~ Binomial(n, p), π converges for all real t. Examples n n = j= 0 j j n j j () t p q t = ( pt + q) n ~ Geometric(p), π () t = j= 1 pq t j 1 j = pt 1 qt converges for qt < 1 i.e. t < 1 q = 1 1 p Note: in this case p j = pq j for j = 1, 2, week 10 11

12 PGF for sums of independent random variables If, Y are independent and Z = +Y then, Z Example Z + Y Y Y () t = E( t ) = E( t ) = E( t t ) = E( t ) E( t ) ( t) π ( t) π = π Let Y ~ Binomial(n, p). Then we can write Y = n. Where i s are i.i.d Bernoulli(p). The pgf of i is π The pgf of Y is then π i 0 1 ( t) = t ( 1 p) + t p = tp q. + Y L+ 1 2 n n () t = E( t ) E( t ) E( t ) E( t ) ( tp q) n = L = +. Y week 10 12

13 Use of PGF to find probabilities Theorem Let be a discrete random variable, whose possible values are the nonnegative integers. Assume π (t 0 ) < for some t 0 > 0. Then π (0) = P( = 0), etc. In general, k where is the k th derivative of π with respect to t. Proof: π ( ) ( 0) = P( 1), ( 0) = 2P( 2), π ' = π '' = π ( k ) ( 0) = k! P( k ), = week 10 13

14 Example Suppose ~ Poisson(λ). The pgf of is given by π j () t t = = j= λ j e λ 0 j! Using this pgf we have that week 10 14

15 Finding Moments from PGFs Theorem Let be a discrete random variable, whose possible values are the nonnegative integers. If π (t) < for t < t 0 for some t 0 > 1. Then etc. In general, k Where is the kth derivative of π with respect to t. Note: E((-1) (-k+1)) is called the kth factorial moment of. Proof: ( ) π ( 1) ( ), ( 1) = E ( 1) π ' = E π '' ( ), ( k π ) ( 1 ) = E( ( 1)( 2) L( K + 1) ), week 10 15

16 Example Suppose ~ Binomial(n, p). The pgf of is π (t) = (pt+q) n. Find the mean and the variance of using its pgf. week 10 16

17 Uniqueness Theorem for PGF Suppose, Y have probability generating function π and π Y respectively. Then π (t) = π Y (t) if and only if P( = k) = P(Y = k) for k = 0,1,2, Proof: Follow immediately from calculus theorem: If a function is expressible as a power series at x=a, then there is only one such series. A pgf is a power series about the origin which we know exists with radius of convergence of at least 1. week 10 17

18 Moment Generating Functions The moment generating function of a random variable is t ( t) E( e ) m = m (t) exists if m (t) < for t < t 0 >0 If is discrete If is continuous m m tx ( t) e p ( x). = x tx () t = e f ( x) dx. Note: m (t) = π (e t ). week 10 18

19 Examples ~ Exponential(λ). The mgf of is m t () ( ) tx t = E e = e λe 0 λx dx = ~ Uniform(0,1). The mgf of is m 1 t tx () t = E( e ) = e dx = 0 week 10 19

20 Generating Moments from MGFs Theorem Let be any random variable. If m (t) < for t < t 0 for some t 0 > 0. Then m (0) = 1 ( 0) E( ), ( 0) 2 E( ), m' = m'' = etc. In general, ( k ) k ( 0) E( ), m = k Where is the kth derivative of m with respect to t. Proof: ( ) m week 10 20

21 Example Suppose ~ Exponential(λ). Find the mean and variance of using its moment generating function. week 10 21

22 Example Suppose ~ N(0,1). Find the mean and variance of using its moment generating function. week 10 22

23 Example Suppose ~ Binomial(n, p). Find the mean and variance of using its moment generating function. week 10 23

24 Properties of Moment Generating Functions m (0) = 1. If Y=a+b, a, b R then the mgf of Y is given by m () ( ty ) ( at+ bt ) at ( bt t E e E e e E e ) at = = = e m ( bt). Y = If,Y independent and Z = +Y then, m tz t + ty t ty t ty () t = E( e ) = E( e ) = E( e e ) = E( e ) E( e ) m ( t) m ( t) Z = Y week 10 24

25 Uniqueness Theorem If a moment generating function m (t) exists for t in an open interval containing 0, it uniquely determines the probability distribution. week 10 25

26 Example Find the mgf of ~ N(μ,σ 2 ) using the mgf of the standard normal random variable. ( ) ( ) 2 2 Suppose, ~ N μ σ, ~ N μ σ independent. 1 1, 1 2 2, Find the distribution of using mgf approach. 2 week 10 26