Discrete Random Variables

Transcription

1 CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary Random Variables A random variable is a real-valued mapping that assigns a numerical value to each possible outcome of an experiment. Formally, a random variable X on a sample space S is a function X : S R that assigns for all s S a real number X(s). Notation: Upper case characters (e.g., X, Y, Z etc) denote random variable; lower case characters (e.g., x, y, z etc.) denote value attained by a random variable. Discrete Random Variable Examples Consider arrival of packets at a router. Let X be the number of packets that arrive per unit time. X is a random variable that can take the values {,,, }. Consider rolling a pair of dice. Let X equal the sum of the dice on a roll. If we think of the sample points as a pair (i, j), where i value rolled by the first die and j value rolled by the second die, we have X(s) i + j. X is a random variable that can take any value between and. Discrete Random Variable 3

2 Another Example Consider tossing two fair coins (i.e., one toss followed by another). Define a random variable X as the number of heads seen in the experiment. For this experiment: The sample space S {(H,H), (H, T), (T,H), (T, T)} The mapping of s S to a real number X(s) is as follows: s X(s) (H,H) (H,T) (T,H) (T,T) Discrete Random Variable 4 Event Space of Random Variables Random variable partitions its sample space into mutually exclusive and collectively exhaustive events. Define A x {X x} to be a subset of S consisting of all sample points s to which the random variable X assigns the value x. A { s S S( s) x} x Clearly, A x satisfies the following properties: A A x y U R A x x, x y S Discrete Random Variable 5 Event Space - An Example Consider again the coin tossing experiment. Enumerate all possible event spaces of the random variable X? A { s S X ( s) } {( T, T )} A { s S X ( s) } {( H, T ),( T, H )} A { s S X ( s) } {( H, H )} Discrete Random Variable 6

3 Discrete Random Variables and PMF A random variable X is said to be discrete if the number of possible values of X is finite, or at most, an infinite sequence of different values. Discrete random variables are characterized by the probabilities of the values attained by the variable. These probabilities are referred to as the Probability Mass Function (PMF) of X. Mathematically, we define PMF as: p X ( x) P( X x) P({ s X ( s) x}) X ( s) x P( s) Discrete Random Variable 7 Properties of PMF px ( x), x R ; this follows from Axioms of Probability. p ; this is true because random variable X x R X ( x) assigns some value x Rto each sample point s S. A discrete random variable can take finite or countably infinite different values, say x, x,. Thus, the condition above can be restated as follows: p i X ( xi ) Terminology: Probability Mass Function is also referred to as discrete density function. Discrete Random Variable 8 Cumulative Distribution Function The PMF is defined for a specific random variable value x. How can we compute the probability of A R? Compute P({ s X ( s) A}) P({ s X ( s) x x A i}) i Write theaboveas P({ s X ( s) A}) P( X A); if A ( a, b), we write P( X A) as P( a < X < b). The cumulative distribution function (CDF) of a random variable X is a function F X (t), < t <, defined as: F ( t) P( < X t) X P( X t) px ( x) x t U Discrete Random Variable 9 3

4 Properties of CDF F(x), <x <. F(x) is a monotone increasing function of x because if x x, then F(x ) F(x ). lim x and lim x -. Note: In the above, the random variable is not explicitly specified. If we are talking about two random variables, say X and Y at the same time, then we should explicitly specify them in the CDFs, e.g., F X (t) and F Y (t). Terminology: Cumulative Distribution Function may be called Probability Distribution Function or simply a Distribution Function. Discrete Random Variable Expectation PMF of a random variable provides several numbers. Expectation (or mean) of X is one way to summarize the information provided by PMF Definition: E[ X ] xp X ( x) μ weighted average of possible values of X. x Discrete Random Variable Properties of Mean E[ cx ] ce[ X ] E[ n i c X ] i i n i c E[ X ] c i s are constants works even if X i s are not independent i i Discrete Random Variable 4

5 Variance Variance of a random variable X, denoted as Var(X) or σ, is defined as the expected value of (X E[X]) : Var( X ) E[( X E[ X ]) Var(X) can also be calculated as: Var( X ) E[ X ] ( E[ X ]) where E[X ] is the second moment of X. Discrete Random Variable 3 ] Properties of Variance Variance is a measure of dispersion of a random variable about its mean. Var( X ) Var( cx ) c Var( x) Var n i xi n i if x i s are independent Var( x ) i Discrete Random Variable 4 Common Discrete Random Variables Bernoulli Random Variable Binomial Random Variable Geometric Random Variable Poisson Random Variable Discrete Random Variable 5 5

6 Bernoulli Random Variable Consider an experiment whose outcome can be either success or failure. If X is a random variable that characterizes this experiment, we can say X for success, and X for failure. The PMF for this random variable is given by: p X () P({X }) p p X () P({X }) p q where p is the probability of success of an experiment. Discrete Random Variable 6 Bernoulli Distribution The CDF of a Bernoulli random variable X with parameter p- q is given by:, x < F( x) q, x <, x Mean and variance: E[ X ] p Var( X ) p( p) Figure : CDF of a Bernoulli Random Variable Discrete Random Variable 7 Binomial Random Variable Consider n Bernoulli trials, where each trial can result in a success with probability p. The number of successes X in such a n-trial sequence is a binomial random variable. The PMF for this random variable is given by: n k p ( p) px ( k) P{( X k)} k, n k, k,,,..., n otherwise where p is the probability of success of a Bernoulli trial. Discrete Random Variable 8 6

7 Binomial PMF Binomial Distribution (n ).3.5 P ({X k }) p / p /4 p 3/4 p / Number of successes (k ) Discrete Random Variable 9 Binomial PMF (large n, small p) Binomial Distribution (n, p.).3.5 P ({X k }) Number of successes (k ) Discrete Random Variable CDF of Binomial R.V. t n i n i FX ( t) p ( p), t i i How did we compute p X (k)? From independent trial assumption, we know p k (-p) n-k is the probability of any sequence of outcomes that results in k successes. There are n such sequences. k Hence, we calculate p X (k) as shown above. Mean and variance: E[ X ] np Var( X ) np( p) Discrete Random Variable 7

8 Geometric Random Variable The number of Bernoulli trials, X, until first success is a Geometric random variable. PMF is given as: k ( p) p, k,,... px ( k), otherwise CDF is given as: F ( t) t i Mean and variance: E[ X ] p X ( p) i p ( p) p Var( X ) p t t Discrete Random Variable, Geometric PMF Geometric Distribution (p.5) P ({X k}) Number of trials until first success (k ) Discrete Random Variable 3 Example: Modeling Packet Loss Geometric r.v. gives number of trials required to get first success It is easy to see p X (k) (-p) k- p, k,, where p is the probability of success of a trial Modeling packet losses seen at a router We can model using a Bernoulli process {Y, Y, Y, } where Y i represents a Bernoulli trial for packet number i We can say: P{Y i } p (i.e., a packet loss) P{Y i } - p (i.e., no loss) So number of successful packet transmissions before first loss, X, is geometrically distributed P{(X n)} p (-p) n-, n,, (good length distribution) Discrete Random Variable 4 8

9 Example: Modeling Packet Loss ( ) Model Suppose each bit transmitted through a channel is received correctly with probability -p and corrupted with probability p. Each transmission is an independent Bernoulli experiment. Assume p is constant over time. S R PKT ACK Timer times out if no ACK is received Assume each packet has length l bits Questions ) How many trials we need to successfully deliver a packet? ) How does () depend on the channel BER (p)? Discrete Random Variable 5 Example: Modeling Packet Loss ( ) P(no error in transmission of packet) (-p) l q P P P 3 are packet transmission trials X number of trials needed to successfully transmit a packet X is geometrically distributed with probability q E[X] / q As p, q E[X], which coincides well with intuition Also, for fixed p, as l E[X] Discrete Random Variable 6 Poisson Random Variable A discrete random variable, X, that takes only nonnegative integer values is said to be Poisson with parameter >, if X has the following PMF: e px ( k), k, k! k,,,... otherwise Poisson PMF with parameter is a good approximation of Binomial PMF with parameters n and p, provided np, n is very large, and p is very small. Discrete Random Variable 7 9

10 Poisson Random Variable ( ) Note that k k z z e, k! X k p ( k) e for any real or complex number e Discrete Random Variable 8 Poisson PMF Poisson Distribution P [X k] Number of events (k) Discrete Random Variable 9 Poisson Approximation to Binomial Binomial Distribution (n, p.) Poisson Distribution ( ) P ({X k })..5. P ({X k }) Number of successes (k ) Number of events (k ) Binomial distribution with large n and small p can be approximated by Poisson distribution with np Discrete Random Variable 3

11 Poisson Random Variable (cont.) CDF of Poisson Random Variable: Mean and variance: t FX ( t) e k E[ X ] k, k! t Var( X ) Consider N independent Poisson random variables X i, i,,3,,n, with parameters X i. Then XX +X + +X N is also a Poisson r.v. with parameter Ν Discrete Random Variable 3 Deriving the Mean of Poisson R.V. Poisson r.v. has PMF : px ( k) e, k,,,... k! E[ X ] can be calculated as E[ X ] k k k e k e k k! k k! k m e e k ( k )! m m! px ( m) m443, from axioms of probability Discrete Random Variable 3 k Example: Job arrivals Consider modeling number of job arrivals at a shop in an interval (,t] Let be the rate of arrival of jobs In an interval Δt P{one arrival in Δt} Δt P{two or more arrivals in Δt} is negligible Divide the interval (,t] into n subintervals of equal lengths Assume arrival of jobs in each interval to be independent of arrivals in another interval Discrete Random Variable 33

12 Example: Job arrivals ( ) If n, the interval can be viewed as a sequence of Bernoulli trials with t p Δt n The number of successes k in n trials can be given by the Binomial distribution s PMF n k p ( p) k n k Discrete Random Variable 34 Example: Job arrivals ( ) Substitute for p t/n to get k n k n t t, k,,..., n k n n Setting n the above reduces to -t k e ( t), k,,,... k! Letting k, the probability of k events in time interval (,] is k - e k! which is the Poisson distribution Discrete Random Variable 35 Example: ALOHA Protocol ALOHA protocol was developed in 97 s at the University of Hawaii It is a Medium Access Control (MAC) layer protocol developed for sharing wireless channels ALOHA protocol is designed to allow multiple users to use a single communication channel Discrete Random Variable 36

13 Example: ALOHA - Basic Idea Very, very simple Let users transmit whenever they have data to be sent If two or more users send data at the same time, a collision occurs and the packets are destroyed Upon collision, the sender waits for a random amount of time and resends the packet Modeling question: What is the throughput of an ALOHA channel? Discrete Random Variable 37 Example: ALOHA - Model Infinite population of users generating N frames per frame time Frame time: amount of time required to transmit a fixed-length frame < N < Poisson model : Poisson distribution predicts the number of events to occur in a time period Question: what is the rate of generation of frames G? Frames generated include new + retransmitted frames At low loads G N At high loads G > N Discrete Random Variable 38 Example: ALOHA - Model ( ) Let S rate of successful frame transmission p probability of successful transmission S/G Question: What is the vulnerable period? t Discrete Random Variable 39 3

14 Example: ALOHA - Model ( ) Mean # of frames generated in vulnerable period t G Why? E[X] G, where X X +X We assume X +X are Poisson distributed with rate G, so X is also Poisson with rate G Probability that no other traffic is initiated in vulnerable period is p X () e -G, which is Poisson model Discrete Random Variable 4 Example: ALOHA - Throughput p S G G G e S G e ds at maximum throughput dg G G e + G( ) e is throughput G e ( G) G (/ ) Smax e.84 e ALOHA protocol' s maximum channel utilization is8.4% Discrete Random Variable 4 Jointly Distributed Random Variables 4

15 Joint PMFs of Multiple R.V. s Probabilistic models may involve more than one random variable These random variables are defined for the same experiment and sample space, and they may have relationships among them Let Z: (X, Y) be defined on sample space S For each sample point s in S, the random variables X and Y take one of its possible values, e.g. X(s)x, Y(s)y Z is then a -dimensional vector satisfying the following relationship: Z: S R with Z(s)z(x, y) Discrete Random Variable 43 Joint PMF (...) The Joint PMF of X and Y (or the Joint PMF of random vector Z) is defined as: p ( z) P({ Z z}) Properties of this PMF Z P({ X x},{ Y y}) pz ( z), z R { z pz ( z) }: a subset of R px ( x) px, Y ( x, y) y py ( y) px, Y ( x, y) x px ( x) and py ( y) :Marginal PMFs of x and y, respectively Discrete Random Variable 44 Conditional PMF 5

16 Conditioning On An Event Look at conditional PMFs given the occurrence of a certain event or given the value of another random variable Conditional PMF of random variable X, conditioned on an event A with P(A)> is defined as: P({ X x} A} p X A ( x) P{( X x A)} P( A) Calculate p X A (x) by adding the probabilities of outcomes that result in Xx and belong to the conditioning event A, and then normalize by dividing with P(A). Discrete Random Variable 46 Example: A Web Surfer A web surfer will repeatedly attempt to connect to a Web server, up to a maximum of n times. Each attempt has a probability p of being successful. What is the PMF of the number of events, given that the surfer successfully connects to the Web server? Discrete Random Variable 47 Example: A Web Surfer ( ) Let A be the event that the web surfer successfully connects (with at most n attempts) Let X be the number of attempts needed to establish a connection assuming unlimited number of attempts could be made. Clearly, X is a geometric random variable with parameter p and A{X n}. k ( p) p, k,,..., n n n j j P( A) ( p) p, and px A( k) ( p) p j j, otherwise Discrete Random Variable 48 6

17 Conditioning on Another R.V. Let X and Y be two random variables associated with the same experiment. Suppose Y equals y, then this provides some information regarding the value of X. This information is captured by the conditional PMF: P({ X x, Y y}) px, Y ( x, y) px Y ( x y) P({ Y y}) py ( y) The conditional PMF p X Y (x,y) satisfies the normalization property px Y ( x y) x Discrete Random Variable 49 7