Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample space S. Defiitio: A radom variable X o a sample space S is a fuctio X : S R that assigs a real umber X( s) to each sample poit s S. We defie the evet space A x to be the subset of S to which the radom variable X assigs the value x. A x { s S X( s) x} () Example: Take 3 Beroulli trials. Sample space has 8 possible outcomes: 000, 00, 00, 00, 0, 0, 0,. But if we oly iterested i the umber of successes, the 00, 00, 00 sample poits map to the value (for umber of s withi the three trials). Similarly, 0, 0, 0 map to 2. The evet space has oly 4 outcomes 0,, 2, 3 istead of the origial sample space which has 8 poits. Typically, we are oly iterested i the evet space rather tha i the whole sample space S if our iterest lies oly i the experimetal values of the rv X. A x Discrete radom variables: A radom variable is discrete if it ca take o values from a discrete set of umbers. Probability mass fuctio (pmf): P( A x ) P( s) P[ X x] p X X( s) x (2)
Probability distributio fuctio or Cumulative distributio fuctio (cdf): F X ( t) P( X t) p X x t (3) Ofte we will see the statemet, X is a discrete rv with pmf P X with o hit or metio of the sample space. Kow the pmf, cdf, mea (expected value) ad variace for the followig types of discrete radom variables:. Beroulli - oe parameter p (probability of success) A discrete radom variable X that is either 0 or has the followig pmf: p X ( 0) p q Mea is p ad variace is pq. 2. Biomial - cout umber of successes i idepedet trials with p as probability of success of each trial - two parameters ad p. Its pmf is as follows: Mea is p ad variace is p( p) or pq. 3. Geometric - cout umber of trials upto first success (while biomial couts umber of successes) - oe parameter p p X ( k) p X ( ) p k k p ( p) 0 k k 0 otherwise p X ( i) p( p) i for i, 2, F X ( t) ( p) t for ( t 0) ad (4) (5) 4. Negative biomial - cout umber of trials util r th success - two parameters p ad r. 5. Poisso distributio: A discrete radom variable X has a Poisso distributio with parameter λ ( λ > 0 ). The pmf o is e λ λ x p X ------------ for x 0,, 2, x! Mea ad variace are both λ (6) 2
6. Hypergeometric - samplig without replacemet while biomial is samplig with replacemet; N compoets of which d are defective. I a sample of compoets what is the probability that k are defective. 7. Uiform - oe parameter - rage ; discrete versio: Mea is ( + ) 2 ad variace is ( 2 ) 2. 8. Costat - oe parameter c p X ( i) 9. Zipf - log-tailed discrete-value distributio -- for i. Cotiuous radom variables: Probability desity fuctio (pdf): df X ----------------- dx Probability distributio fuctio or Cumulative distributio fuctio (cdf): (7) F X ( t) P( X t) ( r) dr Kow the pdf, cdf, mea (expected value) ad variace for the followig types of radom variables: t (8). Expoetial - parameter λ F e λx if( 0 x < ) 0 otherwise (9) f if( x > 0) λe λx 0 otherwise (0) E[ X] -- λ () Var[ X] ---- λ 2 (2) 3
2. Logormal [Devore] A oegative radom variable X is said to have a logormal distributio if the radom variable Y l( X) has a ormal distributio. The resultig pdf of a logormal radom variable whe l ( X) is ormally distributed with mea µ ad variace σ 2 is Mea is. Media is e µ. Variace is ( e σ2 ) ( e ). 3. Gamma fuctio [Devore] For α > 0, the gamma fuctio Γ( α) is defied by Importat properties are: a. For ay α >, Γ( α) ( α )Γ( α ) b. For ay positive iteger,, Γ( ) ( )! c. e µ + σ2 2 Γ( 2) π Gamma distributio [Devore] A cotiuous radom variable X is said to have a gamma distributio if the pdf o is where α ad β are both greater tha 0. The stadard gamma distributio has β. If α ad β λ, the gamma distributio reduces to a expoetial distributio. Mea is αβ ad variace is αβ 2. The parameter β is called the scale parameter because values other tha either stretch or compress the pdf i the x directio (Devore), ad α is called the shape parameter. 4. Chi-Squared distributio [Devore] -----------------e 2πσx [ l µ ] 2 ( 2σ 2 ) x 0 0 x < 0 2µ + σ2 Γ( α) x α e x dx Let ν be a positive iteger. The a radom variable X is said to have a chi-squared distributio with parameter ν if the pdf o is the gamma desity with α ν 2 ad β 2. The parameter ν is called the umber of degrees of freedom (df) o. The symbol χ 2 is used to represet chisquared. Mea is ν ad variace is 2ν. I is a stadard ormal radom variable, ad Y X 2, the Y has a chi-squared distributio with ν degree of freedom [Devore] 0 ------------------x α β α e x β x 0 Γ( α) 0 otherwise 4
5. Cauchy distributio It is the same as the t-distributio with degree of freedom. Mea ad variace are udefied. Media ad mode are both 0. Desity fuctio: ----------------------,. π( + x 2 < x < ) 6. Uiform distributio A cotiuous radom variable X has a uiform distributio o the iterval [A,B] if the pdf o is Mea is ( B + A) 2 ad variace is ( B A) 2 2. 7. Studet-t distributios The t-distributio has oly oe parameter, the degrees of freedom. Type help(pt) i R to see the PDF of the t-distributio. The mea is 0 whe >, ad the variace is ( 2) for > 2. As 40, the t distributio becomes approximately equal to the ormal distributio. P( t α 2, < T < t α 2, ) α P( T < t α, ) α, the value t α, is called a t critical value. 8. Erlag distributio 9. Hypoexpoetial distributio 0. Hyperexpoetial distributio. Normal (Gaussia) distributio: two parameters: mea ad stadard deviatio. 2. Pareto distributio B ------------ ( A A < x ) < B 0 otherwise αθ ------------ α, x θ x α + (3) where α is the shape parameter ad θ is the scale parameter. Mea value is αθ ( α ). 3. Weibull distributio 5
Relatioships betwee radom variables. Mutually exclusive evets P( A B) P( A) + P( B) ; P( A B) 0. (4) I geeral P( A B) P( A) + P( B) P( A B) (5) 2. Idepedet evets P( A B) P( A)P( B) (6) 3. Law of total probability: Let evets be mutually exclusive ad exhaustive. B i 4. Bayes Theorem (Devore) P( A) P( A B i )P( B i ) i (7) Let A, A 2,, A k be a collectio of k mutually exclusive ad exhaustive evets with prior probability P( A i ), where i, 2,, k. The for ay other evet B for which P( B) > 0, the posteriori probability of give that B has occurred is A j P( A j B) 5. Correlatio of two radom variables P( A j B) P( B A ------------------------ j )P( A j ) ------------------------------------------- P( B) k j, 2,, k P( B A i )P( A i ) i (8) r XY E[ XY] (9) E[ XY] xyp X, Y ( x, y) x S X y S Y for a discrete r.v. (20) I ad Y are idepedet radom variables the because P X, Y ( x, y) P X P Y ( y), E[ XY] E[ X]E[ Y] (see [2, page 07]). Secod momet of a sigle radom variable is E[ X 2 ]. 6. Covariace of two radom variables cov( X, Y) E[ X µ X ][ Y µ Y ] E[ XY] µ X µ Y, where µ X E[ X] ad µ Y E[ Y]. Relate this to variace of a sigle radom variable Var[ X] E[ ( X µ X ) 2 ] E[ X 2 2 ] µ X (2) 6
7. Correlatio coefficiet: ρ XY cov( X, Y) -------------------------------------- var( X)var( Y) (22) Memoryless property: Two distributios, the expoetial cotiuous r.v. distributio ad geometric discrete r.v. distributio ejoy this property. Let X be the lifetime of a compoet. Suppose we have observed that the system has bee operatioal for time t. We would like to kow the probability that it will be operatioal for hours. Say Y X t. We will show that y more P( Y y X > t) P( Y y), i other words, how log it lasts beyod some time t is idepedet of how log the compoet has bee operatioal so far, i.e. t. Expoetial distributio: P( Y y X > t) Geometric distributio: P( Y y X > t) P( X t y X > t) P( X y + t X > t) P( X y + t ad X > t) P( t < X y + t) ---------------------------------------------------- ------------------------------------ P( X > t) P( X > t) ( y + t) t t λe λx dx e λt ( ------------------------------- e λy ) -------------------------------- ( e λy ) P( Y y) λe λx dx e λt (23) (24) Let X be a rv with a geometric distributio ad Y X t. We will show that P( Y y X > t) P( Y y). P( Y y X > t) ( p) y + t ( ( p) t --------------------------------------------------------------------------------- ) ( p) t ( ( p) y ) ( p) t ------------------------------------------------------------ ( p) t P( Y y) Note: Y X t has the same distributio as X because t is a costat. 7
Relatio betwee expoetial distributio (for cotiuous rv) ad geometric distributio (for discrete rv): [2] I is a expoetial radom variable with parameter a, the K X is a geometric radom variable with parameter p e a. Uivariate data statistics terms:. what are sample mea, sample media, sample variace? Quitiles, quartiles, percetiles? R book, page 43: The sample mea of the umeric data set, x, x 2,, x is x x + x 2 + + x --------------------------------------- R book, page 44: The sample media, m, of x, x 2,, x is the middle value of the sorted values. Let the sorted data be deoted by x ( ) x ( 2) x ( ). The m x ( k + ) 2k + (odd) -- ( x 2 ( k) + x ( k + ) ) 2k (eve) R book, page 49: The sample variace ad sample stadard deviatio. For a umeric data set x, x 2,, x, the sample variace is s 2 ----------- x, ad ( i x) 2 i sample stadard deviatio, s is the square root of sample variace. R book, page 50: The p th quatile is at positio + p( ) i the sorted data. If this is ot a iteger, a weighted average is used. For example, as the media is the 0.5 quatile, if is eve, the positio is a fractio, ad hece a average is used as show above i the defiitio of media. The p th quatile splits the dataset so that 00 p % is smaller ad 00 ( p) % is larger. R book, page 50: Percetiles do the same thig as quatiles, except thata scale of 0 to 00 is used istead of 0 to. Devore book: Order the sample observatios from smallest to largest. The the i th smallest observatio i the list is take to be the ---------------------------- 00( i 0.5) th sample percetile. R book, page 50. Quartiles refer to the 0, 25, 50, 75 ad 00 percetiles. Quitiles refer to the 0, 20, 40, 60, 80, 00 percetiles. 8
Radom variables vs. sample values: Devore book: Radom variables are customarily deoted by uppercase letters, such as X ad Y, ear the ed of the alphabet. Lowercase letters are used to represet some particular value of the correspodig radom variable, e.g., X x. Refereces [] K. S. Trivedi, Probability, Statistics with Reliability, Queueig ad Computer Sciece Applicatios, Secod Editio, Wiley, 2002, ISBN 0-47-3334-7. [2] R. Yates ad D. Goodma, Probability ad Stochastic Processes, Wiley, ISBN 0-47-7837-3. [3] Jay L. Devore, Probability ad Statistics for Egieerig ad the Scieces. [4] J Verzai, Usig R for Itroductory Statistics, Chapma & Hall. 9