Stat 5101 Notes: Brand Name Distributions
|
|
- Shannon Andrews
- 5 years ago
- Views:
Transcription
1 Stat 5101 Notes: Brand Name Distributions Charles J. Geyer September 5, 2012 Contents 1 Discrete Uniform Distribution 2 2 General Discrete Uniform Distribution 2 3 Uniform Distribution 3 4 General Uniform Distribution 3 5 Bernoulli Distribution 4 6 Binomial Distribution 5 7 Hypergeometric Distribution 6 8 Poisson Distribution 7 9 Geometric Distribution 8 10 Negative Binomial Distribution 9 11 Normal Distribution Exponential Distribution Gamma Distribution Beta Distribution Multinomial Distribution 15 1
2 16 Bivariate Normal Distribution Multivariate Normal Distribution Chi-Square Distribution Student s t Distribution Snedecor s F Distribution Cauchy Distribution Laplace Distribution 25 1 Discrete Uniform Distribution DiscUnif(n). Discrete. Rationale Equally likely outcomes. The interval 1, 2,..., n of the integers. Probability Mass Function f(x) = 1 n, x = 1, 2,..., n E(X) = n var(x) = n General Discrete Uniform Distribution Discrete. Any finite set S. 2
3 Probability Mass Function f(x) = 1 n, x S, where n is the number of elements of S. 3 Uniform Distribution Unif(a, b). Rationale Continuous analog of the discrete uniform distribution. Parameters Real numbers a and b with a < b. The interval (a, b) of the real numbers. Probability Density Function f(x) = 1 b a, a < x < b E(X) = a + b 2 (b a)2 var(x) = 12 Relation to Other Distributions Beta(1, 1) = Unif(0, 1). 4 General Uniform Distribution Any open set S in R n. 3
4 Probability Density Function f(x) = 1 c, x S where c is the measure (length in one dimension, area in two, volume in three, etc.) of the set S. 5 Bernoulli Distribution Ber(p). Discrete. Rationale Any zero-or-one-valued random variable. Parameter Real number 0 p 1. The two-element set {0, 1}. Probability Mass Function f(x) = { p, x = 1 E(X) = p 1 p, x = 0 var(x) = p(1 p) Addition Rule If X 1,..., X k are IID Ber(p) random variables, then X X k is a Bin(k, p) random variable. Degeneracy If p = 0 the distribution is concentrated at 0. If p = 1 the distribution is concentrated at 1. Relation to Other Distributions Ber(p) = Bin(1, p). 4
5 6 Binomial Distribution Bin(n, p). Discrete. Rationale Sum of n IID Bernoulli random variables. Parameters Real number 0 p 1. Integer n 1. The interval 0, 1,..., n of the integers. Probability Mass Function ( ) n f(x) = p x (1 p) n x, x x = 0, 1,..., n E(X) = np var(x) = np(1 p) Addition Rule If X 1,..., X k are independent random variables, X i being Bin(n i, p) distributed, then X X k is a Bin(n n k, p) random variable. Normal Approximation If np and n(1 p) are both large, then Bin(n, p) N ( np, np(1 p) ) Poisson Approximation If n is large but np is small, then Bin(n, p) Poi(np) Theorem The fact that the probability mass function sums to one is equivalent to the binomial theorem: for any real numbers a and b n k=0 ( ) n a k b n k = (a + b) n. k 5
6 Degeneracy If p = 0 the distribution is concentrated at 0. If p = 1 the distribution is concentrated at n. Relation to Other Distributions Ber(p) = Bin(1, p). 7 Hypergeometric Distribution Hypergeometric(A, B, n). Discrete. Rationale Sample of size n without replacement from finite population of B zeros and A ones. The interval max(0, n B),..., min(n, A) of the integers. Probability Mass Function ( A B ) f(x) = x)( n x ), x = max(0, n B),..., min(n, A) where ( A+B n E(X) = np var(x) = np(1 p) N n N 1 p = A A + B N = A + B (7.1) Binomial Approximation If n is small compared to either A or B, then where p is given by (7.1). Hypergeometric(n, A, B) Bin(n, p) 6
7 Normal Approximation If n is large, but small compared to either A or B, then Hypergeometric(n, A, B) N ( np, np(1 p) ) where p is given by (7.1). Theorem The fact that the probability mass function sums to one is equivalent to min(a,n) ( )( ) ( ) A B A + B = x n x n x=max(0,n B) 8 Poisson Distribution Poi(µ) Discrete. Rationale Counts in a Poisson process. Parameter Real number µ > 0. The non-negative integers 0, 1,.... Probability Mass Function f(x) = µx x! e µ, x = 0, 1,... E(X) = µ var(x) = µ Addition Rule If X 1,..., X k are independent random variables, X i being Poi(µ i ) distributed, then X 1 + +X k is a Poi(µ 1 + +µ k ) random variable. Normal Approximation If µ is large, then Poi(µ) N (µ, µ) 7
8 Theorem The fact that the probability mass function sums to one is equivalent to the Maclaurin series for the exponential function: for any real number x x k k! = ex. k=0 9 Geometric Distribution Geo(p). Discrete. Rationales Discrete lifetime of object that does not age. Waiting time or interarrival time in sequence of IID Bernoulli trials. Inverse sampling. Discrete analog of the exponential distribution. Parameter Real number 0 < p 1. The non-negative integers 0, 1,.... Probability Mass Function f(x) = p(1 p) x x = 0, 1,... E(X) = 1 p p var(x) = 1 p p 2 Addition Rule If X 1,..., X k are IID Geo(p) random variables, then X X k is a NegBin(k, p) random variable. 8
9 Theorem The fact that the probability mass function sums to one is equivalent to the geometric series: for any real number s such that s < 1 k=0 s k = 1 1 s. Degeneracy If p = 1 the distribution is concentrated at Negative Binomial Distribution NegBin(r, p). Discrete. Rationale Sum of IID geometric random variables. Inverse sampling. Gamma mixture of Poisson distributions. Parameters Real number 0 < p 1. Integer r 1. The non-negative integers 0, 1,.... Probability Mass Function ( ) r + x 1 f(x) = p r (1 p) x, x = 0, 1,... x E(X) = var(x) = r(1 p) p r(1 p) p 2 Addition Rule If X 1,..., X k are independent random variables, X i being NegBin(r i, p) distributed, then X X k is a NegBin(r r k, p) random variable. 9
10 Normal Approximation If r(1 p) is large, then ( ) r(1 p) r(1 p) NegBin(r, p) N, p p 2 Degeneracy If p = 1 the distribution is concentrated at 0. Extended Definition The definition makes sense for noninteger r if binomial coefficients are defined by ( ) r r (r 1) (r k + 1) = k k! which for integer r agrees with the standard definition. Also ( ) ( ) r + x 1 r = ( 1) x x x which explains the name negative binomial. (10.1) Theorem The fact that the probability mass function sums to one is equivalent to the generalized binomial theorem: for any real number s such that 1 < s < 1 and any real number m k=0 ( ) m s k = (1 + s) m. (10.2) k If m is a nonnegative integer, then ( m k ) is zero for k > m, and we get the ordinary binomial theorem. Changing variables from m to m and from s to s and using (10.1) turns (10.2) into ( ) m + k 1 ( ) m s k = ( s) k = (1 s) m k k k=0 k=0 which has a more obvious relationship to the negative binomial density summing to one. 11 Normal Distribution N (µ, σ 2 ). 10
11 Rationale Limiting distribution in the central limit theorem. Error distribution that turns the method of least squares into maximum likelihood estimation. Parameters Real numbers µ and σ 2 > 0. The real numbers. Probability Density Function f(x) = 1 2πσ e (x µ)2 /2σ 2, < x < E(X) = µ var(x) = σ 2 E{(X µ) 3 } = 0 E{(X µ) 4 } = 3σ 4 Linear Transformations N (aµ + b, a 2 σ 2 ) distributed. If X is N (µ, σ 2 ) distributed, then ax + b is Addition Rule If X 1,..., X k are independent random variables, X i being N (µ i, σi 2) distributed, then X X k is a N (µ µ k, σ σ2 k ) random variable. Theorem The fact that the probability density function integrates to one is equivalent to the integral e z2 /2 dz = 2π Relation to Other Distributions If Z is N (0, 1) distributed, then Z 2 is Gam( 1 2, 1 2 ) = chi2 (1) distributed. Also related to Student t, Snedecor F, and Cauchy distributions (for which see). 11
12 12 Exponential Distribution Exp(λ). Rationales Lifetime of object that does not age. Waiting time or interarrival time in Poisson process. Continuous analog of the geometric distribution. Parameter Real number λ > 0. The interval (0, ) of the real numbers. Probability Density Function f(x) = λe λx, 0 < x < Cumulative Distribution Function F (x) = 1 e λx, 0 < x < E(X) = 1 λ var(x) = 1 λ 2 Addition Rule If X 1,..., X k are IID Exp(λ) random variables, then X X k is a Gam(k, λ) random variable. Relation to Other Distributions Exp(λ) = Gam(1, λ). 13 Gamma Distribution Gam(α, λ). 12
13 Rationales Sum of IID exponential random variables. Conjugate prior for exponential, Poisson, or normal precision family. Parameter Real numbers α > 0 and λ > 0. The interval (0, ) of the real numbers. Probability Density Function f(x) = λα Γ(α) xα 1 e λx, where Γ(α) is defined by (13.1) below. 0 < x < E(X) = α λ var(x) = α λ 2 Addition Rule If X 1,..., X k are independent random variables, X i being Gam(α i, λ) distributed, then X 1 + +X k is a Gam(α 1 + +α k, λ) random variable. Normal Approximation If α is large, then ( α Gam(α, λ) N λ, α ) λ 2 Theorem The fact that the probability density function integrates to one is equivalent to the integral 0 x α 1 e λx dx = Γ(α) λ α the case λ = 1 is the definition of the gamma function Γ(α) = 0 x α 1 e x dx (13.1) 13
14 Relation to Other Distributions Exp(λ) = Gam(1, λ). chi 2 (ν) = Gam( ν 2, 1 2 ). If X and Y are independent, X is Γ(α 1, λ) distributed and Y is Γ(α 2, λ) distributed, then X/(X + Y ) is Beta(α 1, α 2 ) distributed. If Z is N (0, 1) distributed, then Z 2 is Gam( 1 2, 1 2 ) distributed. Facts About Gamma Functions Integration by parts in (13.1) establishes the gamma function recursion formula Γ(α + 1) = αγ(α), α > 0 (13.2) The relationship between the Exp(λ) and Gam(1, λ) distributions gives Γ(1) = 1 and the relationship between the N (0, 1) and Gam( 1 2, 1 2 ) distributions gives Γ( 1 2 ) = π Together with the recursion (13.2) these give for any positive integer n Γ(n + 1) = n! and Γ(n ) = ( n 1 ( ) 2) n π 14 Beta Distribution Beta(α 1, α 2 ). Rationales Ratio of gamma random variables. Conjugate prior for binomial or negative binomial family. 14
15 Parameter Real numbers α 1 > 0 and α 2 > 0. The interval (0, 1) of the real numbers. Probability Density Function f(x) = Γ(α 1 + α 2 ) Γ(α 1 )Γ(α 2 ) xα 1 1 (1 x) α < x < 1 where Γ(α) is defined by (13.1) above. E(X) = α 1 α 1 + α 2 α 1 α 2 var(x) = (α 1 + α 2 ) 2 (α 1 + α 2 + 1) Theorem The fact that the probability density function integrates to one is equivalent to the integral 1 Relation to Other Distributions 0 x α 1 1 (1 x) α 2 1 dx = Γ(α 1)Γ(α 2 ) Γ(α 1 + α 2 ) If X and Y are independent, X is Γ(α 1, λ) distributed and Y is Γ(α 2, λ) distributed, then X/(X + Y ) is Beta(α 1, α 2 ) distributed. Beta(1, 1) = Unif(0, 1). 15 Multinomial Distribution Multi(n, p). Discrete. Rationale Multivariate analog of the binomial distribution. 15
16 Parameters Real vector p in the parameter space { } k p R k : 0 p i, i = 1,..., k, and p i = 1 i=1 (15.1) (real vectors whose components are nonnegative and sum to one). The set of vectors { S = x Z k : 0 x i, i = 1,..., k, and } k x i = n i=1 (15.2) (integer vectors whose components are nonnegative and sum to n). Probability Mass Function where f(x) = ( ) k n p x i i x, i=1 ( ) n = x is called a multinomial coefficient. n! k i=1 x i! x S E(X i ) = np i var(x i ) = np i (1 p i ) cov(x i, X j ) = np i p j, i j (Vector Form) E(X) = np var(x) = nm where M = P pp T where P is the diagonal matrix whose vector of diagonal elements is p. 16
17 Addition Rule If X 1,..., X k are independent random vectors, X i being Multi(n i, p) distributed, then X X k is a Multi(n n k, p) random variable. Normal Approximation If n is large and p is not near the boundary of the parameter space (15.1), then Multi(n, p) N (np, nm) Theorem The fact that the probability mass function sums to one is equivalent to the multinomial theorem: for any vector a of real numbers ( ) k n a x i i = (a a k ) n x i=1 x S Degeneracy If a vector a exists such that Ma = 0, then var(a T X) = 0. In particular, the vector u = (1, 1,..., 1) always satisfies Mu = 0, so var(u T X) = 0. This is obvious, since u T X = k i=1 X i = n by definition of the multinomial distribution, and the variance of a constant is zero. This means a multinomial random vector of dimension k is really of dimension no more than k 1 because it is concentrated on a hyperplane containing the sample space (15.2). Marginal Distributions Every univariate marginal is binomial X i Bin(n, p i ) Not, strictly speaking marginals, but random vectors formed by collapsing categories are multinomial. If A 1,..., A m is a partition of the set {1,..., k} and Y j = X i, i A j q j = p i, i A j j = 1,..., m j = 1,..., m then the random vector Y has a Multi(n, q) distribution. 17
18 Conditional Distributions If {i 1,..., i m } and {i m+1,..., i k } partition the set {1,..., k}, then the conditional distribution of X i1,..., X im given X im+1,..., X ik is Multi(n X im+1 X ik, q), where the parameter vector q has components q j = p ij p i1 + + p im, j = 1,..., m Relation to Other Distributions Each marginal of a multinomial is binomial. If X is Bin(n, p), then the vector (X, n X) is Multi ( n, (p, 1 p) ). 16 Bivariate Normal Distribution See multivariate normal below. Rationales See multivariate normal below. Parameters Real vector µ of dimension 2, real symmetric positive semidefinite matrix M of dimension 2 2 having the form ( ) σ 2 M = 1 ρσ 1 σ 2 ρσ 1 σ 2 σ2 2 where σ 1 > 0, σ 2 > 0 and 1 < ρ < +1. The Euclidean space R 2. Probability Density Function f(x) = 1 2π det(m) 1/2 exp ( 1 2 (x µ)t M 1 (x µ) ) ( [ (x1 ) 1 = 2π 1 µ 2 1 exp 1 ρ 2 σ 1 σ 2 2(1 ρ 2 ) σ 1 ( ) ( ) ( ) ]) x1 µ 1 x2 µ 2 x2 µ 2 2 2ρ +, x R 2 σ 1 σ 2 σ 2 18
19 (Vector Form) E(X i ) = µ i, i = 1, 2 var(x i ) = σ 2 i, i = 1, 2 cov(x 1, X 2 ) = ρσ 1 σ 2 cor(x 1, X 2 ) = ρ E(X) = µ var(x) = M Linear Transformations See multivariate normal below. Addition Rule See multivariate normal below. Marginal Distributions X i is N (µ i, σi 2 ) distributed, i = 1, 2. Conditional Distributions The conditional distribution of X 2 given X 1 is ( N µ 2 + ρ σ ) 2 (x 1 µ 1 ), (1 ρ 2 )σ2 2 σ 1 17 Multivariate Normal Distribution N (µ, M) Rationales Multivariate analog of the univariate normal distribution. Limiting distribution in the multivariate central limit theorem. Parameters Real vector µ of dimension k, real symmetric positive semidefinite matrix M of dimension k k. The Euclidean space R k. 19
20 Probability Density Function If M is (strictly) positive definite, f(x) = (2π) k/2 det(m) 1/2 exp ( 1 2 (x µ)t M 1 (x µ) ), x R k Otherwise there is no density (X is concentrated on a hyperplane). (Vector Form) E(X) = µ var(x) = M Linear Transformations If X is N (µ, M) distributed, then a + BX, where a is a constant vector and B is a constant matrix of dimensions such that the vector addition and matrix multiplication make sense, has the N (a + Bµ, BMB T ) distribution. Addition Rule If X 1,..., X k are independent random vectors, X i being N (µ i, M i ) distributed, then X 1 + +X k is a N (µ 1 + +µ k, M 1 + +M k ) random variable. Degeneracy If a vector a exists such that Ma = 0, then var(a T X) = 0. Partitioned Vectors and Matrices The random vector and parameters are written in partitioned form ( ) X1 X = (17.1a) X 2 ( ) µ1 µ = (17.1b) µ 2 ( ) M11 M M = 12 (17.1c) M 21 M 2 when X 1 consists of the first r elements of X and X 2 of the other k r elements and similarly for µ 1 and µ 2. Marginal Distributions Every marginal of a multivariate normal is normal (univariate or multivariate as the case may be). In partitioned form, the (marginal) distribution of X 1 is N (µ 1, M 11 ). 20
21 Conditional Distributions Every conditional of a multivariate normal is normal (univariate or multivariate as the case may be). In partitioned form, the conditional distribution of X 1 given X 2 is N (µ 1 + M 12 M 22 [X 2 µ 2 ], M 11 M 12 M 22 M 21) where the notation M 22 denotes the inverse of the matrix M 22 if the matrix is invertible and otherwise any generalized inverse. 18 Chi-Square Distribution chi 2 (ν) or χ 2 (ν). Rationales Sum of squares of IID standard normal random variables. Sampling distribution of sample variance when data are IID normal. Asymptotic distribution in Pearson chi-square test. Asymptotic distribution of log likelihood ratio. Parameter Real number ν > 0 called degrees of freedom. The interval (0, ) of the real numbers. Probability Density Function f(x) = ( 1 2 )ν/2 Γ( ν 2 ) xν/2 1 e x/2, 0 < x <. E(X) = ν var(x) = 2ν Addition Rule If X 1,..., X k are independent random variables, X i being chi 2 (ν i ) distributed, then X 1 + +X k is a chi 2 (ν 1 + +ν k ) random variable. 21
22 Normal Approximation If ν is large, then Relation to Other Distributions chi 2 (ν) = Gam( ν 2, 1 2 ). chi 2 (ν) N (ν, 2ν) If X is N (0, 1) distributed, then X 2 is chi 2 (1) distributed. If Z and Y are independent, X is N (0, 1) distributed and Y is chi 2 (ν) distributed, then X/ Y/ν is t(ν) distributed. If X and Y are independent and are chi 2 (µ) and chi 2 (ν) distributed, respectively, then (X/µ)/(Y/ν) is F (µ, ν) distributed. 19 Student s t Distribution t(ν). Rationales Sampling distribution of pivotal quantity n(x n µ)/s n when data are IID normal. Marginal for µ in conjugate prior family for two-parameter normal data. Parameter Real number ν > 0 called degrees of freedom. The real numbers. Probability Density Function f(x) = 1 ν+1 Γ( 2 ) νπ Γ( ν 2 ) ( 1 ) (ν+1)/2, < x < x2 ν 22
23 If ν > 1, then E(X) = 0. Otherwise the mean does not exist. If ν > 2, then var(x) = ν ν 2. Otherwise the variance does not exist. Normal Approximation If ν is large, then Relation to Other Distributions t(ν) N (0, 1) If X and Y are independent, X is N (0, 1) distributed and Y is chi 2 (ν) distributed, then X/ Y/ν is t(ν) distributed. If X is t(ν) distributed, then X 2 is F (1, ν) distributed. t(1) = Cauchy(0, 1). 20 Snedecor s F Distribution F (µ, ν). Rationale Ratio of sums of squares for normal data (test statistics in regression and analysis of variance). Parameters Real numbers µ > 0 and ν > 0 called numerator degrees of freedom and denominator degrees of freedom, respectively. The interval (0, ) of the real numbers. Probability Density Function f(x) = µ+ν Γ( 2 )µµ/2 ν ν/2 x µ/2 1 Γ( µ 2 )Γ( ν 2 ), 0 < x < + (µx + ν) (µ+ν)/2 23
24 If ν > 2, then E(X) = Otherwise the mean does not exist. Relation to Other Distributions ν ν 2. If X and Y are independent and are chi 2 (µ) and chi 2 (ν) distributed, respectively, then (X/µ)/(Y/ν) is F (µ, ν) distributed. If X is t(ν) distributed, then X 2 is F (1, ν) distributed. 21 Cauchy Distribution Cauchy(µ, σ). Rationales Very heavy tailed distribution. Counterexample to law of large numbers. Parameters Real numbers µ and σ > 0. The real numbers. Probability Density Function f(x) = 1 πσ ( x µ σ ) 2, < x < + No moments exist. Addition Rule If X 1,..., X k are IID Cauchy(µ, σ) random variables, then X n = (X X k )/n is also Cauchy(µ, σ). 24
25 Relation to Other Distributions t(1) = Cauchy(0, 1). 22 Laplace Distribution Laplace(µ, σ). Rationales The sample median is the maximum likelihood estimate of the location parameter. Parameters Real numbers µ and σ > 0, called the mean and standard deviation, respectively. The real numbers. Probability Density Function ( 2 f(x) = 2σ exp ) 2 x µ σ, < x < E(X) = µ var(x) = σ 2 25
Stat 5101 Notes: Brand Name Distributions
Stat 5101 Notes: Brand Name Distributions Charles J. Geyer February 14, 2003 1 Discrete Uniform Distribution DiscreteUniform(n). Discrete. Rationale Equally likely outcomes. The interval 1, 2,..., n of
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationChapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationChapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationA Few Special Distributions and Their Properties
A Few Special Distributions and Their Properties Econ 690 Purdue University Justin L. Tobias (Purdue) Distributional Catalog 1 / 20 Special Distributions and Their Associated Properties 1 Uniform Distribution
More informationStat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides: Deck 7 Asymptotics, also called Large Sample Theory Charles J. Geyer School of Statistics University of Minnesota 1 Asymptotic Approximation The last big subject in probability
More informationStat 5101 Lecture Slides: Deck 8 Dirichlet Distribution. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides: Deck 8 Dirichlet Distribution Charles J. Geyer School of Statistics University of Minnesota 1 The Dirichlet Distribution The Dirichlet Distribution is to the beta distribution
More informationProbability Distributions Columns (a) through (d)
Discrete Probability Distributions Columns (a) through (d) Probability Mass Distribution Description Notes Notation or Density Function --------------------(PMF or PDF)-------------------- (a) (b) (c)
More informationLIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
More informationThings to remember when learning probability distributions:
SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationStat 5101 Lecture Slides Deck 5. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides Deck 5 Charles J. Geyer School of Statistics University of Minnesota 1 Joint and Marginal Distributions When we have two random variables X and Y under discussion, a useful shorthand
More informationCommon probability distributionsi Math 217 Probability and Statistics Prof. D. Joyce, Fall 2014
Introduction. ommon probability distributionsi Math 7 Probability and Statistics Prof. D. Joyce, Fall 04 I summarize here some of the more common distributions used in probability and statistics. Some
More information3 Modeling Process Quality
3 Modeling Process Quality 3.1 Introduction Section 3.1 contains basic numerical and graphical methods. familiar with these methods. It is assumed the student is Goal: Review several discrete and continuous
More information1 Uniform Distribution. 2 Gamma Distribution. 3 Inverse Gamma Distribution. 4 Multivariate Normal Distribution. 5 Multivariate Student-t Distribution
A Few Special Distributions Their Properties Econ 675 Iowa State University November 1 006 Justin L Tobias (ISU Distributional Catalog November 1 006 1 / 0 Special Distributions Their Associated Properties
More informationContinuous Random Variables and Continuous Distributions
Continuous Random Variables and Continuous Distributions Continuous Random Variables and Continuous Distributions Expectation & Variance of Continuous Random Variables ( 5.2) The Uniform Random Variable
More informationContinuous Random Variables
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability. Often, there is interest in random variables
More informationAdvanced topics from statistics
Advanced topics from statistics Anders Ringgaard Kristensen Advanced Herd Management Slide 1 Outline Covariance and correlation Random vectors and multivariate distributions The multinomial distribution
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More information15 Discrete Distributions
Lecture Note 6 Special Distributions (Discrete and Continuous) MIT 4.30 Spring 006 Herman Bennett 5 Discrete Distributions We have already seen the binomial distribution and the uniform distribution. 5.
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More informationMULTIVARIATE DISTRIBUTIONS
Chapter 9 MULTIVARIATE DISTRIBUTIONS John Wishart (1898-1956) British statistician. Wishart was an assistant to Pearson at University College and to Fisher at Rothamsted. In 1928 he derived the distribution
More informationACM 116: Lectures 3 4
1 ACM 116: Lectures 3 4 Joint distributions The multivariate normal distribution Conditional distributions Independent random variables Conditional distributions and Monte Carlo: Rejection sampling Variance
More informationExperimental Design and Statistics - AGA47A
Experimental Design and Statistics - AGA47A Czech University of Life Sciences in Prague Department of Genetics and Breeding Fall/Winter 2014/2015 Matúš Maciak (@ A 211) Office Hours: M 14:00 15:30 W 15:30
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationMAS223 Statistical Inference and Modelling Exercises
MAS223 Statistical Inference and Modelling Exercises The exercises are grouped into sections, corresponding to chapters of the lecture notes Within each section exercises are divided into warm-up questions,
More informationStatistical Methods in Particle Physics
Statistical Methods in Particle Physics Lecture 3 October 29, 2012 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2012 / 13 Outline Reminder: Probability density function Cumulative
More informationDepartment of Large Animal Sciences. Outline. Slide 2. Department of Large Animal Sciences. Slide 4. Department of Large Animal Sciences
Outline Advanced topics from statistics Anders Ringgaard Kristensen Covariance and correlation Random vectors and multivariate distributions The multinomial distribution The multivariate normal distribution
More informationSampling Distributions
In statistics, a random sample is a collection of independent and identically distributed (iid) random variables, and a sampling distribution is the distribution of a function of random sample. For example,
More informationChapter 4 Multiple Random Variables
Review for the previous lecture Theorems and Examples: How to obtain the pmf (pdf) of U = g ( X Y 1 ) and V = g ( X Y) Chapter 4 Multiple Random Variables Chapter 43 Bivariate Transformations Continuous
More informationChapter 7: Special Distributions
This chater first resents some imortant distributions, and then develos the largesamle distribution theory which is crucial in estimation and statistical inference Discrete distributions The Bernoulli
More informationMFM Practitioner Module: Quantitative Risk Management. John Dodson. September 23, 2015
MFM Practitioner Module: Quantitative Risk Management September 23, 2015 Mixtures Mixtures Mixtures Definitions For our purposes, A random variable is a quantity whose value is not known to us right now
More informationRandom Vectors 1. STA442/2101 Fall See last slide for copyright information. 1 / 30
Random Vectors 1 STA442/2101 Fall 2017 1 See last slide for copyright information. 1 / 30 Background Reading: Renscher and Schaalje s Linear models in statistics Chapter 3 on Random Vectors and Matrices
More informationt x 1 e t dt, and simplify the answer when possible (for example, when r is a positive even number). In particular, confirm that EX 4 = 3.
Mathematical Statistics: Homewor problems General guideline. While woring outside the classroom, use any help you want, including people, computer algebra systems, Internet, and solution manuals, but mae
More informationProbability Density Functions
Probability Density Functions Probability Density Functions Definition Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f (x) such that
More informationSTATISTICS SYLLABUS UNIT I
STATISTICS SYLLABUS UNIT I (Probability Theory) Definition Classical and axiomatic approaches.laws of total and compound probability, conditional probability, Bayes Theorem. Random variable and its distribution
More informationStat 5101 Lecture Slides Deck 4. Charles J. Geyer School of Statistics University of Minnesota
Stat 5101 Lecture Slides Deck 4 Charles J. Geyer School of Statistics University of Minnesota 1 Existence of Integrals Just from the definition of integral as area under the curve, the integral b g(x)
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationLecture 6: Special probability distributions. Summarizing probability distributions. Let X be a random variable with probability distribution
Econ 514: Probability and Statistics Lecture 6: Special probability distributions Summarizing probability distributions Let X be a random variable with probability distribution P X. We consider two types
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationSampling Distributions
Sampling Distributions In statistics, a random sample is a collection of independent and identically distributed (iid) random variables, and a sampling distribution is the distribution of a function of
More informationCommon ontinuous random variables
Common ontinuous random variables CE 311S Earlier, we saw a number of distribution families Binomial Negative binomial Hypergeometric Poisson These were useful because they represented common situations:
More informationSome Special Distributions (Hogg Chapter Three)
Some Special Distributions Hogg Chapter Three STAT 45-: Mathematical Statistics I Fall Semester 5 Contents Binomial & Related Distributions. The Binomial Distribution............... Some advice on calculating
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationSome Special Distributions (Hogg Chapter Three)
Some Special Distributions Hogg Chapter Three STAT 45-: Mathematical Statistics I Fall Semester 3 Contents Binomial & Related Distributions The Binomial Distribution Some advice on calculating n 3 k Properties
More informationPCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities
PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets
More informationContinuous Distributions
A normal distribution and other density functions involving exponential forms play the most important role in probability and statistics. They are related in a certain way, as summarized in a diagram later
More informationStat410 Probability and Statistics II (F16)
Stat4 Probability and Statistics II (F6 Exponential, Poisson and Gamma Suppose on average every /λ hours, a Stochastic train arrives at the Random station. Further we assume the waiting time between two
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationCHAPTER 6 SOME CONTINUOUS PROBABILITY DISTRIBUTIONS. 6.2 Normal Distribution. 6.1 Continuous Uniform Distribution
CHAPTER 6 SOME CONTINUOUS PROBABILITY DISTRIBUTIONS Recall that a continuous random variable X is a random variable that takes all values in an interval or a set of intervals. The distribution of a continuous
More information1 Review of Probability and Distributions
Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote
More informationBrief Review of Probability
Maura Department of Economics and Finance Università Tor Vergata Outline 1 Distribution Functions Quantiles and Modes of a Distribution 2 Example 3 Example 4 Distributions Outline Distribution Functions
More informationStatistical Methods in Particle Physics
Statistical Methods in Particle Physics. Probability Distributions Prof. Dr. Klaus Reygers (lectures) Dr. Sebastian Neubert (tutorials) Heidelberg University WS 07/8 Gaussian g(x; µ, )= p exp (x µ) https://en.wikipedia.org/wiki/normal_distribution
More informationARCONES MANUAL FOR THE SOA EXAM P/CAS EXAM 1, PROBABILITY, SPRING 2010 EDITION.
A self published manuscript ARCONES MANUAL FOR THE SOA EXAM P/CAS EXAM 1, PROBABILITY, SPRING 21 EDITION. M I G U E L A R C O N E S Miguel A. Arcones, Ph. D. c 28. All rights reserved. Author Miguel A.
More informationProbability Background
CS76 Spring 0 Advanced Machine Learning robability Background Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu robability Meure A sample space Ω is the set of all possible outcomes. Elements ω Ω are called sample
More informationSTAT Chapter 5 Continuous Distributions
STAT 270 - Chapter 5 Continuous Distributions June 27, 2012 Shirin Golchi () STAT270 June 27, 2012 1 / 59 Continuous rv s Definition: X is a continuous rv if it takes values in an interval, i.e., range
More informationLecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
More informationJoint Distributions. (a) Scalar multiplication: k = c d. (b) Product of two matrices: c d. (c) The transpose of a matrix:
Joint Distributions Joint Distributions A bivariate normal distribution generalizes the concept of normal distribution to bivariate random variables It requires a matrix formulation of quadratic forms,
More informationCovariance. Lecture 20: Covariance / Correlation & General Bivariate Normal. Covariance, cont. Properties of Covariance
Covariance Lecture 0: Covariance / Correlation & General Bivariate Normal Sta30 / Mth 30 We have previously discussed Covariance in relation to the variance of the sum of two random variables Review Lecture
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationActuarial Science Exam 1/P
Actuarial Science Exam /P Ville A. Satopää December 5, 2009 Contents Review of Algebra and Calculus 2 2 Basic Probability Concepts 3 3 Conditional Probability and Independence 4 4 Combinatorial Principles,
More information3 Continuous Random Variables
Jinguo Lian Math437 Notes January 15, 016 3 Continuous Random Variables Remember that discrete random variables can take only a countable number of possible values. On the other hand, a continuous random
More informationDefinition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R
Random Variables Definition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R As such, a random variable summarizes the outcome of an experiment
More informationProbability Distributions
Probability Distributions Seungjin Choi Department of Computer Science Pohang University of Science and Technology, Korea seungjin@postech.ac.kr 1 / 25 Outline Summarize the main properties of some of
More informationMultivariate Distributions
IEOR E4602: Quantitative Risk Management Spring 2016 c 2016 by Martin Haugh Multivariate Distributions We will study multivariate distributions in these notes, focusing 1 in particular on multivariate
More informationReview for the previous lecture
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Review for the previous lecture Definition: Several discrete distributions, including discrete uniform, hypergeometric, Bernoulli,
More informationChapter 1. Sets and probability. 1.3 Probability space
Random processes - Chapter 1. Sets and probability 1 Random processes Chapter 1. Sets and probability 1.3 Probability space 1.3 Probability space Random processes - Chapter 1. Sets and probability 2 Probability
More information2 Random Variable Generation
2 Random Variable Generation Most Monte Carlo computations require, as a starting point, a sequence of i.i.d. random variables with given marginal distribution. We describe here some of the basic methods
More informationRandom Vectors and Multivariate Normal Distributions
Chapter 3 Random Vectors and Multivariate Normal Distributions 3.1 Random vectors Definition 3.1.1. Random vector. Random vectors are vectors of random 75 variables. For instance, X = X 1 X 2., where each
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationDistributions of Functions of Random Variables. 5.1 Functions of One Random Variable
Distributions of Functions of Random Variables 5.1 Functions of One Random Variable 5.2 Transformations of Two Random Variables 5.3 Several Random Variables 5.4 The Moment-Generating Function Technique
More informationSTAT:5100 (22S:193) Statistical Inference I
STAT:5100 (22S:193) Statistical Inference I Week 10 Luke Tierney University of Iowa Fall 2015 Luke Tierney (U Iowa) STAT:5100 (22S:193) Statistical Inference I Fall 2015 1 Monday, October 26, 2015 Recap
More informationSTAT 4385 Topic 01: Introduction & Review
STAT 4385 Topic 01: Introduction & Review Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso xsu@utep.edu Spring, 2016 Outline Welcome What is Regression Analysis? Basics
More informationMATH c UNIVERSITY OF LEEDS Examination for the Module MATH2715 (January 2015) STATISTICAL METHODS. Time allowed: 2 hours
MATH2750 This question paper consists of 8 printed pages, each of which is identified by the reference MATH275. All calculators must carry an approval sticker issued by the School of Mathematics. c UNIVERSITY
More information1 Review of Probability
1 Review of Probability Random variables are denoted by X, Y, Z, etc. The cumulative distribution function (c.d.f.) of a random variable X is denoted by F (x) = P (X x), < x
More informationProbability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014
Probability Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh August 2014 (All of the slides in this course have been adapted from previous versions
More information1.12 Multivariate Random Variables
112 MULTIVARIATE RANDOM VARIABLES 59 112 Multivariate Random Variables We will be using matrix notation to denote multivariate rvs and their distributions Denote by X (X 1,,X n ) T an n-dimensional random
More informationPROBABILITY DISTRIBUTIONS. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
PROBABILITY DISTRIBUTIONS Credits 2 These slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Parametric Distributions 3 Basic building blocks: Need to determine given Representation:
More informationFIRST YEAR EXAM Monday May 10, 2010; 9:00 12:00am
FIRST YEAR EXAM Monday May 10, 2010; 9:00 12:00am NOTES: PLEASE READ CAREFULLY BEFORE BEGINNING EXAM! 1. Do not write solutions on the exam; please write your solutions on the paper provided. 2. Put the
More informationTABLE OF CONTENTS CHAPTER 1 COMBINATORIAL PROBABILITY 1
TABLE OF CONTENTS CHAPTER 1 COMBINATORIAL PROBABILITY 1 1.1 The Probability Model...1 1.2 Finite Discrete Models with Equally Likely Outcomes...5 1.2.1 Tree Diagrams...6 1.2.2 The Multiplication Principle...8
More informationPattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions
Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite
More informationStatistics and data analyses
Statistics and data analyses Designing experiments Measuring time Instrumental quality Precision Standard deviation depends on Number of measurements Detection quality Systematics and methology σ tot =
More informationAsymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
More informationStatistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
More informationChapter 2 Continuous Distributions
Chapter Continuous Distributions Continuous random variables For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following
More informationSpring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
More informationProbability Theory and Statistics. Peter Jochumzen
Probability Theory and Statistics Peter Jochumzen April 18, 2016 Contents 1 Probability Theory And Statistics 3 1.1 Experiment, Outcome and Event................................ 3 1.2 Probability............................................
More informationTHE QUEEN S UNIVERSITY OF BELFAST
THE QUEEN S UNIVERSITY OF BELFAST 0SOR20 Level 2 Examination Statistics and Operational Research 20 Probability and Distribution Theory Wednesday 4 August 2002 2.30 pm 5.30 pm Examiners { Professor R M
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
More informationMATH : EXAM 2 INFO/LOGISTICS/ADVICE
MATH 3342-004: EXAM 2 INFO/LOGISTICS/ADVICE INFO: WHEN: Friday (03/11) at 10:00am DURATION: 50 mins PROBLEM COUNT: Appropriate for a 50-min exam BONUS COUNT: At least one TOPICS CANDIDATE FOR THE EXAM:
More informationWeek 9 The Central Limit Theorem and Estimation Concepts
Week 9 and Estimation Concepts Week 9 and Estimation Concepts Week 9 Objectives 1 The Law of Large Numbers and the concept of consistency of averages are introduced. The condition of existence of the population
More informationProbability and Statistics
Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be Chapter 3: Parametric families of univariate distributions CHAPTER 3: PARAMETRIC
More informationSTA 2201/442 Assignment 2
STA 2201/442 Assignment 2 1. This is about how to simulate from a continuous univariate distribution. Let the random variable X have a continuous distribution with density f X (x) and cumulative distribution
More information2 Functions of random variables
2 Functions of random variables A basic statistical model for sample data is a collection of random variables X 1,..., X n. The data are summarised in terms of certain sample statistics, calculated as
More informationThis exam is closed book and closed notes. (You will have access to a copy of the Table of Common Distributions given in the back of the text.
TEST #3 STA 536 December, 00 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. You will have access to a copy
More informationReview of Probabilities and Basic Statistics
Alex Smola Barnabas Poczos TA: Ina Fiterau 4 th year PhD student MLD Review of Probabilities and Basic Statistics 10-701 Recitations 1/25/2013 Recitation 1: Statistics Intro 1 Overview Introduction to
More informationChp 4. Expectation and Variance
Chp 4. Expectation and Variance 1 Expectation In this chapter, we will introduce two objectives to directly reflect the properties of a random variable or vector, which are the Expectation and Variance.
More informationStatistical Distributions
CHAPTER Statistical Distributions In this chapter, we shall present some probability distributions that play a central role in econometric theory First, we shall present the distributions of some discrete
More information