Statistical distributions: Synopsis

Transcription

1 Statistical distributions: Synopsis Basics of Distributions Special Distributions: Binomial, Exponential, Poisson, Gamma, Chi-Square, F, Extreme-value etc Uniform Distribution Empirical Distributions Quantile Normalisation

2 Basics of Distributions A random variable X has (cumulative) distribution function F (x) = Pr(X x) Any increasing function F is a distribution if 0 = F ( ) F (x) F ( ) = 1 F (x) F (y) when x < y f x Figure 1 : Normal Distribution N(1,1)

3 Continuous Distributions If X is continuous then it has density function f (x) = df dx Pr(X x) = F (x) = x f (t)dt f x Figure 2 : Normal Distribution N(1,1)

4 Discrete Distributions If X takes discrete values (usually integers) then it has the mass function f (x) = Pr(X = x) Pr(X x) = F (x) = t x f (t) Poisson Distribution with mean λ: Pr(X = x) = e λ λx x! y x Figure 3 : Poisson Distribution λ = 5

5 Expectation The mean of the distribution E(X ) = µ = xf (x)dx The mean of a sample X = sample size n increases n i=1 X i n converges to E(X ) as the

6 Variance var(x ) = σ 2 = (x E(X )) 2 f (x)dx The standard deviation σ is measured on the same scale as X σ 2 = E(X E(X )) 2 ) Sample variance ˆσ 2 = n i=1 (X i X ) 2 n converges to σ 2 as n increases.

7 Additivity The expectation is additive: E(aX + by ) = ae(x ) + be(y ) E( X ) = E( X 1 + X X n ) = E(X 1) + E(X 2 ) E(X n ) = µ n n The variance is almost additive: var(ax + by ) = a 2 var(x ) + b 2 var(y ) var( X 1 + X X n ) = var(x 1) + var(x 2 ) var(x n ) n n 2 = σ2 n

8 Chebyshev s inequality True for any distribution with a variance. Relates the variance σ to probability of the extreme event X E(X ) Pr( k) σ Let I (V ) be the indicator function for an event V, so Pr(V ) = f (x) = E(I (V )) x V Pr( X E(X ) kσ) = E(I ( X E(X ) kσ)) ( ) X E(X ) 2 = E(I ( 1)) kσ ( ) X E(X ) 2 E( ) kσ = 1 k 2

9 Weak Law of Large Numbers The sample mean X is a random variable with expectation E(X ) and variance σ2 n. Applying Chebyshev s inequality (setting k = ɛ n) gives Pr( X E(X ) ɛ) 1 σ nɛ 2 For a fixed choice of ɛ, we can choose a sample size n so that this probability is as small as we wish. in large samples. Pr( X E(X ) σ ɛ) 0

10 Common Distributions Normal Binomial Poisson Negative Binomial Exponential Gamma Chi-Squared T F Extreme Value

11 Normal Distribution N(µ, σ 2 ) The Mother of all distributions density function f (x) = 1 (x µ)2 e 2σ 2 2πσ 2 Pr(X x) = Φ(x) = x f (t)dt Central Limit Theorem says the distribution of sample mean X of any distribution with a variance tends to N(µ, σ2 n )

12 Binomial Distribution B(n, p) Probability of r successes in n independent trials, each with probability p of success. Pr(X = r) = p r (1 p) n r n! r!(n r)! E(X ) = p var(x ) = p(1 p) As np 0, distribution of X /n N(p, p(1 p)) R functions are rbinom, dbinom, pbinom, qbinom

13 Multinomial Distribution Generalisation of the Binomial to multiple outcomes. A multivariate distribution of count data. Each observation takes one of K categorical values (e.g. 1-D contingency table). Probability of being in group i is p i. Out of a sample of N observations, N i are in group i. Probability of observing the vector (N 1, N 2,..., N K ): Pr(N 1, N 2,..., N K ) = N! i p N i i N i! E(N i ) = p i var(n i ) = p i (1 p i ) cov(n i, N j ) = p i p j

14 Poisson Distribution Po(λ) Probability of r independent events in a given time when the mean rate is λ. Pr(X = r) = e λ λr r! E(X ) = λ var(x ) = λ As λ, Po(λ) N(λ, λ) Additivity: if X Po(λ), Y Po(µ) then X + Y Po(λ + µ). Examples: Radioactive decay, Read coverage in next generation sequencing (not)

15 Negative Binomial Distribution NB(r,p) Distribution of the number of successful trials k until r failures, where each trial has probability p of success. The last trial is always a failure. Pr(X = k) = (1 p) r p k( ) k+r 1 k E(X ) = pr var(x ) = 1 p pr (1 p) 2 E(X ) Often used to model Over-dispersed Poisson distributions where variance exceeds the mean, e.g. in RNAseq data Po(λ) lim r > NB(r, λ λ+r )

16 Exponential Distribution Exp(λ) Distribution of times between Poisson events. Memoryless - the past does not influence future. F (t) = Pr(T < t) = 1 e λt f (t) = λe λt E(T ) = 1 λ var(t ) = 1 λ 2 log p-values Exp(1) waiting times between events in a Poisson Process Note - geometric distribution is discrete analogue of exponential, gives distribution of the number of trials until first failure p r (1 p)

17 Gamma Distribution Γ(n, λ) Distribution of sum of n independent exponential random variables Pr(T < t) = E(T ) = n λ var(t ) = n λ 2 t 0 (λt)n 1 e λt dt Γ(n) NOTE: In R, dgamma() etc parameters are a, s corresponding to n, 1 λ

18 Example: Fisher s Method for Combining P-values Often we only have p-values (not the underlying data) e.g. from a collection of GWASs. We want to test if collections of p-values are significant (e.g. for all SNPs in a gene). We are primarily interested in the smaller p-values, so looking at sums of p-values is not optimal because it is dominated by uninteresting large p-values Instead, consider S = i log(p i), which is dominated by contributions from smaller p-values. If the p-values are independent then s is distributed like a sum of n exponential random variables where λ = 1, i.e. S Γ(n, 1).

19 Chi-squared Distribution χ 2 n Distribution of the sum of n squared Normal random variables. Also a special case of the Gamma distribution: Applications: Likelihood ratio tests χ 2 n Γ(2n, 1) Distribution of sample variance Contingency Tables

20 T Distribution T n Distribution of T = Z S n of the ratio of a standard normal random variable Z N(0, 1) to the square root of Chi-square S χ 2 n on n df. f (x) = n+1 Γ( 2 ) nπγ( n 2 )(1 + x 2 n+1 ) 2 n where Γ(n) = 0 t n 1 e t dt Used in the T-test and to compute confidence intervals Note that when n = 1 then T 1 is the same as the Cauchy distribution and does not have a finite variance

21 F Distribution F (n, m) Distribution of the ratio of two scaled Chi-squared distributions on n, m df. Used in ANOVA and for comparing variances If X 1 χ 2 n, X 2 χ 2 m, then W = X 1m X 2 n (nw) n m m F (n, m). Density is f (w) = (nw+m) n+m w 1 0 tn/2 (1 t) m/2 dt E(W ) = m m 2 F (n, ) χ2 n n F (1, m) T 2 m

22 Extreme-Value Distributions (EVD) Distribution of M n = max(x 1, X 2,...X n ). Three limiting distributions depending on the underlying distribution of X i (compare to Central Limit Theorem...). We can find a n, b n such that Pr( Mn an b n < t) G(t) { exp( ( t a Weibull: G(t) = b )α ) t < a 1 otherwise Gumbel: G(t) = exp( e t a b ) { 0 t a Frechet: G(t) = exp( ( t a b ) α ) otherwise see R package GEV

23 EVD Example: Longest run R of succeses in N trials Consider the number M(r) of runs of length at least r. The Pr(R < r) = Pr(M(r) = 0). Pr(run of at least r at a given position) = (1 p)p r When r is large then long runs are very rare events, and will be Poisson distributed with mean µ = N(1 p)p r Pr(M(r) = 0) = e µ = e N(1 p)pr = exp( e r a b )) where b = log p, a b = log N(1 p) Gumbel distribution This argument can be used to model rare events as Poisson processes

24 Estimating Probability Densities - Kernel Density Estimation Data X 1, X 2,...X N from an unknown distribution. Density function can be estimated by superimposing many tiny distributions, each with variance σ 2 centred on an X i. f (x) = 1 N Often φ(x) is the density of a standard Normal N(0, 1). σ controls the degree of smoothing. see R function density() i φ( x X i σ ) density.default(x = u$mt[u$iscase == 0]) Density N = 5650 Bandwidth = Figure 4 : Estimated Density

25 Fitting a distribution R function fitdistr in library MASS Fits a parametric distribution to a sample by maximum likelihood eg fitdistr ( x, gamma ) density.default(x = u$mt[u$iscase == 0]) Density N = 5650 Bandwidth = Figure 5 : Estimated Density and Gamma fit

26 Empirical Cumulative Distribution Function (ECDF) Sample x 1, x 2,...x n. Define the indicator function I i (t) = 1 if x i t Then the ECDF ˆF (t) = Ii (t) n i.e. fraction of sample t. Asymptotically ˆF (t) F (t) as n R function ecdf() will compute the ECDF of a sample.

27 The Uniform Distribution X is uniformly distributed between 0 and 1. X U(0, 1) X is uniformly distributed between 0 and 1. F (x) = Pr(X < x) = x, (0 x 1) f (x) = 1, (0 x 1) R functions for U(0,1) are punic, dunif, qunif, runif z x

28 Quantile Normalisation Let X have distribution function F (x). Let u = F (x) Then the random variable U = F (X ) is uniformly distributed: Pr(U u) = Pr(F (X ) F (x)) = Pr(X x) = F (x) Pr(U u) = u So we can transform any distribution into the uniform Equally we can transform a uniform to any distribution using the quantile function F 1 (U)

29 Quantile Normalisation To transform a sample to a target distribution with CDF Φ(t): ECDF quantile normalises a sample to the uniform distribution: x i ˆF (x i ) Φ 1 (ˆF (x i )) has the desired distribution R code to quantile normalise a vector X to a Normal: n = length(x)+1 r = rank(x)/n q = qnorm(r) # Fhat # Phi_inv

30 Quantile Normalisation Transform data with awkward distribution to a better one Often stabilises behaviour of P-values, but can lose power Useful in GWAS

31 Quantile-Quantile Plots Does a sample x 1, x 2...x n come from a given distribution F? Sort y 1 y 2... y n and quantile normalise to a uniform F 1 (y 1 ) F 1 (y 2 )... F 1 (y n ) The expected quantiles of a uniform distribution are 1 n + 1, 2 n + 1,... n n + 1 E(F 1 (y k )) = k n + 1 If F is correct then expect to observe linear plots of F 1 (y k )) vs E(F 1 (y k )) log 10 F 1 (y k )) vs log 10 E(F 1 (y k ))

32 QQ Plots Examples observed quantile theoretical quantile Figure 7 : black: sample from N(0,1), red: sample from T 2

33 QQ Plots Examples Most common scenario is when the sample X 1...X n are millions of log p-values from a GWAS. Inflation of P-values can occur because of: Population Structure Non-normality of the phenotype (for quantitative traits) Linkage Disequilibrium Unknown Reasons...

34 Comparing Distributions: Kolmogorov-Smirnov Test Are two continuous ECDFs ˆF 1 (x), ˆF 2 (x) the same? D = max x ˆF 1 (x) ˆF 2 (x) Distribution of D is known, and is independent of ˆF 1 (x), ˆF 2 (x) so can be used to compare them. It can also compare an ECDF to a given known distribution. The distribution of the KS statistic D also provides confidence intervals for QQ plots. The KS test is implemented in the R function ks.test() Other tests for comparing distributions include the Chi-squared test (especially for discrete valued distributions), Anderson-Darling test.