Mean and variance. Compute the mean and variance of the distribution with density

Transcription

1 Mean and variance Compute the mean and variance of the distribution with density > f <- function(x) x^3 * exp(-x)/6 using integrate. Then compute the mean and variance for the distribution with distribution function > F <- function(x) 1 - x^(-0.3) * exp(-0.4 * (x - 1)). p.1/19

2 Solutions > f <- function(x) x^3 * exp(-x)/6 > f.mod <- function(x) x * f(x) > mu <- integrate(f.mod, 0, Inf)$value > mu [1] 4 > f.mod <- function(x) (x - mu)^2 * f(x) > integrate(f.mod, 0, Inf)$value [1] 4. p.2/19

3 Solutions - continued Have to compute the density by differentiation > F <- function(x) 1 - x^(-0.3) * exp(-0.4 * (x - 1)) > f <- function(x) 0.3 * x^(-1.3) * exp(-0.4 * (x - 1)) * + x^(-0.3) * exp(-0.4 * (x - 1)) > f.mod <- function(x) x * f(x) > mu <- integrate(f.mod, 1, Inf)$value > mu [1] > f.mod <- function(x) (x - mu)^2 * f(x) > integrate(f.mod, 1, Inf)$value [1] p.3/19

4 Conditional probabilities P is a probability measure on the sample space E. Definition: If A E is an event with P(A) > 0 we define the conditional probability of B given A as P(B A) = P(B A) P(A).. p.4/19

5 Conditional probabilities P is a probability measure on the sample space E. Definition: If A E is an event with P(A) > 0 we define the conditional probability of B given A as P(B A) = P(B A) P(A). Fixing A we have that B P(B A) is a probability measure the conditional probability measure given A. need to check if the conditions for a probability measure are satisfied.. p.4/19

6 EPO example An EPO test reveals in 99% of the cases a cyclist that has taken EPO. Is this good?. p.5/19

7 EPO example An EPO test reveals in 99% of the cases a cyclist that has taken EPO. Is this good? tp = fp = tn = fn = true positive false positive true negative false negative. p.5/19

8 EPO example An EPO test reveals in 99% of the cases a cyclist that has taken EPO. Is this good? tp = fp = tn = fn = true positive false positive true negative false negative With A 1 = {tp, fn} and B = {tp, fp} we know that P(B A 1 ) = 0.99 We want to know P(A 1 B).. p.5/19

9 Bayes theorem Theorem: If A 1,...,A n are disjoint events in E with E = A 1... A n and if B E is any event (with P(B) > 0) then P(A i B) = for all i = 1,...,n. P(B A i )P(A i ) P(B A 1 )P(A 1 ) P(B A n )P(A n ) (-2). p.6/19

10 EPO example Let A 2 = {fp, tn} then P(A 1 B) = P(B A 1 )P(A 1 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ).. p.7/19

11 EPO example Let A 2 = {fp, tn} then P(A 1 B) = P(B A 1 )P(A 1 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ). Assume P(A 2 B) = If P(A 1 ) = 0.07 (P(A 2 ) = 0.93) P(A 1 B) = = p.7/19

12 EPO example Let A 2 = {fp, tn} then P(A 1 B) = P(B A 1 )P(A 1 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ). Assume P(A 2 B) = If P(A 1 ) = 0.07 (P(A 2 ) = 0.93) P(A 1 B) = = If P(A 1 ) = 0.3 then P(A 1 B) = = p.7/19

13 Independence Natural to say that B is independent of A if P(B A) = P(A). This implies that A is independent of B and that P(A B) = P(A)P(B). p.8/19

14 Independence Natural to say that B is independent of A if P(B A) = P(A). This implies that A is independent of B and that P(A B) = P(A)P(B) Definition: We say that two events A and B are independent if P(A B) = P(A)P(B).. p.8/19

15 Random variables A random variable is a notational convention representing the unrealized outcome of an experiment prior to observing the outcome. Usually denoted X, Y, Z.. p.9/19

16 Random variables A random variable is a notational convention representing the unrealized outcome of an experiment prior to observing the outcome. Usually denoted X, Y, Z. The probability measure governing the experiment is called the distribution of the corresponding random variable.. p.9/19

17 Random variables A random variable is a notational convention representing the unrealized outcome of an experiment prior to observing the outcome. Usually denoted X, Y, Z. The probability measure governing the experiment is called the distribution of the corresponding random variable. A random variable X representing the outcome from a binary experiment with sample space E = {0, 1} is called a Bernoulli variable. The probability p = P(X = 1) is often called the success probability.. p.9/19

18 Transformations If E and E are two sample spaces, a transformation is a map h : E E that assigns the transformed outcome h(x) in E to the outcome x in E. Definition: If P is a probability measure on E the transformed probability measure, h(p), on E is given by h(p)(a) = P(h 1 (A)) = P({x E h(x) A}) for any event A E. If X is a random variable with distribution P, the distribution of h(x) is h(p).. p.10/19

19 Point probabilities If E discrete, P a probability measure on E with point probabilities p(x) for x E and h : E E the probability measure h(p) has point probabilities q(y) = x:h(x)=y p(x), y h(e).. p.11/19

20 Point probabilities If E discrete, P a probability measure on E with point probabilities p(x) for x E and h : E E the probability measure h(p) has point probabilities q(y) = x:h(x)=y p(x), y h(e). If E R then the mean (provided it is defined) under h(p) is µ = x E h(x)p(x). p.11/19

21 Point probabilities If E discrete, P a probability measure on E with point probabilities p(x) for x E and h : E E the probability measure h(p) has point probabilities q(y) = x:h(x)=y p(x), y h(e). If E R then the mean (provided it is defined) under h(p) is µ = x E h(x)p(x) and the variance (provided it is defined) under h(p) is σ 2 = x E (h(x) µ) 2 p(x) = x E h(x) 2 p(x) µ 2.. p.11/19

22 Exponential families Let E be discrete and H : E R a function. Define for θ R the function φ(θ) = x E exp(θh(x)) [0, ].. p.12/19

23 Exponential families Let E be discrete and H : E R a function. Define for θ R the function φ(θ) = x E exp(θh(x)) [0, ]. With Θ = {θ R φ(θ) < } we define for θ Θ the probability measure P θ on E with point probabilities p θ (x) = 1 φ(θ) exp(θh(x)).. p.12/19

24 Exponential families Let E be discrete and H : E R a function. Define for θ R the function φ(θ) = x E exp(θh(x)) [0, ]. With Θ = {θ R φ(θ) < } we define for θ Θ the probability measure P θ on E with point probabilities p θ (x) = 1 φ(θ) exp(θh(x)). The family of measures P θ on E parameterized by θ Θ is called an exponential family.. p.12/19

25 Exponential families Define the following function in R > H <- function(n) -(n-10)^2 and let H be defined on E = N. What is Θ? Compute and plot the function φ. Compute and plot the point probabilities for the probability measure corresponding to θ = 1 and θ = 4.. p.13/19

26 Exponential families Provided that all sums are sensible we have the mean under H(P θ ) is µ(θ) = x E 1 φ(θ) H(x) exp(θh(x)) = φ (θ) φ(θ).. p.14/19

27 Exponential families Provided that all sums are sensible we have the mean under H(P θ ) is µ(θ) = x E 1 φ(θ) H(x) exp(θh(x)) = φ (θ) φ(θ). and the variance under H(P θ ) σ 2 (θ) = x E H(x) 2 φ(θ) exp(θh(x)) φ (θ) 2 φ(θ) 2 = φ (θ)φ(θ) φ (θ) 2 φ(θ) 2 = µ (θ).. p.14/19

28 Special exponential families If E R and c : E R let Θ = {θ R x E exp(θx + c(x)) < } and define for θ Θ the probability measure P θ on E with point probabilities p θ (x) = exp(θx b(θ) + c(x)) where b(θ) = x E exp(θx + c(x)).. p.15/19

29 Special exponential families If E R and c : E R let Θ = {θ R x E exp(θx + c(x)) < } and define for θ Θ the probability measure P θ on E with point probabilities p θ (x) = exp(θx b(θ) + c(x)) where b(θ) = x E exp(θx + c(x)). In the general notation we have b(θ) = log φ(θ) = x E exp(θx)w(x) with w(x) = exp(c(x)).. p.15/19

30 Special exponential families For the mean and variance under P θ we find that µ(θ) = φ (θ) φ(θ) = b (θ). p.16/19

31 Special exponential families For the mean and variance under P θ we find that µ(θ) = φ (θ) φ(θ) = b (θ) and σ 2 (θ) = b (θ).. p.16/19

32 Special exponential families For the mean and variance under P θ we find that µ(θ) = φ (θ) φ(θ) = b (θ) and σ 2 (θ) = b (θ). The Bernoulli distribution and the Poisson distribution are two examples of distributions that fit within the framework of distributions considered here. The importance is that we have established a general relation between the θ parameter and the mean value.. p.16/19

33 Location and scale Consider the transformation h : R R given by h(x) = σx + µ for some constants µ R and σ > 0 is called a scale-location transformation.. p.17/19

34 Location and scale Consider the transformation h : R R given by h(x) = σx + µ for some constants µ R and σ > 0 is called a scale-location transformation. If X is a real valued random variable with distribution function F then the distribution of Y = h(x) = σx + µ has distribution function G(x) = F ( ) x µ. σ. p.17/19

35 Location and scale If F is differentiable with f = F the distribution of Y has density g(x) = G (x) = 1 ( ) x µ σ f. σ. p.18/19

36 Location and scale If F is differentiable with f = F the distribution of Y has density g(x) = G (x) = 1 ( ) x µ σ f. σ If, moreover, X has mean 0 and variance 1 then Y has mean µ and variance σ 2.. p.18/19

37 The normal distribution The density for the normal distribution with location parameter µ and scale parameter σ 2 is ) 1 ( σ 2π exp (x µ)2 2σ 2 = ( ) 1 exp (x µ)2 2πσ 2 2σ 2.. p.19/19

38 The normal distribution The density for the normal distribution with location parameter µ and scale parameter σ 2 is ) 1 ( σ 2π exp (x µ)2 2σ 2 = ( ) 1 exp (x µ)2 2πσ 2 2σ 2. We write X N(µ, σ 2 ) to denote that the distribution of X is the normal distribution with location parameter µ ans scale parameter σ 2.. p.19/19

39 The normal distribution The density for the normal distribution with location parameter µ and scale parameter σ 2 is ) 1 ( σ 2π exp (x µ)2 2σ 2 = ( ) 1 exp (x µ)2 2πσ 2 2σ 2. We write X N(µ, σ 2 ) to denote that the distribution of X is the normal distribution with location parameter µ ans scale parameter σ 2. Since the mean and variance of X N(0, 1) is 0 and 1, respectively, it follows that if X N(µ, σ 2 ) is µ and σ 2, respectively.. p.19/19