Advanced Herd Management Probabilities and distributions Anders Ringgaard Kristensen Slide 1
Outline Probabilities Conditional probabilities Bayes theorem Distributions Discrete Continuous Distribution functions Sampling from distributions Estimation Hypotheses Confidence intervals Slide 2
Probabilities: Basic concepts The probability concept is used in daily language. What do we mean when we say: The probability of the outcome 5 when rolling a dice is 1/6? The probability that cow no. 543 is pregnant is 0.40? The probability that USA will attack North Korea within 5 years is 0.05? Slide 3
Interpretations of probabilities At least 3 different interpretations are observed: A frequentist interpretation: The probability expresses how frequent we will observe a given outcome if exactly the same experiment is repeated a large number of times. The value is rather objective. An objective belief interpretation: The probability expresses our belief in a certain (unobservable) state or event. The belief may be based on an underlying frequentist interpretation of similar cases and thus be rather objective. A subjective belief interpretation: The probability expresses our belief in a certain unobservable (or not yet observed) event. Slide 4
Experiments An experiment may be anything creating an outcome we can observe. The sample space, S, is the set of all possible outcomes. An event, A, is a subset of S, i.e. A S Two events A 1 and A 2 are called disjoint, if they have no common outcomes, i.e. if A 1 A 2 = Slide 5
Example of experiment Rolling a dice: The sample space is S = {1, 2, 3, 4, 5, 6} Examples of events: A 1 = {1} A 2 = {1, 5} A 3 = {4, 5, 6} Since A 1 A 3 =, A 1 and A 3 are disjoint. A 1 and A 2 are not disjoint, because A 1 A 2 = {1} Slide 6
A simplified definition Let S be the sample space of an experiment. A probability distribution P on S is a function, so that P(S) = 1. For any event A S, 0 P(A) 1 For any two disjoint events A 1 and A 2, P(A 1 A 2 ) = P(A 1 ) + P(A 2 ) Slide 7
Example: Rolling a dice Like before: S = {1, 2, 3, 4, 5, 6} A valid probability function on S is, for A S: P(A) = A /6 where A is the size of A (i.e. the number of elements it contains) P({1}) = P({2}) = P({3}) = P({4}) = P({5}) = P({6}) = 1/6 P({1, 5}) = 2/6 = 1/3 P({1, 2, 3}) = 3/6 = 1/2 Notice, that many other valid probability functions could be defined (even though the one above is the only one that makes sense from a frequentist point of view). Slide 8
Independence If two events A and B are independent, then P(A B) = P(A)P(B). Example: Rolling two dices S = {(1, 1), (1, 2),, (1, 6),, (6, 6)} For any A S: P(A) = A /36 A = {(6, 1), (6, 2),, (6, 6)} P(A) = 6/36 = 1/6 B = {(1, 6), (2, 6),, (6, 6)} P(B) = 6/36 = 1/6 A B = {(6, 6)} and P(A B) = (1/6)(1/6) = 1/36 Slide 9
Conditional probabilities Let A and B be two events, where P(B) > 0 The conditional probability of A given B is written as P(A B), and it is by definition Slide 10
Example: Rolling a dice Again, let S = {1, 2, 3, 4, 5, 6}, and P(A) = A /6. Define B = {1, 2, 3}, and A = {2}. Then A B = {2}, and The logical result: If you know the outcome is 1, 2 or 3, it is reasonable to assume that all 3 values are equally probable. Slide 11
Conditional sum rule Let A 1, A 2, A n be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then Slide 12
Sum rule: Dice example Define the 3 disjoint events A 1 = {1, 2}, A 2 = {3, 4}, A 3 = {5, 6} Thus A 1 A 2 A 3 = S Define B = {1, 3, 5} (we know that P(B) = ½) P(B A 1 ) = P(B A 1 )/P(A 1 ) = (1/6)/(1/3) = ½ P(B A 2 ) = P(B A 2 )/P(A 2 ) = (1/6)/(1/3) = ½ P(B A 3 ) = P(B A 3 )/P(A 3 ) = (1/6)/(1/3) = ½ Thus Slide 13
Bayes theorem Let A 1, A 2, A n be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then Bayes theorem is extremely important in all kinds of reasoning under uncertainty. Updating of belief. Slide 14
Updating of belief, I In a dairy herd, the conception rate is known to be 0.40. Define M as the event mating for a cow. Define Π + as the event pregnant for the same cow, and Π - as the event not pregnant. Thus P(Π + M) = 0.40 is a conditional probability. Given that the cow has been mated, the probability of pregnancy is 0.40. Correspondingly, P(Π - M) = 0.60 After 3 weeks the farmer observes the cow for heat. The farmer s heat detection rate is 0.55. Define H + as the event that the farmer detects heat. Thus, P(H + Π - ) = 0.55, and P(H - Π - ) = 0.45 There is a slight risk that the farmer erroneously observes a pregnant cow to be in heat. We assume, that P(H + Π + ) = 0.01 Notice, that all probabilities are figures that makes sense and are estimated on a routine basis (except P(H + Π + ) which is a guess) Slide 15
Updating of belief, II Now, let us assume that the farmer observes the cow, and concludes, that it is not in heat. Thus, we have observed the event H - and we would like to know the probability, that the cow is pregnant, i.e. we wish to calculate P(Π + H - ) We apply Bayes theorem: We know all probabilities in the formula, and get In other words, our belief in the event pregnant increases from 0.40 to 0.59 based on a negative heat observation result Slide 16
Summary of probabilities Probabilities may be interpreted As frequencies As objective or subjective beliefs in certain events The belief interpretation enables us to represent uncertain knowledge in a concise way. Bayes theorem lets us update our belief (knowledge) as new observations are done. Slide 17
Discrete distributions In some cases the probability is defined by a certain function defined over the sample space. In those cases, we say that the outcome is drawn from a standard distribution. There exist standard distributions for many natural phenomena. If the sample space is a countable set, we denote the corresponding distribution as discrete. Slide 18
Discrete distributions If X is the random variable representing the outcome, the expected value of a discrete distribution is defined as The variance is defined as We shall look at two important discrete distributions: The binomial distribution The Poisson distribution. Slide 19
The binomial distribution I Consider an experiment with binary outcomes: Success (s) or failure (f) Mating of a sow Pregnant (s), not pregnant (f) Tossing a coin Heads (s), tails (f) Testing for a disease Present (s), not present (f) Assume that the probability of success is p and that the experiment is repeated n times. Let X be the total number of successes observed in the n experiments. The sample space of the compound n experiments is S = {0, 1, 2,, n} The random variable X is then said to be binomially distributed with parameters p and n. Slide 20
The binomial distribution II The probability function P(X = k) is (by objective frequentist interpretation) given by where is the binomial coefficient which may be calculated or looked up in a table. Slide 21
The binomial distribution III The mean (expected value) of a binomial distribution is simply E(X) = np. The variance is Var(X) = np(1-p) The binomial distribution is one of the most frequently used distribution for natural phenomena. Slide 22
The binomial distribution IV Three binomial distributions with n = 10 P(k ) 0,35 0,3 0,25 0,2 0,15 0,1 0,05 0 0 1 2 3 4 5 6 7 8 9 10 k 0,2 0,5 0,8 Three binomial distributions, where n = 10, and p = 0.2, 0.5 and 0.8, respectively. Slide 23
The Poisson distribution I If a certain phenomenon occurs a random with a constant intensity (but independently of each others) the total number of occurrences X in a time interval of a given length (or in a space of a given area) is Poisson distributed with parameter λ Examples: Number of (non-infectious) disease cases per month Number of feeding system failures per year Number of labor incidents per year Slide 24
The Poisson distribution II The sample space for Y is S = {0, 1, 2, } The probability function P(X = k) is (by objective frequentist interpretation) given by The expected value is E(X) = λ The variance is Var(X) = λ The Poisson distribution may be used as an approximation for a binomial distribution with small p and large n Slide 25
The Poisson distribution III Three poisson distributions P(k ) 0,3 0,25 0,2 0,15 0,1 0,05 0 0 5 10 15 20 25 k 2 6 12 Three Poisson distributions with λ = 2, 6 and 12, respectively. Slide 26
Continuous distributions In some cases, the sample space S of a distribution is not countable. If, furthermore, S is an interval on R, the random variable X taking values in S is said to have a continuous distribution. For any x S, we have P(X = x) = 0. Thus, no probability function exists for a continuous distribution. Instead, the distribution is defined by a density function f(x). Slide 27
Density functions The density function f has the following properties (for a, b R and a b) Thus, for a continuous distribution, f can only be interpreted as a probability when integrated over an interval. Slide 28
Continuous distributions For a continuous distribution, the expected value E(X) is defined as And the variance is (just like the discrete case) We shall here look at 3 important distributions: The uniform distribution The normal distribution The exponential distributions Slide 29
The uniform distribution If S = [a; b], and the random variable X has a uniform distribution on S, then the density function is The expected value and the variance are Uniform f(x) 1 0,8 0,6 0,4 0,2 Slide 30 0 0 0,5 1 1,5 2 x
The normal distribution I If S = R, and the random variable X has a normal distribution on S, then the density function is The expected value and the variance simply turn out to be E(X) = µ, and Var(X) = σ 2 We say that X is N(µ, σ 2 ), or X N(µ, σ 2 ) Slide 31
The normal distribution II The normal distribution may be used to represent almost all kinds of random outcome on the continuous scale in the real world. Exceptions are phenomena that are bounded in some sense (e.g. the waiting time to be served in a queue cannot be negative) It can be showed (central limit theorems) that if X 1, X 2,, X n are random variables of (more or less) any kind, then the sum Y n = X 1 + X 2 + + X n is normally distributed for n sufficiently large. The normal distribution is the cornerstone among statistical distributions. Slide 32
Normal distributions III Three normal distributions 0,5 f(x ) 0,4 0,3 0,2 0,1 0-10 -5 0 5 10 x m=0, s=3 m=-5, s=1 m=0, s=1 Three normal distributions with mean m and standard deviation s Slide 33
Normal distributions IV The normal distribution with µ = 0, and σ = 1 is called the standard normal distribution. A random variable being standard normally distributed is often denoted as Z The density function of the standard normal distribution is often denoted as φ. It follows that Slide 34
Normal distributions V Let X 1 N(µ 1, σ 1 2), X 2 N(µ 2, σ 2 2), and X 1 and X 2 are independent. Define Y 1 = X 1 + X 2 and Y 1 = X 1 X 2. Then Y 1 N(µ 1 + µ 2, σ 2 1 + σ 2 2) Y 2 N(µ 1 µ 2, σ 2 1 + σ 2 2) Let a and b be arbitrary real numbers, and let X N(µ, σ2). Define Y =ax + b. Then, Y N(aµ + b, a 2 σ 2 ) Slide 35
Normal distributions VI From the previous slide it follows in particular, that if X N(µ, σ2), then So, if f is the density function of X N(µ, σ2), then Thus, we can calculate the value of any density function for a normal distribution from the density distribution of the Slide standard 36 normal distribution.
Exponential distribution I If S = R + = ]0; [, and the random variable X has an exponential distribution on S, then the density function is The expected value and the variance are E(X) = λ -1, and Var(X) = λ -2 We say that X is exponentially distributed with parameter λ. Slide 37
Exponential distribution II The exponential distribution is in many ways the complimentary to the Poisson distribution. If something happens at random at constant intensity, the number of events within a fixed time interval is Poisson distributed, and the waiting time between two events is exponentially distributed. Less frequently used in herd management. Slide 38
Exponential distribution III Three exponential distributions 1 f(x x ) 0,8 0,6 0,4 0,2 1 0,5 0,2 0 0 2 4 6 8 10 x Three exponential distributions with mean 1, 2 and 5, respectively. Slide 39
Distribution functions I The distributions presented have all been defined by their probability functions (discrete distributions) and density functions (continuous distributions). We might just as well have used the distribution function F, which is defined in the same way for both classes of distributions: F(x) = P(X x) Slide 40
Distribution functions II Even though the definition is the same, the value of the distribution function is calculated in different ways for the two classes of distributions. For discrete distributions For continuous distributions Slide 41
Distribution functions III It follows directly, that for a continuous distribution, F (x) = f(x) The distribution function of the standard normal distribution is often denoted as Φ, and naturally Φ (z) = φ(z). No closed form (formula) exists for Φ, it must be looked up in tables. For discrete distributions, the distribution function most often doesn t have a closed form, so it must be looked up in tables. Slide 42
Distribution functions IV Any distribution function F has the following two properties: F(x) 0 for x - F(x) 1 for x Slide 43
Distribution function, Binomial Three binomial distributions with n = 10 Three binomial distributions with n = 10 0,35 1,2 P (k ) 0,3 0,25 0,2 0,15 0,1 0,05 0 0 1 2 3 4 5 6 7 8 9 10 0,2 0,5 0,8 P(k ) 1 0,8 0,6 0,4 0,2 0 0 1 2 3 4 5 6 7 8 9 10 0,2 0,5 0,8 k k Probability functions to the left, distribution functions to the right. Slide 44
Distribution function, Poisson Three poisson distributions Probability function P(k ) 0,3 0,25 0,2 0,15 0,1 0,05 0 0 5 10 15 20 25 k 2 6 12 P(k ) 1,2 1 0,8 0,6 0,4 0,2 0 Slide 45 Three poisson distributions 0 5 10 15 20 25 k 2 6 12 Distribution function
Distribution function, uniform Uniform Uniform f(x) 1 0,8 0,6 0,4 0,2 0 0 0,5 1 1,5 2 x f(x) 1 0,8 0,6 0,4 0,2 0 0 0,5 1 1,5 2 x Density function to the left Distribution function to the right Slide 46
Distribution function, normal Three normal distributions Three normal distributions 0,5 1 f(x ) 0,4 0,3 0,2 0,1 m=0, s=3 m=-5, s=1 m=0, s=1 f(x ) 0,8 0,6 0,4 0,2 m=0, s=3 m=-5, s=1 m=0, s=1 0-10 -5 0 5 10 x 0-10 -5 0 5 10 x Density function to the left Distribution function to the right Slide 47
Distribution function, exponential Three exponential distributions Three exponential distributions 1 1 f(x ) 0,8 0,6 0,4 0,2 1 0,5 0,2 f(x ) 0,8 0,6 0,4 0,2 1 0,5 0,2 0 0 2 4 6 8 10 x 0 0 2 4 6 8 10 x Density function to the left Distribution function to the right Slide 48
Sampling from a distribution Assume that X 1, X 2,, X n are sampled independently from the same distribution having the known expectation µ and the known standard deviation σ Then the mean of the sample has the expected value µ and the standard deviation In particular, if the X i s are N(µ, σ 2 ) then the sample mean is N(µ, σ 2 /n) Slide 49
Sampling from a normal distribution Assume that X 1, X 2,, X n are sampled independently from the same normal distribution N(µ, σ 2 ) where µ is unknown and σ is known. For some reason we expect (hope) that µ has a certain value µ 0, and we would therefore like to test the following hypothesis: H 0 : µ = µ 0 How can we do that? Well, we know that the sample mean is N(µ, σ 2 /n) Slide 50
Hypothesis testing, normal dist. I A normal distribution with standard deviation 3 A normal distribution with standard deviation 3 0,2 1 0,8 f(x) 0,1 m=0, s=3 f(x) 0,6 0,4 m=0, s=3 0,2 0-1 -9-8 -7-6 -5-4 -3-2 -1 0 1 2 3 4 5 6 7 8 9 10 0 0-1 -9-8 -7-6 -5-4 -3-2 -1 0 1 2 3 4 5 6 7 8 9 10 0 x x Observations close to the mean are far more likely than distant observations. From the distribution function we can calculate the likelihood that an observation falls within the interval µ ± σ The likelihood that an observation falls within the interval µ ± 2σ Rule of thumb: 2/3 of the observations falls within ±σ and 95% within ±2σ Slide 51
Hypothesis testing, normal dist. II We can test our hypothesis H 0 for instance by calculating a confidence interval for the mean. A 95% confidence interval for the sample mean (distributed as N(µ, σ 2 /n)) under H 0 is calculated as If the sample mean is included in the interval, we accept H 0, otherwise we reject. If neither µ nor σ are known, the sample mean becomes student-t distributed (with n-1 degrees of freedom) instead. Then the confidence interval becomes wider as consequence of the uncertainty on σ. For large n the student-t distribution converges towards a standard normal Slide distribution. 52
Hypothesis testing, binomial Assume that we have observed the outcome of X successes out of n in a binomial trial. We would like to test the hypothesis: H 0 : p = p 0 Under H 0, the expected number of successes is E 0 (X) = np 0 and the variance is Var 0 = np 0 (1-p 0 ) How likely is it that the observed value of X is drawn from a binomial distribution with parameters p 0 and n? Basically two approaches may be used: Approximate with the normal distribution N(np 0,Var 0 ). This is a reasonable approach if n is big. Remember that n now has a different meaning! We have only one observation from the distribution Use the distribution function of the binomial distribution Slide 53 directly. Only valid approach for small n.
Other distributions Used as hyper distributions for parameters of other distributions in order to represent uncertainty: The Gamma distribution (hyper distribution for the mean and variance of a poisson) The Beta distribution (hyper distribution for the p parameter of a binomial distribution) Will be discussed briefly under advanced topics. Distributions for statistical tests: The χ 2 distribution. The student-t distribution The F distribution Those distributions will not be discussed very much in this course. Many other distributions are described in literature Slide 54
What distribution can I use to represent: Litter size in sheep? Litter size in sows? Number of cows/sows conceiving after first service. Time to first estrus? Milk yield of dairy cows? Daily gain of slaughter pigs? Slide 55