1 Presessional Probability

Transcription

1 1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional material summarises some basic facts, most of which you have seen before, and are given for future reference. 1.1 Finite Probability Spaces Consider a random experiment with a finite set of possible outcomes. This set is called the sample space and denoted Ω. Probability measure on Ω is given by a function that assigns to each ω Ω a number P(ω [0, 1], so that P(ω = 1. ω Ω The pair (Ω, P is a finite probability space. The subsets of Ω are called events. Probability of an event A is defined as P(A = ω A P(ω. The addition rule of probabilities holds: if A B = (events A and B are disjoint, or incompatible, then P(A B = P(A + P(B. In general, we have the inclusion-exclusion formula P(A B = P(A + P(B P(A B. The event complement to A is denoted A c = Ω \ A. Note that P(A c = 1 P(A. Example 1.1. Flipping a coin three times. The sample space is Ω = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }. An individual element of Ω is a sequence of three tosses ω = ω 1 ω 2 ω 3 (this is a shorthand notation for (ω 1, ω 2, ω 3. We assume that p is probability of a head and q = 1 p probability of a tail, and that the tosses are independent. Then P(HHH = p 3, P(HHT = p 2 q,..., P(T T T = q 3. Let A be the event first toss is a head. probability of A is p: We check from the definitions that the P(A = P(ω ω Ω =P(HHH + P(HHT + P(HT H + P(HT T =p 3 + p 2 q + p 2 q + pq 2 = p 2 (p + q + pq(p + q =p 2 + pq = p(p + q = p. The complement event A c is first toss is a tail. We have P(A c = 1 p = q. 1

2 Example 1.2. Let a, b, c be three companies. Let (i, j, k be the outcome that in 2014 company i makes more profit than company j and that company j makes more profit than company k. Then the sample space is Ω = {(a, b, c, (a, c, b, (b, a, c, (b, c, a, (c, a, b, (c, b, a}. Define a probability measure on Ω by letting P(ω = 1/6 for every ω Ω. Let A be the event that a makes maximum profit in Then A = {(a, b, c, (a, c, b} and P(A = 1/6 + 1/6 = 1/3. Example 1.3. Let the probability that the FTSE100 increases today be 0.52 and the probability that it increases tomorrow be 0.52 as well. Suppose that the probability that it increases both today and tomorrow is What is the probability that the FTSE100 increases neither today nor tomorrow? Solution. Let A be the event that the FTSE100 increases today and let B be the event that the FTSE100 increases tomorrow. We are given that P(A = P(B = 0.52 and P(A B = 0.28 and we want to find P((A B c. Now P(A B = P(A + P(B P(A B = 0.76, so P((A B c = = 0.24 is the sought answer. Two events A and B are called independent if P(A B = P(AP(B. Note: independent does not mean disjoint. Disjoint (incompatible events distinct from, Ω are not independent! If P(B > 0 the conditional probability of A given B is P(A B = P(A B P(B. If A and B are events with P(B > 0, then P(A = P(A B is the same at that A and B are independent. Exercise 1.4. Two dice are rolled. 1. Let A be the event the first die turns up an odd number, B the total is even. Are the events A and B independent? Give a detailed answer by calculating the probabilities. 2. Let C be the total is seven, D first die turns up three. Calculate P(C D, P(D C. More generally, two or more events A 1, A 2,..., A k are said to be (mutually independent if ( k k P B j = P(B j, where each B j is either A j or A c j. Equivalently, if for every possible selection A i1,..., A im from the list A 1,..., A k it holds P (A i1... A im = P(A i1 P(A im. 2

3 It is not enough for the mutual independence that the events are pairwise independent. It might happen that P(A i A j = P(A i P(A j for every pair i j, but the independence fails. In particular, three events A, B, C are independent if the following four equalities hold: P(A B C = P(AP(BP(C P(A B = P(AP(B, P(A C = P(AP(C, P(B C = P(BP(C. 1.2 General probability spaces Finite probability spaces are by far not sufficient to describe many real-life phenomena. Sometimes it is enough to adopt a countably infinite probability space, e.g. for the experiment of tossing a coin until first head. More often, however, we need to model continuous quantities like velocity, stock prices, distance, etc, for which a larger probability space is required. A general probability space is a triple (Ω, F, P, where Ω is a sample space of all possible outcomes, P is a probability measure, and F is a collection of events A Ω for which probability P(A is defined. The simple rule of addition of probabilities needs to be replaced by a stronger countableadditivity rule ( P P(A j = P(A j, A j F. required to hold for pairwise disjoint events A 1, A 2,... (with A j F. Example 1.5. For infinite series of tosses of a coin, a natural sample space is Ω = {ω 1 ω 2... : ω j {H, T }}. An elementary outcome is an infinite sequence of heads and tails which might start like T HHHT T H.... Assuming that coin has probability p of a head we can calculate probabilities of events like A = {ω = ω 1 ω 2... : ω 1 = T, ω 2 = H, ω 3 = H}, (these are elements ω 1 ω 2... of ω with fixed few initial coordinates. Probabilities of more complex events are determined from these using the laws of probability (addition rule and the complement rule. Example 1.6. Experiment random point in a circle is described by a circle of radius 1, taken for Ω, the probability measure P(A = area of A, π and F the system of sets, to which area can be assigned. We need a more complex probability space to model stock market data in time. 3

4 1.3 Random Variables A random variable is a numerical quantity X associated with a random experiment. Formally, X is a R-valued function defined on the sample space. (Sometimes it is useful to also permit values ±. A random variable is characterized by a probability distribution which is described differently for discrete and continuous random variables. We speak of a discrete random variable if X has a finite (or countably infinite set of possible values {x i }. The probability distribution in this case is specified by probabilities of individual values, often denoted p i = P(X = x i (another possible notation: p X (i. Under P(X = x i we mean probability of the event that X assumes the value x i, so we may also write p i = P({ω Ω : X(ω = x i }, but the argument ω in X(ω is usually omitted. Sometimes the collection of probabilities p i is called probability function or probability mass function. For named discrete distributions the set {x i } is some collection of integer numbers. Example 1.7. Let X be the number of heads in a series of n coin tosses, with probability p for a head in each toss. Possible values of X are i = 0, 1,..., n. For instance, for n = 3, X(ω = 2 if ω {HHT, HT H, T HH}, so P(X = 2 = 3p 2 q. The general formula is ( n p i = P(X = i = p i q n i for i = 0,..., n. i Due to its intrinsic connection to the binomial formula, this probability mass function is called the binomial distribution, denoted Binomial(n, p. In the case n = 1 this is Bernoulli(p (or distribution, with P(X = 1 = p and P(X = 0 = 1 p. Example 1.8. Let X be the number of tails before the first head. Possible values of X are i = 0, 1,..., and p i = P(X = i = pq i. This is known as the geometric distribution (denoted Geometric(p, due to the connection with infinite geometric series q i = 1 1 q. i=0 Note: sometimes the range of geometric distribution is taken i = 1, 2,..., interpreted as possible number of trials to get the first head (including the trial with head. Example 1.9. Poisson distribution (denoted Poisson(λ with parameter λ > 0 is defined by the formula p i = e λ λi, i = 0, 1,... i! 4

5 This distribution appears as approximation to the binomial distribution Binomial(n, p for n large and np λ. For discrete random variable X with P(X = x i = p i, its expectation (aka expected value is given by EX = x i p i = x i P(X = x i. i i If this is an infinite series, the expectation is defined if i x i p i < (otherwise undefined. There is an equivalent formula in terms of the probabilities of elementary outcomes EX = ω Ω X(ωP(ω. Example Suppose that a certain company makes 1,000,000 with probability 1/4; loses 500,000 with probability 1/4; and makes 2,000,000 with probability 1/4. If X the profit of the company, then X is a random variable with x 1 = p 1 = 1/4 x 2 = p 2 = 1/4 x 3 = p 3 = 1/2 In particular, the expected profit of the company in pounds is given by EX = x 1 p 1 + x 2 p 2 + x 3 p 3 = Using notation for distributed as, we have EX = np for X Binomial(n, p, EX = λ for X Poisson(λ, EX = (1 p/p for X Geometric(p. We speak of a continuous random variable when X takes values in R, R + (set of nonnegative real numbers, or in some interval and the probability of every individual value is zero. In this course, continuous rv s will be only considered with a probability density function (p.d.f. or just density f X, so that P(X [a, b] = b a f X (xdx. The density is nonnegative, f X (x 0 and has the total integral equal 1 f X (xdx = 1. 5

6 The function F X (x = P(X x = x f X (ydy is called (cumulative distribution function, which is also related to the density via the standard formulas of calculus d dx F X(x = f X (x, For continuous rv the expected value is b a f X (xdx = F X (b F X (a. EX = (it is well-defined if x f X(xdx <. xf X (xdx Example For A > 0 a random variable X is said to have a Cauchy distribution if its density is A f X (x = π(a 2 + x 2. It turns out that the expectation for Cauchy distributed rv does not exist, because for x it holds that x f X (x c/ x (c a constant, hence E X = x f X (x dx = 2 An important property of the expectation is linearity: 0 A x π(a 2 + x 2 dx =, Proposition If X 1,..., X m are random variables and α 1,..., α m are constants, then ( m m E α j X j = α j EX j. Definition The variance of a random variable X is given by or by the equivalent alternative formula: Var(X = E(X EX 2, Var(X = EX 2 (EX 2. The variance is well defined if EX 2 < (we sometimes say that the second moment EX 2 exists. The standard deviation is given by σ X = Var(X. 6

7 Example We say X is uniform on [0, 1], written as X Uniform(0, 1, if { 1 if 0 x 1 f X (x = 0 otherwise The moments of X are E[X k ] = x k f X (x dx = 1 From this, Var(X = 1/3 (1/2 2 = 1/12. 0 x k dx = [ ] x=1 1 k + 1 xk+1 = 1 x=0 k + 1. Unlike the expectation, the variance is not linear, as the following result shows. Lemma If X is a random variable and a and b are constants, then Var(aX + b = a 2 Var(X. Example A random variable X is said to be normal (aka normally distributed, or Gaussian if X has pdf f X (x = 1 (x µ2 exp (. 2πσ 2σ 2 The parameters µ R and σ 2 > 0 are the mean (expectation and the variance of X, respectively. We write X N(µ, σ 2. If µ = 0 and σ = 1, we say that X is standard normal, in which case the density is denoted as φ(x = 1 2π exp ( x2 2 The distribution function of a standard normal random variable is denoted Φ(x = x φ(y dy. Although there is no elementary formula for Φ, the values can be found in statistical tables of via standard software. Due to the symmetry of the bell-shaped curve about zero ( φ(x = φ( x we have Moreover, for a b Φ(x = P(X x = P(X x = 1 P(X x = 1 Φ( x.. P(a X b = P(X b P(X a = Φ(b Φ(a. Every normal rv can be transformed to a standard normal rv as follows: Lemma If X N(µ, σ 2 then X µ σ N(0, 1. 7

8 It is often needed to derive a density of rv variable Y = g(x from the density of X, where g is some function. Theorem 1.18 (Transformation Formula. Let X be a continuous rv and let Y = g(x, where g is a differentiable function which is (i either strictly monotonically increasing (so g (x > 0 x R (ii or strictly monotonically decreasing (so g (x < 0 x R. Then { fx (g 1 (y d dy f Y (y = g 1 (y for all y for which g 1 (y exists 0 for all other y Lognormal distribution introduced in the next example is particularly important in finance. Example A rv Y is said to be lognormal with parameters µ and σ 2 where µ R and σ 2 > 0, if log Y N(µ, σ 2. We write Y LogNormal(µ, σ 2. Equivalently, Y satisfies Y = exp(x, where X N(µ, σ 2. The expectation and the variance are EY = exp(µ σ2 and Var(Y = exp(2µ + σ 2 (e σ2 1. We will now use the Transformation Formula to determine the density of lognormal distribution: 1 (log y µ2 f Y (y = exp (, y > 0 2πσy 2σ 2 (f Y (y = 0 for y < 0 Proof. Write g(x = e x. Then Y = g(x. Now, if y = e x, then x = log y, so and Moreover, since X N(µ, σ 2, g 1 (y = log y for y > 0, d dy g 1 (y = 1 y f X (x = 1 2πσ exp for y > 0. ( (x µ2. 2σ 2 Using the Transformation Formula we see that if y > 0 then f Y (y = f X (log y 1 y = 1 (log y µ2 exp (, 2πσy 2σ 2 while f Y (y = 0 if y 0. 8

9 We can make calculation with the lognormal distribution function using the tables for the normal Φ. Example Suppose Y LogNormal(µ, σ 2 with µ = 0.20 and σ = Determine y such that P(Y y = Solution. Note that P(Y y = P(log Y log y where log Y N(µ, σ 2. Thus ( log Y µ 0.95 = P(Y y = P(log Y log y = P log y µ ( log y µ = Φ. σ σ σ From the table for Φ we find so log y µ σ = 1.645, y = exp(µ σ = Independence, covariance and correlation We start with a general definition/theorem. Definition Two random variables X and Y are said to be independent if the events {X x} and {Y y} are independent for all x, y, that is P(X x, Y y = P(X xp(y y. Equivalently, if for all functions g 1, g 2 (provided the expectations are defined. E[g 1 (Xg 2 (Y ] = E[g 1 (X]E[g 2 (X] Furthermore, random variables X 1, X 2,..., X k are said to be independent if the events {X 1 x 1 },..., {X k x k } are independent for all x 1,..., x k ; equivalently E[g 1 (X 1 g k (X k ] = E[g 1 (X 1 ] E[g k (X k ] for any functions g 1,..., g k. In more practical terms, the independence for discrete X, Y means that for all possible values x i, y j of X, Y and for continuous X, Y that for all x, y P(X = x i, Y = y j = P(X = x i P(Y = y j, f X,Y (x, y = f X (xf Y (y 9

10 where the joint density f X,Y is defined by P(X [a, b], Y [c, d] = b d a c f X,Y (x, ydxdy. Likewise for independent X 1,..., X k, the joint probability mass function (respectively, joint density function should factorise in marginal probability mass functions (respectively, density functions in the discrete (respectively, continuous case. The following addition rules are useful: Lemma Let X and Y be independent rv s 1. If X Binomial(n, p, Y Binomial(m, p then X + Y Binomial(n + m, p, 2. If X Poisson(λ, Y Poisson(µ then X + Y Poisson(λ + µ, 3. If X N(µ 1, σ 2 1, Y N(µ 2, σ 2 2 then X + Y N(µ 1 + µ 2, σ σ 2 2. A standard way to check the addition rules is to appeal to the characteristic function ϕ X (t = Ee itx (where i = 1. The product formula ϕ X+Y (t = ϕ X (tϕ Y (t holds if and only if X and Y are independent. Recall that the covariance of two rv s X and Y is defined by cov(x, Y = E[(X EX(Y EY ]. Note that cov(x, Y = cov(y, X and that cov(x, X = Var(X. The following is a useful reformulation, cov(x, Y = E[XY ] EXEY. Covariance turns out to be linear in each of its arguments. We shall state a special case first. Lemma Let X 1, X 2 and Y be three rv s. Then cov(x 1 + X 2, Y = cov(x 1, Y + cov(x 2, Y. Repeated application of the previous lemma, together with the fact that cov(x, Y = cov(y, X yields the following result. Proposition Let X i, i = 1, 2,..., m and Y j, j = 1, 2,... n be two sequences of random variables. Then ( m n m n cov X i, Y j = cov(x i, Y j. i=1 i=1 10

11 The correlation coefficient for two rv s X and Y is given by It is always the case that ρ X,Y = cov(x, Y σ(xσ(y. 1 ρ X,Y 1. (provided the variances, hence the correlation coefficient are defined. We now recall that two random variables X and Y are said to uncorrelated if cov(x, Y = 0. Lemma If two rv s X and Y are independent, then they are uncorrelated. Proof. By independence E[XY ] = EXEY. Note: the converse of the lemma is false, that is, X and Y uncorrelated does not imply that X and Y are independent. Proposition For rv s X 1, X 2,..., X n we have ( n n n Var X i = cov(x i, X j. i=1 i=1 If the random variables X 1, X 2,..., X n are pairwise uncorrelated (that is X i and X j are uncorrelated whenever i j, then ( n n Var X i = Var(X i. i=1 Proposition For pairwise independent rv s X 1, X 2,..., X n (i.e. such that X i and X j are independent for i j i=1 ( m Var X j = m Var(X j. 1.5 The Law of Large Numbers and the Central Limit Theorem Theorem Suppose X 1, X 2,... are independent identically distributed rv s with EX i = µ. Let S n = n X j. Then with probability one S n lim n n = µ. The normal distribution appears in the applications due to the Central Limit Theorem (CLT, one of the main results of Probability Theory. CLT quantifies how fast S n /n converges to µ. 11

12 Theorem 1.29 (Central Limit Theorem. Let X 1, X 2,... be independent identically distributed rv s with mean EX i = µ and variance Var(X i = σ 2, where µ R and σ 2 > 0. Define S n = n i=1 X i. Then lim P n ( Sn nµ nσ x = Φ(x ( x R. Example Historically, the CLT was first shown for Bernoulli trials, as approximation the binomial distribution. In the n-times coin-tossing experiment set X j = 1(ω j = H, where ω 1... ω n {H, T } (1( is 1 when true and is 0 otherwise. Then S n = X X n is the number of heads in n tosses, and S n Binomial(n, p, The CLT in this case says that (S n np/ npq is approximately normally distributed for n large. The LLN only says that S n /n is approximately p. Example The exact distribution of S n is typically complicated. However, suppose X 1, X 2,..., X n are independent N(µ, σ 2 -distributed rv s. Then, by the addition theorem for the normal distribution S n N(nµ, nσ 2, and so in this case S n nµ nσ N(0, 1. Example Let X j be Exponential(λ, i.e. with density f Xj (x = λe λx for x > 0. The moments are EX j = λ, Var(X j = 1/λ 2. The sum S n has Gamma distribution with density λ k f Sn (x = (n 1! xn 1 e λx, x > 0. The CLT in this case says that the distribution of (λs n n/ n is approximately N(0, 1. Gambler s ruin problem Simple random walk provides a model of the wealth by playing head-or-tail game with unit bets. Let X 1, X 2, be iid (independent, identically distributed rv s with P(X j = 1 = P(X j = 1 = 1/2. With some (integer initial capital S 0, the random walk S n = S 0 + X X n models the fortune of a gambler in n rounds (with S n S 0 being the net winnings. If two players start with initial capital of A and B pounds, respectively, each betting a pound, what is the probability that either of them get broken (so the other wins? In terms of the random walk the question is: what the probability that the random walk reaches level A before level B? Consider the random time τ = min{n 0 : S n = A or S n = B}, 12

13 and define S τ = S n 1(τ = n n=0 to be the value of random walk at time τ. Clearly, S τ is either A or B, and we want to determine the conditional probability Consider a more general quantity P(S τ = A S 0 = 0. π(k = P(S τ = A S 0 = k, B k A, the probability to reach A before B when starting at S 0 = k. Looking at what happens at the first step, we get a recursion π(k = 1 2 π(k π(k + 1, 2 B < k < A with the boundary conditions π(a = 1, π( B = 0. From this equation, setting π( B + 1 = x, we get π( B + 2 = 2x, π( B + 3 = 3x,..., π( B + B + A = (B + Ax. From the boundary condition, x = 1/(A + B. Finally, π(0 = π( B + B = B A + B is the probability to reach A before B, equal to the probability to get ruined when gambling with A pounds against a player who started with B pounds. 13