Your pure maths needs to be far stronger for S4 than in any other Statistics module. You must be strong on general binomial expansion from C4.

Transcription

1 OCR Statistics Module Revision Sheet The S exam is hour 0 minutes long. You are allowed a graphics calculator. Before you go into the exam make sureyou are fully aware of the contents of theformula booklet you receive. Also be sure not to panic; it is not uncommon to get stuck on a question (I ve been there!). Just continue with what you can do and return at the end to the question(s) you have found hard. If you have time check all your work, especially the first question you attempted...always an area prone to error. Preliminaries J.M.S. Your pure maths needs to be far stronger for S than in any other Statistics module. You must be strong on general binomial expansion from C. (+x) n = +nx+ n(n ) x + n(n )(n )! x + n(n )(n )(n ) x + This is valid only for x <. This is important for probability/moment generating functions. In particular you must be good at plucking out specific coefficients (which may represent probabilities). For example find the x coefficient in x(+x ) +x. x(+x ) +x = (x+x )(+x) = (x+x ) ((+ x ) ) = ( (x+x ) = (x+x ) ) + x ( x x , x So the x coefficient will be 9 6 ( 6, + 9 ) = 0 thinking ahead about what coefficients you are going to need. 5, ). It helps hugely to be Recall from S that E(g(X)) = g(x i )p i for discrete random variables and E(g(X)) = g(x)f(x)dx for continuous random variables. Recall also that Var(X) E(X ) (E(X)). Probability There are three very useful ways of representing information in probability questions. Venn diagrams, tree diagrams and two-way tables. You must think hard about which approach is going to be most helpful in the question you are to answer. Read the whole question before you start! Set theory is very important in probability. Know the following J.M.Stone

2 A B is the intersection of the sets A and B. The overlap between the two sets. AND A B is the union of the sets A and B. Anything that lies in either A or B (or both). OR A means not A. Everything outside A. {} (or ) denotes the empty set. For example A A = {} Events A and B are mutually exclusive if both A and B cannot both happen. Represented by a Venn diagram of non-overlapping circles. Here P(A B) = P(A)+P(B). However in the general case where A and B are not mutually exclusive we have P(A B) = P(A)+P(B) P(A B). This is because we are overcounting the overlap. It is called the addition law. For three events the addition law becomes A, B and C we have (in general) P(A B C) = P(A)+P(B)+P(C) P(A B) P(A C) P(B C)+P(A B C). Again this drops out easily from a Venn diagram. Events A,A,... are said to be exhaustive if P(A A...) =. In other words the events A,A,... contain all the possibilities. If A and B are independent events then P(A B) = P(A) P(B). We read P(A B) as the probability of A given that B has occured. It is defined P(A B) = P(A B). P(B) However this formula is not always easy to apply, so Mr Stone s patented collapsing universes approach from a Venn or tree diagram is often more intuitive. Using P(A B) = P(A B) P(B) and P(B A) = P(A B) P(A) we discover P(A B) = P(A)P(B A) = P(B)P(A B). This is called the multiplication law of probability and is incredibly useful in converting P(A B) into P(B A) and vice versa. The multiplication law drops out readily from a tree diagram. Bayes Theorem states P(A j B) = P(A j)p(b A j ) P(B) = P(A j )P(B A j ) all i P(A i)p(b A i ). This looks scary, but drops out from a tree diagram. The formal statement is not required for S, but is very important. Reverend Thomas Bayes from my home town of Tunbridge Wells. Wrote a document defending Newton s calculus hence a rather good bloke. J.M.Stone

3 Non-Parametric Tests AllofthehypothesistestsstudiedinStats&requiredknowledge(orattheveryleast an assumption) of some kind of underlying distribution for you to carry out the test. However sometimes you have no knowledge about the underlying population. Statisticians therefore developed a series of non-parametric tests for situations where you have no knowledge of the underlying population. The sign test is a test about the median (i.e. the point at which you have an equal number of data points either side). If H 0 : median = 0, say, then under H 0, whether a random piece of data lies above or below 0 has probability. For n pieces of data we therefore have a binomial B(n, ). Rather than work out critical values, the best approach is probably to calculate (under H 0 ) the probability of what you have observed and anything more extreme. For example test at the 5% level whether the median of the data,,,, 6, 7,, 9, 9, 9, 0, 0,, is 5. Note that there are four pieces of data less than 5. H 0 : The median of the data is 5. H : The median of the data is not 5. α = 5%. Two tailed test. Under H 0, X B(, ). P(X ) = 0.09 > 0.05, so at the 5% level there is insufficient evidence to reject H 0 and we conclude that the median of the data is probably 5. [You could have also gone through the rigmarole of demonstrating that the critical value is (or ) but my way is quicker and life s short.] Although there is no example in your textbook I see no reason why they couldn t ask a question where you had a large enough sample to require the normal approximation to B(n, )...don t forget your continuity correction. The sign test is a very crude test because it takes absolutely no account of how far away the data lies on either side of the median. If you want to take account of the magnitude of the deviations you need to use......the Wilcoxon signed-rank test. Here it is assumed that the data is symmetric; therefore it is a test about both the median or the mean because for symmetric data the median and mean are the same. You calculate the deviations from the median/mean, rank the size of the deviations and then sum the positive ranks to get P and sum the negative ranks to get Q. The test statistic is T, where T is the smaller of P or Q. For example test at the 5% level whether the mean of.,., 7.,.9,.,.6, 5.6, 5.7 is. The data sort of looks symmetric, so OK to proceed with Wilcoxon. H 0 : The mean of the data is. H : The mean of the data is greater than. α = 5%. One tailed test. J.M.Stone

4 Data Deviation Rank Signed Rank So P = 7, Q = 9, so T obs = 9. The lower T is, the worse it is for H 0 and the tables give the largest value at which you would reject H 0. T crit = 5. 9 > 5, so at the 5% level we have insufficient evidence to reject H 0 and conclude that the mean is probably. For large samples (i.e. when the tables don t give the values you want; running out of values) a normal approximation can be used where Z = T +0.5 n(n+). n(n+)(n+) Note that because T is the smaller of P and Q that Z will always be negative (both Z crit and Z obs ). For example if you had 00 pieces of data and you were testing at the % level whether the mean was some value (against H of the mean not being some value) and P = 000 and Q = 050 then T = 000. So Z obs = T obs +0.5 n(n+) n(n+)(n+) = =.0 Because it is a two-tailed % test we reverse look-up to obtain Z crit =.576. Finally.0 >.576, so at the % level there is insufficient evidence to reject H 0 and conclude that the mean is probably whatever we thought it was under H 0. The Wilcoxon rank-sum test is the non-parametric equivalent of the two-sample t-test from S. It tests whether two different sets of data are drawn from identical populations. The central idea for the theory is that if X and Y are drawn from identical distributions, then P(X < Y) =. The tables are then constructed from tedious consideration of all the possible arrangements of the ranks (called the sampling distribution ). Given two sets of data, let m be the number of pieces of data from the smaller data set and n be the number of pieces of data from the larger data set (if they are both the same size it s up to you which is m and which n). Then rank all the data and sum the ranks of the m population; call this total R m. Also calculate m(n+m+) R m and let the test statistic W be the smaller of R m and m(n+m+) R m. The smaller W is, the more likely we are to reject H 0 and the tables give the largest W at which we reject H 0. For example test at the 5% level whether the following are drawn from identical populations. A 0 0 B H 0 : Data drawn from identical distributions. H : Data not drawn from identical distributions. α = 5%. Two tailed test. J.M.Stone

5 Data Rank So m =, n = 6, R m =, m(n+m+) R m = 6, W obs =. Looking at the tables we see W crit =, and >, so at the 5% level there is insufficient evidence to reject H 0 and we conclude that the data is probably drawn from identical distributions. For large samples (i.e. when the tables don t give the values you want; running out of values) a normal approximation can be used where Probability Generating Functions Z = W +0.5 m(m+n+). mn(m+n+) In Stats & you met discrete random variables (DRVs) such that each outcome had a probability attached. Sometimes there were rules which related the probability to the outcome (binomial, geometric, Poisson). However, in general we had: x x x x x... P(X = x) p p p p... Recall that p i = becausethesumof all theprobabilities must total and that E(X) = pi x i. Also E(f(X)) = p i f(x i ) from Stats and Var(X) = E(X ) (E(X)) = pi x i ( p i x i ) from Stats. At some point some bright spark decided to consider the properties of G X (t) = E(t X ) = p i t x i = p t x +p t x +p t x +p t x + where t is a dummy variable unrelated to x. You can see that this will create either a finite or infinite series. This is called the probability generating function of X. It is a single function that contains within it all of the (potentially infinite) probabilities of X. For example given x 0 P(X = x) 6 the generating function is G X (t) = p t x + p t x + p t x + p t x + = 6 t + t + + t + t. We can therefore see that if (say) we saw a term 5 t6, then we can see that P(X = 6) = 5. Note that if you see a constant term then that tells you P(X = 0) because t 0 =. An important property is that G X () = because G X () is just the sum of all the probabilities of X, i.e. p i. Another useful thing to do is consider the derivative G X (t) with respect to t; G X(t) = p i x i t x i = p x t x +p x t x +p x t x + Again, if we consider G () we obtain G () = p i x i = p x +p x +p x +p x + = E(X). 5 J.M.Stone

6 Variances can also be calculated by Var(X) = G X()+G X() (G X()). Some standard pgfs are given in the formula book: Distribution B(n, p) P o(λ) Geo(p) pgf ( p+pt) n eλ(t ) pt Any good candidate should be able to derive these... ( p)t For two independent random variables X and Y (with pgfs G X (t) and G Y (t) respectively) the pgf of X+Y is G X+Y (t) = G X (t) G Y (t). This extends to three or more independent random variables. Moment Generating Functions You will recall from FP that the Maclaurin expansion for e x is e x = +x+ x! + x! + x! + x5 5! +. This is valid for all values of x (and you should know why from your Pure teachings). An alternative notation used is e x exp(x). The nth moment of a distribution is E(X n ). So the first moment is just E(X). The second moment is E(X ), which is useful in calculating variances. The zeroth moment is E(X 0 ) = E() =. The moment generating function(mgf) is defined for by x x x x x... P(X = x) p p p p... M X (t) = E(e tx ) = p i e x it = p e x t +p e x t +p e x t +p e x t + = p +p x t+p x t! +p +p x t+p x t! +p +p x t+p x t! +p x t! +p x t! +p x t! x +p +p x t+p t x +p t +!! = (p +p +p +p + ) +(p x +p x +p x +p x + ) t + ( p x +p x +p x +p x + ) t! + ( p x +p x +p x +p x + ) t + = E()+E(X)t+E ( X ) t!! +E( X ) t! +E( X ) t! J.M.Stone

7 So you can see that the constant term of M X (t) should always be E() = because it represents thesum of theprobabilities. Thecoefficient of twill bee(x) andthe coefficient of t! (not just the coefficient of t ) will be E(X ). In general the coefficient of tn n! will be E(X n ), that is, the nth moment. As with pgfs, differentiating mgfs (with respect to t) is a good thing. However, instead of letting t = we let t = 0 (because a 0 = ). So differentiating M X (t) we find: So M X (0) = E(X). Differentiating again we find: M X (t) = p e x t +p e x t +p e x t +p e x t + M X (t) = p x e x t +p x e x t +p x e x t +p x e x t + M X(0) = p x +p x +p x +p x + = x i p i = E(X). M X (t) = p x e x t +p x e x t +p x e x t +p x e x t + M X(t) = p x e x t +p x e x t +p x e x t +p x e x t + M X (0) = p x +p x +p x +p x + = x ip i = E(X ). So using Var(X) = E(X ) (E(X)) we find Var(X) = M X (0) (M X (0)). Notice that with mgfs there are two ways to obtain the expectation and variance of your random variable. All things being equal I would choose the differentiation method, but you must ensure that your mgf is defined for t = 0. Also read the question carefully to see what they are wanting. Moment generating functions can also be defined for continuous random variables: M X (t) = f(x)e tx dx. As before M X (0) =, M X (0) = E(X), M X (0) = E(X ). Convergence issues can arise EXAMPLE!!!!!! Some standard mgfs are given in the formula book: Distribution Uniform on [a,b] Exponential N(µ,σ ) mgf e bt e at (b a)t Any good candidate should be able to derive these too... λ λ t e µt+ σ t As with pgfs, for two independent random variables X and Y (with mgfs G X (t) and G Y (t) respectively) the mgf of X +Y is M X+Y (t) = M X (t) M Y (t). This extends to three or more independent random variables. Estimators It is vital to recall here that E(X) = µ and Var(X) = σ (by definition). 7 J.M.Stone

8 Given a population there may be many parameters that we may wish to know. For example we might like to know the mean µ, the variance σ, the median M, the maximum or minimum, the IQR, etc. In general we shall call this parameter θ. Usually we will never know θ because we won t have the whole population. But we will be able to take a random sample from the population. From this sample we can calculate a quantity U which we shall use to estimate θ. We call U an estimator of θ. U is said to be an unbiased estimator of θ if E(U) = θ. i.e. if we take an average of all possible U (remember that U is a random variable) we will get the desired θ. If E(U) θ then the estimator is said to be biased (not giving the desired result on average). For example to show that K = X +X +5X is an unbiased estimator of µ we merely consider E(K) and keep whittling down as far as we can go (using S expectation and variance algebra) ( ) X +X +5X E(K) = E ( X = E + X + 5X ) = E(X )+ E(X )+ 5 E(X ) = µ+ µ+ 5 µ = µ. For continuous random variables just remember that E(X) = xf(x)dx. For example find the value of k which makes L = k(x +X ) an unbiased estimator of θ for { ( ) f(x) = θ x θ 0 x θ 0 otherwise First calculate E(X) à la S: E(X) = xf(x)dx = θ 0 x θ So for L to be unbiased we need E(L) = θ, so ( x ) [ ] x θ dx = θ θ x θ = θ 0. E(L) = θ E(k(X +X ) = θ k(e(x )+E(X )) = θ k( E(X)) = θ ( ) θ k = θ k =. Given two unbiased estimators the most efficient estimator (of the two) is the one where Var(U) is smaller. A smaller variance is a good thing. J.M.Stone

9 Sometimes you may need calculus to work out the most efficient estimator from an infinite family. For example X, X and X are three independent measurements of X. S = ax +X +X a+6 (with a 6) is suggested as an estimator for µ. Prove that S is unbiased whatever the value of a and find the value of a which makes S most efficient. So ( ) ax +X +X E(S) = E a+6 = a+6 E(aX +X +X ) = a+6 [ae(x )+E(X )+E(X )] = a+6 [aµ+µ+µ] = µ (a+6) = µ. a+6 So S is unbiased for all values of a. Now consider ( ) ax +X +X Var(S) = Var a+6 = (a+6) Var(aX +X +X ) = (a+6) [a Var(X )+Var(X )+6Var(X )] = a +0 (a+6) σ. To minimise Var(S) we need d davar(s) = 0. So 0 = d ( a ) +0 da (a+6) σ = a(a+6) (a+6)(a +0) (a+6) σ So a = 6 or a = 0 So 0 = a(a+6) (a+6)(a +0) 0 = (a+6)[a(a+6) (a +0)] 0 = (a+6)(6a 0)., but a 6 so a = 0 is the value of a that makes S most efficient. Here is a tough type of problem that caught me out the first two (or three (or four (...))) times I saw it. Slot away the method just in case. For example consider f(x) = { x θ 0 x θ 0 otherwise An estimate of θ is required and a suggestion is made to calculate 5L where L is the maximum of two independent observations of X (X and X ). Show that this estimator is unbiased. The thing to remember is that for L to be the maximum of X and X, then X and X must both be less than or equal to L; i.e. we are going to calculate a cdf. So P(L l) = P(X l) P(X l). I suppose we should consider the second derivative to show that this value of a minimises rather than maximises the variance, but life s too short J.M.Stone

10 (This can be extended to three or more independent samplings of X.) By sketching f(x) we can see that the probability that one observation is less than or equal to l is given by a triangle in this case of area l θ (or by the integral l 0 f(x)dx for a more general f(x)). So P(L l) = P(X l) P(X l) = l l = l. Differentiating θ θ θ wrt to l we find the pdf of l to be f(l) = { l θ 0 l θ 0 otherwise Therefore we calculate E ( ) 5L as follows: ( ) 5L E = 5 E(L) θ = 5 l l 0 θ dl = 5 [ ] l 5 θ 5θ = θ. Therefore 5L is an unbiased estimator of θ. I will leave it as an exercise for the reader to demonstrate that Var ( ) 5L = θ. Discrete Bivariate Distributions The discrete random variables you have met thus far have been in one variable only. For example x 5 7 P(X = x) However we can have discrete bivariate distributions. For example 9 0 X 5 0 Y From this we can see, say, P(X =,Y = 5) = 0. The marginal distribution is what one obtains if one of the variables is ignored. In the above example the marginal distribution of X can be written 0 0 x 5 P(X = x) 5 This can be added to the bivariate distribution thus: 0 X Y J.M.Stone

11 E(X) and Var(X) can be calculated in the usual way obtaining E(X) = 0 and Var(X) = (do it!). Similarly you can work out the marginal distribution of Y if you are so inclined. The conditional distribution of a bivariate distribution can be calculated given that one of the variables (X or Y) has taken a specific value. For the above example the distribution of X conditional on Y = is calculated by rewriting the row with all the values divided by P(Y = ) = 5. x 5 5 P(X = x Y = ) This is all from our friend P(A B) = P(A B) P(B). A way to check whether X and Y are independent of each other in a bivariate distribution is to check whether every entry in the distribution is the product of the two relevant marginal probabilities. For example Y X Here we see P(X =,Y = ) = 9 is the same as P(X = ) P(Y = ) = = 9. The same is true for every entry in the table, so X and Y are independent. It only takes one entry not to satisfy this to ensure X and Y are not independent. The covariance of a discrete bivariate distribution is defined Cov(X,Y) E((X µ X )(Y µ Y )). However this tends to be cumbersome to calculate so we use the equivalent formula Cov(X,Y) = E(XY) µ X µ Y. The covariance can be thought of as the correlation coefficient (r from Stats ) for two probability distributions (sort of). The covariance can be both positive or negative (like the correlation coefficient). To calculate the covariance, first create the marginal distributions: Y 5 6 X 0 Y 5 X Then use the marginal distributions to calculate µ X and µ Y. µ X = E(X) = xp = + + = 7. µ Y = E(Y) = yp = =. Now we use this to calculate the covariance thus: Cov(X,Y) = E(XY) µ X µ Y = ( )+( 5 6 )+( )+( 5 )+( 0)+( 5 ) 7 = 5. J.M.Stone

12 If X and Y are independent then Cov(X,Y) = 0. However, if Cov(X,Y) = 0 this does not necessarily mean that X and Y are independent. But if Cov(X,Y) 0 then X and Y cannot be independent. With an understanding of covariance we can write the relationship for Var(aX ±by) when X and Y are not independent: Var(aX ±by) = a Var(X)+b Var(Y)±abCov(X,Y). Notice the extra term at the end of the formula we are used to from S for independent X and Y. J.M.Stone