SBAOD Statistical Methods & their Applications - II. Unit : I - V

Transcription

1 SBAOD Statistical Methods & their Applications - II Unit : I - V

2 SBAOD Statistical Methods & their applications -II 2 Unit I - Syllabus Random Variable Mathematical Expectation Moments Moment generating function Chebyshev's Theorem

3 Random Variables Random Variable (RV): A numeric outcome that results from an experiment For each element of an experiment s sample space, the random variable can take on exactly one value Discrete Random Variable: An RV that can take on only a finite or count ably infinite set of outcomes Continuous Random Variable: An RV that can take on any value along a continuum (but may be reported discretely SBAOD Statistical Methods & their Applications - II 3

4 Mathematical Expectation: Let X be a random variable with a probability distribution f(x). The mean (or expected value) of X is denoted by X (or E(X)) and is defined by: E(X) μ X x all x x f ( x) ; f ( x) dx ; if if X is discrete X is continuous Example: A shipment of 8 similar microcomputers to a retail outlet contains 3 that are defective and 5 are non-defective. If a school makes a random purchase of 2 of these computers, find the expected number of defective computers purchased SBAOD Statistical Methods & their Applications - II 4

5 EXAMPLE A random variable X has the probability distribution P x (X = x) p 2 Solution: (a) Since X is a discrete r.v., p 1 2 Find (a) the value of p and (b) the mean of X. (b) mean, Discrete Random Variables p 1 4 P ( X x ) xp ( X x )

6 Solution: Let X = the number of defective computers purchased. In Example 3.3, we found that the probability distribution of X is: x F(x)=p(X=x) or: f ( x) P( X x) 3 5 x 2 x ; 8 2 0; otherwise x 0, 1, 2 SBAOD Statistical Methods & their Applications - II 6

7 The expected value of the number of defective computers purchased is the mean of X, which is: E(X) μ X x0 = (0) f(0) + (1) f(1) +(2) f(2) 2 x f ( x) ( 0) (1) (2) (computers) Example Let X be a continuous random variable that represents the life (in hours) of a certain electronic device. The pdf of X is given by: 20,000 ; x 100 f ( x) 3 x 0 ; elsewhere Find the expected life of this type of devices. SBAOD Statistical Methods & their Applications - II 7

8 Solution: E(X) μ X 100 x f ( x) dx x x dx 1 x 2 1 x dx x x (hours) Therefore, we expect that this type of electronic devices to last, on average, 200 hours. SBAOD Statistical Methods & their Applications - II 8

9 Moments The raw moments (or 'moments about zero') of a distribution are defined as for continuous distributions with PDF f(x) and for discrete distributions with PMF pi. SBAOD Statistical Methods & their Applications - II 9

10 MOMENT GENERATING FUNCTION The m.g.f. of random variable X is defined as M X (t) E(e tx ) all all e x x tx e tx f (x)dx f (x) if if X is cont. X is discrete for t Є (-h,h) for some h>0. SBAOD Statistical Methods & their Applications - II 10

11 Properties of m.g.f. M(0)=E[1]=1 If a r.v. X has m.g.f. M(t), then Y=aX+b has a m.g.f. e bt M(at) m.g.f does not always exists (e.g. Cauchy distribution) E(X k ) M (k) (0) where M (k) is the k th derivative. SBAOD Statistical Methods & their Applications - II 11

12 Chebyshev's Theorem: Suppose that X is any random variable with mean E(X)= and variance 2 Var(X)= and standard deviation. Chebyshev's Theorem gives a conservative estimate of the probability that the random variable X assumes a value within k standard deviations (k) of its mean, which is P( k <X< +k). 1 P( k <X< +k) 1 2 k SBAOD Statistical Methods & their Applications - II 12

13 Example Let X be a random variable having an unknown distribution with mean =8 and variance 2 =9 (standard deviation =3). Find the following probability: P(4 <X< 20) Solution: (a) P(4 <X< 20)= P( k <X< +k) k (4 <X< 20)= ( k <X< +k) 4= k 4= 8 k(3) or 20= + k 20= 8+ k(3) 4= 8 3k 20= 8+ 3k 3k=12 3k=12 k=4 k=4 15 Therefore, P(4 <X< 20), and hence,p(4 <X< 20) 16 (approximately) SBAOD Statistical Methods & their Applications - II 13

14 Unit II - Syllabus Binomial distibution Poisson distribution Normal distribution Distribution Fitting SBAOD Statistical Methods & their Applications - II 14

15 Standard Distributions: The following are the standard distributions. Binomial distribution Poisson distribution Normal distribution SBAOD Statistical Methods & their Applications - II 15

16 The Binomial Distribution Bernoulli Random Variables Imagine a simple trial with only two possible outcomes Success (S) Failure (F) Examples Toss of a coin (heads or tails) Sex of a newborn (male or female) Survival of an organism in a region (live or die) Jacob Bernoulli ( ) SBAOD Statistical Methods & their Applications - II 16

17 The Binomial Distribution However, if order is not important, then P(x) = n! x!(n x)! p x q n x where n! x!(n x)! is the number of ways to obtain x successes in n trial. SBAOD Statistical Methods & their Applications - II 17

18 Example for binomial distribution Toss a die 5 times. X=# of sixes.find P(X=2) S=six N=not a six SSNNN1/6*1/6*5/6*5/6*5/6=(1/6) 2 (5/6) 3 SNSNN1/6*5/6*1/6*5/6*5/6=(1/6) 2 (5/6) 3 SNNSN1/6*5/6*5/6*1/6*5/6=(1/6) 2 (5/6) 3 SNNNS NSSNNetc. NSNSN NSNNs NNSSN NNSNS NNNSS Px ( 2) 10* 6 6 [P(S)] # of S SBAOD Statistical Methods & their Applications - II 10 ways to choose 2 of 5 places for S. 5 5! 5! 5*4*3! !(5 2)! 2!3! 2*1*3! [1-P(S)] 5 - # of S 18

19 The Poisson Distribution When there is a large number of trials, but a small probability of success, binomial calculation becomes impractical Example: Number of deaths from horse kicks in the Army in different years The mean number of successes from n trials is µ = np Example: 64 deaths in 20 years from thousands of soldiers Simeon D. Poisson ( ) SBAOD Statistical Methods & their Applications - II 19

20 The Poisson Distribution Poisson distribution is applied where random events in space or time are expected to occur P(x) = e -µ µ x x! Poisson distribution is applied where random events in space or time are expected to occur Deviation from Poisson distribution may indicate some degree of nonrandomness in the events under study Investigation of cause may be of interest SBAOD Statistical Methods & their Applications - II 20

21 Example for poisson distribution Radioactive decay X=# of particles/hour λ =2 particles/min * 60min/hour=120 particles/hr e Px ( 125), x=0, 1, 2, ! SBAOD Statistical Methods & their Applications - II 21

22 The Normal Distribution Discovered in 1733 by de Moivre as an approximation to the binomial distribution when the number of trails is large Derived in 1809 by Gauss Importance lies in the Central Limit Theorem, which states that the sum of a large number of independent random variables (binomial, Poisson, etc.) will approximate a normal distribution Abraham de Moivre ( ) Example: Human height is determined by a large number of factors, both genetic and environmental, which are additive in their effects. Thus, it follows a normal distribution. Karl F. Gauss ( ) SBAOD Statistical Methods & their Applications - II 22

23 The Normal Distribution A continuous random variable is said to be normally distributed with mean and variance 2 if its probability density function is f (x) = 1 2 (x ) 2 /2 2 e f(x) is not the same as P(x) P(x) would be 0 for every x because the normal distribution is continuous x 2 However, P(x 1 < X x 2 ) = f(x)dx x 1 SBAOD Statistical Methods & their Applications - II 23

24 Example of normal distribution For example: What s the probability of getting a math SAT score of 575 or less, =500 and =50? Z i.e., A score of 575 is 1.5 standard deviations above the mean 575 x500 ( ) Z P( X 575) e dx e dz (50) look up Z= 1.5 in standard normal chart =.9332 SBAOD Statistical Methods & their Applications - II 24

25 Standard Normal Distribution Useful properties of the normal distribution μ=0 and σ 2 =1 1. The normal distribution has useful properties: 5000 E(X+Y)= E(X)+E(Y) and σ 2 (X+Y)= σ 2 (X)+ σ 2 (Y) Can be transformed with shift and change of scale operations SBAOD Statistical Methods & their Applications - II 25

26 Distribution Fitting Distribution fitting is the procedure of selecting a statistical distribution that best fits to a data set generated by some random process. In other words, if you have some random data available, and would like to know what particular distribution can be used to describe your data, then distribution fitting is what you are looking for. SBAOD Statistical Methods & their Applications - II 26

27 Goodness-of-Fit Tests Two values are involved, an observed value, which is the frequency of a category from a sample, and the expected frequency, which is calculated based upon the claimed distribution. The idea is that if the observed frequency is really close to the claimed (expected) frequency, then the square of the deviations will be small. The square of the deviation is divided by the expected frequency to weight frequencies. A difference of 10 may be very significant if 12 was the expected frequency, but a difference of 10 isn't very significant at all if the expected frequency was If the sum of these weighted squared deviations is small, the observed frequencies are close to the expected frequencies and there would be no reason to reject the claim that it came from that distribution. Only when the sum is large there is a reason to question the distribution. Therefore, the chi-square goodness-of-fit test is always a right tail test. SBAOD Statistical Methods & their Applications - II 27

28 Unit III - Syllabus Sampling distributions Steps for hypothesis Procedure t-test F-test Chi-square 2x2 Contingency table SBAOD Statistical Methods & their Applications - II 28

29 Sampling Distribution A theoretical frequency distribution of the scores for or values of a statistic, such as a mean. Any statistic that can be computed for a sample has a sampling distribution. A sampling distribution is the distribution of statistics that would be produced in repeated random sampling (with replacement) from the same population. It is all possible values of a statistic and their probabilities of occurring for a sample of a particular size. Standard error The standard error is the standard deviation of a sampling distribution. SBAOD Statistical Methods & their Applications - II 29

30 Aims of sampling Reduces cost of research (e.g. political polls) Generalize about a larger population In some cases (e.g. industrial production) analysis may be destructive, so sampling is needed SBAOD Statistical Methods & their Applications - II 30

31 Sampling Process The process of statistical inference involves using information from a sample to draw conclusions about a wider population. Different random samples yield different statistics. We need to be able to describe the sampling distribution of possible statistic values in order to perform statistical inference. We can think of a statistic as a random variable because it takes numerical values that describe the outcomes of the random sampling process. Population Sample Collect data from a representative Sample... Make an Inference about the Population. SBAOD Statistical Methods & their Applications - II 31

32 Parameters and Statistics As we begin to use sample data to draw conclusions about a wider population, we must be clear about whether a number describes a sample or a population. Definition: A parameter is a number that describes some characteristic of the population. In statistical practice, the value of a parameter is usually not known because we cannot examine the entire population. A statistic is a number that describes some characteristic of a sample. The value of a statistic can be computed directly from the sample data. We often use a statistic to estimate an unknown parameter. Remember s and p: statistics come from samples and parameters come from populations SBAOD Statistical Methods & their Applications - II 32

33 Steps for Hypothesis Testing Formulate H 0 and H 1 Select Appropriate Test Choose Level of Significance Calculate Test Statistic TS CAL Determine Prob Assoc with Test Stat Compare with Level of Significance, Determine Critical Value of Test Stat TS CR Determine if TS CR falls into (Non) Rejection Region Reject/Do not Reject H 0 Draw Marketing Research Conclusion SBAOD Statistical Methods & their Applications - II 33

34 Hypothesis Testing for Differences Hypothesis Tests Parametric Tests (Metric) Non-parametric Tests (Nonmetric) One Sample * t test * Z test * Two-Group t test * Z test Two or More Samples Independent Samples * Paired t test SBAOD Statistical Methods & their Applications - II 34

35 The t Distribution For example, using the normal curve, 1.96 is the cut-off for a twotailed test at the.05 level of significance. On a t distribution with 3 degrees of freedom (a sample size of 4), the cutoff is 3.18 for a two-tailed test at the.05 level of significance. If your estimate is based on a larger sample of 7, the cutoff is 2.45, a critical score closer to that for the normal curve. SBAOD Statistical Methods & their Applications - II 35

36 SBAOD Statistical Methods & their Applications - II 36

37 Testing Exact Hypotheses about a Variance Test the null that the population variance has some specific value. Pick alpha and rejection region. Then: H : Plug hypothesized population variance and sample variance into equation along with sample size we used to estimate variance. Compare to chi-square distribution. 2 ( N 1) ( N 1) s SBAOD Statistical Methods & their Applications - II 37

38 Hypothesis Test to Compare Two Variances F- test SBAOD Statistical Methods & their Applications - II 38

39 Example of an F-distribution 0.34 F24, 19 1% region (0.5% x2) 2.99 To use just the upper tail value, ensure F-ratio is calculated so it is >1, then use upper tail of the 2½% F-tabled value when testing at 5% significance SBAOD Statistical Methods & their Applications - II 39

40 Definition for a 2X2 contingency table Let X and Y denote two categorical variables Variable X (Explanatory/Independent variable) can have one of two values: X = 1 or X = 2 Variable Y (Response/Dependent variable) can have one of two values: Y = 1 or Y = 2 n ij denotes the count of responses in a cell in a table SBAOD Statistical Methods & their Applications - II 40

41 Example for contingency table H O : Horned lizards eat equal amounts of leaf cutter, carpenter and black ants. H A : Horned lizards eat more amounts of one species of ants than the others. Leaf Cutter Ants Carpenter Ants Black Ants Total Observed Expected O-E (O-E) 2 E χ 2 = 1.90 χ 2 = Sum of all: (O-E) 2 E Calculate degrees of freedom: (c-1)(r-1 = 3-1 = 2 Under a critical value of your choice (e.g. α = 0.05 or 95% confidence), look up Chi-square statistic on a Chi-square distribution table. SBAOD Statistical Methods & their Applications - II 41

42 Unit IV Syllabus: Experimental Design ANOVA CRD RBD LSD Concepts of factorial experiments SBAOD Statistical Methods & their Applications - II 42

43 Experimental Design Experimental Design. :a method of research in the social sciences (such as sociology or psychology) in which a controlled experimental factor is subjected to special treatment for purposes of comparison with a factor kept constant. The basic principles of experimental design are (i) Randomization, (ii) Replication and (iii) Local Control. Randomization: Ensures that each subject in a population or each site used for sampling has an equal opportunity of being selected. Replication: This is necessary to estimate the degree of chance variation among samples. SBAOD Statistical Methods & their Applications - II 43

44 Local control (error control) The replication is used with local control to reduce the experimental error. For example, if the experimental units are divided into different groups such that they are homogeneous within the blocks, than the variation among the blocks is eliminated and ideally the error component will contain the variation due to the treatments only. This will in turn increase the efficiency SBAOD Statistical Methods & their Applications - II 44

45 Experimental Design Sample Size: The larger the sample size, the better. A larger sample size tends to give you a closer estimate of the true population mean. SBAOD Statistical Methods & their Applications - II 45

46 Definition of Anova ANOVA: analysis of variation in an experimental outcome and especially of a statistical variance in order to determine the contributions of given factors or variables to the variance. Remember: RA Fischer, Evolutionary Biology Analysis of Variance is a widely used statistical technique that partitions the total variability in our data into components of variability that are used to test hypotheses. SBAOD Statistical Methods & their Applications - II 46

47 Analysis of Variance Analysis of Variance is a widely used statistical technique that partitions the total variability in our data into components of variability that are used to test hypotheses. In ANOVA, we compare the between-group variation with the within-group variation to assess whether there is a difference in the population means Types 1.One way ANOVA 2.Two way ANOVA SBAOD Statistical Methods & their Applications - II 47

48 One Way Anova The one-way analysis of variance (ANOVA) is used to determine whether the mean of a dependent variable is the same in two or more unrelated, independent groups. The following link explains the one way anova in detail. SBAOD Statistical Methods & their Applications - II 48

49 Two Way ANOVA The two-way ANOVA compares the mean differences between groups that have been split on two independent variables (called factors). The primary purpose of a two-way ANOVA is to understand if there is an interaction between the two independent variables on the dependent variable The following link explains the two way anova in detail. SBAOD Statistical Methods & their Applications - II 49

50 Completely Randomized Design CRD generalizes the two-sample t-test (pooled variances). Experimental units are relatively homogeneous. Experiment will use very few replicates. Treatments are assigned to experimental units at random. Each treatment is replicated the same number of times (balanced design). No accommodation made for disturbing variables (extraneous sources of variation). High probability that a large fraction of the experimental units set out at the beginning of the study may be lost or unavailable for measurement at the appropriate time. SBAOD Statistical Methods & their Applications - II 50

51 Randomized Block Design (RBD) Any experimental design in which the randomization of treatments is restricted to groups of experimental units within a predefined block of units assumed to be internally homogeneous is called a randomized block design. Blocks of units are created to control known sources of variation in expected (mean) response among experimental units. There are two classifications or factors in an RBD: block effects and treatment effects. Rules for blocking: Carefully examine the situation at hand and identify those factors which are known to affect the proposed response. Choose one or two of these factors as the basis for creating blocks. Blocking factors are sometimes referred to as disturbing factors. SBAOD Statistical Methods & their Applications - II 51

52 The ANOVA table for the Completely Randomized Design Source df Sum of Squares Treatments t - 1 SS Tr Error t(n 1) SS Error Total tn - 1 SS Total The ANOVA table for the Randomized Block Design Source df Sum of Squares Blocks n - 1 SS Blocks Treatments t - 1 SS Tr Error (t 1) (n 1) SS Error Total tn - 1 SS Total SBAOD Statistical Methods & their Applications - II 52

53 Latin Square Design If you can block on two (perpendicular) sources of variation (rows x columns) you can reduce experimental error when compared to the RBD More restrictive than the RBD The total number of plots is the square of the number of treatments Each treatment appears once and only once in each row and column A B C D B C D A C D A B D A B C SBAOD Statistical Methods & their Applications - II 53

54 ANOVA for LSD Source df SS MS F Total t 2-1 SSTot = Y Y 2 i j ij Row t-1 SSR= SSR/(t-1) MSR/MSE i 2 i t Y Y Column t-1 SSC= SSC/(t-1) MSC/MSE j 2 j t Y Y Treatment t-1 SST = SST/(t-1) MST/MSE k 2 k t Y Y Error (t-1)(t-2) SSE = SSE/(t-1)(t-2) SSTot-SSR- SSC-SST SBAOD Statistical Methods & their Applications - II 54

55 Factorial Experiment In statistics, a full factorial experiment is an experiment whose design consists of two or more factors, each with discrete possible values or "levels", and whose experimental units take on all possible combinations of these levels across all such factors. A full factorial design may also be called a fully crossed design. Such an experiment allows the investigator to study the effect of each factor on the response variable, as well as the effects of interactions between factors on the response variable. SBAOD Statistical Methods & their Applications - II 55

56 Unit V Syllabus: Non-parametric test Comparison between parametric and Non-parametric test Sign test Run test Wilcoxon signed Rank test Mann Whitney U test SBAOD Statistical Methods & their Applications - II 56

57 Nonparametric tests Parametric tests involve estimating parameters such as the mean, and assume that distribution of sample means are normally distributed Often data does not follow a Normal distribution eg number of cigarettes smoked, cost to NHS etc. Positively skewed distributions SBAOD Statistical Methods & their Applications - II 57

58 Parametric vs Nonparametric Statistics Parametric Statistics are statistical techniques based on assumptions about the population from which the sample data are collected. Assumption that data being analyzed are randomly selected from a normally distributed population. Requires quantitative measurement that yield interval or ratio level data. Nonparametric Statistics are based on fewer assumptions about the population and the parameters. Sometimes called distribution-free statistics. A variety of nonparametric statistics are available for use with nominal or ordinal data. SBAOD Statistical Methods & their Applications - II 58

59 Advantages of Nonparametric Techniques Sometimes there is no parametric alternative to the use of nonparametric statistics. Certain nonparametric test can be used to analyze nominal data. Certain nonparametric test can be used to analyze ordinal data. The computations on nonparametric statistics are usually less complicated than those for parametric statistics, particularly for small samples. Probability statements obtained from most nonparametric tests are exact probabilities. SBAOD Statistical Methods & their Applications - II 59

60 Disadvantages of Nonparametric Statistics Nonparametric tests can be wasteful of data if parametric tests are available for use with the data. Nonparametric tests are usually not as widely available and well know as parametric tests. For large samples, the calculations for many nonparametric statistics can be tedious. SBAOD Statistical Methods & their Applications - II 60

61 The Sign test: A nonparametric test for the central location of a distribution The test statistic: S = the number of observations that exceed µ 0 Comment: If H 0 : median = 0 is true we would expect 50% of the observations to be above 0, and 50% of the observations to be below 0 SBAOD Statistical Methods & their Applications - II 61

62 Runs Test Test for randomness - is the order or sequence of observations in a sample random or not Each sample item possesses one of two possible characteristics Run - a succession of observations which possess the same characteristic Example with two runs: F, F, F, F, F, F, F, F, M, M, M, M, M, M, M Example with fifteen runs: F, M, F, M, F, M, F, M, F, M, F, M, F, M, F SBAOD Statistical Methods & their Applications - II 62

63 Runs Test: Large Sample If either n 1 or n 2 is > 20, the sampling distribution of R is approximately normal. R R 2 n n n n 1 n n n n n n 2 (2 ) ( n1n 2) ( n 1) 1 n2 Z R R R SBAOD Statistical Methods & their Applications - II 63

64 Wilcoxon Matched-Pairs Signed Rank Test A nonparametric alternative to the t test for related samples before and after studies Studies in which measures are taken on the same person or object under different conditions Studies or twins or other relatives SBAOD Statistical Methods & their Applications - II 64

65 Wilcoxon Matched-Pairs Signed Rank Test: Large Sample Formulas nn 1 T T Z where : n n T 4 n 12 n 1 T T 24 = number of pairs T= total rank for either + or differences, whichever is less SBAOD Statistical Methods & their Applications - II 65

66 Mann-Whitney U Test Nonparametric counterpart of the t test for independent samples Does not require normally distributed populations May be applied to ordinal data Assumptions Independent Samples At Least Ordinal Data SBAOD Statistical Methods & their Applications - II 66

67 Mann-Whitney U Test: Sample Size Consideration Size of sample one: n 1 Size of sample two: n 2 If both n 1 and n 2 are 10, the small sample procedure is appropriate. If either n 1 or n 2 is greater than 10, the large sample procedure is appropriate. SBAOD Statistical Methods & their Applications - II 67

68 in group1 values sum or the ranks of number in group 2 number in group1 : W n n W n n n n where U U U U U U Z n n n n n n Mann-Whitney U Test: Formulas for Large Sample Case SBAOD Statistical Methods & their Applications - II 68