Factors affecting the Type II error and Power of a test The factors that affect the Type II error, and hence the power of a hypothesis test are 1. Deviation of truth from postulated value, 6= 0 2 2. Variability in population, 3. Significance level, 4. Sample size, n. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 11
Two questions Given pre-specified risks and, determine 1. how small a? = µ a µ 0 can be detected at a given n? 2. how large n should be so as to detect a given?? Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 12
Two questions Given pre-specified risks and, determine 1. how small a? = µ a µ 0 can be detected at a given n? 2. how large n should be so as to detect a given?? Exact results can be derived for upper- and lower-tailed tests, with an approximation for the two-tailed test. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 13
Upper tailed test: Choosing the sample size Define first, a critical value z, such that (by convention) Pr(Z >z )= OR Pr(Z < z )= (9) where the second definition follows by symmetry. Then, it follows from (8) (for the upper-tailed test) that, z = z =) z + z = (10) Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 14
Upper tailed test: Choosing the sample size Define first, a critical value z, such that (by convention) Pr(Z >z )= OR Pr(Z < z )= (9) where the second definition follows by symmetry. Then, it follows from (8) (for the upper-tailed test) that, z = z =) z + z = (10) For a fixed, n and, Observe that increasing has a lowering effect on and vice versa! Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 15
Choosing the sample size Equation (10) can be now used to answer the two burning questions! For pre-specified and risks and, 1.? = (z + z ) p n 2. n = z + z? / 2 (round up to the next integer) where z,z are the standard normal variates corresponding to and risks respectively. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 16
Choosing the sample size Equation (10) can be now used to answer the two burning questions! For pre-specified and risks and, 1.? = (z + z ) p n 2. n = z + z? / 2 (round up to the next integer) where z,z are the standard normal variates corresponding to and risks respectively. Example: =0.05, =0.1, n =(2.925 /? ) 2. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 17
Useful viewpoint Useful viewpoint: Introduce signal-to-noise ratio SNR =? /. Then, For specified risks and, the number of samples required to detect mean-shifts relative to standard deviation of the noise / 1/SNR 2. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 18
Lower-tailed and Two-tailed tests For the lower-tailed test, we have an exact relation: z = z =) z + z = (11) On the other hand, for the two-tailed test, we have an approximate result =) z /2 + z (12) when F z /2 p n Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 19
Example Propellant burning rate Suppose for the propellant burning rate example, it is required to determine for the (two-tailed) test: 1. the magnitude of deviation it can detect for the given sample size n = 25 2. the sample size for detecting a deviation of =1cm/s with a probability of 0.9. Solution: : Use Equation (12) to solve by hand. Use power.t.test in R with appropriate options Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 20
Closing remarks Topics for further reading: 1. Operating characteristic (OC) curves 2. Non-parametric tests 3. Likelihood ratio tests 4. Hypothesis testing using bootstrapping 5. Multiple hypotheses tests 6.... Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 21
Bibliography I Bendat, J. S. and A. G. Piersol (2010). Random Data: Analysis and Measurement Procedures. 4 th edition. New York, USA: John Wiley & Sons, Inc. Johnson, R. A. (2011). Miller and Freund s: Probability and Statistics for Engineers. Upper Saddle River, NJ, USA: Prentice Hall. Montgomery, D. C. and G. C. Runger (2011). Applied Statistics and Probability for Engineers. 5 th edition. New York, USA: John Wiley & Sons, Inc. Ogunnaike, B. A. (2010). Random Phenomena: Fundamentals of Probability and Statistics for Engineers. Raton, FL, USA: CRC Press, Taylor & Francis Group. Boca Tangirala, A. K. (2014). Principles of System Identification: Theory and Practice. Group. CRC Press, Taylor & Francis Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 22