Introduction to Statistical Hypothesis Testing

Transcription

1 Introduction to Statistical Hypothesis Testing Arun K. Tangirala Statistics for Hypothesis Testing - Part 1 Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 1

2 Learning objectives I Sampling and Estimation I Statistics I Sample mean, variance, proportion and correlation. I Sampling distributions: concepts I Distribution of sample mean Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 2

3 Practical aspects and limitations of rigorous analysis Limitations of dealing with probability distributions I Sufficient process knowledge and the entire outcome space not easily available I Theoretical construction and estimation of f(x) is therefore hampered I Alternatives? I Available: Finite observations of phenomena I Can we say something about the ensemble behaviour? Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 3

4 From probability to statistics Probability Enables analysis of random phenomena and their expected behaviour using f(x). Moving from ensemble space to the finite observation space. I Essentially shifting from outcomes to observations I What kind of analysis is necessary? I Systematic and efficient way of extracting information I Importantly, what type of data and how should it be acquired? I Observations should contain sufficient" information about f(x) Statistical analysis provides answers to both questions Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 4

5 Central concepts 1. Population: Complete collection of all possibilities corresponding to a random experiment and characterized by the p.d.f. f(x ). 2. Samples: Specific set of observations obtained from a single experiment. Also known as a realization. 3. Inference: A conclusion made about the population from the data. I Finite data set only a representative of the population. Hence, every inference has a degree of uncertainty. Example: average solid propellant burning rate, defective parts Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 5

6 The three entities Statistics Aids in analysing random phenomena using finite data sets by either partly or fully reconstructing f(x). 1. Random variable, X: Actual variable(s) of interest 2. Observation set (data), {x i } n i=1: Available entity 3. Ensemble description, f(x): Not available. Has to be inferred or estimated from data. I We shall use the notation f(x ) to indicate its dependence on the parameter vector (e.g., =, h = µ 2i T ). Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 6

7 Important Assumption Finite observation set is representative of the population (informative data) - achieved through a proper design of experiment. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 7

8 Statistical Analysis 1. Analysis of data (a.k.a. Descriptive Statistics) Exploratory Data Analysis i. Graphical: Organizing, presenting and summarizing data, e.g., Boxplots, ii. Numerical: Compute descriptive statistics, e.g., mean, median, range 2. Drawing inferences (a.k.a Inductive Statistics) Inferential Statistics i. Estimation: Determine unknowns (parameters) from data, e.g., mean, variance. ii. Hypothesis testing: Validating a postulation regarding f(x) or the parameters using the data as a source of evidence. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 8

9 Sampling Fundamental to statistical analysis is the know-how of how to sample" and the effects of sampling on the estimates (not to be confused with sampling in signal processing). Sampling - a study of variability inherent in samples from population and sampling distribution - a study of the distributions of key statistics (mean, variance) Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 9

10 Three central concepts 1. Random sample: x = {x 1,x 2,,x n }. Essentially a single data record. I All subsets should have equal chances of being selected. No particular preference towards a section of the population! 2. Statistic: Function of the observation set, g(x), constructed for the purpose of estimating parameters. Sometimes denoted as ˆ (x) (estimator function). 3. Distribution of the statistic: How the statistic g(x) varies across samples or data records. Also known as the sampling distribution. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 10

11 Random Sample Aim: Obtain a sample (subset) of n (n finite) observations from a population of size N (N could be infinite) Primary consideration: All subsets should have equal chances of being selected. No particular preference towards a section of the population! Definition Let X 1,X 2,,X n constitute n mutually, stochastically independent random variables, each of which stems from the same but possibly unknown pdf f(x). Then, this set of variables is a random sample. The joint pdf of the random sample above is then f(x 1,x 2,,x n )= ny f(x i ) i=1 Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 11

12 Statistic The purpose of collecting data is to make inferences by a suitable processing of the data. For this purpose, introduce a processor" function known as the statistic. Statistic Any function of the random sample with a p.d.f. f(x; ) ˆ = g(x 1,X 2,,X n ) and which does not require the knowledge of the unknown parameters of f(x) is said to be a statistic. Examples: (i) X = 1 nx X i (ii) Z =( X µ)/ (only if µ and are known). n i=1 Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 12

13 Points to note I Knowing g(.), one can construct the p.d.f. of ˆ, which will then tell us the probabilistic characteristics of the statistic I Although ˆ does not depend on the unknown parameters, its distribution does. I Distributions of statistics are in general known as sampling distributions. I Very useful for estimation and hypothesis testing. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 13

14 Statistics of interest Three popular statistics in univariate analysis: 1. Sample mean: To provide an estimate of the true average µ. 2. Sample variance: As an estimate of the true variability of X, Sample proportion: As an estimation of the population proportion. and in multivariate analysis: 1. Sample correlation: Provides and estimate of the true correlation between two random variables X and Y (or the respective populations). Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 14

15 Estimation At the heart of any statistical analysis is an estimator. The role of the estimator is to produce an estimate given information and other user inputs. Two popular estimation problems: I Estimation of parameters of a distribution I Estimation of model parameters (regression) Subject is very broad - applies to all broad fields of engineering, medicine, econometrics, etc. Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 15

16 Post-estimation analysis Any estimate is a random variable and a function of the sample size. Statistical properties of the estimate qualify the goodness of an estimator: I Accuracy: How accurate is the estimate on the average? I Precision: What is the variability of the estimates obtained from different records? I Does the given estimator produce an estimate with the least variability? I What can we confidently say about the true value of (call it 0 ) from the obtained estimate? I Will the estimate converge (to the truth) as we increase the sample size? Arun K. Tangirala, IIT Madras Intro to Statistical Hypothesis Testing 16