Last Lecture. Distinguish Populations from Samples. Knowing different Sampling Techniques. Distinguish Parameters from Statistics

Transcription

1

2 Last Lecture Distinguish Populations from Samples Importance of identifying a population and well chosen sample Knowing different Sampling Techniques Distinguish Parameters from Statistics Knowing different Scales of Measurement to represent data Distinguish Descriptive from Inferential Statistics

3 Lecture Goals Importance of Statistical Measurements Distinguish Categorical from Numerical variables Knowing different ways of statistically describing data

4 Notations Characteristic Population (Parameter) Sample (Statistic) Mean 1 1 Variance Standard Deviation (SD)

5 Parameters vs. Statistics Parameters represent truth Statistics have associated error Population to make inferences about the population Sample figure out descriptive statistics about the sample

6 Statistical Description of Data Statistics describes a numeric set of data by its - Center - Variability - Shape Statistics describes a categorical set of data by - Frequency, percentage or proportion of each category

7 Data Collection Variables Gathering a sample requires measuring some physical quantity along a scale A variable is a characteristic or condition, about each individual element of a population or sample, that can change or take on different values. Example: Network Congested Arrival Rate Packet Drop

8 Categorical Variable Number of Subjects Lists the categories and presents the percent or count of individuals who fall in each category Figure 1: Bar Chart of Subjects in Treatment Groups Treatment Group Treatment Group Frequency Proportion Percent (%) 1 15 (15/60)= (25/60)= (20/60)= Total Figure 2: Pie Chart of Subjects in Treatment Groups 33% 25% % 3

9 Numerical Variable Overall pattern can be described by its shape, center, and spread. The following age distribution is right skewed. The center lies between 80 to 100. Number of Subjects Figure 3: Age Distribution More Age in Month Mean Standard Error Median 84 Mode 84 Standard Deviation Sample Variance Kurtosis Skewness Range 95 Minimum 48 Maximum 143 Sum 5425 Count 60

10 Methods of Center Measurement Center measurement is a summary measure of the overall level of a dataset Methods: mean, median, mode, geometric mean etc. Returns a central value for a set of observations and the extent to which the central value characterizes the whole set of data

11 a a

12 Scales of Measurement 0 4 types of scales: Nominal scale: categorical scale can not be mutually compared e.g. transport protocol type Ordinal scale: defines an ordering of the variable distances along the scale are insignificant e.g. Likert scale, Mean Opinion Score (MOS)

13 a a

14 Law of Large Numbers In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.

15 a

16 Effect of Outliers on the Mean sample mean excluding the last value: sample mean including the last value:

17 Methods of Center Measurement Median: the median value of a sample is the value such that 50% of the samples are larger than this value. - Find the sample median: {9, 3, 6, 7, 5, 2} if the observations have more than 1 median value the sample set is bimodal Mode: The value that is observed most frequently. The mode is undefined for sequences in which no observation is repeated.

18 Effect of Outliers What is a better measurement of the sample: Mean or Median? Cumulative Frequency sample mean excluding the last value: 17 sample mean (including the last value): 61.1 sample median (including the last value): 2

19 Methods of Center Measurement Measures of central value such as the mean or median must be coupled with measures of data dispersion (e.g., average distance from the mean) to indicate how well the central value characterizes the data as a whole. E.g. Consider two data sets: A: 30, 50, 70 B: 40, 50, 60 The mean of data set B is a better representation of the data set than is the case for set A.

20 Methods of Variability Measurement Variability (or dispersion) measures the amount of scatter in a dataset Methods: range, variance, standard deviation, interquartile range, coefficient of variation etc. 20

21 Methods of Variability Measurement Range is the difference between the largest and the smallest observations E.g. The range of 10, 5, 2, 100 is (100 2)=98 - Simplest (crude) measure of variability - Susceptible to outliers and therefore not always reliable - A better measurement is the Inter quartile Range

22 Methods of Variability Measurement Quartiles: Data can be divided into four regions that cover the total range of observed values. Cut points for these regions are known as quartiles.

23 Methods of Variability Measurement Quartiles: Data can be divided into four regions that cover the total range of observed values. Cut points for these regions are known as quartiles.

24 Methods of Variability Measurement Quartiles: In notations, quartiles of a data is the ((n+1)/4)q th observation of the data, where q is the desired quartile and n is the number of observations of data

25 Methods of Variability Measurement Quartiles : In example Q1= ((15+1)/4)1 =4 th observation of the data. The 4 th observation is 11. So Q1 is of this data is 11. The second quartile is Q2=40 (This is also the Median.) The third quartile is Q3=61.

26 Methods of Variability Measurement Inter Quartile Range: - Difference between Q3 and Q1 (but not necessarily fixed)* *can be calculated for any q% & 1 q% (percentiles) - The range of values in which the central 1 2q% of the sample lies - Conveys nearly the same information as the range but less sensitive to outliers

27 Methods of Variability Measurement Percentiles: If data is ordered and divided into 100 parts, then cut points are called Percentiles. 25 th percentile is the Q1, 50 th percentile is the Median (Q2) and the 75 th percentile of the data is Q3. In notations, percentiles of a data is the ((n+1)/100)pth observation of the data, where p is the desired percentile and n is the number of observations of data.

28 Next Lecture More on variability measurement Statistical Descriptions with shapes - Skewness - Kurtosis Measuring Statistical Error - Confidence Intervals