# Chapter 1 Statistical Inference

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations to populations are justified by statistics alone under random sampling. Otherwise, generalizations may be speculative and can only be justified by subjective judgment. null and alternative hypotheses A null hypothesis is usually a simple statement that there is no effect or no difference between groups, and is formally an equation about a parameter. We might find data to be consistent with a null hypothesis, but we do not prove a null hypothesis to be true. We may fail to reject a null hypothesis, but we do not accept it based on data. An alternative hypothesis is what we accept if we are able to reject a null hypothesis. Formally, it is usually an inequality about one or more parameters. test statistic a test statistic is something that can be calculated from sample data. Statistical inferences are based on the null distribution of the test statistic, the random distribution the statistic would have if we were able to take several samples of the same size and if the null hypothesis were true. In an experiment where individuals are randomly allocated, we can also consider the randomization distribution of the test statistic. p-value A p-value is the probability that the test statistic would take on a value at least as extreme (in reference to the alternative hypothesis) as that from the actual data, assuming that the null hypothesis is true. Small p-values indicate evidence against the null hypothesis. P-values are probabilities about the values of test statistics, but are not probabilities of hypotheses directly. permutation and randomization tests In a randomization test, we ask how unusual the results of an experiment are compared to all possible randomizations. A permutation test is identical in practice, but differs in interpretation because the groups are observed instead of randomly assigned. confounding Two variables are confounded when we cannot distinguish between their effects. Chapter 2 t-distribution Methods standard error The standard error of a statistic is the estimated standard deviation of its sampling distribution. Z-ratio and t-ratio The Z-ratio (z-score) of a statistic that is an estimator for a parameter is the (estimate - parameter)/(standard deviation). In many situations the Z-ratio will have an (approximate) standard normal distribution. The t-ratio is (estimate - parameter)/(standard error). The difference is that the t-ratio has variability in both the numerator (estimate) and the denominator (standard error), so the t-ratio is a more variable statistic than the Z-ratio. In many situations the t-ratio will have a t distribution with a number of degrees of freedom that is a function of the sample size. paired samples versus two independent samples In a paired setting, pairs of observations are sampled together. It is appropriate to take differences within the pair and to base inferences on the distribution of these differences. When there are two independent samples, inferences are based on the distribution of the difference of sample means, which has a different standard error than the

2 mean of paired differences. In a paired design, the effects of confounding factors are often reduced more efficiently than by randomization. pooled standard deviation If the model assumes that there is a common standard deviation, then it makes sense to pool information from all samples to estimate it. This is an assumption of ANOVA and regression as well. However, if in a two-sample setting there is a large discrepancy in variability in the two samples, it is best not to pool. confidence intervals A 95% confidence interval is made from a procedure that will contain the parameter for 95% of all samples. The values in a 95% confidence interval are precisely those for which the corresponding two-sided hypothesis test would have a p-value larger than Chapter 3 Assumptions robustness A statistical procedure is robust if it is still valid when assumptions are not met. resistant A statistical procedure is resistant when it does not change much when part of the data changes, perhaps drastically. outlier An outlier is an observation that is far away from the rest of the group. transformations Transformed variables (logs are a common choice) often come closer to fitting model assumptions. Chapter 4 Alternative Methods ranks Transforming observations to ranks is an alternative to t tools. This transformation may be especially useful with censored data, where an observation may not be known exactly, but its value may be known to be larger than some value. (For example, if the variable being measured is survival time and the individual has not died by the end of the study, the survival time is censored.) permutation test In a permutation test, the p-value is a proportion of regroupings. sign test For paired data, we can consider randomizing the sign of each paired difference (or permuting the groups within each pair) to find a p-value. Normal approximation P-values from permutation tests can be found by a normal approximation. Chapter 5 One-way ANOVA pooled estimates of SD If there are two or more samples and we assume constant variance in the populations, the best estimate of the common variance is the pooled variance, s 2 p, which is a weighted average of the sample variances, weighted by their degrees of freedom. degrees of freedom For a random sample, the degrees of freedom are one less than the sample size, n 1. One-way ANOVA In a one-way ANalysis Of VAriance, there is a single categorical explanatory variable and a quantitative response variable. The null hypothesis is that all population means are equal. To test this hypothesis, we use an F -test. The ANOVA table is an accounting method for computing the F test statistic.

3 F -distribution An F distribution has both numerator and denominator degrees of freedom. The mean is near one. P-values in F -tests are areas to the right. Extra-sum-of-squares F -test An extra sum of squares F -test is for comparing nested models that differ by one or more parameters having values fixed at zero. residual plots A plot of residuals versus fitted values (or group in an ANOVA) can be useful for detecting deviations from constant variance assumptions or skewness. random effects In a random effects model, the groups are thought to be sampled from some larger population of interest about which inferences are desired. In a fixed effects model, the groups in the study are the only ones of interest. Chapter 6 Multiple Comparisons linear combination A linear combination of variables is a sum of the variables where each variable may be multiplied by a coefficient. simultaneous inference Simultaneous inference is when more than one inferences are made at the same time. familywise confidence level The familywise confidence level is the proportion of samples for which a set of more than one confidence intervals will all contain the true parameter values. multiple comparisons When there are more than two groups, it is often of interest to make multiple comparisons. To control for multiple comparisons, it is best to increase the width of standard confidence intervals by using a larger multiplier than the standard t-distribution one to be confident hat all intervals are correct. Tukey-Kramer The Tukey-Kramer procedure assumes that sample sizes are equal, an is based on the Studentized range statistic, a normalized difference between the maximum and minimum sample mean. Tukey-Kramer is only valid for comparing sample means. Scheffé The Scheffé procedure uses a multiplier based on the F distribution and is based on the distribution of the most extreme linear contrast. This procedure is very conservative, but valid for searching for comparisons after the fact. Bonferroni The Bonferroni procedure is valid for a predetermined number of comparisons of any type. The multiplier comes from a t distribution, but with increased confidence level based on the number of comparisons. Chapter 7 Simple Linear Regression simple linear regression In simple linear regression, there is a single quantitative explanatory variable and a quantitative response variable. interpolation Interpolation is making predictions or estimates of the mean response within the range of the explanatory variable. extrapolation Extrapolation is making predictions or estimates of the mean response outside the range of the explanatory variable.

4 model assumptions the four model assumptions are (1) normality of the responses around their means; (2) linearity of the means as a function of the explanatory variable; (3) constant variance for each value of the explanatory variable; and (4) independence among all observations. fitted values A fitted value is the estimated mean for a set of explanatory variables. residual The residual is the difference between the actual value and the fitted value. least squares estimates The principle of least squares says that the best estimated regression coefficients minimize the sum of squared residuals. (In simple linear regression and in regular multiple regression, these are also maximum likelihood estimates.) degrees of freedom the degrees of freedom for a set of residuals are the number of observations minus the number of parameters for the mean. estimate of σ The standard estimate of σ is the square root of the residual sum of squares over the square root of the degrees of freedom. The square of this is an unbiased estimate of the variance. (The maximum likelihood estimate divides by n instead of n p where there are n observations and p parameters.) standard error The standard error associated with an estimate is an estimate of the standard deviation of the distribution of the estimate. the regression effect the regression effect, or regression to the mean is the tendency for future observations to be more average than the previous observation, on average. This is a necessary result of correlations being less than 1 (in situations where there is not a perfect linear relationship). correlation The correlation coefficient measures the strength of the linear relationship between two quantitative variables. Graphs are important: r near 1 or 1 does not imply that a linear fit is better than a nonlinear fit, just that the linear fit is better than a fit of a horizontal line. An r near 0 does not imply no relationship between the variables; there could be a very strong nonlinear relationship. Chapter 8 Regression Assumptions testing linearity A curve in the plot of residuals versus fitted values can indicate nonlinearity. testing constant variance A wedge-shaped pattern of residuals is evidence that variance is large for larger means than for smaller means. testing normality Skewness in residual plots can indicate lack of normality, as can a lack of straightness in a normal quantile plot of residuals. R 2 The R 2 test statistic can be interpreted as the proportion of variance in the response variable explained by the regression. Chapter 9 Multiple Regression multiple regression In multiple regression, there is a single quantitative response variable and at least two explanatory variables.

5 dummy variables categorical variables can be included in a regression analysis by coding the presence of each level but one in a separate variable. interaction two variables display interaction if the size of the effect of one depends on the value of the other. This can be modeled in multiple regression by including the product of the variables in the model. coded scatterplots It is often useful to use different plotting symbols (or colors) for each different population. interpreting coefficients A coefficient in a regression model only has meaning in the context of all of the other coefficients from the same model. In a strict mathematical sense, a regression coefficient is a measure of how the response changes per unit change in the explanatory variable keeping other variables fixed. This interpretation is often faulty, because it may not be possible or plausible in the context to keep all other variables fixed. This is especially obvious when, for example, X and X 2 are two variables, but can also occur if X 1 and X 2 are strongly correlated. Chapter 10 Inference in Multiple Regression tests of single coefficients Tests of single coefficients in a regression model are made using a t-test, and confidence intervals are based on multipliers from the t distribution. tests of linear combinations Tests or confidence intervals for linear combinations of coefficients requires first finding the standard error of the estimate (the linear combination of coefficients). This will be the square root of a linear combination of variances and covariances of the coefficients. covariance The covariance of two estimates is a measure of how they vary together. Chapter 11 Model Checking dealing with outliers If an observation is an outlier, it is worth fitting the model with and without the observation. If the inferences of main interest are the same, include the observation. If the inferences of main interest are different, you have a choice. You can include the observation, but indicate in the summary of the analysis that the results are strongly dependent on the inclusion of an influential observation. Or, you can drop the observation and limit the analysis to a portion of the population of interest. influence Cook s distance is one way to measure the influence of an observation. It is a scaled measure of how all of the fitted values change when a single observation is left out of a fit. A very rough guide is that a Cook s distance near one or larger indicates an influential observation. Cook s distance only measures influence one observation at a time, so it is not a perfect measure. For example, a pair of observations could be highly influential, but each might individually be not so influential. leverage Leverage is a measure of the explanatory variables alone that measures how far the explanatory variable values are from the rest. Observations with high leverage have the potential to be influential, but may not be. The leverage values are the diagonals of the hat matrix from the matrix approach to regression.

6 Chapter 12 Model Selection Strategies The curse of dimensionality I have not used this phrase in the course, but the general idea is that to maintain a set level of precision, the number of data points needs to increase exponentially with the number of parameters. What this means in a multiple regression setting is that, even if there are a lot of important variables and we want to include polynomial terms and interactions to better model what is really happening, a lot of data is necessary to model all of the parameters precisely. Bias and variance are in conflict adding more parameters to a model decreases bias but adds variability to the estimates of the parameters from data, and the amount of data new data necessary to maintain variability increases exponentially. A solution is to accept a certain amount of bias and fit models that are simpler, but that provide more reliable estimates. variable selection There are several methods for deciding which variables to include in a model. Different methods need not result in the same model. This is okay as long as we realize that it is rare for there to be a single model that is much better than all others. Variable selection strategies can be thought of as means to find useful models. forward selection In forward selection, variables are added one at a time, adding the best available variable at each step, until there are no more variables for which the improvement in fit is worth the penalty for adding the variable. In the forward selection procedure, once a variable is in, it cannot be removed. backward elimination In backward elimination, a model with many parameters is fit first and then variables are eliminated one at a time, always removing the least beneficial variable until removing any additional variables results in a worse model by the objective. Once a variable is removed, it is gone for good. stepwise regression Stepwise regression alternates between forward and backward steps. all subsets regression If there are p variables, there are 2 p possible linear models with these variables. In all subsets regression, the model that optimizes some objective criterion from all of the 2 p models is selected as the best. In practice, this is only feasible for relatively small p, which is why sequential procedures (forward selection, backward elimination, stepwise regression) are used as alternative procedures. Cp The Cp statistic picks models for which the total mean square error is estimated to be small. AIC AIC is based on a likelihood model and is the deviance plus a penalty of 2 per parameter. BIC BIC is also based on likelihood models and is the deviance plus a penalty of log n per parameter. Because log n > 2 for all but the tiniest of data sets, BIC is less likely than AIC to include too many variables. Chapter 13 Two-Way ANOVA two-way ANOVA Two-way ANOVA is multiple regression with two categorical explanatory variables (or factors). In general, ANOVA is the special case of regression where there is a quantitative response variable and one or more categorical explanatory variables. The response variable is modeled as varying normally around a mean that is a linear combination of the explanatory variables.

8 Chapter 14 Two-way ANOVA without replication replication A design has replication if there are multiple observations for some of the treatment combinations. Without replication, there is only one observation per treatment combination. As a consequence all the data would be used to estimate a saturated model, and there are no degrees of freedom left over for inference. The saturated model will predict fit the data exactly and all residuals would be 0. But there are situations where replication is impossible. Without replication, there is little choice but to adopt an additive model. repeated measures In a repeated measures design, the same individual is measured several times. This introduces a special kind of blocking. random effects versus fixed effects If a factor contains all of the levels of interest, it is a fixed effect. Examples include sex for which there are no other choices or dosage where an experimenter may select specific levels for the analysis. A factor is a random effect if we want to control for differences among the levels, but see these levels as a sample from some larger population of possible levels of interest. A common example is an animal study where we need to model differences among the animals in the study, but are interested in the effects of the other variables on other animals that may have selected instead for the study. When a factor is included in a model as a random effect, is brings with it another source of variation. Estimates of model parameters for the mean are the same whether or not the factors are considered to be fixed or random, but the resulting inference can be different. In this course we did not explore these differences in analysis. Chapter 20 Logistic Regression generalized linear model In a generalized linear model, the mean of the response variable is a function of a linear combination of the explanatory variables. Examples in this course are logistic regression and binomial regression where the responses of individuals are 0/1 binary variables (or categorical variables with two levels). In these cases, we use the logit function, the log of the odds ratio, as the function of the mean to model as a linear function. link function In a generalized linear model, the link function is the function that has a linear model. You can think of link functions as special transformations so that the linearity assumption of the regular linear model holds. (Constant variance and normality probably will not hold.) maximum likelihood Maximum likelihood is a method of parameter estimation in which the parameter values that produce the largest probability of the observed data are the estimates. likelihood function The probability density function or probability mass function describes the probability distribution of a random variable. For fixed parameter values and variable possible values of the random variable, these functions integrate (or sum) to one. If we fix the possible values at the observed data and treat the parameters as variables, this is the likelihood function. A likelihood function will usually not integrate (or sum) to one. properties of maximum likelihood estimators (1) bias approaches zero as sample size increase, (2) SDs of sampling distribution of estimators can be found analytically (often), (3) maximum likelihood estimators do as well as possible in most situations, (4) sampling distributions of maximum likelihood estimators are approximately normal (which means there are simple approximate statistical inferential tools available).

9 likelihood ratio test In a likelihood ratio test, we reject the null hypothesis if the ratio of the maximum likelihood of a reduced model over the maximum likelihood of a full model is too small. In other words, reject when the full model explains the data much better than a reduced model. On the log scale, this is equivalent to a difference of log-likelihoods. It turns out that for large enough samples, twice the difference in log-likelihoods (full - reduced) has an approximate chi-square distribution with degrees of freedom equal to the difference in the number of parameters. deviance The deviance is a constant minus twice the log-likelihood. Thus, differences in deviances between models are equivalent to likelihood ratio tests, and differences in deviances can have chisquare distributions if sample sizes are large enough. other types of regression for binary response The important qualitative aspect of the logit function is that is an increasing function from (0, 1) to the real line. Other functions, such as the cumulative distribution function of the standard normal curve would work as well. Chapter 21 Binomial Regression binomial distribution The binomial distribution counts the number of successes in a fixed number of independent trials when there is the same success probability for each trial. binomial regression In a logistic regression setting where there are multiple observations for each set of values of the explanatory variables, binomial regression can be appropriate. The estimated regression coefficients are identical. The deviance will be off by a constant factor because in binomial regression, the order of the observations does not matter, but it does for logistic regression. (The probability that the first two of five observations are successs is different than the probability that two of five are successes.) deviance residuals Deviance residuals have a functional form that is based on the likelihood function. The sum of squared deviance residuals is the deviance. If many deviance residuals are larger than 2 in absolute value, this is an indivation that extra-binomial variability may be present. maximum likelihood The principle of maximum likelihood says to estimate parameters with values that make the probability (likelihood) of the observed data to be as high as possible. Chapter 22 Poisson Regression poisson distribution The Poisson distribution is a probability distribution on the nonnegative integers. It may be derived as a limit fo binomial distributions for which n tends to infinity but the mean np tends to a constant µ. It also results from generally plausible assumption on a distribution of points where: (1) the mean is proportional to the length (or area or volume) of a region; (2) the counts in disjoint regions are independent; and (3) the probability of two or more points in a small region gets small quickly enough. The mean and variance of the Poisson distribution are the same. comparison between Poisson and binomial regression Poisson regression and binomial regression each have a nonnegative count as the response. The main difference is that in binomial regression, it is neccesary for the response in each case to be modeled as a sum of a known number of independent 0/1 random variables, so there is known maximum value. In Poisson regression, there is no theoretical maximum value.

10 link function In a log-linear model for Poisson counts, the conventional choice of link function is the log. extra-poisson variation In practice, it is often the case that observed variability is larger than that predicted by the Poisson distribution. In this case, a more general model that assumes that the variance is a scalar multiple of the mean can be fit. This is called a quasilikelihood model.

### Stat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010

1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of

### Glossary for the Triola Statistics Series

Glossary for the Triola Statistics Series Absolute deviation The measure of variation equal to the sum of the deviations of each value from the mean, divided by the number of values Acceptance sampling

### 11. Generalized Linear Models: An Introduction

Sociology 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Copyright 2014 by John Fox Generalized Linear Models: An Introduction 1 1. Introduction I A synthesis due to Nelder and

### Linear Models 1. Isfahan University of Technology Fall Semester, 2014

Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and

### ˆπ(x) = exp(ˆα + ˆβ T x) 1 + exp(ˆα + ˆβ T.

Exam 3 Review Suppose that X i = x =(x 1,, x k ) T is observed and that Y i X i = x i independent Binomial(n i,π(x i )) for i =1,, N where ˆπ(x) = exp(ˆα + ˆβ T x) 1 + exp(ˆα + ˆβ T x) This is called the

### Multiple Linear Regression

Andrew Lonardelli December 20, 2013 Multiple Linear Regression 1 Table Of Contents Introduction: p.3 Multiple Linear Regression Model: p.3 Least Squares Estimation of the Parameters: p.4-5 The matrix approach

### Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

### Model Estimation Example

Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions

### Interpret Standard Deviation. Outlier Rule. Describe the Distribution OR Compare the Distributions. Linear Transformations SOCS. Interpret a z score

Interpret Standard Deviation Outlier Rule Linear Transformations Describe the Distribution OR Compare the Distributions SOCS Using Normalcdf and Invnorm (Calculator Tips) Interpret a z score What is an

### Outline of GLMs. Definitions

Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density

### 8 Nominal and Ordinal Logistic Regression

8 Nominal and Ordinal Logistic Regression 8.1 Introduction If the response variable is categorical, with more then two categories, then there are two options for generalized linear models. One relies on

### Probability and Statistics

Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be CHAPTER 4: IT IS ALL ABOUT DATA 4a - 1 CHAPTER 4: IT

### Course in Data Science

Course in Data Science About the Course: In this course you will get an introduction to the main tools and ideas which are required for Data Scientist/Business Analyst/Data Analyst. The course gives an

### Unit 27 One-Way Analysis of Variance

Unit 27 One-Way Analysis of Variance Objectives: To perform the hypothesis test in a one-way analysis of variance for comparing more than two population means Recall that a two sample t test is applied

### LISA Short Course Series Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R. Liang (Sally) Shan Nov. 4, 2014

LISA Short Course Series Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R Liang (Sally) Shan Nov. 4, 2014 L Laboratory for Interdisciplinary Statistical Analysis LISA helps VT researchers

### Prentice Hall Stats: Modeling the World 2004 (Bock) Correlated to: National Advanced Placement (AP) Statistics Course Outline (Grades 9-12)

National Advanced Placement (AP) Statistics Course Outline (Grades 9-12) Following is an outline of the major topics covered by the AP Statistics Examination. The ordering here is intended to define the

### * Tuesday 17 January :30-16:30 (2 hours) Recored on ESSE3 General introduction to the course.

Name of the course Statistical methods and data analysis Audience The course is intended for students of the first or second year of the Graduate School in Materials Engineering. The aim of the course

### Niche Modeling. STAMPS - MBL Course Woods Hole, MA - August 9, 2016

Niche Modeling Katie Pollard & Josh Ladau Gladstone Institutes UCSF Division of Biostatistics, Institute for Human Genetics and Institute for Computational Health Science STAMPS - MBL Course Woods Hole,

### CHAPTER 6: SPECIFICATION VARIABLES

Recall, we had the following six assumptions required for the Gauss-Markov Theorem: 1. The regression model is linear, correctly specified, and has an additive error term. 2. The error term has a zero

### Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

### Stat 705: Completely randomized and complete block designs

Stat 705: Completely randomized and complete block designs Timothy Hanson Department of Statistics, University of South Carolina Stat 705: Data Analysis II 1 / 16 Experimental design Our department offers

### Model comparison and selection

BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)

### For Bonnie and Jesse (again)

SECOND EDITION A P P L I E D R E G R E S S I O N A N A L Y S I S a n d G E N E R A L I Z E D L I N E A R M O D E L S For Bonnie and Jesse (again) SECOND EDITION A P P L I E D R E G R E S S I O N A N A

### Correlation and regression

1 Correlation and regression Yongjua Laosiritaworn Introductory on Field Epidemiology 6 July 2015, Thailand Data 2 Illustrative data (Doll, 1955) 3 Scatter plot 4 Doll, 1955 5 6 Correlation coefficient,

### Generalized linear models

Generalized linear models Douglas Bates November 01, 2010 Contents 1 Definition 1 2 Links 2 3 Estimating parameters 5 4 Example 6 5 Model building 8 6 Conclusions 8 7 Summary 9 1 Generalized Linear Models

### 11. Generalized Linear Models: An Introduction

Sociolog 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Generalized Linear Models: An Introduction 1 1. Introduction I A snthesis due to Nelder and Wedderburn, generalized linear

### 1-Way ANOVA MATH 143. Spring Department of Mathematics and Statistics Calvin College

1-Way ANOVA MATH 143 Department of Mathematics and Statistics Calvin College Spring 2010 The basic ANOVA situation Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative

### 9 Generalized Linear Models

9 Generalized Linear Models The Generalized Linear Model (GLM) is a model which has been built to include a wide range of different models you already know, e.g. ANOVA and multiple linear regression models

### , (1) e i = ˆσ 1 h ii. c 2016, Jeffrey S. Simonoff 1

Regression diagnostics As is true of all statistical methodologies, linear regression analysis can be a very effective way to model data, as along as the assumptions being made are true. For the regression

### Linear regression. Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1,X 2,...X p is linear.

Linear regression Linear regression is a simple approach to supervised learning. It assumes that the dependence of Y on X 1,X 2,...X p is linear. 1/48 Linear regression Linear regression is a simple approach

### Ron Heck, Fall Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October 20, 2011)

Ron Heck, Fall 2011 1 EDEP 768E: Seminar in Multilevel Modeling rev. January 3, 2012 (see footnote) Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October

### Background to Statistics

FACT SHEET Background to Statistics Introduction Statistics include a broad range of methods for manipulating, presenting and interpreting data. Professional scientists of all kinds need to be proficient

### Hierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture!

Hierarchical Generalized Linear Models ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models Introduction to generalized models Models for binary outcomes Interpreting parameter

### Interactions and Factorial ANOVA

Interactions and Factorial ANOVA STA442/2101 F 2017 See last slide for copyright information 1 Interactions Interaction between explanatory variables means It depends. Relationship between one explanatory

### Data Set 1A: Algal Photosynthesis vs. Salinity and Temperature

Data Set A: Algal Photosynthesis vs. Salinity and Temperature Statistical setting These data are from a controlled experiment in which two quantitative variables were manipulated, to determine their effects

### Statistical Analysis of Engineering Data The Bare Bones Edition. Precision, Bias, Accuracy, Measures of Precision, Propagation of Error

Statistical Analysis of Engineering Data The Bare Bones Edition (I) Precision, Bias, Accuracy, Measures of Precision, Propagation of Error PRIOR TO DATA ACQUISITION ONE SHOULD CONSIDER: 1. The accuracy

### Chapter 14 Student Lecture Notes 14-1

Chapter 14 Student Lecture Notes 14-1 Business Statistics: A Decision-Making Approach 6 th Edition Chapter 14 Multiple Regression Analysis and Model Building Chap 14-1 Chapter Goals After completing this

### Psychology Seminar Psych 406 Dr. Jeffrey Leitzel

Psychology Seminar Psych 406 Dr. Jeffrey Leitzel Structural Equation Modeling Topic 1: Correlation / Linear Regression Outline/Overview Correlations (r, pr, sr) Linear regression Multiple regression interpreting

### Exam C Solutions Spring 2005

Exam C Solutions Spring 005 Question # The CDF is F( x) = 4 ( + x) Observation (x) F(x) compare to: Maximum difference 0. 0.58 0, 0. 0.58 0.7 0.880 0., 0.4 0.680 0.9 0.93 0.4, 0.6 0.53. 0.949 0.6, 0.8

### Applied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur

Applied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 29 Multivariate Linear Regression- Model

### UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013

UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013 Exam policy: This exam allows two one-page, two-sided cheat sheets; No other materials. Time: 2 hours. Be sure to write your name and

### Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

### Orthogonal, Planned and Unplanned Comparisons

This is a chapter excerpt from Guilford Publications. Data Analysis for Experimental Design, by Richard Gonzalez Copyright 2008. 8 Orthogonal, Planned and Unplanned Comparisons 8.1 Introduction In this

### Regression Model Specification in R/Splus and Model Diagnostics. Daniel B. Carr

Regression Model Specification in R/Splus and Model Diagnostics By Daniel B. Carr Note 1: See 10 for a summary of diagnostics 2: Books have been written on model diagnostics. These discuss diagnostics

### Fractional Polynomial Regression

Chapter 382 Fractional Polynomial Regression Introduction This program fits fractional polynomial models in situations in which there is one dependent (Y) variable and one independent (X) variable. It

### Machine Learning Linear Classification. Prof. Matteo Matteucci

Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)

### Modeling Overdispersion

James H. Steiger Department of Psychology and Human Development Vanderbilt University Regression Modeling, 2009 1 Introduction 2 Introduction In this lecture we discuss the problem of overdispersion in

### Economics 471: Econometrics Department of Economics, Finance and Legal Studies University of Alabama

Economics 471: Econometrics Department of Economics, Finance and Legal Studies University of Alabama Course Packet The purpose of this packet is to show you one particular dataset and how it is used in

### Business Statistics. Lecture 9: Simple Regression

Business Statistics Lecture 9: Simple Regression 1 On to Model Building! Up to now, class was about descriptive and inferential statistics Numerical and graphical summaries of data Confidence intervals

### Algebra II. A2.1.1 Recognize and graph various types of functions, including polynomial, rational, and algebraic functions.

Standard 1: Relations and Functions Students graph relations and functions and find zeros. They use function notation and combine functions by composition. They interpret functions in given situations.

### Investigating Models with Two or Three Categories

Ronald H. Heck and Lynn N. Tabata 1 Investigating Models with Two or Three Categories For the past few weeks we have been working with discriminant analysis. Let s now see what the same sort of model might

### Sigmaplot di Systat Software

Sigmaplot di Systat Software SigmaPlot Has Extensive Statistical Analysis Features SigmaPlot is now bundled with SigmaStat as an easy-to-use package for complete graphing and data analysis. The statistical

### [y i α βx i ] 2 (2) Q = i=1

Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation

### Linear Regression Models P8111

Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started

### Multinomial Logistic Regression Models

Stat 544, Lecture 19 1 Multinomial Logistic Regression Models Polytomous responses. Logistic regression can be extended to handle responses that are polytomous, i.e. taking r>2 categories. (Note: The word

### Multicollinearity occurs when two or more predictors in the model are correlated and provide redundant information about the response.

Multicollinearity Read Section 7.5 in textbook. Multicollinearity occurs when two or more predictors in the model are correlated and provide redundant information about the response. Example of multicollinear

### What If There Are More Than. Two Factor Levels?

What If There Are More Than Chapter 3 Two Factor Levels? Comparing more that two factor levels the analysis of variance ANOVA decomposition of total variability Statistical testing & analysis Checking

### Notes 3: Statistical Inference: Sampling, Sampling Distributions Confidence Intervals, and Hypothesis Testing

Notes 3: Statistical Inference: Sampling, Sampling Distributions Confidence Intervals, and Hypothesis Testing 1. Purpose of statistical inference Statistical inference provides a means of generalizing

### Chapter 10 Nonlinear Models

Chapter 10 Nonlinear Models Nonlinear models can be classified into two categories. In the first category are models that are nonlinear in the variables, but still linear in terms of the unknown parameters.

### FEEG6017 lecture: Akaike's information criterion; model reduction. Brendan Neville

FEEG6017 lecture: Akaike's information criterion; model reduction Brendan Neville bjn1c13@ecs.soton.ac.uk Occam's razor William of Occam, 1288-1348. All else being equal, the simplest explanation is the

From Practical Data Analysis with JMP, Second Edition. Full book available for purchase here. Contents About This Book... xiii About The Author... xxiii Chapter 1 Getting Started: Data Analysis with JMP...

### 36-309/749 Experimental Design for Behavioral and Social Sciences. Dec 1, 2015 Lecture 11: Mixed Models (HLMs)

36-309/749 Experimental Design for Behavioral and Social Sciences Dec 1, 2015 Lecture 11: Mixed Models (HLMs) Independent Errors Assumption An error is the deviation of an individual observed outcome (DV)

### Hotelling s One- Sample T2

Chapter 405 Hotelling s One- Sample T2 Introduction The one-sample Hotelling s T2 is the multivariate extension of the common one-sample or paired Student s t-test. In a one-sample t-test, the mean response

### Linear Models (continued)

Linear Models (continued) Model Selection Introduction Most of our previous discussion has focused on the case where we have a data set and only one fitted model. Up until this point, we have discussed,

### A Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,

A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. Heckman-Type

### Conditions for Regression Inference:

AP Statistics Chapter Notes. Inference for Linear Regression We can fit a least-squares line to any data relating two quantitative variables, but the results are useful only if the scatterplot shows a

### Hypothesis testing, part 2. With some material from Howard Seltman, Blase Ur, Bilge Mutlu, Vibha Sazawal

Hypothesis testing, part 2 With some material from Howard Seltman, Blase Ur, Bilge Mutlu, Vibha Sazawal 1 CATEGORICAL IV, NUMERIC DV 2 Independent samples, one IV # Conditions Normal/Parametric Non-parametric

### Chapter 13 Section D. F versus Q: Different Approaches to Controlling Type I Errors with Multiple Comparisons

Explaining Psychological Statistics (2 nd Ed.) by Barry H. Cohen Chapter 13 Section D F versus Q: Different Approaches to Controlling Type I Errors with Multiple Comparisons In section B of this chapter,

### QUANTITATIVE TECHNIQUES

UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION (For B Com. IV Semester & BBA III Semester) COMPLEMENTARY COURSE QUANTITATIVE TECHNIQUES QUESTION BANK 1. The techniques which provide the decision maker

### Lecture 6: Linear Regression (continued)

Lecture 6: Linear Regression (continued) Reading: Sections 3.1-3.3 STATS 202: Data mining and analysis October 6, 2017 1 / 23 Multiple linear regression Y = β 0 + β 1 X 1 + + β p X p + ε Y ε N (0, σ) i.i.d.

### CS 5014: Research Methods in Computer Science

Computer Science Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Fall 2010 Copyright c 2010 by Clifford A. Shaffer Computer Science Fall 2010 1 / 207 Correlation and

### 26:010:557 / 26:620:557 Social Science Research Methods

26:010:557 / 26:620:557 Social Science Research Methods Dr. Peter R. Gillett Associate Professor Department of Accounting & Information Systems Rutgers Business School Newark & New Brunswick 1 Overview

### Bivariate Data Summary

Bivariate Data Summary Bivariate data data that examines the relationship between two variables What individuals to the data describe? What are the variables and how are they measured Are the variables

### MODULE 6 LOGISTIC REGRESSION. Module Objectives:

MODULE 6 LOGISTIC REGRESSION Module Objectives: 1. 147 6.1. LOGIT TRANSFORMATION MODULE 6. LOGISTIC REGRESSION Logistic regression models are used when a researcher is investigating the relationship between

### Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

Title stata.com logistic postestimation Postestimation tools for logistic Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

### Generalized Linear Models (GLZ)

Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the

### Dr. Maddah ENMG 617 EM Statistics 11/28/12. Multiple Regression (3) (Chapter 15, Hines)

Dr. Maddah ENMG 617 EM Statistics 11/28/12 Multiple Regression (3) (Chapter 15, Hines) Problems in multiple regression: Multicollinearity This arises when the independent variables x 1, x 2,, x k, are

### 2 Prediction and Analysis of Variance

2 Prediction and Analysis of Variance Reading: Chapters and 2 of Kennedy A Guide to Econometrics Achen, Christopher H. Interpreting and Using Regression (London: Sage, 982). Chapter 4 of Andy Field, Discovering

### Statistical methods and data analysis

Statistical methods and data analysis Teacher Stefano Siboni Aim The aim of the course is to illustrate the basic mathematical tools for the analysis and modelling of experimental data, particularly concerning

### Statistics 135 Fall 2008 Final Exam

Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations

### Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics

Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).

### Correlation and Regression

Correlation and Regression October 25, 2017 STAT 151 Class 9 Slide 1 Outline of Topics 1 Associations 2 Scatter plot 3 Correlation 4 Regression 5 Testing and estimation 6 Goodness-of-fit STAT 151 Class

### A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL

Discussiones Mathematicae Probability and Statistics 36 206 43 5 doi:0.75/dmps.80 A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Tadeusz Bednarski Wroclaw University e-mail: t.bednarski@prawo.uni.wroc.pl

### Lecture 10: Alternatives to OLS with limited dependent variables. PEA vs APE Logit/Probit Poisson

Lecture 10: Alternatives to OLS with limited dependent variables PEA vs APE Logit/Probit Poisson PEA vs APE PEA: partial effect at the average The effect of some x on y for a hypothetical case with sample

### College Algebra Through Problem Solving (2018 Edition)

City University of New York (CUNY) CUNY Academic Works Open Educational Resources Queensborough Community College Winter 1-25-2018 College Algebra Through Problem Solving (2018 Edition) Danielle Cifone

### Linear Regression Models

Linear Regression Models Model Description and Model Parameters Modelling is a central theme in these notes. The idea is to develop and continuously improve a library of predictive models for hazards,

### 388 Index Differencing test ,232 Distributed lags , 147 arithmetic lag.

INDEX Aggregation... 104 Almon lag... 135-140,149 AR(1) process... 114-130,240,246,324-325,366,370,374 ARCH... 376-379 ARlMA... 365 Asymptotically unbiased... 13,50 Autocorrelation... 113-130, 142-150,324-325,365-369

### ANOVA Situation The F Statistic Multiple Comparisons. 1-Way ANOVA MATH 143. Department of Mathematics and Statistics Calvin College

1-Way ANOVA MATH 143 Department of Mathematics and Statistics Calvin College An example ANOVA situation Example (Treating Blisters) Subjects: 25 patients with blisters Treatments: Treatment A, Treatment

### Chapter 12 Summarizing Bivariate Data Linear Regression and Correlation

Chapter 1 Summarizing Bivariate Data Linear Regression and Correlation This chapter introduces an important method for making inferences about a linear correlation (or relationship) between two variables,

### Hypothesis Testing. We normally talk about two types of hypothesis: the null hypothesis and the research or alternative hypothesis.

Hypothesis Testing Today, we are going to begin talking about the idea of hypothesis testing how we can use statistics to show that our causal models are valid or invalid. We normally talk about two types

### Regression With a Categorical Independent Variable

Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent

### The University of Hong Kong Department of Statistics and Actuarial Science STAT2802 Statistical Models Tutorial Solutions Solutions to Problems 71-80

The University of Hong Kong Department of Statistics and Actuarial Science STAT2802 Statistical Models Tutorial Solutions Solutions to Problems 71-80 71. Decide in each case whether the hypothesis is simple

### 16.400/453J Human Factors Engineering. Design of Experiments II

J Human Factors Engineering Design of Experiments II Review Experiment Design and Descriptive Statistics Research question, independent and dependent variables, histograms, box plots, etc. Inferential

### Finding Relationships Among Variables

Finding Relationships Among Variables BUS 230: Business and Economic Research and Communication 1 Goals Specific goals: Re-familiarize ourselves with basic statistics ideas: sampling distributions, hypothesis

### Topic 1. Definitions

S Topic. Definitions. Scalar A scalar is a number. 2. Vector A vector is a column of numbers. 3. Linear combination A scalar times a vector plus a scalar times a vector, plus a scalar times a vector...

### Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions

Journal of Modern Applied Statistical Methods Volume 8 Issue 1 Article 13 5-1-2009 Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error

### Regression Analysis: Exploring relationships between variables. Stat 251

Regression Analysis: Exploring relationships between variables Stat 251 Introduction Objective of regression analysis is to explore the relationship between two (or more) variables so that information

### Applied Regression Modeling

Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS