Optimal design of experiments
|
|
- Clementine Harrington
- 5 years ago
- Views:
Transcription
1 Optimal design of experiments Session 4: Some theory Peter Goos / 40 Optimal design theory continuous or approximate optimal designs implicitly assume an infinitely large number of observations are available is mathematically convenient exact or discrete designs finite number of observations fewer theoretical results / 40
2 Continuous versus exact designs continuous { } x x ξ =... x h w w... w h x,x,...,x h : design points or support points w,w,...,w h : weights (w i 0, i w i = ) h: number of different points exact { } x x ξ =... x h n n... n h n,n,...,n h : (integer) numbers of observations at x,...,x n i n i = n h: number of different points 3 / 40 Information matrix all criteria to select a design are based on information matrix model matrix f T (x ) f T (x ) X = = f T (x 3 ) f T (x n ) I x x x x x x 4 / 40
3 Information matrix (total) information matrix M = σ XT X = σ n f(x i )f T (x i ) i= per observation information matrix σ f(x i)f T (x i ) 5 / 40 Information matrix industrial example σ ( X T X ) = / 40
4 Information matrix exact designs M = h n i f(x i )f T (x i ) i= where h = number of different points n i = number of replications of point i continuous designs h M = w i f(x i )f T (x i ) i= 7 / 40 D-optimality criterion seeks designs that minimize variance-covariance matrix of ˆβ... that minimize σ (X T X)... that minimize (X T X)... that maximize X T X or M D-optimal designs minimize generalized variance of ˆβ volume of confidence ellipsoid about unknown β D stands for determinant 8 / 40
5 D s -optimality criterion useful when the interest is only in a subset of the parameters useful when intercept or block effects are of no interest seeks designs that minimize determinant of variance-covariance matrix corresponding to subset of β Y = Xβ + ɛ = X β + X β + ɛ where β is the set of parameters of interest 9 / 40 D s -optimality criterion [ X M = (X T T X) = X X T X ] X T X X T X [ ] cov( ˆβ) = (X T X) A B = C D part of the variance-covariance matrix corresponding to β minimizing A is the same as minimizing ( X T X X T X (X T X ) X T X ) and as maximizing X T X X T X (X T X ) X T X 0 / 40
6 A-optimality criterion seeks designs that minimize average variance of parameter estimates seeks designs that minimize p var ( ) ˆβi /p i= p = number of model parameters seeks designs that minimize trace σ ( X T X ) or trace ( X T X ) / 40 V- and G-optimality criteria V-optimality (also Q-, IV-, I-optimality) seeks designs that minimize average prediction variance over design region χ... that minimize σ f T (x)(x T X) f(x)dx G-optimality χ seeks designs that minimize maximum prediction variance over design region χ... that minimize max χ var{ŷ(x)} = max χ f T (x)(x T X) f(x) / 40
7 Discussion problem: in general, all optimality criteria lead to different designs exception: D-optimal continuous designs = G-optimal continuous designs general equivalence theorem prediction variance is maximal in design points maximum = number of model parameters what optimality criterion to use? 3 / 40 D-optimality criterion most popular criterion D-optimal designs are usually quite good w.r.t. other criteria D-optimal designs are not affected by linear transformations of levels of the experimental variables computational advantages: update formulas 4 / 40
8 Linear transformations of factor levels X XA = Z max X T X max Z T Z = (XA) T XA = A T X T XA = A T X T X A = A X T X 5 / 40 Quadratic regression in one variable χ = [,] Y = β 0 + β x + β x + ɛ D-optimal continuous design { } 0 + Design = /3 /3 /3 information matrix 0 /3 M = 0 /3 0 /3 0 /3 6 / 40
9 Quadratic regression in one variable general equivalence theorem implies that D-optimal continuous design is also G-optimal D-optimal design minimizes the maximum prediction variance maximum is equal to p check by plotting prediction variance for all x [,] var{ŷ(x)} = f T (x)m f(x) 7 / 40 Prediction variance var{ŷ(x)} = f T (x)m f(x) = [ x x ] = [ 3 3x 3 x x] = 3 3x + 3 x 3x + 9 x4 x x x x = 3 9 x + 9 x4 8 / 40
10 Prediction variance var{ŷ(x)} p = / 40 Quadratic regression in one variable consider other design Design = { 0 } + /4 / /4 information matrix 0 M = / 40
11 Prediction variance var{ŷ(x)} = f T (x)m f(x) = [ x x ] = [ x x + 4x ] = x + x x + 4x 4 = x + 4x 4 x x x x / 40 Prediction variance var{ŷ(x)} 4 p = Design is not D-optimal / 40
12 Optimal exact designs A-optimal continuous design ξ = { } what if n = 4,8,? multiply weights of continuous design by 4, 8 or integer numbers of runs at each point what if n = 5,6,7? multiply weights by 5, 6, or 7 non-integer numbers of runs at some of the points not useful in practice 3 / 40 Polynomial regression in one variable D-optimal design points for the dth order polynomial regression in one variable d x x x 3 x 4 x 5 x 6 x 7 weight / 0 /3 3 a 3 a 3 /4 4 a 4 0 a 4 /5 5 a 5 b 5 b 5 a 5 /6 6 a 6 b 6 0 b 6 a 6 /7 a 3 = /5 b 5 = (7 7)/ a 4 = 3/7 a 6 = (5 + 5)/33 a 5 = (7 + 7)/ b 6 = (5 5)/33 4 / 40
13 Quadratic regression in one variable Design has covariance matrix (X T X) = 0 3/ / with trace(x T X) = 3 + 3/ + 9/ = 9 Design has covariance matrix 0 (X T X) = with trace(x T X) = = 8 Design is A-optimal 5 / 40 3 Factorial design main-effects model Y = β 0 + β x + β x + β 3 x 3 + ɛ model matrix X = / 40
14 3 factorial design (total) information matrix M = X T X = per observation information matrix M = n (XT X) = / 40 Prediction variance var{ŷ(x)} = f T (x) { M } f(x) = [ ]{ x x x } 3 M x x x 3 = + x + x + x 3 if χ = [,] 3, then this is maximal when x = ±, x = ±, x 3 = ± maximum = 4 = p general equivalence theorem is satisfied 3 factorial design is D- and G-optimal 8 / 40
15 More Plackett-Burman designs: optimal for main-effects model factorial designs: optimal for main-effects-plus-interactions models some remarks off-diagonal elements of information matrix often zero (impossible when quadratic terms) optimal designs are often symmetric these properties are more difficult to achieve for exact designs 9 / 40 Quadratic regression in two variables Y = β 0 +β x +β x +β x x +β x +β x +ɛ D-optimal continuous design + x x D-optimal discrete designs n = 9: one observation at each point of the continuous D-optimal design n = 6: D-optimal exact design does not resemble D-optimal continuous one 30 / 40
16 Information matrix D-optimal continuous design / 40 Quadratic regression in two variables n = 6 n = 7 n = 8 n = 9 D-optimal exact designs 3 / 40
17 Quadratic design in two variables n = 6 n = 7 n = 8 n = 9 D-optimal exact three-level designs 33 / 40 Quadratic regression in variables run, runs, 3 runs D-optimal exact design n = 8 34 / 40
18 An industrial example experiment to investigate effect of amount of glycerine (%), x ( x 3) speed temperature reduction ( F/min), x ( x 3) response: amount of surviving biological material (%), y context: freeze-dried coffee retaining volatile compounds in freeze-dried coffee is important for its smell and taste Excel file: quadratic3.xls 35 / 40 An industrial example Y i = β 0 + β x i + β x i +β x i x i + β x i + β x i + ɛ 9 observations / tests 3 factorial design is optimal what if combination of high x and high x are not allowed? 36 / 40
19 Constrained design region 3 x x + x 5 constraint x 3 x : glycerine, x : temperature reduction 37 / 40 Constrained design region: solution I scale 3 factorial design down so that it fits in constrained design region 3 x x 3 det(x T X) = / 40
20 Constrained design region: solution II move forbidden point (3,3) inward 3 x x 3 det(x T X) = / 40 Constrained design region: solution III use optimal design approach 3 x (.9,.9) x 3 det(x T X) = / 40
8 RESPONSE SURFACE DESIGNS
8 RESPONSE SURFACE DESIGNS Desirable Properties of a Response Surface Design 1. It should generate a satisfactory distribution of information throughout the design region. 2. It should ensure that the
More informationOptimal design of experiments
Optimal design of experiments Session 5: Design construction Peter Goos 1 / 33 Constructing exact optimal designs direct approaches to maximize X T X Microsoft Excel mathematical programming run into problems
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationRegression: Lecture 2
Regression: Lecture 2 Niels Richard Hansen April 26, 2012 Contents 1 Linear regression and least squares estimation 1 1.1 Distributional results................................ 3 2 Non-linear effects and
More informationProperties of the least squares estimates
Properties of the least squares estimates 2019-01-18 Warmup Let a and b be scalar constants, and X be a scalar random variable. Fill in the blanks E ax + b) = Var ax + b) = Goal Recall that the least squares
More information1 One-way analysis of variance
LIST OF FORMULAS (Version from 21. November 2014) STK2120 1 One-way analysis of variance Assume X ij = µ+α i +ɛ ij ; j = 1, 2,..., J i ; i = 1, 2,..., I ; where ɛ ij -s are independent and N(0, σ 2 ) distributed.
More informationResponse Surface Methodology IV
LECTURE 8 Response Surface Methodology IV 1. Bias and Variance If y x is the response of the system at the point x, or in short hand, y x = f (x), then we can write η x = E(y x ). This is the true, and
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationRegression. Estimation of the linear function (straight line) describing the linear component of the joint relationship between two variables X and Y.
Regression Bivariate i linear regression: Estimation of the linear function (straight line) describing the linear component of the joint relationship between two variables and. Generally describe as a
More informationI-Optimal Versus D-Optimal Split-Plot Response Surface Designs
I-Optimal Versus D-Optimal Split-Plot Response Surface Designs BRADLEY JONES SAS Institute and Universiteit Antwerpen PETER GOOS Universiteit Antwerpen and Erasmus Universiteit Rotterdam Response surface
More informationA General Criterion for Factorial Designs Under Model Uncertainty
A General Criterion for Factorial Designs Under Model Uncertainty Steven Gilmour Queen Mary University of London http://www.maths.qmul.ac.uk/ sgg and Pi-Wen Tsai National Taiwan Normal University Fall
More informationMa 3/103: Lecture 24 Linear Regression I: Estimation
Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the
More informationWeighted Least Squares
Weighted Least Squares The standard linear model assumes that Var(ε i ) = σ 2 for i = 1,..., n. As we have seen, however, there are instances where Var(Y X = x i ) = Var(ε i ) = σ2 w i. Here w 1,..., w
More informationProbability and Statistics Notes
Probability and Statistics Notes Chapter Seven Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Seven Notes Spring 2011 1 / 42 Outline
More informationAppendix A: Review of the General Linear Model
Appendix A: Review of the General Linear Model The generallinear modelis an important toolin many fmri data analyses. As the name general suggests, this model can be used for many different types of analyses,
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationRandomisation, Replication, Response Surfaces. and. Rosemary
Randomisation, Replication, Response Surfaces and Rosemary 1 A.C. Atkinson a.c.atkinson@lse.ac.uk Department of Statistics London School of Economics London WC2A 2AE, UK One joint publication RAB AND ME
More information1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM.
Université du Sud Toulon - Var Master Informatique Probabilistic Learning and Data Analysis TD: Model-based clustering by Faicel CHAMROUKHI Solution The aim of this practical wor is to show how the Classification
More informationA Study on Minimal-point Composite Designs
A Study on Minimal-point Composite Designs by Yin-Jie Huang Advisor Ray-Bing Chen Institute of Statistics, National University of Kaohsiung Kaohsiung, Taiwan 8 R.O.C. July 007 Contents Introduction Construction
More informationThe Hilbert Space of Random Variables
The Hilbert Space of Random Variables Electrical Engineering 126 (UC Berkeley) Spring 2018 1 Outline Fix a probability space and consider the set H := {X : X is a real-valued random variable with E[X 2
More informationRegression, Ridge Regression, Lasso
Regression, Ridge Regression, Lasso Fabio G. Cozman - fgcozman@usp.br October 2, 2018 A general definition Regression studies the relationship between a response variable Y and covariates X 1,..., X n.
More informationA-optimal Minimax Design Criterion for. Two-level Fractional Factorial Designs
A-optimal Minimax Design Criterion for Two-level Fractional Factorial Designs by Yue Yin BA. Beijing Institute of Technology 2011 A Thesis Submitted in Partial Fullfillment of the Requirements for the
More informationBayesian Regressions in Experimental Design
Bayesian Regressions in Experimental Design Stanley Sawyer Washington University Vs. April 9, 8. Introduction. The purpose here is to derive a formula for the posterior probabilities given observed data
More information2. Regression Review
2. Regression Review 2.1 The Regression Model The general form of the regression model y t = f(x t, β) + ε t where x t = (x t1,, x tp ), β = (β 1,..., β m ). ε t is a random variable, Eε t = 0, Var(ε t
More information9. Model Selection. statistical models. overview of model selection. information criteria. goodness-of-fit measures
FE661 - Statistical Methods for Financial Engineering 9. Model Selection Jitkomut Songsiri statistical models overview of model selection information criteria goodness-of-fit measures 9-1 Statistical models
More informationWeighted Least Squares
Weighted Least Squares The standard linear model assumes that Var(ε i ) = σ 2 for i = 1,..., n. As we have seen, however, there are instances where Var(Y X = x i ) = Var(ε i ) = σ2 w i. Here w 1,..., w
More information2.1 Linear regression with matrices
21 Linear regression with matrices The values of the independent variables are united into the matrix X (design matrix), the values of the outcome and the coefficient are represented by the vectors Y and
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationStochastic Design Criteria in Linear Models
AUSTRIAN JOURNAL OF STATISTICS Volume 34 (2005), Number 2, 211 223 Stochastic Design Criteria in Linear Models Alexander Zaigraev N. Copernicus University, Toruń, Poland Abstract: Within the framework
More informationLinear Models for Regression CS534
Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict
More informationTwo-Stage Computing Budget Allocation. Approach for Response Surface Method PENG JI
Two-Stage Computing Budget Allocation Approach for Response Surface Method PENG JI NATIONAL UNIVERSITY OF SINGAPORE 2005 Two-Stage Computing Budget Allocation Approach for Response Surface Method PENG
More informationF9 F10: Autocorrelation
F9 F10: Autocorrelation Feng Li Department of Statistics, Stockholm University Introduction In the classic regression model we assume cov(u i, u j x i, x k ) = E(u i, u j ) = 0 What if we break the assumption?
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationSTAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
More informationUseful Numerical Statistics of Some Response Surface Methodology Designs
Journal of Mathematics Research; Vol. 8, No. 4; August 20 ISSN 19-9795 E-ISSN 19-9809 Published by Canadian Center of Science and Education Useful Numerical Statistics of Some Response Surface Methodology
More informationOptimal experimental design, an introduction, Jesús López Fidalgo
Optimal experimental design, an introduction Jesus.LopezFidalgo@uclm.es University of Castilla-La Mancha Department of Mathematics Institute of Applied Mathematics to Science and Engineering Books (just
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationExample: Suppose Y has a Poisson distribution with mean
Transformations A variance stabilizing transformation may be useful when the variance of y appears to depend on the value of the regressor variables, or on the mean of y. Table 5.1 lists some commonly
More information. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
More informationMultilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do
More informationLinear Models for Regression CS534
Linear Models for Regression CS534 Example Regression Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict
More information2 Introduction to Response Surface Methodology
2 Introduction to Response Surface Methodology 2.1 Goals of Response Surface Methods The experimenter is often interested in 1. Finding a suitable approximating function for the purpose of predicting a
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2
MA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2 1 Ridge Regression Ridge regression and the Lasso are two forms of regularized
More informationBusiness Statistics. Tommaso Proietti. Model Evaluation and Selection. DEF - Università di Roma 'Tor Vergata'
Business Statistics Tommaso Proietti DEF - Università di Roma 'Tor Vergata' Model Evaluation and Selection Predictive Ability of a Model: Denition and Estimation We aim at achieving a balance between parsimony
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #3 1 / 42 Outline 1 2 3 t-test P-value Linear
More informationCointegrated VAR s. Eduardo Rossi University of Pavia. November Rossi Cointegrated VAR s Financial Econometrics / 56
Cointegrated VAR s Eduardo Rossi University of Pavia November 2013 Rossi Cointegrated VAR s Financial Econometrics - 2013 1 / 56 VAR y t = (y 1t,..., y nt ) is (n 1) vector. y t VAR(p): Φ(L)y t = ɛ t The
More informationStatistics 910, #5 1. Regression Methods
Statistics 910, #5 1 Overview Regression Methods 1. Idea: effects of dependence 2. Examples of estimation (in R) 3. Review of regression 4. Comparisons and relative efficiencies Idea Decomposition Well-known
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationRegression #4: Properties of OLS Estimator (Part 2)
Regression #4: Properties of OLS Estimator (Part 2) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #4 1 / 24 Introduction In this lecture, we continue investigating properties associated
More informationTransductive Experiment Design
Appearing in NIPS 2005 workshop Foundations of Active Learning, Whistler, Canada, December, 2005. Transductive Experiment Design Kai Yu, Jinbo Bi, Volker Tresp Siemens AG 81739 Munich, Germany Abstract
More informationModel Selection and Geometry
Model Selection and Geometry Pascal Massart Université Paris-Sud, Orsay Leipzig, February Purpose of the talk! Concentration of measure plays a fundamental role in the theory of model selection! Model
More informationAn Introduction to Parameter Estimation
Introduction Introduction to Econometrics An Introduction to Parameter Estimation This document combines several important econometric foundations and corresponds to other documents such as the Introduction
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationSelecting an Orthogonal or Nonorthogonal Two-Level Design for Screening
Selecting an Orthogonal or Nonorthogonal Two-Level Design for Screening David J. Edwards 1 (with Robert W. Mee 2 and Eric D. Schoen 3 ) 1 Virginia Commonwealth University, Richmond, VA 2 University of
More informationSolutions to Exercises
1 c Atkinson et al 2007, Optimum Experimental Designs, with SAS Solutions to Exercises 1. and 2. Certainly, the solutions to these questions will be different for every reader. Examples of the techniques
More informationLecture 4: Heteroskedasticity
Lecture 4: Heteroskedasticity Econometric Methods Warsaw School of Economics (4) Heteroskedasticity 1 / 24 Outline 1 What is heteroskedasticity? 2 Testing for heteroskedasticity White Goldfeld-Quandt Breusch-Pagan
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationLecture 1: January 12
10-725/36-725: Convex Optimization Fall 2015 Lecturer: Ryan Tibshirani Lecture 1: January 12 Scribes: Seo-Jin Bang, Prabhat KC, Josue Orellana 1.1 Review We begin by going through some examples and key
More informationUnit 12: Response Surface Methodology and Optimality Criteria
Unit 12: Response Surface Methodology and Optimality Criteria STA 643: Advanced Experimental Design Derek S. Young 1 Learning Objectives Revisit your knowledge of polynomial regression Know how to use
More informationDesign Moments. Many properties of experimental designs are quantified by design moments. Given a model matrix 1 x 11 x 21 x k1 1 x 12 x 22 x k2 X =
8.5 Rotatability Recall: Var[ŷ(x)] = σ 2 x (m) () 1 x (m) is the prediction variance and NVar[ŷ(x)]/σ 2 = Nx (m) () 1 x (m) is the scaled prediction variance. A design is rotatable if the prediction variance
More informationChapter 1. Linear Regression with One Predictor Variable
Chapter 1. Linear Regression with One Predictor Variable 1.1 Statistical Relation Between Two Variables To motivate statistical relationships, let us consider a mathematical relation between two mathematical
More informationApplied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
More informationStatistical Models. ref: chapter 1 of Bates, D and D. Watts (1988) Nonlinear Regression Analysis and its Applications, Wiley. Dave Campbell 2009
Statistical Models ref: chapter 1 of Bates, D and D. Watts (1988) Nonlinear Regression Analysis and its Applications, Wiley Dave Campbell 2009 Today linear regression in terms of the response surface geometry
More informationLecture 13: Simple Linear Regression in Matrix Format
See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 13: Simple Linear Regression in Matrix Format 36-401, Section B, Fall 2015 13 October 2015 Contents 1 Least Squares in Matrix
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationLecture 2. The Simple Linear Regression Model: Matrix Approach
Lecture 2 The Simple Linear Regression Model: Matrix Approach Matrix algebra Matrix representation of simple linear regression model 1 Vectors and Matrices Where it is necessary to consider a distribution
More informationRestricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model
Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives
More informationMultiple Linear Regression
Multiple Linear Regression University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html 1 / 42 Passenger car mileage Consider the carmpg dataset taken from
More informationPharmaceutical Experimental Design and Interpretation
Pharmaceutical Experimental Design and Interpretation N. ANTHONY ARMSTRONG, B. Pharm., Ph.D., F.R.Pharm.S., MCPP. KENNETH C. JAMES, M. Pharm., Ph.D., D.Sc, FRSC, F.R.Pharm.S., C.Chem. Welsh School of Pharmacy,
More informationMultiplication of Polynomials
Summary 391 Chapter 5 SUMMARY Section 5.1 A polynomial in x is defined by a finite sum of terms of the form ax n, where a is a real number and n is a whole number. a is the coefficient of the term. n is
More informationIntroduction to Statistical modeling: handout for Math 489/583
Introduction to Statistical modeling: handout for Math 489/583 Statistical modeling occurs when we are trying to model some data using statistical tools. From the start, we recognize that no model is perfect
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 51 Outline 1 Matrix Expression 2 Linear and quadratic forms 3 Properties of quadratic form 4 Properties of estimates 5 Distributional properties 3 / 51 Matrix
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationOPTIMAL DESIGN INPUTS FOR EXPERIMENTAL CHAPTER 17. Organization of chapter in ISSO. Background. Linear models
CHAPTER 17 Slides for Introduction to Stochastic Search and Optimization (ISSO)by J. C. Spall OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS Organization of chapter in ISSO Background Motivation Finite sample
More informationAdvanced Engineering Statistics - Section 5 - Jay Liu Dept. Chemical Engineering PKNU
Advanced Engineering Statistics - Section 5 - Jay Liu Dept. Chemical Engineering PKNU Least squares regression What we will cover Box, G.E.P., Use and abuse of regression, Technometrics, 8 (4), 625-629,
More informationFinal Review. Yang Feng. Yang Feng (Columbia University) Final Review 1 / 58
Final Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Final Review 1 / 58 Outline 1 Multiple Linear Regression (Estimation, Inference) 2 Special Topics for Multiple
More informationarxiv: v1 [stat.co] 7 Sep 2017
Computing optimal experimental designs with respect to a compound Bayes risk criterion arxiv:1709.02317v1 [stat.co] 7 Sep 2017 Radoslav Harman, and Maryna Prus September 8, 2017 Abstract We consider the
More informationMidterm: CS 6375 Spring 2015 Solutions
Midterm: CS 6375 Spring 2015 Solutions The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run out of room for an
More informationComments on algorithmic design
Comments on algorithmic design Robert E. Wheeler Copyright 2004-2009 by Robert E. Wheeler 1 Contents 1 Introduction 4 2 The varieties of designs 5 2.1 One-way layouts................................. 5
More information401 Review. 6. Power analysis for one/two-sample hypothesis tests and for correlation analysis.
401 Review Major topics of the course 1. Univariate analysis 2. Bivariate analysis 3. Simple linear regression 4. Linear algebra 5. Multiple regression analysis Major analysis methods 1. Graphical analysis
More informationNeed for Several Predictor Variables
Multiple regression One of the most widely used tools in statistical analysis Matrix expressions for multiple regression are the same as for simple linear regression Need for Several Predictor Variables
More informationWiley. Methods and Applications of Linear Models. Regression and the Analysis. of Variance. Third Edition. Ishpeming, Michigan RONALD R.
Methods and Applications of Linear Models Regression and the Analysis of Variance Third Edition RONALD R. HOCKING PenHock Statistical Consultants Ishpeming, Michigan Wiley Contents Preface to the Third
More informationLecture 13: Data Modelling and Distributions. Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1
Lecture 13: Data Modelling and Distributions Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1 Why data distributions? It is a well established fact that many naturally occurring
More informationLecture 24: Weighted and Generalized Least Squares
Lecture 24: Weighted and Generalized Least Squares 1 Weighted Least Squares When we use ordinary least squares to estimate linear regression, we minimize the mean squared error: MSE(b) = 1 n (Y i X i β)
More informationThe Standard Linear Model: Hypothesis Testing
Department of Mathematics Ma 3/103 KC Border Introduction to Probability and Statistics Winter 2017 Lecture 25: The Standard Linear Model: Hypothesis Testing Relevant textbook passages: Larsen Marx [4]:
More informationLINEAR REGRESSION MODELS W4315
LINEAR REGRESSION MODELS W431 HOMEWORK ANSWERS March 9, 2010 Due: 03/04/10 Instructor: Frank Wood 1. (20 points) In order to get a maximum likelihood estimate of the parameters of a Box-Cox transformed
More informationInverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
More informationHigh-dimensional regression
High-dimensional regression Advanced Methods for Data Analysis 36-402/36-608) Spring 2014 1 Back to linear regression 1.1 Shortcomings Suppose that we are given outcome measurements y 1,... y n R, and
More informationTWO-LEVEL FACTORIAL EXPERIMENTS: IRREGULAR FRACTIONS
STAT 512 2-Level Factorial Experiments: Irregular Fractions 1 TWO-LEVEL FACTORIAL EXPERIMENTS: IRREGULAR FRACTIONS A major practical weakness of regular fractional factorial designs is that N must be a
More informationLINEAR MODELS FOR CLASSIFICATION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
LINEAR MODELS FOR CLASSIFICATION Classification: Problem Statement 2 In regression, we are modeling the relationship between a continuous input variable x and a continuous target variable t. In classification,
More informationUsing Estimating Equations for Spatially Correlated A
Using Estimating Equations for Spatially Correlated Areal Data December 8, 2009 Introduction GEEs Spatial Estimating Equations Implementation Simulation Conclusion Typical Problem Assess the relationship
More informationConsequences of measurement error. Psychology 588: Covariance structure and factor models
Consequences of measurement error Psychology 588: Covariance structure and factor models Scaling indeterminacy of latent variables Scale of a latent variable is arbitrary and determined by a convention
More informationb) since the remainder is 0 I need to factor the numerator. Synthetic division tells me this is true
Section 5.2 solutions #1-10: a) Perform the division using synthetic division. b) if the remainder is 0 use the result to completely factor the dividend (this is the numerator or the polynomial to the
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More information5.1 Polynomial Functions
5.1 Polynomial Functions In this section, we will study the following topics: Identifying polynomial functions and their degree Determining end behavior of polynomial graphs Finding real zeros of polynomial
More informationQuestion 1. Find the coordinates of the y-intercept for. f) None of the above. Question 2. Find the slope of the line:
of 4 4/4/017 8:44 AM Question 1 Find the coordinates of the y-intercept for. Question Find the slope of the line: of 4 4/4/017 8:44 AM Question 3 Solve the following equation for x : Question 4 Paul has
More informationFirst Year Examination Department of Statistics, University of Florida
First Year Examination Department of Statistics, University of Florida August 20, 2009, 8:00 am - 2:00 noon Instructions:. You have four hours to answer questions in this examination. 2. You must show
More informationTopic 4: Model Specifications
Topic 4: Model Specifications Advanced Econometrics (I) Dong Chen School of Economics, Peking University 1 Functional Forms 1.1 Redefining Variables Change the unit of measurement of the variables will
More informationLinear Models Review
Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
More information