Introduction to Matrix Algebra and the Multivariate Normal Distribution
|
|
- Alaina James
- 6 years ago
- Views:
Transcription
1 Introduction to Matrix Algebra and the Multivariate Normal Distribution Introduction to Structural Equation Modeling Lecture #2 January 18, 2012 ERSH 8750: Lecture 2
2 Motivation for Learning the Multivariate Normal Distribution The univariate and multivariate normal distributions serve as the backbone of the most frequently used statistical procedures They are very robust and very useful Understanding their form and function will help you learn about a whole host of statistical routines immediately This course is essentially a course based in normal distributions Most of structural equation modeling happens with continuous variables with assumptions of multivariate normality To learn about the MVN distribution, one must first learn about matrix algebra ERSH 8750: Lecture 2 2
3 Introduction and Motivation for Matrix Algebra Structural equation modeling and nearly all other multivariate statistical techniques are described with matrix algebra It is the language of modern statistics When new methods are developed, the first published work typically involves matrices It makes technical writing more concise formulae are smaller Have you seen: (from the general linear model) (from confirmatory factor analysis) Useful tip: matrix algebra is a great way to get out of conversations and other awkward moments ERSH 8750: Lecture 2 3
4 Why Learn Matrix Algebra Matrix algebra can seem very abstract from the purposes of this class (and statistics in general) Learning matrix algebra is important for: Understanding how statistical methods work And when to use them (or not use them) Understanding what statistical methods mean Reading and writing results from new statistical methods Today s class is the first lecture of learning the language of structural equation modeling ERSH 8750: Lecture 2 4
5 Definitions We begin this class with some general definitions (from dictionary.com): Matrix: 1. A rectangular array of numeric or algebraic quantities subject to mathematical operations 2. The substrate on or within which a fungus grows Algebra: 1. A branch of mathematics in which symbols, usually letters of the alphabet, represent numbers or members of a specified set and are used to represent quantities and to express general relationships that hold for all members of the set 2. A set together with a pair of binary operations defined on the set. Usually, the set and the operations include an identity element, and the operations are commutative or associative ERSH 8750: Lecture 2 5
6 A Guiding Example To demonstrate matrix algebra, we will make use of data Imagine that somehow I collected data SAT test scores for both the Math (SATM) and Verbal (SATV) sections of 1,000 students The descriptive statistics of this data set are given below: ERSH 8750: Lecture 2 6
7 The Data In Excel: ERSH 8750: Lecture 2 7
8 THE BASICS: DEFINITIONS OF MATRICES, VECTORS, AND SCALARS ERSH 8750: Lecture 2 8
9 Matrices A matrix is a rectangular array of data Used for storing numbers Matrices can have unlimited dimensions For our purposes all matrices will have two dimensions: Row Columns Matrices are symbolized by boldface font in text, typically with capital letters Size (r rows x c columns) ERSH 8750: Lecture 2 9
10 Vectors A vector is a matrix where one dimension is equal to size 1 Column vector: a matrix of size Row vector: a matrix of size Vectors are typically written in boldface font text, usually with lowercase letters ERSH 8750: Lecture 2 10
11 Scalars A scalar is just a single number (as we have known before) The name scalar is important: the number scales a vector it can make a vector longer or shorter ERSH 8750: Lecture 2 11
12 Matrix Elements A matrix (or vector) is composed of a set of elements Each element is denoted by its position in the matrix (row and column) For our matrix of data (size 1000 rows and 2 columns), each element is denoted by: The first subscript is the index for the rows: i= 1,,r (= 1000) The second subscript is the index for the columns: j = 1,,c (= 2),, ERSH 8750: Lecture 2 12
13 Matrix Transpose The transpose of a matrix is a reorganization of the matrix by switching the indices for the rows and columns An element in the original matrix is now in the transposed matrix ERSH 8750: Lecture 2 13
14 Types of Matrices Square Matrix: A matrix that has the same number of rows and columns Correlation/covariance matrices are square matrices Diagonal Matrix: A diagonal matrix is a square matrix with non zero diagonal elements ( and zeros on the off diagonal elements ( ): We will use diagonal matrices to form correlation matrices Symmetric Matrix: A symmetric matrix is a square matrix where all elements are reflected across the diagonal ( Correlation and covariance matrices are symmetric matrices ERSH 8750: Lecture 2 14
15 MATRIX ALGEBRA ERSH 8750: Lecture 2 15
16 Moving from Vectors to Matrices A matrix can be thought of as a collection of vectors Matrix operations are vector operations on steroids Matrix algebra defines a set of operations and entities on matrices I will present a version meant to mirror your previous algebra experiences Definitions: Identity matrix Zero vector Ones vector Basic Operations: Addition Subtraction Multiplication Division ERSH 8750: Lecture 2 16
17 Matrix Addition and Subtraction Matrix addition and subtraction are much like vector addition/subtraction Rules: Matrices must be the same size (rows and columns) Method: The new matrix is constructed of element by element addition/subtraction of the previous matrices Order: The order of the matrices (pre and post ) does not matter ERSH 8750: Lecture 2 17
18 Matrix Addition/Subtraction ERSH 8750: Lecture 2 18
19 Matrix Multiplication Matrix multiplication is a bit more complicated The new matrix may be a different size from either of the two multiplying matrices Rules: Pre multiplying matrix must have number of columns equal to the number of rows of the post multiplying matrix Method: Order: The elements of the new matrix consist of the inner (dot) product of the row vectors of the pre multiplying matrix and the column vectors of the post multiplying matrix The order of the matrices (pre and post ) matters ERSH 8750: Lecture 2 19
20 Matrix Multiplication ERSH 8750: Lecture 2 20
21 Multiplication in Statistics Many statistical formulae with summation can be re expressed with matrices A common matrix multiplication form is: Diagonal elements: Off diagonal elements: For our SAT example: 251,797, ,928, ,928, ,862,700 ERSH 8750: Lecture 2 21
22 Identity Matrix The identity matrix is a matrix that, when pre or postmultiplied by another matrix results in the original matrix: The identity matrix is a square matrix that has: Diagonal elements = 1 Off diagonal elements = 0 ERSH 8750: Lecture 2 22
23 Zero Vector The zero vector is a column vector of zeros When pre or post multiplied the result is the zero vector: ERSH 8750: Lecture 2 23
24 Ones Vector A ones vector is a column vector of 1s: The ones vector is useful for calculating statistical terms, such as the mean vector and the covariance matrix Next class we will discuss what these matrices are, how we compute them, and what the mean ERSH 8750: Lecture 2 24
25 Matrix Division : The Inverse Matrix Division from algebra: First: Second: 1 Division in matrices serves a similar role For square and symmetric matrices, an inverse matrix is a matrix that when pre or post multiplied with another matrix produces the identity matrix: Calculation of the matrix inverse is complicated Even computers have a tough time Not all matrices can be inverted Non invertable matrices are called singular matrices In statistics, singular matrices are commonly caused by linear dependencies ERSH 8750: Lecture 2 25
26 The Inverse In data: the inverse shows up constantly in statistics Models which assume some type of (multivariate) normality need an inverse covariance matrix Using our SAT example Our data matrix was size (1000 x 2), which is not invertable However was size (2 x 2) square, and symmetric 251,797, ,928, ,928, ,862,700 The inverse is: ERSH 8750: Lecture 2 26
27 Matrix Algebra Operations For such that exists: ERSH 8750: Lecture 2 27
28 UNIVARIATE STATISTICS ERSH 8750: Lecture 2 28
29 Today s Second Example For today, we will use hypothetical data on the heights and weights of 50 students picked at random from the University of Georgia You can follow along in Mplus using the example syntax and the data file heightweight.csv Weight (in pounds) Height (in inches) ERSH 8750: Lecture 2 29
30 Example Mplus Syntax PLOT command Requests various plots PLOT1 is for histograms and scatterplots OUTPUT command Requests additional output SAMPSTAT: descriptive sample statistics STANDARDIZED: standardized estimates ERSH 8750: Lecture 2 30
31 Random Variables A random variable is a variable whose outcome depends on the result of chance Can be continuous or discrete Example of continuous: represents the height of a person, drawn at random Example of discrete: represents the gender of a person, drawn at random Had a density function that indicates relative frequency of occurrence A density function is a mathematical function that gives a rough picture of the distribution from which a random variable is drawn We will discuss the ways that are used to summarize a random variable ERSH 8750: Lecture 2 31
32 Mean The mean is a measure of central tendency For a single variable, (put into a column vector): ( is a column vector with elements all 1 size N x 1) The sampling distribution of Converges to normal as N increases (central limit theorem) In our data: mean height: (in inches) ERSH 8750: Lecture 2 32
33 Variance The variance is a measure of variability (or spread/range) For a single variable, (put into a column vector): 1 1 Note, if divided by N, maximum likelihood estimate is used MLE is biased, so often N 1 is used If the original data are normally distributed, then the sampling distribution of the variance is: 1 For our data: Variance of height: 21.8(in inches squared) Standard deviation of height: 4.7(in inches) ERSH 8750: Lecture 2 33
34 Linear Combinations of If we create a new variable based on a linear combination of our previous variable: The mean of the linear combination is The variance of the linear combination is In our data: If we wanted to change height from inches to centimeters, we would make the linear combination 2.54 Mean of height in CM: Variance of height in CM 2 : Standard deviation of height in CM: ERSH 8750: Lecture 2 34
35 UNIVARIATE STATISTICAL DISTRIBUTIONS ERSH 8750: Lecture 2 35
36 Univariate Normal Distribution For a continuous random variable (ranging from to ) the univariate normal distribution function is: The shape of the distribution is governed by two parameter: The mean The variance The skewness (lean) and kurtosis (peakedness) are fixed Standard notation for normal distributions is ERSH 8750: Lecture 2 36
37 Univariate Normal Distribution For any value of, gives the height of the curve (relative frequency) ERSH 8750: Lecture 2 37
38 Chi Square Distribution Another frequently used univariate distribution is the Chi Square distribution Previously we mentioned the sampling distribution of the variance followed a Chi Square distribution For a continuous (i.e., quantitative) random variable (ranging from to ), the chi square distribution is given by: is called the gamma function The chi square distribution is governed by one parameter: (the degrees of freedom) The mean is equal to ; the variance is equal to 2 ERSH 8750: Lecture 2 38
39 (Univariate) Chi Square Distribution ERSH 8750: Lecture 2 39
40 Uses of Distributions Statistical models make distributional assumptions on various parameters and/or parts of data These assumptions govern: How models are estimated How inferences are made How missing data may be imputed If data do not follow an assumed distribution, inferences may be inaccurate Sometimes a problem, other times not so much Therefore, it can be helpful to check distributional assumptions prior to (or while) running statistical analyses ERSH 8750: Lecture 2 40
41 BIVARIATE STATISTICS AND DISTRIBUTIONS ERSH 8750: Lecture 2 41
42 Bivariate Statistics Up to this point, we have focused on only one of our variables: height Looked at its marginal distribution (the distribution of height independent of that of weight) Could have looked at weight, marginally Structural equation modeling is about exploring joint distributions of multiple variables How sets of variables relate to each other As such, we will now look at the joint distributions of two variables or in matrix form: (size N x 2) Beginning with two, then moving to anything more than two ERSH 8750: Lecture 2 42
43 Multiple Means: The Mean Vector We can use a vector to describe the set of means for our data Here is a N x 1 vector of 1s The resulting mean vector is a p x 1 vector of means For our data: From Mplus: ERSH 8750: Lecture 2 43
44 Mean Vector: Graphically The mean vector is the center of the distribution of both variables Weight (in pounds) Height (in inches) ERSH 8750: Lecture 2 44
45 Covariance of a Pair of Variables The covariance is a measure of the relatedness Expressed in the product of the units of the two The covariance between height and weight was (in inch pounds) The denominator N is the ML version unbiased is N 1 Because the units of the covariance are difficult to understand, we more commonly describe association (correlation) between two variables with correlation Covariance divided by the product of each variable s standard deviation ERSH 8750: Lecture 2 45
46 Covariance in Mplus Because Mplus defaults to the multivariate normal distribution, covariance is the type of parameter natively estimated by the program Additionally, the MODEL RESULTS are maximum likelihood estimates (MLEs, see next week s lecture) This means the denominator is N, not N 1 ERSH 8750: Lecture 2 46
47 Correlation of a Pair of Variables Correlation is covariance divided by the product of the standard deviation of each variable: The correlation between height and weight was 0.72 Correlation is unitless it only ranges between 1 and 1 If and had variances of 1, the covariance between them would be a correlation Covariance of standardized variables = correlation ERSH 8750: Lecture 2 47
48 Covariance and Correlation in Matrices The covariance matrix (for any number of variables p) is found by: 1 If we take the SDs (the square root of the diagonal of the covariance matrix) and put them into a diagonal matrix,the correlation matrix is found by: 1 1 ERSH 8750: Lecture 2 48
49 Example Covariance Matrix For our data, the covariance matrix was: , The diagonal matrix was: , The correlation matrix was: , , , ERSH 8750: Lecture 2 49
50 Correlation and Covariance in Mplus Mplus does not directly estimate correlation But you can obtain it in two ways with the output section: SAMPSTAT STANDARDIZED Cov is, Mean is ERSH 8750: Lecture 2 50
51 Generalized Variance The determinant of the sample covariance matrix is called the generalized variance It is a measure of spread across all variables Reflecting how much overlap (covariance) in variables occurs Amount of overlap reduces the generalized sample variance The generalized sample variance is: Largest when variables are uncorrelated Zero when variables form a linear dependency In data: The generalized variance is seldom used descriptively, but shows up more frequently in maximum likelihood functions ERSH 8750: Lecture 2 51
52 Total Sample Variance The total sample variance is the sum of the variances of each variable in the sample The sum of the diagonal elements of the sample covariance matrix The trace of the sample covariance matrix tr The total sample variance does not take into consideration the covariances among the variables Will not equal zero if linearly dependency exists In data: The total sample variance is commonly used as the denominator (target) when calculating variance accounted for measures ERSH 8750: Lecture 2 52
53 BIVARIATE NORMAL DISTRIBUTION ERSH 8750: Lecture 2 53
54 Bivariate Normal Distribution The bivariate normal distribution is a statistical distribution for two variables Both variable is normally distributed marginally (by itself) Together, they form a bivariate normal distribution The bivariate normal density provides the relatively frequency of observing any pair of observations, Where ERSH 8750: Lecture 2 54
55 Bivariate Normal Plot #1 Density Surface (3D) Density Surface (2D): Contour Plot ERSH 8750: Lecture 2 55
56 Bivariate Normal Plot #2 Density Surface (3D) Density Surface (2D): Contour Plot ERSH 8750: Lecture 2 56
57 MULTIVARIATE DISTRIBUTIONS (VARIABLES 2) ERSH 8750: Lecture 2 57
58 Multivariate Normal Distribution The multivariate normal distribution is the generalization of the univariate normal distribution to multiple variables The bivariate normal distribution just shown is part of the MVN The MVN provides the relative likelihood of observing all p variables for a subject i simultaneously: The multivariate normal density function is: ERSH 8750: Lecture 2 58
59 The Multivariate Normal Distribution The mean vector is The covariance matrix is The covariance matrix must be non singular (invertable) ERSH 8750: Lecture 2 59
60 Multivariate Normal Notation Standard notation for the multivariate normal distribution of p variables is Our bivariate normal would have been, In data: The multivariate normal distribution serves as the basis for most every statistical technique commonly used in the social and educational sciences General linear models (ANOVA, regression, MANOVA) General linear mixed models (HLM/multilevel models) Factor and structural equation models (EFA, CFA, SEM, path models) Multiple imputation for missing data Simply put, the world of commonly used statistics revolves around the multivariate normal distribution Understanding it is the key to understanding many statistical methods ERSH 8750: Lecture 2 60
61 Bivariate Normal Plot #2 (Multivariate Normal) Density Surface (3D) Density Surface (2D): Contour Plot ERSH 8750: Lecture 2 61
62 Multivariate Normal Properties The multivariate normal distribution has some useful properties that show up in statistical methods If is distributed multivariate normally: 1. Linear combinations of are normally distributed 2. All subsets of are multivariate normally distributed 3. A zero covariance between a pair of variables of implies that the variables are independent 4. Conditional distributions of are multivariate normal ERSH 8750: Lecture 2 62
63 Sampling Distributions of MVN Statistics Just like in univariate statistics, there is a multivariate central limit theorem If the set of observations on variables is multivariate normal or not: The distribution of the mean vector is: The distribution of (covariance matrix) is Wishart (a multivariate chi square) with degrees of freedom ERSH 8750: Lecture 2 63
64 The Wishart Distribution The Wishart distribution is a multivariate chi square distribution Input: S (model predicted covariance matrix) Output: Likelihood value Fixed: (sample value of covariance matrix) In statistics, it appears whenever: Data are assumed multivariate normal Only covariance matrix based inferences are needed Mean vector ignored Mainly: Initial ML factor analysis and structural equation modeling ERSH 8750: Lecture 2 64
65 Sufficient Statistics The sample estimates and are called sufficient statistics All of the information contained in the data can be summarized by these two statistics alone No data is needed for analysis only these statistics Only true when: Data truly follow a multivariate normal distribution No missing data are present ERSH 8750: Lecture 2 65
66 ASSESSING UNIVARIATE NORMALITY IN MPLUS ERSH 8750: Lecture 2 66
67 Assessing Multivariate Normality of Data Up until the last week of class, we will assume our data follow a multivariate normal distribution Data should be multivariate normal then Methods exist for getting a rough picture of whether or not data are multivariate normal I will highlight a graphical method using the Q Q plot Q Q plot: actual data with what normal distribution would predict Mplus will provide Q Q plots for each variable the first step in assessing multivariate normality If one or more variables are not normal, then the whole set cannot be multivariate normal In reality: Determining exact distribution of data is difficult if not impossible Methods that assume multivariate normality are fairly robust to small deviations from that assumption ERSH 8750: Lecture 2 67
68 Q Q Plots and Normal Distribution Overlaid Histograms in Mplus If you specify the PLOT section, you can go to Graphs View Graphs and plot your variables: Q Q Plot: Points should fall onto line Histogram: Distribution should be symmetric, unimodal ERSH 8750: Lecture 2 68
69 When and Why to Check Normality Good news: most methods are fairly robust to violations of normality For homework we will have to check this assumption for each variable If normality does not hold: Check your data for input errors (i.e., 99 = missing) Plot every variable visualization is the most important thing Do not clean your data Use a model with a different link function, if possible (generalized model) ERSH 8750: Lecture 2 69
70 Overall Practice of Data Inspection Overall, checking your data in Mplus is a good thing Helps make sure variables are input properly Makes sure missing codes are properly read You would not believe how often this happens in practice Compare easily obtainable statistics Sample means [mean vector] Covariances Variances If values do not match those of another program, check your data for issues in importing/exporting ERSH 8750: Lecture 2 70
71 WRAPPING UP ERSH 8750: Lecture 2 71
72 Concluding Remarks Matrix algebra plays a role in concisely depicting structural equation models Many SEM texts and articles are described by The univariate and multivariate normal distributions serve as the backbone of the most frequently used statistical procedures They are very robust and very useful Understanding their form and function will help you learn about a whole host of statistical routines immediately This course is essentially a course based in normal distributions ERSH 8750: Lecture 2 72
A Introduction to Matrix Algebra and the Multivariate Normal Distribution
A Introduction to Matrix Algebra and the Multivariate Normal Distribution PRE 905: Multivariate Analysis Spring 2014 Lecture 6 PRE 905: Lecture 7 Matrix Algebra and the MVN Distribution Today s Class An
More informationAn Introduction to Matrix Algebra
An Introduction to Matrix Algebra EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #8 EPSY 905: Matrix Algebra In This Lecture An introduction to matrix algebra Ø Scalars, vectors, and matrices
More informationVectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
More informationUnivariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation
Univariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation PRE 905: Multivariate Analysis Spring 2014 Lecture 4 Today s Class The building blocks: The basics of mathematical
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 1: August 22, 2012
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model EPSY 905: Multivariate Analysis Lecture 1 20 January 2016 EPSY 905: Lecture 1 -
More informationStatistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationUnivariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation
Univariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture 3: February 10, 2016 Today s Class The building blocks: The
More informationMultilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do
More informationReview of Multiple Regression
Ronald H. Heck 1 Let s begin with a little review of multiple regression this week. Linear models [e.g., correlation, t-tests, analysis of variance (ANOVA), multiple regression, path analysis, multivariate
More informationMaximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood
Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood PRE 906: Structural Equation Modeling Lecture #3 February 4, 2015 PRE 906, SEM: Estimation Today s Class An
More informationMATH 320, WEEK 7: Matrices, Matrix Operations
MATH 320, WEEK 7: Matrices, Matrix Operations 1 Matrices We have introduced ourselves to the notion of the grid-like coefficient matrix as a short-hand coefficient place-keeper for performing Gaussian
More informationGlossary. The ISI glossary of statistical terms provides definitions in a number of different languages:
Glossary The ISI glossary of statistical terms provides definitions in a number of different languages: http://isi.cbs.nl/glossary/index.htm Adjusted r 2 Adjusted R squared measures the proportion of the
More informationMean Vector Inferences
Mean Vector Inferences Lecture 5 September 21, 2005 Multivariate Analysis Lecture #5-9/21/2005 Slide 1 of 34 Today s Lecture Inferences about a Mean Vector (Chapter 5). Univariate versions of mean vector
More informationIntroduction to Confirmatory Factor Analysis
Introduction to Confirmatory Factor Analysis Multivariate Methods in Education ERSH 8350 Lecture #12 November 16, 2011 ERSH 8350: Lecture 12 Today s Class An Introduction to: Confirmatory Factor Analysis
More informationAn Introduction to Path Analysis
An Introduction to Path Analysis PRE 905: Multivariate Analysis Lecture 10: April 15, 2014 PRE 905: Lecture 10 Path Analysis Today s Lecture Path analysis starting with multivariate regression then arriving
More informationReview from Bootcamp: Linear Algebra
Review from Bootcamp: Linear Algebra D. Alex Hughes October 27, 2014 1 Properties of Estimators 2 Linear Algebra Addition and Subtraction Transpose Multiplication Cross Product Trace 3 Special Matrices
More informationBusiness Analytics and Data Mining Modeling Using R Prof. Gaurav Dixit Department of Management Studies Indian Institute of Technology, Roorkee
Business Analytics and Data Mining Modeling Using R Prof. Gaurav Dixit Department of Management Studies Indian Institute of Technology, Roorkee Lecture - 04 Basic Statistics Part-1 (Refer Slide Time: 00:33)
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Introduction Edps/Psych/Stat/ 584 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c Board of Trustees,
More informationMatrices and Vectors
Matrices and Vectors James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 11, 2013 Outline 1 Matrices and Vectors 2 Vector Details 3 Matrix
More informationPrincipal Components Analysis
Principal Components Analysis Lecture 9 August 2, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #9-8/2/2011 Slide 1 of 54 Today s Lecture Principal Components Analysis
More informationCourse Number 432/433 Title Algebra II (A & B) H Grade # of Days 120
Whitman-Hanson Regional High School provides all students with a high- quality education in order to develop reflective, concerned citizens and contributing members of the global community. Course Number
More informationLinear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.
POLI 7 - Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes - Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems
More informationPath Analysis. PRE 906: Structural Equation Modeling Lecture #5 February 18, PRE 906, SEM: Lecture 5 - Path Analysis
Path Analysis PRE 906: Structural Equation Modeling Lecture #5 February 18, 2015 PRE 906, SEM: Lecture 5 - Path Analysis Key Questions for Today s Lecture What distinguishes path models from multivariate
More informationIntroduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation. EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016
Introduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016 EPSY 905: Intro to Bayesian and MCMC Today s Class An
More informationHistograms, Central Tendency, and Variability
The Economist, September 6, 214 1 Histograms, Central Tendency, and Variability Lecture 2 Reading: Sections 5 5.6 Includes ALL margin notes and boxes: For Example, Guided Example, Notation Alert, Just
More informationLinear Algebra V = T = ( 4 3 ).
Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional
More informationAn Introduction to Mplus and Path Analysis
An Introduction to Mplus and Path Analysis PSYC 943: Fundamentals of Multivariate Modeling Lecture 10: October 30, 2013 PSYC 943: Lecture 10 Today s Lecture Path analysis starting with multivariate regression
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationA Primer on Statistical Inference using Maximum Likelihood
A Primer on Statistical Inference using Maximum Likelihood November 3, 2017 1 Inference via Maximum Likelihood Statistical inference is the process of using observed data to estimate features of the population.
More informationClass Introduction and Overview; Review of ANOVA, Regression, and Psychological Measurement
Class Introduction and Overview; Review of ANOVA, Regression, and Psychological Measurement Introduction to Structural Equation Modeling Lecture #1 January 11, 2012 ERSH 8750: Lecture 1 Today s Class Introduction
More information[ Here 21 is the dot product of (3, 1, 2, 5) with (2, 3, 1, 2), and 31 is the dot product of
. Matrices A matrix is any rectangular array of numbers. For example 3 5 6 4 8 3 3 is 3 4 matrix, i.e. a rectangular array of numbers with three rows four columns. We usually use capital letters for matrices,
More information1 A Review of Correlation and Regression
1 A Review of Correlation and Regression SW, Chapter 12 Suppose we select n = 10 persons from the population of college seniors who plan to take the MCAT exam. Each takes the test, is coached, and then
More informationA User's Guide To Principal Components
A User's Guide To Principal Components J. EDWARD JACKSON A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Brisbane Toronto Singapore Contents Preface Introduction 1. Getting
More informationPsych Jan. 5, 2005
Psych 124 1 Wee 1: Introductory Notes on Variables and Probability Distributions (1/5/05) (Reading: Aron & Aron, Chaps. 1, 14, and this Handout.) All handouts are available outside Mija s office. Lecture
More informationMath 360 Linear Algebra Fall Class Notes. a a a a a a. a a a
Math 360 Linear Algebra Fall 2008 9-10-08 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are
More information401 Review. 6. Power analysis for one/two-sample hypothesis tests and for correlation analysis.
401 Review Major topics of the course 1. Univariate analysis 2. Bivariate analysis 3. Simple linear regression 4. Linear algebra 5. Multiple regression analysis Major analysis methods 1. Graphical analysis
More informationWe use the overhead arrow to denote a column vector, i.e., a number with a direction. For example, in three-space, we write
1 MATH FACTS 11 Vectors 111 Definition We use the overhead arrow to denote a column vector, ie, a number with a direction For example, in three-space, we write The elements of a vector have a graphical
More informationStat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables.
Stat 260 - Lecture 20 Recap of Last Class Last class we introduced the covariance and correlation between two jointly distributed random variables. Today: We will introduce the idea of a statistic and
More informationLINEAR SYSTEMS, MATRICES, AND VECTORS
ELEMENTARY LINEAR ALGEBRA WORKBOOK CREATED BY SHANNON MARTIN MYERS LINEAR SYSTEMS, MATRICES, AND VECTORS Now that I ve been teaching Linear Algebra for a few years, I thought it would be great to integrate
More informationStructural Equation Modeling and Confirmatory Factor Analysis. Types of Variables
/4/04 Structural Equation Modeling and Confirmatory Factor Analysis Advanced Statistics for Researchers Session 3 Dr. Chris Rakes Website: http://csrakes.yolasite.com Email: Rakes@umbc.edu Twitter: @RakesChris
More informationLesson U2.1 Study Guide
Lesson U2.1 Study Guide Sunday, June 3, 2018 2:05 PM Matrix operations, The Inverse of a Matrix and Matrix Factorization Reading: Lay, Sections 2.1, 2.2, 2.3 and 2.5 (about 24 pages). MyMathLab: Lesson
More informationRegression With a Categorical Independent Variable
Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent
More informationModule 03 Lecture 14 Inferential Statistics ANOVA and TOI
Introduction of Data Analytics Prof. Nandan Sudarsanam and Prof. B Ravindran Department of Management Studies and Department of Computer Science and Engineering Indian Institute of Technology, Madras Module
More informationNumerical Methods Lecture 2 Simultaneous Equations
Numerical Methods Lecture 2 Simultaneous Equations Topics: matrix operations solving systems of equations pages 58-62 are a repeat of matrix notes. New material begins on page 63. Matrix operations: Mathcad
More informationa11 a A = : a 21 a 22
Matrices The study of linear systems is facilitated by introducing matrices. Matrix theory provides a convenient language and notation to express many of the ideas concisely, and complicated formulas are
More informationObservations Homework Checkpoint quizzes Chapter assessments (Possibly Projects) Blocks of Algebra
September The Building Blocks of Algebra Rates, Patterns and Problem Solving Variables and Expressions The Commutative and Associative Properties The Distributive Property Equivalent Expressions Seeing
More informationEPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7
Introduction to Generalized Univariate Models: Models for Binary Outcomes EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7 EPSY 905: Intro to Generalized In This Lecture A short review
More informationA Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
More informationModern Algebra Prof. Manindra Agrawal Department of Computer Science and Engineering Indian Institute of Technology, Kanpur
Modern Algebra Prof. Manindra Agrawal Department of Computer Science and Engineering Indian Institute of Technology, Kanpur Lecture 02 Groups: Subgroups and homomorphism (Refer Slide Time: 00:13) We looked
More informationLecture Notes Part 2: Matrix Algebra
17.874 Lecture Notes Part 2: Matrix Algebra 2. Matrix Algebra 2.1. Introduction: Design Matrices and Data Matrices Matrices are arrays of numbers. We encounter them in statistics in at least three di erent
More informationReview of Statistics 101
Review of Statistics 101 We review some important themes from the course 1. Introduction Statistics- Set of methods for collecting/analyzing data (the art and science of learning from data). Provides methods
More information36-309/749 Experimental Design for Behavioral and Social Sciences. Dec 1, 2015 Lecture 11: Mixed Models (HLMs)
36-309/749 Experimental Design for Behavioral and Social Sciences Dec 1, 2015 Lecture 11: Mixed Models (HLMs) Independent Errors Assumption An error is the deviation of an individual observed outcome (DV)
More informationRepeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models
Repeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models EPSY 905: Multivariate Analysis Spring 2016 Lecture #12 April 20, 2016 EPSY 905: RM ANOVA, MANOVA, and Mixed Models
More informationDesigning Information Devices and Systems I Fall 2017 Official Lecture Notes Note 2
EECS 6A Designing Information Devices and Systems I Fall 07 Official Lecture Notes Note Introduction Previously, we introduced vectors and matrices as a way of writing systems of linear equations more
More informationGetting Started with Communications Engineering. Rows first, columns second. Remember that. R then C. 1
1 Rows first, columns second. Remember that. R then C. 1 A matrix is a set of real or complex numbers arranged in a rectangular array. They can be any size and shape (provided they are rectangular). A
More information6348 Final, Fall 14. Closed book, closed notes, no electronic devices. Points (out of 200) in parentheses.
6348 Final, Fall 14. Closed book, closed notes, no electronic devices. Points (out of 200) in parentheses. 0 11 1 1.(5) Give the result of the following matrix multiplication: 1 10 1 Solution: 0 1 1 2
More informationCHAPTER 9 EXAMPLES: MULTILEVEL MODELING WITH COMPLEX SURVEY DATA
Examples: Multilevel Modeling With Complex Survey Data CHAPTER 9 EXAMPLES: MULTILEVEL MODELING WITH COMPLEX SURVEY DATA Complex survey data refers to data obtained by stratification, cluster sampling and/or
More informationExploratory Factor Analysis and Principal Component Analysis
Exploratory Factor Analysis and Principal Component Analysis Today s Topics: What are EFA and PCA for? Planning a factor analytic study Analysis steps: Extraction methods How many factors Rotation and
More informationDesigning Information Devices and Systems I Spring 2019 Lecture Notes Note 2
EECS 6A Designing Information Devices and Systems I Spring 9 Lecture Notes Note Vectors and Matrices In the previous note, we introduced vectors and matrices as a way of writing systems of linear equations
More informationChapter 1 Statistical Inference
Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations
More informationPreliminary Statistics course. Lecture 1: Descriptive Statistics
Preliminary Statistics course Lecture 1: Descriptive Statistics Rory Macqueen (rm43@soas.ac.uk), September 2015 Organisational Sessions: 16-21 Sep. 10.00-13.00, V111 22-23 Sep. 15.00-18.00, V111 24 Sep.
More informationB553 Lecture 5: Matrix Algebra Review
B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations
More informationTHE SUMMATION NOTATION Ʃ
THE SUMMATION NOTATION Ʃ Single Subscript Notation Most of the calculations we perform in statistics are repetitive operations on lists of numbers. For example, we compute the sum of a set of numbers,
More informationMadison County Schools Suggested 4 th Grade Math Pacing Guide,
Madison County Schools Suggested 4 th Grade Math Pacing Guide, 2016 2017 The following Standards have changes from the 2015-16 MS College- and Career-Readiness Standards: Significant Changes (ex: change
More informationNotes on Mathematics Groups
EPGY Singapore Quantum Mechanics: 2007 Notes on Mathematics Groups A group, G, is defined is a set of elements G and a binary operation on G; one of the elements of G has particularly special properties
More informationHigh School Algebra I Scope and Sequence by Timothy D. Kanold
High School Algebra I Scope and Sequence by Timothy D. Kanold First Semester 77 Instructional days Unit 1: Understanding Quantities and Expressions (10 Instructional days) N-Q Quantities Reason quantitatively
More informationESS 265 Spring Quarter 2005 Time Series Analysis: Linear Regression
ESS 265 Spring Quarter 2005 Time Series Analysis: Linear Regression Lecture 11 May 10, 2005 Multivariant Regression A multi-variant relation between a dependent variable y and several independent variables
More informationVectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1. x 2. x =
Linear Algebra Review Vectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1 x x = 2. x n Vectors of up to three dimensions are easy to diagram.
More informationy response variable x 1, x 2,, x k -- a set of explanatory variables
11. Multiple Regression and Correlation y response variable x 1, x 2,, x k -- a set of explanatory variables In this chapter, all variables are assumed to be quantitative. Chapters 12-14 show how to incorporate
More informationLecture 11 Multiple Linear Regression
Lecture 11 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 11-1 Topic Overview Review: Multiple Linear Regression (MLR) Computer Science Case Study 11-2 Multiple Regression
More informationDesigning Information Devices and Systems I Fall 2018 Lecture Notes Note 2
EECS 6A Designing Information Devices and Systems I Fall 08 Lecture Notes Note Vectors and Matrices In the previous note, we introduced vectors and matrices as a way of writing systems of linear equations
More informationLanguage as a Stochastic Process
CS769 Spring 2010 Advanced Natural Language Processing Language as a Stochastic Process Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu 1 Basic Statistics for NLP Pick an arbitrary letter x at random from any
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationHow can you solve a multistep. How can you solve an absolute value equation? How can you solve and absolute value. inequality?
WDHS Curriculum Map Course: Algebra 1 June 2015 Time Interval/ Content Standards/ Strands Essential Questions Skills Assessment Unit 1 Transfer Goal: Recognize and solve practical or theoretical problems
More informationAlgebra , Martin-Gay
A Correlation of Algebra 1 2016, to the Common Core State Standards for Mathematics - Algebra I Introduction This document demonstrates how Pearson s High School Series by Elayn, 2016, meets the standards
More informationAP Statistics Cumulative AP Exam Study Guide
AP Statistics Cumulative AP Eam Study Guide Chapters & 3 - Graphs Statistics the science of collecting, analyzing, and drawing conclusions from data. Descriptive methods of organizing and summarizing statistics
More informationStat 206: Sampling theory, sample moments, mahalanobis
Stat 206: Sampling theory, sample moments, mahalanobis topology James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Notation My notation is different from the book s. This is partly because
More informationLesson 1: Inverses of Functions Lesson 2: Graphs of Polynomial Functions Lesson 3: 3-Dimensional Space
Table of Contents Introduction.............................................................. v Unit 1: Modeling with Matrices... 1 Lesson 2: Solving Problems Using Matrices.................................
More informationMath 123, Week 2: Matrix Operations, Inverses
Math 23, Week 2: Matrix Operations, Inverses Section : Matrices We have introduced ourselves to the grid-like coefficient matrix when performing Gaussian elimination We now formally define general matrices
More informationContents. Acknowledgments. xix
Table of Preface Acknowledgments page xv xix 1 Introduction 1 The Role of the Computer in Data Analysis 1 Statistics: Descriptive and Inferential 2 Variables and Constants 3 The Measurement of Variables
More informationMadison County Schools Suggested 3 rd Grade Math Pacing Guide,
Madison County Schools Suggested 3 rd Grade Math Pacing Guide, 2016 2017 The following Standards have changes from the 2015-16 MS College- and Career-Readiness Standards: Significant Changes (ex: change
More informationCSCI 239 Discrete Structures of Computer Science Lab 6 Vectors and Matrices
CSCI 239 Discrete Structures of Computer Science Lab 6 Vectors and Matrices This lab consists of exercises on real-valued vectors and matrices. Most of the exercises will required pencil and paper. Put
More informationModule 1: Equations and Inequalities (30 days) Solving Equations: (10 Days) (10 Days)
Module 1: Equations and Inequalities (30 days) Word Problems Literal Equations (Scientific Applications) Justifying solutions Algebraic Proofs Represent constraints by equations and inequalities Graphing
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationMaths for Signals and Systems Linear Algebra for Engineering Applications
Maths for Signals and Systems Linear Algebra for Engineering Applications Lectures 1-2, Tuesday 11 th October 2016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE LONDON
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationIntroduction to Factor Analysis
to Factor Analysis Lecture 10 August 2, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #10-8/3/2011 Slide 1 of 55 Today s Lecture Factor Analysis Today s Lecture Exploratory
More informationMilford Public Schools Curriculum. Department: Mathematics Course Name: Algebra 1 Level 2
Milford Public Schools Curriculum Department: Mathematics Course Name: Algebra 1 Level 2 UNIT 1 Unit Title: Intro to Functions and Exponential Expressions Unit Description: Students explore the main functions
More informationA Re-Introduction to General Linear Models (GLM)
A Re-Introduction to General Linear Models (GLM) Today s Class: You do know the GLM Estimation (where the numbers in the output come from): From least squares to restricted maximum likelihood (REML) Reviewing
More informationMatrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
More informationSPSS LAB FILE 1
SPSS LAB FILE www.mcdtu.wordpress.com 1 www.mcdtu.wordpress.com 2 www.mcdtu.wordpress.com 3 OBJECTIVE 1: Transporation of Data Set to SPSS Editor INPUTS: Files: group1.xlsx, group1.txt PROCEDURE FOLLOWED:
More informationPOLI270 - Linear Algebra
POLI7 - Linear Algebra Septemer 8th Basics a x + a x +... + a n x n b () is the linear form where a, b are parameters and x n are variables. For a given equation such as x +x you only need a variable and
More informationAPPENDIX: MATHEMATICAL INDUCTION AND OTHER FORMS OF PROOF
ELEMENTARY LINEAR ALGEBRA WORKBOOK/FOR USE WITH RON LARSON S TEXTBOOK ELEMENTARY LINEAR ALGEBRA CREATED BY SHANNON MARTIN MYERS APPENDIX: MATHEMATICAL INDUCTION AND OTHER FORMS OF PROOF When you are done
More informationDeciphering Math Notation. Billy Skorupski Associate Professor, School of Education
Deciphering Math Notation Billy Skorupski Associate Professor, School of Education Agenda General overview of data, variables Greek and Roman characters in math and statistics Parameters vs. Statistics
More informationAlgebra 1. Mathematics Course Syllabus
Mathematics Algebra 1 2017 2018 Course Syllabus Prerequisites: Successful completion of Math 8 or Foundations for Algebra Credits: 1.0 Math, Merit The fundamental purpose of this course is to formalize
More informationImage Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More informationI&C 6N. Computational Linear Algebra
I&C 6N Computational Linear Algebra 1 Lecture 1: Scalars and Vectors What is a scalar? Computer representation of a scalar Scalar Equality Scalar Operations Addition and Multiplication What is a vector?
More information