Math Camp. Justin Grimmer. Associate Professor Department of Political Science Stanford University. September 9th, 2016
|
|
|
- Beverley Hoover
- 5 years ago
- Views:
Transcription
1 Math Camp Justin Grimmer Associate Professor Department of Political Science Stanford University September 9th, 2016 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
2 Where We ve Been, Where We re Going Calculus: Analyze behavior of functions on real line - Convergence - Differentiation - Integration Linear Algebra - Data stored in matrices - Higher dimensional spaces - complex world, condition on many factors - flood of big data, store in many dimensions - Linear Algebra: - Algebra of matrices - Geometry of high dimensional space - Calculus (multivariable) in many dimensions Very important for regression(!!!!) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
3 Points + Vectors. - A point in R π - e - An ordered pair in R 2 = R R - (1, 2) - (0, 0) - (π, e) - An ordered triple in R 3 = R R R - (3.1, 4.5, ) - An ordered n-tuple in R n = R R... R - (a 1, a 2,..., a n ) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
4 Points and Vectors Definition A point x R n is an ordered n-tuple, (x 1, x 2,..., x n ). The vector x R n is the arrow pointing from the origin (0, 0,..., 0) to x. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
5 One Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
6 One Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
7 One Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
8 Two Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
9 Two Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
10 Two Dimensional Example Justin Grimmer (Stanford University) Methodology I September 9th, / 61
11 Three Dimensional Example - (Latitude, Longitude, Elevation) - (1, 2, 3) - (0, 1, 0) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
12 N-Dimensional Example - Individual campaign donation records x = (1000, 0, 10, 50, 15, 4, 0, 0, 0,..., ) - Counties have proportion of vote for Obama y = (0.8, 0.5, 0.6,..., 0.2) - Run experiment, assess feeling thermometer of elected official t = (0, 100, 50, 70, 80,..., 100) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
13 Arithmetic with Vectors Definition Suppose u and v are vectors u, v R n, u = (u 1, u 2, u 3,..., u n ) v = (v 1, v 2, v 3,..., v n ) We will say u = v if u 1 = v 1, u 2 = v 2,..., u n = v n Define the sum of u + v as u + v = (u 1 + v 1, u 2 + v 2, u 3 + v 3,..., u n + v n ) Suppose k R. We will call k a scalar. Define ku as the scalar product ku = (ku 1, ku 2,..., ku n ) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
14 Examples Suppose: u = (1, 2, 3, 4, 5) v = (1, 1, 1, 1, 1) k = 2 Then, u + v = (1 + 1, 2 + 1, 3 + 1, 4 + 1, 5 + 1) = (2, 3, 4, 5, 6) ku = (2 1, 2 2, 2 3, 2 4, 2 5) = (2, 4, 6, 8, 10) kv = (2 1, 2 1, 2 1, 2 1, 2 1) = (2, 2, 2, 2, 2) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
15 Properties of Arithmetic Challenge Proofs we can do these! Theorem Suppose that u, v, w R n and k and l are scalars. a) u + v = v + u Justin Grimmer (Stanford University) Methodology I September 9th, / 61
16 Properties of Arithmetic Challenge Proofs we can do these! Theorem Suppose that u, v, w R n and k and l are scalars. a) u + v = v + u Proof. u + v = (u 1 + v 1, u 2 + v 2,..., u n + v n ) = (v 1 + u 1, v 2 + u 2,..., v n + u n ) = v + u Justin Grimmer (Stanford University) Methodology I September 9th, / 61
17 Properties of Arithmetic Challenge Proofs we can do these! Theorem Suppose that u, v, w R n and k and l are scalars. b) u + 0 = 0 + u = u Justin Grimmer (Stanford University) Methodology I September 9th, / 61
18 Properties of Arithmetic Challenge Proofs we can do these! Theorem Suppose that u, v, w R n and k and l are scalars. b) u + 0 = 0 + u = u Proof. u + 0 = (u 1 + 0, u 2 + 0,..., u n + 0) = (0 + u 1, 0 + u 2,..., 0 + u n ) = 0 + u = (u 1, u 2,..., u n ) = u Justin Grimmer (Stanford University) Methodology I September 9th, / 61
19 Properties of Arithmetic Challenge Proofs we can do these! Theorem Suppose that u, v, w R n and k and l are scalars. c) (l + k)u = l(u) + k(u) Proof. How can we show this? Justin Grimmer (Stanford University) Methodology I September 9th, / 61
20 Challenge Proofs - Show that 1u = u - Show that u + 1u = 0 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
21 Inner Product Definition Suppose u R n and v R n then define u v, u v = u 1 v 1 + u 2 v u n v n N = u i v i i=1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
22 Examples Suppose u = (1, 2, 3) and v = (2, 3, 1). Then, u v = = = 11 Suppose y = (y 1, y 2,..., y N ) and 1 = (1, 1, 1,..., 1). Then, y 1 = y 1 + y y n n = i=1 y i Justin Grimmer (Stanford University) Methodology I September 9th, / 61
23 R Code Create a vector in R Justin Grimmer (Stanford University) Methodology I September 9th, / 61
24 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
25 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() Justin Grimmer (Stanford University) Methodology I September 9th, / 61
26 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
27 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
28 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
29 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
30 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
31 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
32 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
33 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
34 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 add [1] Justin Grimmer (Stanford University) Methodology I September 9th, / 61
35 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 add [1] scalar<- 10 *x1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
36 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 add [1] scalar<- 10 *x1 scalar [1] Justin Grimmer (Stanford University) Methodology I September 9th, / 61
37 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 add [1] scalar<- 10 *x1 scalar [1] output<- x1 %*% x2 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
38 R Code Create a vector in R vec <- c(1, 2, 3, 4, 5) vec<- c() vec[1]<- 1 vec[2]<- 2 vec[3]<- 3 vec[4]<- 4 vec[5]<- 5 x1<- c(2, 2, 3, 2) x2<- c(5, 3, 1, 3) add <- x1 + x2 add [1] scalar<- 10 *x1 scalar [1] output<- x1 %*% x2 output [,1] [1,] 25 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
39 Challenge Problems - Suppose v = (1, 4, 1, 4) and w = (4, 1, 4, 1). Calculate: v w - Prove v w = w v - Super hard: prove v v 0 and v v = 0 if and only if v = 0. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
40 Vector Length x_2 x_1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
41 Vector Length x_2 - Pythogorean Theorem: Side with length a a x_1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
42 Vector Length x_2 - Pythogorean Theorem: Side with length a - Side with length b and right triangle b a x_1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
43 Vector Length x_2 c = (a^2 + b^2)^(1/2) b - Pythogorean Theorem: Side with length a - Side with length b and right triangle - c = a 2 + b 2 a x_1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
44 Vector Length x_2 c = (a^2 + b^2)^(1/2) b - Pythogorean Theorem: Side with length a - Side with length b and right triangle - c = a 2 + b 2 - This is generally true a x_1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
45 Vector Length Definition Suppose v R n. Then, we will define its length as v = (v v) 1/2 = (v v v v 2 n ) 1/2 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
46 Calculating a Length Example 1: suppose x = (1, 1, 1). x = (x x) 1/2 = ( ) 1/2 = 3 Example 2: R code for length function len.vec<- function(x) { p1< sqrt(x% %x) return(p1) } x <- c(1,1,1) len.vec(x) [,1] [1,] Justin Grimmer (Stanford University) Methodology I September 9th, / 61
47 Coding Problem Let s calculate the length of some vectors - Write a function to assess the length of a vector. - Use it to calculate the length of: - y<- c(10, 20, 30, 40) - x<- seq(1, 1000*pi, len=1000) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
48 Texts in Space Justin Grimmer (Stanford University) Methodology I September 9th, / 61
49 Texts in Space Doc1 = (1, 1, 3,..., 5) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
50 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
51 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Justin Grimmer (Stanford University) Methodology I September 9th, / 61
52 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Provides many operations that will be useful Justin Grimmer (Stanford University) Methodology I September 9th, / 61
53 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Provides many operations that will be useful Inner Product between documents: Justin Grimmer (Stanford University) Methodology I September 9th, / 61
54 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Provides many operations that will be useful Inner Product between documents: Doc1 Doc2 = (1, 1, 3,..., 5) (2, 0, 0,..., 1) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
55 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Provides many operations that will be useful Inner Product between documents: Doc1 Doc2 = (1, 1, 3,..., 5) (2, 0, 0,..., 1) = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
56 Texts in Space Doc1 = (1, 1, 3,..., 5) Doc2 = (2, 0, 0,..., 1) Doc1, Doc2 R M Provides many operations that will be useful Inner Product between documents: Doc1 Doc2 = (1, 1, 3,..., 5) (2, 0, 0,..., 1) = = 7 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
57 Length of document: Justin Grimmer (Stanford University) Methodology I September 9th, / 61
58 Length of document: Doc1 Doc1 Doc1 = (1, 1, 3,..., 5) (1, 1, 3,..., 5) = = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
59 Length of document: Doc1 Doc1 Doc1 = (1, 1, 3,..., 5) (1, 1, 3,..., 5) = = 6 Cosine of the angle between documents: Justin Grimmer (Stanford University) Methodology I September 9th, / 61
60 Length of document: Doc1 Doc1 Doc1 = (1, 1, 3,..., 5) (1, 1, 3,..., 5) = = 6 Cosine of the angle between documents: cos θ ( Doc1 Doc1 = = 0.52 ) ( ) Doc2 Doc2 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
61 Measuring Similarity Documents in space measure similarity/dissimilarity Justin Grimmer (Stanford University) Methodology I September 9th, / 61
62 Measuring Similarity Documents in space measure similarity/dissimilarity What properties should similarity measure have? Justin Grimmer (Stanford University) Methodology I September 9th, / 61
63 Measuring Similarity Documents in space measure similarity/dissimilarity What properties should similarity measure have? - Maximum: document with itself Justin Grimmer (Stanford University) Methodology I September 9th, / 61
64 Measuring Similarity Documents in space measure similarity/dissimilarity What properties should similarity measure have? - Maximum: document with itself - Minimum: documents have no words in common (orthogonal ) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
65 Measuring Similarity Documents in space measure similarity/dissimilarity What properties should similarity measure have? - Maximum: document with itself - Minimum: documents have no words in common (orthogonal ) - Increasing when more of same words used Justin Grimmer (Stanford University) Methodology I September 9th, / 61
66 Measuring Similarity Documents in space measure similarity/dissimilarity What properties should similarity measure have? - Maximum: document with itself - Minimum: documents have no words in common (orthogonal ) - Increasing when more of same words used -? s(a, b) = s(b, a). Justin Grimmer (Stanford University) Methodology I September 9th, / 61
67 Measuring Similarity Word Word 1 Measure 1: Inner product Justin Grimmer (Stanford University) Methodology I September 9th, / 61
68 Measuring Similarity Word Word 1 Measure 1: Inner product (2, 1) (1, 4) = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
69 Word Word 1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
70 Word Word 1 Problem(?): length dependent Justin Grimmer (Stanford University) Methodology I September 9th, / 61
71 Word Word 1 Problem(?): length dependent (4, 2) (1, 4) = 12 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
72 Word θ Word 1 Problem(?): length dependent (4, 2) (1, 4) = 12 a b = a b cos θ Justin Grimmer (Stanford University) Methodology I September 9th, / 61
73 Cosine Similarity cos θ: removes document length from similarity measure Justin Grimmer (Stanford University) Methodology I September 9th, / 61
74 Cosine Similarity cos θ: removes document length from similarity measure cos θ = ( ) ( ) a b a b Justin Grimmer (Stanford University) Methodology I September 9th, / 61
75 Cosine Similarity cos θ: removes document length from similarity measure cos θ = (4, 2) (4, 2) ( ) ( ) a b a b = (0.89, 0.45) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
76 Cosine Similarity cos θ: removes document length from similarity measure cos θ = (4, 2) (4, 2) (2, 1) (2, 1) ( ) ( ) a b a b = (0.89, 0.45) = (0.89, 0.45) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
77 Cosine Similarity cos θ: removes document length from similarity measure cos θ = (4, 2) (4, 2) (2, 1) (2, 1) (1, 4) (1, 4) ( ) ( ) a b a b = (0.89, 0.45) = (0.89, 0.45) = (0.24, 0.97) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
78 Cosine Similarity cos θ: removes document length from similarity measure cos θ = (4, 2) (4, 2) (2, 1) (2, 1) (1, 4) (1, 4) ( ) ( ) a b a b = (0.89, 0.45) = (0.89, 0.45) = (0.24, 0.97) (0.89, 0.45) (0.24, 0.97) = 0.65 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
79 Cosine Similarity Word θ Word 1 cos θ: removes document length from similarity measure Justin Grimmer (Stanford University) Methodology I September 9th, / 61
80 Cosine Similarity Word θ Word 1 cos θ: removes document length from similarity measure Project onto Hypersphere Justin Grimmer (Stanford University) Methodology I September 9th, / 61
81 Cosine Similarity Word θ Word 1 cos θ: removes document length from similarity measure Project onto Hypersphere cos θ Inverse distance on Hypersphere Justin Grimmer (Stanford University) Methodology I September 9th, / 61
82 Cosine Similarity Word θ Word 1 cos θ: removes document length from similarity measure Project onto Hypersphere cos θ Inverse distance on Hypersphere von Mises Fisher distribution : distribution on sphere surface Justin Grimmer (Stanford University) Methodology I September 9th, / 61
83 Matrices Definition A Matrix is a rectangular array of numbers A = a 11 a a 1n a 21 a a 2n a m1 a m2... a mn If A has m rows n columns we will say that A is an m n matrix. Suppose X and Y are m n matrices. Then X = Y if x ij = y ij for all i and j Justin Grimmer (Stanford University) Methodology I September 9th, / 61
84 Simple Examples I = If I is an n n matrix we will call an identity matrix. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
85 Simple Examples X = ( 1 2 ) X is an 2 3 matrix Justin Grimmer (Stanford University) Methodology I September 9th, / 61
86 Matrix Algebra Definition Suppose X and Y are m n matrices. Then define x 11 x x 1n y 11 y y 1n x 21 x x 2n X + Y = y 21 y y 2n x m1 x m2... x mn y m1 y m2... y mn x 11 + y 11 x 12 + y x 1n + y 1n x 21 + y 21 x 22 + y x 2n + y 2n = x m1 + y m1 x m2 + y m2... x mn + y mn Justin Grimmer (Stanford University) Methodology I September 9th, / 61
87 Matrix Algebra Definition Suppose X is an m n matrix and k R. Then, kx 11 kx kx 1n kx 21 kx kx 2n kx = kx m1 kx m2... kx mn Prove theorems about this tonight Justin Grimmer (Stanford University) Methodology I September 9th, / 61
88 R Code Using matrix command mat1<- matrix(na, nrow=3, ncol=2) ## Creating matrix mat1[1,1]<- 1 mat1[1,2]<- 2 mat1[2,1]<- 1 mat1[2,2]<- 4 mat1[3,1]<- exp(1) mat1[3,2]<- 4 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
89 R Code Using rbind r1<- c(1, 2) r2<- c(1, 4) r3<- c(exp(1), 4) mat1<- rbind(r1, r2, r3) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
90 R Code Using cbind c1<- c(1, 1, exp(1) ) c2<- c(2, 4, 4) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
91 R Code dim(mat1) [1] 3 2 mat1 + mat1 [,1] [,2] [1,] [2,] [3,] Justin Grimmer (Stanford University) Methodology I September 9th, / 61
92 R Code What if the matrices are of different dimension mat1<- matrix(1, nrow=3, ncol=2) mat2<- matrix(2, nrow=10, ncol=3) mat1 + mat2 Error in mat1 + mat2 : non-conformable arrays Justin Grimmer (Stanford University) Methodology I September 9th, / 61
93 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix Justin Grimmer (Stanford University) Methodology I September 9th, / 61
94 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn Justin Grimmer (Stanford University) Methodology I September 9th, / 61
95 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn X = x 11 x 12. x 1n Justin Grimmer (Stanford University) Methodology I September 9th, / 61
96 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn X = x 11 x 21 x 12 x 22.. x 1n x 2n Justin Grimmer (Stanford University) Methodology I September 9th, / 61
97 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn X = x 11 x x 12 x x 1n x 2n... Justin Grimmer (Stanford University) Methodology I September 9th, / 61
98 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn x 11 x x m1 x 12 x x m2... x 1n x 2n... x mn Justin Grimmer (Stanford University) Methodology I September 9th, / 61
99 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn x 11 x x m1 x 12 x x m2... x 1n x 2n... x mn If X is an m n then X is n m. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
100 Matrix Transpose We will use matrix transpose to flip the dimensionality of a matrix X = X = x 11 x x 1n x 21 x x 2n x m1 x m2... x mn x 11 x x m1 x 12 x x m2... x 1n x 2n... x mn If X is an m n then X is n m. If X = X then we say X is symmetric. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
101 Matrix Transpose Example 1: X = ( ) then X = In R mat1<- matrix(c(1, 2, 3), nrow=3, ncol=2) mat2<- t(mat1) dim(mat1) 3 2 dim(mat2) 2 3 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
102 Matrix Multiplication How do we multiply matrices? Justin Grimmer (Stanford University) Methodology I September 9th, / 61
103 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Justin Grimmer (Stanford University) Methodology I September 9th, / 61
104 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose we have two matrices Justin Grimmer (Stanford University) Methodology I September 9th, / 61
105 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
106 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: Justin Grimmer (Stanford University) Methodology I September 9th, / 61
107 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X Y Justin Grimmer (Stanford University) Methodology I September 9th, / 61
108 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
109 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
110 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
111 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
112 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = = Justin Grimmer (Stanford University) Methodology I September 9th, / 61
113 Matrix Multiplication How do we multiply matrices? Because we want to use matrix multiplication to solve equations we won t use an intuitive definition Suppose ( we) have two ( matrices ) X =, Y = We will create a new matrix A by matrix multiplication: A = X ( Y ) ( ) = = = ( ) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
114 Matrix Multiplication Definition Suppose X is an m n matrix and Y is an n k matrix. Then define the matrix A an m k matrix that obtains from multiplying X and Y as, A = X Y x 11 x x 1n y 11 y y 1k x 21 x x 2n y 21 y y 2k = x m1 x m2... x mn y n1 y n2... y nk x 11 y 11 + x 12 y x 1n y n1... x 11 y 1k + x 12 y 2k x 1n y nk =..... x m1 y 11 + x m2 y x mn y n1... x m1 y 11 + x m2 y x mn y nk Justin Grimmer (Stanford University) Methodology I September 9th, / 61
115 Definition Suppose X is an m n matrix and Y is an n k matrix. Write the row x 1 x 2 vectors of X =. and Y as column vector Y = ( ) y 1 y 2... y k. x m Then the m k matrix A = X Y can be written as x 1 y 1 x 1 y 2... x 1 y k x 2 y 1 x 2 y 2... x 2 y k A = x m y 1 x m y 2... x m y k Justin Grimmer (Stanford University) Methodology I September 9th, / 61
116 Matrix Multiplication Let s ( work on an) example together! X = Y = 1 5 What is X Y? 3 5 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
117 Matrix Multiplication Let s ( work on an) example together! X = Y = 1 5 What is X Y? 3 5 Not all matrices can be multiplied. Matrix AB exists only if the number of columns in A = number of rows in B. If AB exists we will say the matrices are conformable Justin Grimmer (Stanford University) Methodology I September 9th, / 61
118 Matrix Multiplication with a Vector Suppose X = a 3 4 matrix and that v = matrix (or a column vector) what is X v? What is X v? a 4 1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
119 Algebraic Properties Suppose X is an m n matrix and Y is an n k matrix. Suppose that I =..... as the identity matrix and that k Re X Y Y X in general!!!! (but it could) - X I = X (let s talk it out!) - (X ) = X - (X Y ) = Y X - (kx ) = kx - (X + Y ) = X + Y Justin Grimmer (Stanford University) Methodology I September 9th, / 61
120 Examples, Implenting in R R and matrix multiplication X<- matrix(na, nrow=2, ncol=3) Y<- matrix(na, nrow=3, ncol=2) X[1,]<- c(1, 4, 5) X[2,]<- c(10, 2, 3) Y[1,]<- c(2, 3) Y[2,]<- c(1, 5) Y[3,]<- c(3, 5) A<- X%*%Y > A [,1] [,2] [1,] [2,] Justin Grimmer (Stanford University) Methodology I September 9th, / 61
121 Matrix Inversion Big topic: suppose X is an n n matrix. We want to find the matrix X 1 such that where I is the n n identity matrix. Why? - Regression - Solving systems of equations X 1 X = X X 1 = I - Will provide intuition about colinearity, fixed effects, treatment designs and what we can learn as social scientists Calculate Properties of Inverses when do inverses exist Application to regression analysis Justin Grimmer (Stanford University) Methodology I September 9th, / 61
122 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 0x 1 + 0x 2 + x 3 = 5 (0.1) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
123 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 x 1 + x 2 + 0x 3 = 0 0x 1 + x 2 + x 3 = 0 x 1 + 0x 2 + x 3 = 0 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
124 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
125 Some Motivating Examples Consider the following equations: A = x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
126 Some Motivating Examples Consider the following equations: A = x = (x 1, x 2, x 3 ) x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
127 Some Motivating Examples Consider the following equations: A = x = (x 1, x 2, x 3 ) b = (0, 5, 6) x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = 6 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
128 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = A = x = (x 1, x 2, x 3 ) b = (0, 5, 6) The system of equations are now, Justin Grimmer (Stanford University) Methodology I September 9th, / 61
129 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = A = x = (x 1, x 2, x 3 ) b = (0, 5, 6) The system of equations are now, Ax = b Justin Grimmer (Stanford University) Methodology I September 9th, / 61
130 Some Motivating Examples Consider the following equations: x 1 + x 2 + x 3 = 0 0x 1 + 5x 2 + 0x 3 = 5 0x 1 + 0x 2 + 3x 3 = A = x = (x 1, x 2, x 3 ) b = (0, 5, 6) The system of equations are now, Ax = b A 1 exists if and only if Ax = b has only one solution. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
131 Matrix Inversion, Definition Definition Suppose X is an n n matrix. We will call X 1 the inverse of X if X 1 X = X X 1 = I If X 1 exists then X is invertible. If X 1 does not exist, then we will say X is singular. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
132 Matrix Inversion You ll never invert a matrix by hand. We re going to use R X<- matrix(na, nrow=3, ncol=3) X[1,]<- c(2, 3, 4) X[2,]<- c(3, 1, 3) X[3,]<- c(2, 4, 2) X.inv<- solve(x) > X.inv [,1] [,2] [,3] [1,] [2,] [3,] X.inv%*%X [,1] [,2] [,3] [1,] e e-16 [2,] e e+00 [3,] e e+00 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
133 Matrix Inversion 1) Calculate Inverses 2) Properties of Inverses Justin Grimmer (Stanford University) Methodology I September 9th, / 61
134 Matrix Inversion 1) Calculate Inverses 2) Properties of Inverses Theorem The inverse of matrix X, X 1, is unique Justin Grimmer (Stanford University) Methodology I September 9th, / 61
135 Matrix Inversion 1) Calculate Inverses 2) Properties of Inverses Theorem The inverse of matrix X, X 1, is unique Proof. By way of contradiction, suppose not. Then there are at least two matrices A and C such that AX = I and CX = I This implies that, AXC = (AX )C = I C = C Justin Grimmer (Stanford University) Methodology I September 9th, / 61
136 Matrix Inversion But it also implies that AXC = A(X C) = A(I ) = A So C = AXC = A or C = A but this contradicts our assumption that there are two unique inverses. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
137 Matrix Inversion Theorem Suppose A has inverse A 1 and B has inverse B 1. Then, (AB) 1 = B 1 A 1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
138 Matrix Inversion Theorem Suppose A has inverse A 1 and B has inverse B 1. Then, (AB) 1 = B 1 A 1 Proof. We need to show that (B 1 A 1 )(AB) = (AB)(B 1 A 1 ) = I. (B 1 A 1 )(AB) = B 1 (A 1 A)B = B 1 I B = B 1 B = I Justin Grimmer (Stanford University) Methodology I September 9th, / 61
139 Matrix Inversion (AB)(B 1 A 1 ) = A(BB 1 )A 1 = AI A 1 = AA 1 = I So AB is invertible and (AB) 1 = B 1 A 1. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
140 Challenge Inversion Proofs - Show that (A 1 ) 1 = A. - Show that (ka) 1 = 1 k A 1 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
141 Matrix Inversion 1) How to Calculate an Inverse 2) Inversion properties 3) When do inverses exist? Linear Independence: not repeated information in matrix will be the key (for both inversion and regressions) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
142 Matrix Inversion: Existence Definition Suppose we have a set of vectors S = {v 1, v 2,..., v r } And consider the system of equations k 1 v 1 + k 2 v k r v r = 0 If the only solution is k 1 = 0, k 2 = 0, k 3 = 0,..., k r = 0 then we say that the set is linearly independent. If there are other solutions, then the set is linearly dependent. Justin Grimmer (Stanford University) Methodology I September 9th, / 61
143 Matrix Inversion: Existence Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1) Can we write this as a combination of other vectors? Justin Grimmer (Stanford University) Methodology I September 9th, / 61
144 Matrix Inversion: Existence Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1) Can we write this as a combination of other vectors? no! Justin Grimmer (Stanford University) Methodology I September 9th, / 61
145 Matrix Inversion: Existence Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1) Can we write this as a combination of other vectors? no! Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1), v 4 = (1, 2, 3). Can we write this as a combination of other vectors? Justin Grimmer (Stanford University) Methodology I September 9th, / 61
146 Matrix Inversion: Existence Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1) Can we write this as a combination of other vectors? no! Consider v 1 = (1, 0, 0), v 2 = (0, 1, 0), v 3 = (0, 0, 1), v 4 = (1, 2, 3). Can we write this as a combination of other vectors? v 4 = v 1 + 2v 2 + 3v 3 Justin Grimmer (Stanford University) Methodology I September 9th, / 61
147 Matrix Inversion: Existence Theorem Suppose v 1, v 2,..., v K R n. If K > n then the set is linearly dependent Justin Grimmer (Stanford University) Methodology I September 9th, / 61
148 Matrix Inversion: Existence Theorem Suppose v 1, v 2,..., v K R n. If K > n then the set is linearly dependent - v 1 = (v 11, v 21,..., v n1 ) Justin Grimmer (Stanford University) Methodology I September 9th, / 61
149 Matrix Inversion: Existence Theorem Suppose v 1, v 2,..., v K R n. If K > n then the set is linearly dependent - v 1 = (v 11, v 21,..., v n1 ) - Says that if there are more vectors in the set than elements in each vector, one must be linearly dependent Justin Grimmer (Stanford University) Methodology I September 9th, / 61
150 Matrix Inversion: Existence We care because of the following theorem Theorem x 2 Suppose X is an n n matrix. Recall we can write this matrix as.. Then X has an inverse if and only if S = {x 1, x 2,..., x n } is linearly independent If this is true, we say X has full rank x 1 x n Justin Grimmer (Stanford University) Methodology I September 9th, / 61
151 Linear Regression Justin Grimmer (Stanford University) Methodology I September 9th, / 61
152 Linear Regression In 350a you learn about linear regression. For each i (individual) we observe covariates x i1, x i2,..., x ik and independent variable Y i. Then, Justin Grimmer (Stanford University) Methodology I September 9th, / 61
153 Linear Regression In 350a you learn about linear regression. For each i (individual) we observe covariates x i1, x i2,..., x ik and independent variable Y i. Then, Y i = β 0 + β 1 x i1 + β 2 x i β k x ik Justin Grimmer (Stanford University) Methodology I September 9th, / 61
154 Linear Regression In 350a you learn about linear regression. For each i (individual) we observe covariates x i1, x i2,..., x ik and independent variable Y i. Then, Y 1 = β 0 + β 1 x 11 + β 2 x β k x 1k Y 2 = β 0 + β 1 x 21 + β 2 x β k x 2k... Y i = β 0 + β 1 x i1 + β 2 x i β k x ik... Y n = β 0 + β 1 x n1 + β 2 x n β k x nk Justin Grimmer (Stanford University) Methodology I September 9th, / 61
155 Linear Regression - Define x i = (1, x i1, x i2,..., x ik ) x 1 x 2 - Define X =. x n - Define β = (β 0, β 1,..., β k ) - Define Y = (Y 1, Y 2,..., Y n ). Then we can write Y = X β Justin Grimmer (Stanford University) Methodology I September 9th, / 61
156 Linear Regression Y = X β X Y = X X β (X X ) 1 X Y = (X X ) 1 X X β (X X ) 1 X Y = β Big question: is (X X ) 1 invertible? We ll investigate in homework! Justin Grimmer (Stanford University) Methodology I September 9th, / 61
Review: Linear and Vector Algebra
Review: Linear and Vector Algebra Points in Euclidean Space Location in space Tuple of n coordinates x, y, z, etc Cannot be added or multiplied together Vectors: Arrows in Space Vectors are point changes
Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
Linear Algebra V = T = ( 4 3 ).
Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional
Knowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
Matrix Arithmetic. j=1
An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column
Matrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix
Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix
Chapter 3: Theory Review: Solutions Math 308 F Spring 2015
Chapter : Theory Review: Solutions Math 08 F Spring 05. What two properties must a function T : R m R n satisfy to be a linear transformation? (a) For all vectors u and v in R m, T (u + v) T (u) + T (v)
Review of Basic Concepts in Linear Algebra
Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra
M. Matrices and Linear Algebra
M. Matrices and Linear Algebra. Matrix algebra. In section D we calculated the determinants of square arrays of numbers. Such arrays are important in mathematics and its applications; they are called matrices.
1 Matrices and matrix algebra
1 Matrices and matrix algebra 1.1 Examples of matrices A matrix is a rectangular array of numbers and/or variables. For instance 4 2 0 3 1 A = 5 1.2 0.7 x 3 π 3 4 6 27 is a matrix with 3 rows and 5 columns
MATRICES. a m,1 a m,n A =
MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of
Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
Math Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
22A-2 SUMMER 2014 LECTURE Agenda
22A-2 SUMMER 204 LECTURE 2 NATHANIEL GALLUP The Dot Product Continued Matrices Group Work Vectors and Linear Equations Agenda 2 Dot Product Continued Angles between vectors Given two 2-dimensional vectors
Vectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
Linear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
Chapter 2. Matrix Arithmetic. Chapter 2
Matrix Arithmetic Matrix Addition and Subtraction Addition and subtraction act element-wise on matrices. In order for the addition/subtraction (A B) to be possible, the two matrices A and B must have the
Math 360 Linear Algebra Fall Class Notes. a a a a a a. a a a
Math 360 Linear Algebra Fall 2008 9-10-08 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are
We could express the left side as a sum of vectors and obtain the Vector Form of a Linear System: a 12 a x n. a m2
Week 22 Equations, Matrices and Transformations Coefficient Matrix and Vector Forms of a Linear System Suppose we have a system of m linear equations in n unknowns a 11 x 1 + a 12 x 2 + + a 1n x n b 1
Things we can already do with matrices. Unit II - Matrix arithmetic. Defining the matrix product. Things that fail in matrix arithmetic
Unit II - Matrix arithmetic matrix multiplication matrix inverses elementary matrices finding the inverse of a matrix determinants Unit II - Matrix arithmetic 1 Things we can already do with matrices equality
Properties of Linear Transformations from R n to R m
Properties of Linear Transformations from R n to R m MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Topic Overview Relationship between the properties of a matrix transformation
Linear Algebra March 16, 2019
Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented
A primer on matrices
A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous
Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
x 1 x 2. x 1, x 2,..., x n R. x n
WEEK In general terms, our aim in this first part of the course is to use vector space theory to study the geometry of Euclidean space A good knowledge of the subject matter of the Matrix Applications
Applied Matrix Algebra Lecture Notes Section 2.2. Gerald Höhn Department of Mathematics, Kansas State University
Applied Matrix Algebra Lecture Notes Section 22 Gerald Höhn Department of Mathematics, Kansas State University September, 216 Chapter 2 Matrices 22 Inverses Let (S) a 11 x 1 + a 12 x 2 + +a 1n x n = b
Linear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.
POLI 7 - Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes - Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems
MATH.2720 Introduction to Programming with MATLAB Vector and Matrix Algebra
MATH.2720 Introduction to Programming with MATLAB Vector and Matrix Algebra A. Vectors A vector is a quantity that has both magnitude and direction, like velocity. The location of a vector is irrelevant;
Phys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
A FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic
A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Matrix Arithmetic Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
Elementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
Basic Linear Algebra in MATLAB
Basic Linear Algebra in MATLAB 9.29 Optional Lecture 2 In the last optional lecture we learned the the basic type in MATLAB is a matrix of double precision floating point numbers. You learned a number
Introduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
Linear Algebra Homework and Study Guide
Linear Algebra Homework and Study Guide Phil R. Smith, Ph.D. February 28, 20 Homework Problem Sets Organized by Learning Outcomes Test I: Systems of Linear Equations; Matrices Lesson. Give examples of
4 Linear Algebra Review
Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the text 1 Vectors and Matrices For the context of data analysis, the
CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33
Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct
Kevin James. MTHSC 3110 Section 2.1 Matrix Operations
MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j
. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim
Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its
Overview. Motivation for the inner product. Question. Definition
Overview Last time we studied the evolution of a discrete linear dynamical system, and today we begin the final topic of the course (loosely speaking) Today we ll recall the definition and properties of
Linear Algebra (Review) Volker Tresp 2018
Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c
Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
Matrix Algebra for Engineers Jeffrey R. Chasnov
Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon
CS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
Matrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
7.5 Operations with Matrices. Copyright Cengage Learning. All rights reserved.
7.5 Operations with Matrices Copyright Cengage Learning. All rights reserved. What You Should Learn Decide whether two matrices are equal. Add and subtract matrices and multiply matrices by scalars. Multiply
Stat 206: Linear algebra
Stat 206: Linear algebra James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Vectors We have already been working with vectors, but let s review a few more concepts. The inner product of two
Review of linear algebra
Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of
Basic Concepts in Linear Algebra
Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear
Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
An Introduction to Matrix Algebra
An Introduction to Matrix Algebra EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #8 EPSY 905: Matrix Algebra In This Lecture An introduction to matrix algebra Ø Scalars, vectors, and matrices
Linear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
A matrix over a field F is a rectangular array of elements from F. The symbol
Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted
Properties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
[POLS 8500] Review of Linear Algebra, Probability and Information Theory
![[POLS 8500] Review of Linear Algebra, Probability and Information Theory](/thumbs/72/67586683.jpg "[POLS 8500] Review of Linear Algebra, Probability and Information Theory")
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
Lecture 3 Linear Algebra Background
Lecture 3 Linear Algebra Background Dan Sheldon September 17, 2012 Motivation Preview of next class: y (1) w 0 + w 1 x (1) 1 + w 2 x (1) 2 +... + w d x (1) d y (2) w 0 + w 1 x (2) 1 + w 2 x (2) 2 +...
A Introduction to Matrix Algebra and the Multivariate Normal Distribution
A Introduction to Matrix Algebra and the Multivariate Normal Distribution PRE 905: Multivariate Analysis Spring 2014 Lecture 6 PRE 905: Lecture 7 Matrix Algebra and the MVN Distribution Today s Class An
MAC Module 2 Systems of Linear Equations and Matrices II. Learning Objectives. Upon completing this module, you should be able to :
MAC 0 Module Systems of Linear Equations and Matrices II Learning Objectives Upon completing this module, you should be able to :. Find the inverse of a square matrix.. Determine whether a matrix is invertible..
MATH 22A: LINEAR ALGEBRA Chapter 4
MATH 22A: LINEAR ALGEBRA Chapter 4 Jesús De Loera, UC Davis November 30, 2012 Orthogonality and Least Squares Approximation QUESTION: Suppose Ax = b has no solution!! Then what to do? Can we find an Approximate
Lecture 23: 6.1 Inner Products
Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such
Matrices MA1S1. Tristan McLoughlin. November 9, Anton & Rorres: Ch
Matrices MA1S1 Tristan McLoughlin November 9, 2014 Anton & Rorres: Ch 1.3-1.8 Basic matrix notation We have studied matrices as a tool for solving systems of linear equations but now we want to study them
Dot Products, Transposes, and Orthogonal Projections
Dot Products, Transposes, and Orthogonal Projections David Jekel November 13, 2015 Properties of Dot Products Recall that the dot product or standard inner product on R n is given by x y = x 1 y 1 + +
CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra /34
Linear Algebra /34 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct axes)
Fundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
Linear Algebra. Preliminary Lecture Notes
Linear Algebra Preliminary Lecture Notes Adolfo J. Rumbos c Draft date May 9, 29 2 Contents 1 Motivation for the course 5 2 Euclidean n dimensional Space 7 2.1 Definition of n Dimensional Euclidean Space...........
LS.1 Review of Linear Algebra
LS. LINEAR SYSTEMS LS.1 Review of Linear Algebra In these notes, we will investigate a way of handling a linear system of ODE s directly, instead of using elimination to reduce it to a single higher-order
Pre-sessional Mathematics for Big Data MSc Class 2: Linear Algebra
Pre-sessional Mathematics for Big Data MSc Class 2: Linear Algebra Yuri Kalnishkan September 22, 2018 Linear algebra is vitally important for applied mathematics We will approach linear algebra from a
Unit 3: Matrices. Juan Luis Melero and Eduardo Eyras. September 2018
Unit 3: Matrices Juan Luis Melero and Eduardo Eyras September 2018 1 Contents 1 Matrices and operations 4 1.1 Definition of a matrix....................... 4 1.2 Addition and subtraction of matrices..............
Matrices. Chapter Definitions and Notations
Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which
Matrix representation of a linear map
Matrix representation of a linear map As before, let e i = (0,..., 0, 1, 0,..., 0) T, with 1 in the i th place and 0 elsewhere, be standard basis vectors. Given linear map f : R n R m we get n column vectors
Lecture 3: Linear Algebra Review, Part II
Lecture 3: Linear Algebra Review, Part II Brian Borchers January 4, Linear Independence Definition The vectors v, v,..., v n are linearly independent if the system of equations c v + c v +...+ c n v n
Linear Algebra. James Je Heon Kim
Linear lgebra James Je Heon Kim (jjk9columbia.edu) If you are unfamiliar with linear or matrix algebra, you will nd that it is very di erent from basic algebra or calculus. For the duration of this session,
Vector Spaces, Orthogonality, and Linear Least Squares
Week Vector Spaces, Orthogonality, and Linear Least Squares. Opening Remarks.. Visualizing Planes, Lines, and Solutions Consider the following system of linear equations from the opener for Week 9: χ χ
4 Linear Algebra Review
4 Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the course 41 Vectors and Matrices For the context of data analysis,
Linear Equations and Matrix
1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
Matrix Arithmetic. a 11 a. A + B = + a m1 a mn. + b. a 11 + b 11 a 1n + b 1n = a m1. b m1 b mn. and scalar multiplication for matrices via.
Matrix Arithmetic There is an arithmetic for matrices that can be viewed as extending the arithmetic we have developed for vectors to the more general setting of rectangular arrays: if A and B are m n
MATH 167: APPLIED LINEAR ALGEBRA Least-Squares
MATH 167: APPLIED LINEAR ALGEBRA Least-Squares October 30, 2014 Least Squares We do a series of experiments, collecting data. We wish to see patterns!! We expect the output b to be a linear function of
Machine Learning for Large-Scale Data Analysis and Decision Making A. Week #1
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Week #1 Today Introduction to machine learning The course (syllabus) Math review (probability + linear algebra) The future
Lecture 6: Geometry of OLS Estimation of Linear Regession
Lecture 6: Geometry of OLS Estimation of Linear Regession Xuexin Wang WISE Oct 2013 1 / 22 Matrix Algebra An n m matrix A is a rectangular array that consists of nm elements arranged in n rows and m columns
Linear Algebra. Preliminary Lecture Notes
Linear Algebra Preliminary Lecture Notes Adolfo J. Rumbos c Draft date April 29, 23 2 Contents Motivation for the course 5 2 Euclidean n dimensional Space 7 2. Definition of n Dimensional Euclidean Space...........
Linear Models Review
Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
MATH 106 LINEAR ALGEBRA LECTURE NOTES
MATH 6 LINEAR ALGEBRA LECTURE NOTES FALL - These Lecture Notes are not in a final form being still subject of improvement Contents Systems of linear equations and matrices 5 Introduction to systems of
Systems of Linear Equations
Systems of Linear Equations Math 108A: August 21, 2008 John Douglas Moore Our goal in these notes is to explain a few facts regarding linear systems of equations not included in the first few chapters
3 (Maths) Linear Algebra
3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra
Linear Algebra (Review) Volker Tresp 2017
Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.
Lecture 3: Vectors. In Song Kim. September 1, 2011
Lecture 3: Vectors In Song Kim September 1, 211 1 Solving Equations Up until this point we have been looking at different types of functions, often times graphing them. Each point on a graph is a solution
ECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am
ECS130 Scientific Computing Lecture 1: Introduction Monday, January 7, 10:00 10:50 am About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ecs130/ Today
ME751 Advanced Computational Multibody Dynamics
ME751 Advanced Computational Multibody Dynamics Review: Elements of Linear Algebra & Calculus September 9, 2016 Dan Negrut University of Wisconsin-Madison Quote of the day If you can't convince them, confuse
Matrix & Linear Algebra
Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A
2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
Matrix Arithmetic. (Multidimensional Math) Institute for Advanced Analytics. Shaina Race, PhD
Matrix Arithmetic Multidimensional Math) Shaina Race, PhD Institute for Advanced Analytics. TABLE OF CONTENTS Element-wise Operations Linear Combinations of Matrices and Vectors. Vector Multiplication
MATH Topics in Applied Mathematics Lecture 12: Evaluation of determinants. Cross product.
MATH 311-504 Topics in Applied Mathematics Lecture 12: Evaluation of determinants. Cross product. Determinant is a scalar assigned to each square matrix. Notation. The determinant of a matrix A = (a ij
Lecture 7. Econ August 18
Lecture 7 Econ 2001 2015 August 18 Lecture 7 Outline First, the theorem of the maximum, an amazing result about continuity in optimization problems. Then, we start linear algebra, mostly looking at familiar
I&C 6N. Computational Linear Algebra
I&C 6N Computational Linear Algebra 1 Lecture 1: Scalars and Vectors What is a scalar? Computer representation of a scalar Scalar Equality Scalar Operations Addition and Multiplication What is a vector?
Vectors and their uses
Vectors and their uses Sharon Goldwater Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh DRAFT Version 0.95: 3 Sep 2015. Do not redistribute without permission.
Row Space, Column Space, and Nullspace
Row Space, Column Space, and Nullspace MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Every matrix has associated with it three vector spaces: row space