PCA SVD LDA MDS, LLE, CCA. Data mining. Dimensionality reduction. University of Szeged. Data mining
|
|
- Solomon Cannon
- 6 years ago
- Views:
Transcription
1 Dimesioality reductio Uiversity of Szeged
2 The role of dimesioality reductio We ca spare computatioal costs (or simply fit etire datasets ito mai memory) if we represet data i fewer dimesios Visualizatio of datasets (i 2 or 3 dimesios) Elimiatio of oise from data, feature selectio Key idea: try to represet data poits i lower dimesios Depedig our ojective fuctio with respect the lower dimesioal represetatio,,,...
3 ricipal Compoet Aalysis Trasform multidimesioal data ito lower dimesios i such a way that we lose as little proportio of the origial variatio of the data as possile Assumptio: data poits of the origial m-dimesioal space lie at (or at least very close to) a m -dimesioal suspace we shall express data poits with respect this suspace What that m -dimesioal suspace might e? We would like to miimize the recostructio error X k (xi xi ) k2 i=1, where xi is a approximatio for poit xi
4 Covariace Remider Quatifies how much radom variales Y ad Z chage together cov (Y, Z ) = E[(Y µy )(Z µz )] µy = 1 i=1 yi ad µz = 1 zi i=1 Colums i, j of data matrix X (i.e. X:,i, X:,j ) ca e regarded as oservatios from two radom variales
5 Scatter ad covariace matrix Scatter matrix: S = (xi µ)(xi µ) k=1 (U)iased covariace matrix: Σ = 1 S (Σ = 1 1 S) cov (X:,1, X:,1 ) cov (X:,1, X:,2 )... cov (X:,1, X:,m ) cov (X:,2, X:,1 ) cov (X:,2, X:,2 )... cov (X:,2, X:,m ) Σ= cov (X, X ). :,i :,j cov (X:,m, X:,1 ) cov (X:,m, X:,m ) Σi,j is the covariace of variales i ad j (cov (X:,i, X:,j )) What values are icluded i the mai diagoal?
6 Characteristics of scatter ad covariace matrices Claim: matrices S ad Σ are symmetric ad positive defiite Bizoyítás. S= (xi µ)(xi µ) =! (xi µ)(xi µ) = S k=1 k=1 a Sa = k=1 (a (xi µ))((xi µ) a) = (a (xi µ))2 k=1 Cosequece: the eigevalues of S ad C are λ1 λ2... λm The m -dimesioal projectio which preserves most of the variatio of the data ca e otaied y projectig data poits usig the eigevectors elogig to the m highest eigevalues of either matrix S (or C ) (proof: see tale)
7 Lagrage multipliers rovides a schema for solvig (o-)liear optimizatio prolems f (x) mi/max such that gi (x) = i {1,..., } Lagrage fuctio: L(x, λ) = f(x) X λi gi (x) i=1 Karush-Kuh-Tucker (KKT) coditios: ecessity coditios for a optimum L(x, λ) = (1) λi gi (x) = i {1,..., } (2) λi (3)
8 ractical issues Its worth hadlig all the features o similar scales mi-max ormalizatio: xi,j = stadardizatio: xi,j = xi,j mi(x,j ) max(x,j ) mi(x,j ) xi,j µj σj How to choose the reduced dimesioality (m )? m m si2 λi = Hit : i=1 i=1 k i=1 λi m = arg mi m t threshold 1 k m i=1 λi m 1 X λi > λj 1 i m m m = arg max j=1 m = arg max (λi λi+1 ) 1 i m 1
9 Summarizig Sutract the mea vector from data X ad also ormalize it somehow Calculate the scatter/covariace matrix of the ormalized data Calculate its eigevalues Form projectio matrix from the eigevectors correspodig to the m largest eigevalues X = X gives the trasformed data X 1 gives a approximatio o the origial positios of the data poits A useful tutorial o
10 Sigular Value Decompositio X = U x Sigma x V t rak(x ) σi ui vi i=1 s s rak(x m ) 2 xij2 = kx kf = σi X = UΣV = i=1 j=1 i=1 Low(er) rak approximatio of X is X = U Σ V We rely o the top m < m largest sigular value of Σ upo recostructig X This is the est possile m -dimesioal approximatio of X if we look for the approximatio which miimizes kx X kfroeius
11 Sigular Value Decompositio ad Eigedecompositio Remider Ay symmetric matrix A is decomposale as A = X ΛX 1, where X = [x1 x2... xm ] comprises of the orthogoal eigevectors of A ad Λ = diag ([λ1 λ2... λm ]) cotaiig the correspodig eigevalues i its mai diagoal. Why? Ay m matrix X ca e uiquely decomposed ito the product of three matrices of the form UΣV where U = [u1 u2... u ] is the orthoormal matrix cosistig of the eigevectors of XX Σ m = diag ( λ1, λ2,..., λm ) Vm m = [v1 v2... vm ] is the orthoormal matrix cosistig of the eigevectors of X X Why? Orthogoal matrices: M M = I (a trasformatio which preserves distace i the trasformed space as well) Why?
12 Relatio etwee ad Froeius-orm Suppose M = Q R, i.e. mij = k 58 M = = pik qkl rlj l 3 m32 = pik qkl rlj The kmk2f = (mij )2 = i j i j k l 2 Also pik qkl rlj = pik qkl rlj pi qm rmj k l k l m From where kmk2f = pik qkl rlj pi qm rmj i j k l m Give tha matrices, Q, R origiate from a decompositio, kmk2f = pik qkk rkj pi q rj = qkk rkj qkk rkj = (qkk )2. i,j,k, j,k Why? The error of the approximatig Xy X = U Σ V is kx X k2 = ku(σ Σ )V k2 = Data (σ miig σ )2 k
13 CUR The drawack of is that a typically sparse matrix is decomposed ito a products of dese matrices (i.e. U ad V ) Oe alterative is to use CUR decompositio This time oly matrix U happes to e dese Matrices C ad R are composed of the rows ad colums of the matrix X, thus they preserve the sparsity of X is uique ulike CUR
14 CUR decompositio producig C ad R Choose k colums (rows) from the data matrix with replacemet A colum (row) ca e selected more tha oce ito C (R) The proaility of selectig a colum (row) should e proportioal to the sum of squared elemets i it Scale the elemets of the selected colum (row) y 1/ kpi
15 CUR decompositio producig U Create W Rkxk from the itersectio of the k selected rows ad colums erform o W such that W = X ΣY Let U = Y (Σ+ )2 X where Σ+ is the pseudoiverse of Σ Σ is diagoal, so calculatig its pseudoiverse is easy Just take the reciprocal of the o-zero elemets i the diagoal Zeros ca e left itact
16 Alie Matrix Casalaca Titaic CUR decompositio example aul Sam Martha Bo Sarah C = U= 7.1 R=
17 Liear Discrimiat Aalysis Trasform data poits ito lower dimesios i such a way that poits of the same class have as little dispersio as possile whereas poits of differet classes mix as little as possile How should we choose w, i.e. the directio of the projectio? µ ~i = w µi µ~1 µ~2 = w (µ1 µ2 ) X s i = (w x ~ µi )2 x with lael i w = arg max J(w) = arg max w w ~ µ1 ~ µ2 2 2 s 1 + s 22 (1)
18 Withi ad outer scatter matrices The withi-scatter matrix of poits with class lael i: X Si = (x µi )(x µi ) x with lael i Aggregated withi scatter matrix: SW = S1 + S2 Scatter of the poits with class lael i: X X s i 2 = (w x w µi )2 = w (x µi )(x µi ) w = w Si w x with lael i Scatter matrix of the poits etwee differet classes: SB = (µ1 µ2 )(µ1 µ2 ) Scatter of the poits etwee differet classes: (µ 1 µ 2 )2 = (w µ1 w µ2 )2 = w SB w
19 A equivalet ojective with Eq. (1) is w = arg maxw J(w) = arg maxw ww SSWB ww w SB w w SW w is the so-called geeralized Rayleigh-coefficiet J(w) is maximal ww SSWB ww = SB w = λsw w 1 1 SW SB w = λw w = SW (µ1 µ2 ) Remider f (x) (x)g (x) = f (x)g (x) f g (x) g 2 (x) x x Ax = 2Ax, give that A = A xx y = xi yi x (i.e. a vector poitig i=1 of x) i the directio
20 vs
21 Multi-Dimesioal Scalig (MDS) Goal: give cotaiig pair-wise cost/distaces of poits fid the positios of the poits for which k xi xj k δij Trasforms multidimesioal poits ito lower dimesios such that the pairwise distaces get preserved as much as possile
22 Locally Liear Emeddig (LLE), ad all assume liear relatioship etwee variales No-liear dimesioality reductio techique Idea: defie the earest eighors for all poits ad defie them as their liear comiatio Wij = 1, és Wij xj k2, such that kxi J(W ) = i=1 j=1 j=1 wij > xj eighors(xi )
23 Caoical Correlatio Aalysis (CCA) Our data poits have two distict represetatios (coordiate systems) Goal: fid a commo coordiate system (with reduced dimesioality) such that the correlatio etwee the trasformed poits get maximized E [xy ] = E [x 2 ] E [y 2 ] wx Cxy w y wx Cxx wx wy Cyy wy ρ= E[wx xy wy ] E[wx xx wx ] E [wy yy wy ] = arg max ρ is idepedet from the legth of wx ad wy arg max ρ = arg max wx Cxy wy " # " # Σxx Σxy x x Σ= =E y y Σyx Σyy
Machine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationBIOINF 585: Machine Learning for Systems Biology & Clinical Informatics
BIOINF 585: Machie Learig for Systems Biology & Cliical Iformatics Lecture 14: Dimesio Reductio Jie Wag Departmet of Computatioal Medicie & Bioiformatics Uiversity of Michiga 1 Outlie What is feature reductio?
More informationFor a 3 3 diagonal matrix we find. Thus e 1 is a eigenvector corresponding to eigenvalue λ = a 11. Thus matrix A has eigenvalues 2 and 3.
Closed Leotief Model Chapter 6 Eigevalues I a closed Leotief iput-output-model cosumptio ad productio coicide, i.e. V x = x = x Is this possible for the give techology matrix V? This is a special case
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture 9: Pricipal Compoet Aalysis The text i black outlies mai ideas to retai from the lecture. The text i blue give a deeper uderstadig of how we derive or get
More informationCov(aX, cy ) Var(X) Var(Y ) It is completely invariant to affine transformations: for any a, b, c, d R, ρ(ax + b, cy + d) = a.s. X i. as n.
CS 189 Itroductio to Machie Learig Sprig 218 Note 11 1 Caoical Correlatio Aalysis The Pearso Correlatio Coefficiet ρ(x, Y ) is a way to measure how liearly related (i other words, how well a liear model
More informationTopics in Eigen-analysis
Topics i Eige-aalysis Li Zajiag 28 July 2014 Cotets 1 Termiology... 2 2 Some Basic Properties ad Results... 2 3 Eige-properties of Hermitia Matrices... 5 3.1 Basic Theorems... 5 3.2 Quadratic Forms & Noegative
More information6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition
6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More information5.1 Review of Singular Value Decomposition (SVD)
MGMT 69000: Topics i High-dimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of
More informationDistributional Similarity Models (cont.)
Distributioal Similarity Models (cot.) Regia Barzilay EECS Departmet MIT October 19, 2004 Sematic Similarity Vector Space Model Similarity Measures cosie Euclidea distace... Clusterig k-meas hierarchical
More informationDistributional Similarity Models (cont.)
Sematic Similarity Vector Space Model Similarity Measures cosie Euclidea distace... Clusterig k-meas hierarchical Last Time EM Clusterig Soft versio of K-meas clusterig Iput: m dimesioal objects X = {
More informationLinear Classifiers III
Uiversität Potsdam Istitut für Iformatik Lehrstuhl Maschielles Lere Liear Classifiers III Blaie Nelso, Tobias Scheffer Cotets Classificatio Problem Bayesia Classifier Decisio Liear Classifiers, MAP Models
More informationIntroduction to Optimization Techniques. How to Solve Equations
Itroductio to Optimizatio Techiques How to Solve Equatios Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually
More informationLecture 8: October 20, Applications of SVD: least squares approximation
Mathematical Toolkit Autum 2016 Lecturer: Madhur Tulsiai Lecture 8: October 20, 2016 1 Applicatios of SVD: least squares approximatio We discuss aother applicatio of sigular value decompositio (SVD) of
More informationState Space Representation
Optimal Cotrol, Guidace ad Estimatio Lecture 2 Overview of SS Approach ad Matrix heory Prof. Radhakat Padhi Dept. of Aerospace Egieerig Idia Istitute of Sciece - Bagalore State Space Represetatio Prof.
More informationOutline. Linear regression. Regularization functions. Polynomial curve fitting. Stochastic gradient descent for regression. MLE for regression
REGRESSION 1 Outlie Liear regressio Regularizatio fuctios Polyomial curve fittig Stochastic gradiet descet for regressio MLE for regressio Step-wise forward regressio Regressio methods Statistical techiques
More informationMachine Learning for Data Science (CS4786) Lecture 4
Machie Learig for Data Sciece (CS4786) Lecture 4 Caoical Correlatio Aalysis (CCA) Course Webpage : http://www.cs.corell.edu/courses/cs4786/2016fa/ Aoucemet We are gradig HW0 ad you will be added to cms
More informationMachine Learning for Data Science (CS4786) Lecture 9
Machie Learig for Data Sciece (CS4786) Lecture 9 Pricipal Compoet Aalysis Course Webpage : http://www.cs.corell.eu/courses/cs4786/207fa/ DIM REDUCTION: LINEAR TRANSFORMATION x > y > Pick a low imesioal
More informationAbstract Vector Spaces. Abstract Vector Spaces
Astract Vector Spaces The process of astractio is critical i egieerig! Physical Device Data Storage Vector Space MRI machie Optical receiver 0 0 1 0 1 0 0 1 Icreasig astractio 6.1 Astract Vector Spaces
More informationMon Apr Second derivative test, and maybe another conic diagonalization example. Announcements: Warm-up Exercise:
Math 2270-004 Week 15 otes We will ot ecessarily iish the material rom a give day's otes o that day We may also add or subtract some material as the week progresses, but these otes represet a i-depth outlie
More informationMatrix Algebra from a Statistician s Perspective BIOS 524/ Scalar multiple: ka
Matrix Algebra from a Statisticia s Perspective BIOS 524/546. Matrices... Basic Termiology a a A = ( aij ) deotes a m matrix of values. Whe =, this is a am a m colum vector. Whe m= this is a row vector..2.
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationIntroduction to Optimization Techniques
Itroductio to Optimizatio Techiques Basic Cocepts of Aalysis - Real Aalysis, Fuctioal Aalysis 1 Basic Cocepts of Aalysis Liear Vector Spaces Defiitio: A vector space X is a set of elemets called vectors
More informationFactor Analysis. Lecture 10: Factor Analysis and Principal Component Analysis. Sam Roweis
Lecture 10: Factor Aalysis ad Pricipal Compoet Aalysis Sam Roweis February 9, 2004 Whe we assume that the subspace is liear ad that the uderlyig latet variable has a Gaussia distributio we get a model
More informationSymmetric Matrices and Quadratic Forms
7 Symmetric Matrices ad Quadratic Forms 7.1 DIAGONALIZAION OF SYMMERIC MARICES SYMMERIC MARIX A symmetric matrix is a matrix A such that. A = A Such a matrix is ecessarily square. Its mai diagoal etries
More informationTMA4205 Numerical Linear Algebra. The Poisson problem in R 2 : diagonalization methods
TMA4205 Numerical Liear Algebra The Poisso problem i R 2 : diagoalizatio methods September 3, 2007 c Eiar M Røquist Departmet of Mathematical Scieces NTNU, N-749 Trodheim, Norway All rights reserved A
More informationR is a scalar defined as follows:
Math 8. Notes o Dot Product, Cross Product, Plaes, Area, ad Volumes This lecture focuses primarily o the dot product ad its may applicatios, especially i the measuremet of agles ad scalar projectio ad
More informationSummary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector
Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short
More informationA widely used display of protein shapes is based on the coordinates of the alpha carbons - - C α
Nice plottig of proteis: I A widely used display of protei shapes is based o the coordiates of the alpha carbos - - C α -s. The coordiates of the C α -s are coected by a cotiuous curve that roughly follows
More informationDefinitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More informationLecture 22: Review for Exam 2. 1 Basic Model Assumptions (without Gaussian Noise)
Lecture 22: Review for Exam 2 Basic Model Assumptios (without Gaussia Noise) We model oe cotiuous respose variable Y, as a liear fuctio of p umerical predictors, plus oise: Y = β 0 + β X +... β p X p +
More informationFFTs in Graphics and Vision. The Fast Fourier Transform
FFTs i Graphics ad Visio The Fast Fourier Trasform 1 Outlie The FFT Algorithm Applicatios i 1D Multi-Dimesioal FFTs More Applicatios Real FFTs 2 Computatioal Complexity To compute the movig dot-product
More informationLECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)
LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum
More informationMath 5311 Problem Set #5 Solutions
Math 5311 Problem Set #5 Solutios March 9, 009 Problem 1 O&S 11.1.3 Part (a) Solve with boudary coditios u = 1 0 x < L/ 1 L/ < x L u (0) = u (L) = 0. Let s refer to [0, L/) as regio 1 ad (L/, L] as regio.
More information10-701/ Machine Learning Mid-term Exam Solution
0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it
More information36-755, Fall 2017 Homework 5 Solution Due Wed Nov 15 by 5:00pm in Jisu s mailbox
Poits: 00+ pts total for the assigmet 36-755, Fall 07 Homework 5 Solutio Due Wed Nov 5 by 5:00pm i Jisu s mailbox We first review some basic relatios with orms ad the sigular value decompositio o matrices
More informationMath E-21b Spring 2018 Homework #2
Math E- Sprig 08 Homework # Prolems due Thursday, Feruary 8: Sectio : y = + 7 8 Fid the iverse of the liear trasformatio [That is, solve for, i terms of y, y ] y = + 0 Cosider the circular face i the accompayig
More informationChapter 12 EM algorithms The Expectation-Maximization (EM) algorithm is a maximum likelihood method for models that have hidden variables eg. Gaussian
Chapter 2 EM algorithms The Expectatio-Maximizatio (EM) algorithm is a maximum likelihood method for models that have hidde variables eg. Gaussia Mixture Models (GMMs), Liear Dyamic Systems (LDSs) ad Hidde
More informationSection 14. Simple linear regression.
Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo
More informationSupplemental Material: Proofs
Proof to Theorem Supplemetal Material: Proofs Proof. Let be the miimal umber of traiig items to esure a uique solutio θ. First cosider the case. It happes if ad oly if θ ad Rak(A) d, which is a special
More informationMath 778S Spectral Graph Theory Handout #3: Eigenvalues of Adjacency Matrix
Math 778S Spectral Graph Theory Hadout #3: Eigevalues of Adjacecy Matrix The Cartesia product (deoted by G H) of two simple graphs G ad H has the vertex-set V (G) V (H). For ay u, v V (G) ad x, y V (H),
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationBivariate Sample Statistics Geog 210C Introduction to Spatial Data Analysis. Chris Funk. Lecture 7
Bivariate Sample Statistics Geog 210C Itroductio to Spatial Data Aalysis Chris Fuk Lecture 7 Overview Real statistical applicatio: Remote moitorig of east Africa log rais Lead up to Lab 5-6 Review of bivariate/multivariate
More informationSession 5. (1) Principal component analysis and Karhunen-Loève transformation
200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image
More informationNotes The Incremental Motion Model:
The Icremetal Motio Model: The Jacobia Matrix I the forward kiematics model, we saw that it was possible to relate joit agles θ, to the cofiguratio of the robot ed effector T I this sectio, we will see
More information5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.
40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig
More informationEECS 442 Computer vision. Multiple view geometry Affine structure from Motion
EECS 442 Computer visio Multiple view geometry Affie structure from Motio - Affie structure from motio problem - Algebraic methods - Factorizatio methods Readig: [HZ] Chapters: 6,4,8 [FP] Chapter: 2 Some
More information1 1 2 = show that: over variables x and y. [2 marks] Write down necessary conditions involving first and second-order partial derivatives for ( x0, y
Questio (a) A square matrix A= A is called positive defiite if the quadratic form waw > 0 for every o-zero vector w [Note: Here (.) deotes the traspose of a matrix or a vector]. Let 0 A = 0 = show that:
More informationSoo King Lim Figure 1: Figure 2: Figure 3: Figure 4: Figure 5: Figure 6: Figure 7:
0 Multivariate Cotrol Chart 3 Multivariate Normal Distributio 5 Estimatio of the Mea ad Covariace Matrix 6 Hotellig s Cotrol Chart 6 Hotellig s Square 8 Average Value of k Subgroups 0 Example 3 3 Value
More informationNBHM QUESTION 2007 Section 1 : Algebra Q1. Let G be a group of order n. Which of the following conditions imply that G is abelian?
NBHM QUESTION 7 NBHM QUESTION 7 NBHM QUESTION 7 Sectio : Algebra Q Let G be a group of order Which of the followig coditios imply that G is abelia? 5 36 Q Which of the followig subgroups are ecesarily
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More information1 Last time: similar and diagonalizable matrices
Last time: similar ad diagoalizable matrices Let be a positive iteger Suppose A is a matrix, v R, ad λ R Recall that v a eigevector for A with eigevalue λ if v ad Av λv, or equivaletly if v is a ozero
More informationChimica Inorganica 3
himica Iorgaica Irreducible Represetatios ad haracter Tables Rather tha usig geometrical operatios, it is ofte much more coveiet to employ a ew set of group elemets which are matrices ad to make the rule
More informationLast time: Moments of the Poisson distribution from its generating function. Example: Using telescope to measure intensity of an object
6.3 Stochastic Estimatio ad Cotrol, Fall 004 Lecture 7 Last time: Momets of the Poisso distributio from its geeratig fuctio. Gs () e dg µ e ds dg µ ( s) µ ( s) µ ( s) µ e ds dg X µ ds X s dg dg + ds ds
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that
More informationWhy learn matrix algebra? Vectors & Matrices with statistical applications. Brief history of linear algebra
R Vectors & Matrices with statistical applicatios x RXX RXY y RYX RYY Why lear matrix algebra? Simple way to express liear combiatios of variables ad geeral solutios of equatios. Liear statistical models
More informationECON 3150/4150, Spring term Lecture 3
Itroductio Fidig the best fit by regressio Residuals ad R-sq Regressio ad causality Summary ad ext step ECON 3150/4150, Sprig term 2014. Lecture 3 Ragar Nymoe Uiversity of Oslo 21 Jauary 2014 1 / 30 Itroductio
More informationAH Checklist (Unit 3) AH Checklist (Unit 3) Matrices
AH Checklist (Uit 3) AH Checklist (Uit 3) Matrices Skill Achieved? Kow that a matrix is a rectagular array of umbers (aka etries or elemets) i paretheses, each etry beig i a particular row ad colum Kow
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSEMS Versio ECE II, Kharagpur Lesso 8 rasform Codig & K-L rasforms Versio ECE II, Kharagpur Istructioal Oectives At the ed of this lesso, the studets should e ale to:.
More information18.S096: Homework Problem Set 1 (revised)
8.S096: Homework Problem Set (revised) Topics i Mathematics of Data Sciece (Fall 05) Afoso S. Badeira Due o October 6, 05 Exteded to: October 8, 05 This homework problem set is due o October 6, at the
More informationPhysics 324, Fall Dirac Notation. These notes were produced by David Kaplan for Phys. 324 in Autumn 2001.
Physics 324, Fall 2002 Dirac Notatio These otes were produced by David Kapla for Phys. 324 i Autum 2001. 1 Vectors 1.1 Ier product Recall from liear algebra: we ca represet a vector V as a colum vector;
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More information4. Hypothesis testing (Hotelling s T 2 -statistic)
4. Hypothesis testig (Hotellig s T -statistic) Cosider the test of hypothesis H 0 : = 0 H A = 6= 0 4. The Uio-Itersectio Priciple W accept the hypothesis H 0 as valid if ad oly if H 0 (a) : a T = a T 0
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationGRAPHING LINEAR EQUATIONS. Linear Equations ( 3,1 ) _x-axis. Origin ( 0, 0 ) Slope = change in y change in x. Equation for l 1.
GRAPHING LINEAR EQUATIONS Quadrat II Quadrat I ORDERED PAIR: The first umer i the ordered pair is the -coordiate ad the secod umer i the ordered pair is the y-coordiate. (,1 ) Origi ( 0, 0 ) _-ais Liear
More informationOrthogonal transformations
Orthogoal trasformatios October 12, 2014 1 Defiig property The squared legth of a vector is give by takig the dot product of a vector with itself, v 2 v v g ij v i v j A orthogoal trasformatio is a liear
More informationV. Nollau Institute of Mathematical Stochastics, Technical University of Dresden, Germany
PROBABILITY AND STATISTICS Vol. III - Correlatio Aalysis - V. Nollau CORRELATION ANALYSIS V. Nollau Istitute of Mathematical Stochastics, Techical Uiversity of Dresde, Germay Keywords: Radom vector, multivariate
More informationA) is empty. B) is a finite set. C) can be a countably infinite set. D) can be an uncountable set.
M.A./M.Sc. (Mathematics) Etrace Examiatio 016-17 Max Time: hours Max Marks: 150 Istructios: There are 50 questios. Every questio has four choices of which exactly oe is correct. For correct aswer, 3 marks
More informationCLRM estimation Pietro Coretto Econometrics
Slide Set 4 CLRM estimatio Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Thursday 24 th Jauary, 2019 (h08:41) P. Coretto
More informationThe Jordan Normal Form: A General Approach to Solving Homogeneous Linear Systems. Mike Raugh. March 20, 2005
The Jorda Normal Form: A Geeral Approach to Solvig Homogeeous Liear Sstems Mike Raugh March 2, 25 What are we doig here? I this ote, we describe the Jorda ormal form of a matrix ad show how it ma be used
More information11 Correlation and Regression
11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record
More informationMathematics 3 Outcome 1. Vectors (9/10 pers) Lesson, Outline, Approach etc. This is page number 13. produced for TeeJay Publishers by Tom Strang
Vectors (9/0 pers) Mathematics 3 Outcome / Revise positio vector, PQ = q p, commuicative, associative, zero vector, multiplicatio by a scalar k, compoets, magitude, uit vector, (i, j, ad k) as well as
More informationThe Basic Space Model
The Basic Space Model Let x i be the ith idividual s (i=,, ) reported positio o the th issue ( =,, m) ad let X 0 be the by m matrix of observed data here the 0 subscript idicates that elemets are missig
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More informationQuestions and answers, kernel part
Questios ad aswers, kerel part October 8, 205 Questios. Questio : properties of kerels, PCA, represeter theorem. [2 poits] Let F be a RK defied o some domai X, with feature map φ(x) x X ad reproducig kerel
More informationWhere do eigenvalues/eigenvectors/eigenfunctions come from, and why are they important anyway?
Where do eigevalues/eigevectors/eigeuctios come rom, ad why are they importat ayway? I. Bacgroud (rom Ordiary Dieretial Equatios} Cosider the simplest example o a harmoic oscillator (thi o a vibratig strig)
More informationAffine Structure from Motion
Affie Structure from Motio EECS 598-8 Fall 24! Foudatios of Computer Visio!! Istructor: Jaso Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f4!! Readigs: FP 8.2! Date: /5/4!! Materials o these slides
More informationVariable selection in principal components analysis of qualitative data using the accelerated ALS algorithm
Variable selectio i pricipal compoets aalysis of qualitative data usig the accelerated ALS algorithm Masahiro Kuroda Yuichi Mori Masaya Iizuka Michio Sakakihara (Okayama Uiversity of Sciece) (Okayama Uiversity
More informationMatrix Representation of Data in Experiment
Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y
More informationAssumptions. Motivation. Linear Transforms. Standard measures. Correlation. Cofactor. γ k
Outlie Pricipal Compoet Aalysis Yaju Ya Itroductio of PCA Mathematical basis Calculatio of PCA Applicatios //04 ELE79, Sprig 004 What is PCA? Pricipal Compoets Pricipal Compoet Aalysis, origially developed
More informationALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES.
ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES. ANDREW SALCH 1. The Jacobia criterio for osigularity. You have probably oticed by ow that some poits o varieties are smooth i a sese somethig
More informationTridiagonal reduction redux
Week 15: Moday, Nov 27 Wedesday, Nov 29 Tridiagoal reductio redux Cosider the problem of computig eigevalues of a symmetric matrix A that is large ad sparse Usig the grail code i LAPACK, we ca compute
More informationCorrelation Regression
Correlatio Regressio While correlatio methods measure the stregth of a liear relatioship betwee two variables, we might wish to go a little further: How much does oe variable chage for a give chage i aother
More informationStochastic Matrices in a Finite Field
Stochastic Matrices i a Fiite Field Abstract: I this project we will explore the properties of stochastic matrices i both the real ad the fiite fields. We first explore what properties 2 2 stochastic matrices
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationMATH10212 Linear Algebra B Proof Problems
MATH22 Liear Algebra Proof Problems 5 Jue 26 Each problem requests a proof of a simple statemet Problems placed lower i the list may use the results of previous oes Matrices ermiats If a b R the matrix
More informationNonlinear regression
oliear regressio How to aalyse data? How to aalyse data? Plot! How to aalyse data? Plot! Huma brai is oe the most powerfull computatioall tools Works differetly tha a computer What if data have o liear
More informationFilter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and
Filter bas Separately, the lowpass ad highpass filters are ot ivertible T removes the highest frequecy / ad removes the lowest frequecy Together these filters separate the sigal ito low-frequecy ad high-frequecy
More informationClassification of DT signals
Comlex exoetial A discrete time sigal may be comlex valued I digital commuicatios comlex sigals arise aturally A comlex sigal may be rereseted i two forms: jarg { z( ) } { } z ( ) = Re { z ( )} + jim {
More information10/2/ , 5.9, Jacob Hays Amit Pillay James DeFelice
0//008 Liear Discrimiat Fuctios Jacob Hays Amit Pillay James DeFelice 5.8, 5.9, 5. Miimum Squared Error Previous methods oly worked o liear separable cases, by lookig at misclassified samples to correct
More information15.083J/6.859J Integer Optimization. Lecture 3: Methods to enhance formulations
15.083J/6.859J Iteger Optimizatio Lecture 3: Methods to ehace formulatios 1 Outlie Polyhedral review Slide 1 Methods to geerate valid iequalities Methods to geerate facet defiig iequalities Polyhedral
More informationMachine Learning Regression I Hamid R. Rabiee [Slides are based on Bishop Book] Spring
Machie Learig Regressio I Hamid R. Rabiee [Slides are based o Bishop Book] Sprig 015 http://ce.sharif.edu/courses/93-94//ce717-1 Liear Regressio Liear regressio: ivolves a respose variable ad a sigle predictor
More informationChapter 3 Inner Product Spaces. Hilbert Spaces
Chapter 3 Ier Product Spaces. Hilbert Spaces 3. Ier Product Spaces. Hilbert Spaces 3.- Defiitio. A ier product space is a vector space X with a ier product defied o X. A Hilbert space is a complete ier
More informationCMSE 820: Math. Foundations of Data Sci.
Lecture 17 8.4 Weighted path graphs Take from [10, Lecture 3] As alluded to at the ed of the previous sectio, we ow aalyze weighted path graphs. To that ed, we prove the followig: Theorem 6 (Fiedler).
More informationBrief Review of Functions of Several Variables
Brief Review of Fuctios of Several Variables Differetiatio Differetiatio Recall, a fuctio f : R R is differetiable at x R if ( ) ( ) lim f x f x 0 exists df ( x) Whe this limit exists we call it or f(
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More information