Some multivariate methods
|
|
- Ashlee McCoy
- 5 years ago
- Views:
Transcription
1 /7/ Outline Some multivrite methods VERIE CRDENS, PH.D. SSOCIE DJUNC PROFESSOR DEPREN OF RDIOOGY ND BIOEDIC IGING Useful liner lgebr Principl Components nlysis (PC) Independent Components nlysis (IC) Joint IC Prllel IC Prtil est Squres (PS) Cnonicl Correltion nlysis (CC) Ridge regression Vector vector is defined s n ordered rry of numbers, of dimensions p, Below is vector c of dimensions c Nottion: vectors re typiclly denoted by lowercse bold letters tri mtri is defined s n ordered rry of numbers, of dimensions p, q (p rows, q columns) Below is mtri of dimensions Nottion: mtrices re typiclly denoted by uppercse bold letters
2 /7/ ore mtri nottion You cn think of mtri s collection of column vectors of dimension p, c c c You cn think of mtri s collection of row vectors of dimension, q r r r ore mtri nottion he elements of mtri re denoted by i,j, where i refers to the row position nd j to the column position he elements of vector re denoted by c i where i refers to the row position c c c c ypes of mtrices rectngulr p q squre digonl ij, i j 6 4 squre p q ij, i j digonl ii symmetric ij ji rnspose of mtri/vector he mtri is composed of elements ij rnspose of, denoted or hs elements ji [ ] v v v
3 /7/ Vector/tri ddition nd Sclr ultipliction If vectors nd mtrices hve sme number of rows/columns, they cn be dded (or subtrcted) element by element. Vectors nd mtrices cn be multiplied by sclr l t b l t element by element. + +, tri multipliction iner Combintion of Columns For B, ech column of B genertes column of the product B Ech column of B contins set of liner weights hese liner weights re pplied to the columns of to produce single column of numbers B B tri multipliction iner Combintion of Rows For B, ech row of genertes row of the product B Ech row of contins set of liner weights hese liner weights re pplied to the rows of B to produce single row vector of numbers
4 /7/ Determinnt of squre mtri B [ ] + [ ] [ 9 8] [ ] + [ ] [ 8 4 9] B he determinnt of mtri, denoted, is sclr function tht is zero if n only if mtri is of deficient rnk. he rnk is the number of linerly independent rows nd columns of. linerly independent column is one tht is not liner combintion of other columns in the mtri If ny columns of re liner combintion of some other columns of, then is not full rnk. Eigenvlues nd eigenvectors Eigenvectors of symmetric mtri For squre mtri, sclr c nd non-zero vector v re n eigenvlue nd ssocited eigenvector if nd only if they stisfy the eqution, v cv For symmetric, for distinct eigenvlues c i, c j with ssocited eigenvectors v i, v j, v i v j v i nd v j re orthogonl v i nd v j re linerly independent Interprettion: multipliction of n eigenvector by the mtri does not chnge the direction, but only the mgnitude of the originl vector. he eigenvlue is the fctor by which the eigenvector chnges when multiplied by the mtri. 4
5 /7/ Eigendecomposition of symmetric mtri et be rel nd symmetric. here eists mtri Q such tht QΛQ where Q is the squre n n mtri whose i th column is the bsis eigenvector q i of nd Λ is the digonl mtri whose digonl elements re the corresponding eigenvlues. tri pproimtion If the eigenvectors nd eigenvlues of re ordered in the mtrices Λ nd Q in descending order, such tht the first element in Λ is the lrgest eigenvlue of, nd the first column in Q is its corresponding eigenvector. Define Q* s the first m columns of Q, nd D* s n m m digonl mtri with the corresponding m eigenvlues s digonl entries. hen * * * Q D Q i.e., mtri of rnk m tht is the best rnk m pproimtion of. Singulr Vlue Decomposition Our imging problem tri fctoriztion good for rel or comple mtri et be n m n mtri, the SVD tkes the form * UV U is n m m rel or comple unitry mtri V* is n n n rel or comple unitry mtri he digonl entries ij of re the singulr vlues If is positive semi-definite, then the SVD if n eigendecomposition of Often used to compute pseudoinverse of, or s low rnk pproimtion of Given rectngulr nd the SVD of, the following holds V( )V U( )U Sttisticl nlysis of medicl imges common Underdetermined problem housnds to millions of sptil vribles (voels) Usully < observtions (subjects) ypicl solution: divide id into subproblems bl Ech subproblem reltes single voel to clinicl vrible Known s voel-wise, univrite, or pointwise regression pproch populrized by SP (Friston, 99) Dependencies between sptil vribles neglected!!
6 /7/ Emple problem: observtions, 4 sptil vribles Solutions? β β β β 4 4 p ; p ; p ; coefficient mp -sttistic mp outcome y outcome y outcome y y β 4 β y 4β y 4 β 4 Reduce dimensionlity PC IC CC dd constrint t to sum of squres Ridge regression SSO techniques ethods to reduce dimensionlity PC IC JOIN IC PRE IC PS CC PC: Principl Components nlysis Procedure to convert set of observtions of possibly correlted vribles into set of uncorrelted vribles clled principl components We know voels in our imges re sptilly correlted PC ims to trnsform millions of vribles (voels) to few Projection of dt: high to low dimension Emple: n p dt, subjects, imges with voels X,,,,,, O,,, PC y ) y Y y,,, y y y,,, O y, y, y, 6
7 /7/ Wht re principl components? Principl components re liner combintions of the observed vribles y b + b + + b where is column vector of i the originl dt mtri X. In our imging emples, liner combintions of the dt columns (voels) he coefficients of these principl components re chosen to meet three criteri Wht re the three criteri? criteri of Principl Components here re ectly p principl components (PCs), ech being liner combintion of the observed vribles p is number of vribles (columns) in originl dt he PCs re mutully orthogonl ogo (i.e., perpendiculr nd uncorrelted) he components re etrcted in order of decresing vrince he first PC eplins s much of the vribility in the full dt set s possible he second PC eplins s much vribility s possible fter the vribility from the first PC hs been removed, etc. Usul steps in PC Eigenfces emple Hve t lest two vribles (usully you think tht these vribles re inter-relted) Generte correltion or vrince-covrince mtri en center dt mtri X (subtrct men from ech vrible) hen (/(n-))x X is the vrince-covrince mtri Obtin eigenvlues nd eigenvectors Use SVD or other mtri decomposition he first eigenvector is the direction eplining the most vrince in the dt mtri X he first eigenvlue is the mount of vrince eplined Select subset of the eigenvectors (principl components) Sum eigenvlues until threshold (9%?) is reched Generte PC scores Reduced spce Cn be used in subsequent regression/visuliztion Fce recognition, efficient storge Prepre trining set of fce imges sme resolution, lighting, normlized so tht fetures lign Store trining set in mtri F, where ech row is trining fce Imge, row Imge, row Imge, row n Imge, row Imge, row Imge, row n F Imge, row Imge, row Imge, row n Imge, row Imge, row Imge, row n Imge m, row Imge m, row Imge m, row n 7
8 /7/ rining fces Eigenfces, cont. PC of F he principl components, v i, of F re eigenfces he principl component scores re obtined by FV, where V is the mtri of principl components he score is the contribution of ech principl component to the originl fce Cn store the eigenfces nd scores, insted of entire imge If imge hs, voels nd 4 eigenfces describe 98% of the vribility in fces, then for ech new fce need only record 4 scores (not, voels) Principl component regression Independent components nlysis Principl components cn be used for dt reduction prior to regression Y β,, O,,, In PC, the PCs re orthogonl (uncorrelted) In IC, the components re defined to be mimlly sttisticlly independent stronger requirement Independence: knowing gives you no informtion bout y If fdt re Gussin then uncorrelted dimplies independence d Do PC on, then do regression on the scores Y PC PC scores scores β pc pc pc, pc pc pc, O pc, pc, pc, Uncorrelted but not independent Vr Vr 8
9 /7/ IC, cont. IC tends to do better for etrcting useful ptterns in sets of imges, becuse high-dimensionl dtsets typiclly hve strong non-gussinity Computtionlly more chllenging (not simple mtri ti decomposition) No inherent order of components Components my lso be scled IC lso known s BSS: blind source seprtion Forml sttement of problem N independent sources Z (m n) iing mtri. (n n) Produces set of observtions X (m n) X Z Wnt to demi observtions X into Y WX Y Z W IC is trying to estimte W PC vs. IC PC solution: PCs eplin m vrince 9
10 /7/ IC solution: projecting dt onto IC nd IC gives bck two sinusoids Principl nd Independent components IC eploits the non-gussinity of source signls IC: he bsic ide esures of sttisticl independence ssume underlying source signls (Z ) re independent. ssume liner miing mtri ( ) X Z in order to find Y ( Z ), find W, ( - )... Y WX Requires mesure of sttisticl independence which we mimize between ech of the components Non-gussinity (mimize kurtosis) utul informtion (minimize between components) Entropy mimize rndomness Entropy-mimize rndomness imum log likelihood How? Initilise W nd itertively updte W to minimise or mimise cost function tht mesures the (sttisticl) independence between the columns of the Y Cnnot solve using mtri decomposition
11 /7/ Joint IC Prllel IC Vrition on IC to look for components tht pper jointly cross fetures or modlities D imge Discover independent components from two modlities, in ddition to the reltionship between them D imge, control F imge, control F imge X D imge, control F imge, control Observed dt D imge, ptient n D imge, ptient Fetures/imges cross subjects X W Y Joint independent components Component weights/profile Sources dd constrint to independence m { H ( Y ) + H ( Y ) + Corr(, } ) imizing the entropy of the sources in ech modlity nd the correltion between columns of the miing mtri Prtil lest squres (PS) Relted to PC regression YXβ PC of X, keep some PCs nd predict Y PCs eplin vribility in X only hese components my not eplin Y t ll PS finds components of X tht re lso relevnt to Y ltent vectors re components tht simultneously decompose X nd Y tent vectors eplin the covrince between X nd Y Find two sets of weights to crete liner combintions of columns of X nd Y to mimize covrince
12 /7/ PS steps PS Emple Compute X Y-covrince of X nd Y Do SVD of X Y cn clculte the first ltent vector nd lodings from this Subtrct or prtil out the effect of the first ltent vector from X nd Y to crete X nd Y Repet until X is null 6 cognitive mesures on ptients with mild cognitive impirment -weighted imges normlized to tls PS between cognitive mesures nd moment Similr to PC, cn choose subset of ltent vectors to pproimte the prediction of Y nd chieve substntil dt reduction. First ltent vrible tent vrible scores Regions of reltive contrction (blue) nd epnsion (red) relted to V. Scores cn be computed for ech subject, perhps on reduced number of ltent vribles, nd used in regression nlysis.
13 /7/ Cnonicl Correltion nlysis CC, cont. Investigte the reltionship between two sets of vribles X D imge fetures, control D imge fetures, control Y F imge fetures, control F imge fetures, control Find pirs of liner combintions of vribles tht re uncorrelted hese pirs re the cnonicl vrites he dt: set of p independent vribles X, X,, X p nd q dependent d vribles Y, Y,, Y q, mesured on smple of N objects, from which we cn derive (p + q) X (p + q) correltion mtri. D imge fetures, ptient n F imge fetures, ptient n D imge D imge, control F imge, control D imge F imge F imge D imge, control F imge, control D imge, ptient n F imge, ptient n r rp r rq CC correltion mtri Within set (X) correltion between set (X,Y) correltion r p r r q O O r pp rp rpq r p r r q O O r qp rq rqq XX YX XY YY Within set (Y) correltion Wht re cnonicl vrites? Cnonicl vrites re the eigenvectors of the corresponding correltion mtri Orthogonl Spn vribility in either X or Y X ξ U ( U ) ξ U ( U ) X X X Y X XX YX Y XY YY Y ξ V ξ ( V ) Y V ( V )
14 /7/ Estimting cnonicl vrites Estimting cnonicl vrites, cont. he first cnonicl vrite is obtined by finding coefficients of the liner functions p U j X j j q V b Y j j j which mimizes the correltion between U nd V { r( U, )} r ( U, V ) m V he second cnonicl vrite is obtined by finding coefficients of the liner functions p X j j U j q V which mimizes the correltion between U nd V r U, V ) m{ r( U, )} Subject to the following constrints r( U, U ) r( V, V ) j b jy j ( V r( U, V ) r( U, V ) Clculting cnonicl vrites he end result is n eigenvector of: b is n eigenvector of: XX XY YY YY YX XX he squred cnonicl correltion r i is the corresponding eigenvlue YX XY set of r min(p,q) cnonicl vrites, one for the dependent vrible set {V}, the other for the independent vrible set {U} set of r cnonicl correltions C r(u,v) ech representing the correltion between pirs of cnonicl vrites U U High first cnonicl correltion X V ow second cnonicl correltion V 4
15 /7/ Significnce testing Interprettion: cnonicl coefficients Ech CV (cnonicl vrite) is tested in hierrchicl fshion by first testing significnce of ll CVs If ll CVs combined NS, then no CV is significnt If ll CVs combined re significnt, then remove first CV, reclculte test sttistic nd test Continue until test sttistic NS Emine stndrdized coefficients of cnonicl vrites Inference: vribles with lrge (in bsolute vlue) coefficients re most importnt U.9X.9 X +.48X +.9X 4 U minly contrst between X nd X 4 on the one hnd, nd X on the other Interprettion: cnonicl lodings Considertions Emine correltions of originl vribles with cnonicl vrites Inference: vribles with lrge (in bsolute vlue) correltions re most importnt Cnonicl vrite Vrible U U X -.9. X X.9 -. X he vrince of U nd V will be influenced by the scling dopted, but the cnonicl correltions will be unffected he rtio of smple size to totl number of vribles should be lrge (> for cnonicl vrites, > 4 for two cnonicl vrites) X 4 is not relted to U
16 /7/ Ridge regression Constrined sum of squres RIDGE REGRESSION SSO ECHNIQUES Well estblished, widespred method Hoerl nd Kennrd, 97; rqurdt, 97 pplictions to neuroimging (Vldes-Sos, ) Regulrizes underdetermined problem by dding constrint to prmeter sum-of-squres Generliztion of pointwise regression Ordinry lest squres y Xβ+ ε, min ε ε min y-xβ β - β (X X) X y y: n observtions, subjects X: n p independent vribles Solution vlid if X X full-rnk β β: p regression coefficients ε: n residuls Ridge regression solution ( y-xβ λ β ) min +, where λ β - β ridge (X X+ λi) X y λ shrinks bsolute size of coefficients β Shrinkge introduces bis s λ grows, coefficients driven to If p>n, X X will never be full rnk! 6
17 /7/ Ridge trce Computtion X U % D % V % nd X U D V n n n p p p n k k k k p U, V orthonorml columns k is rnk of X, nd number of nonzero d D digonl with elements d i i - β ridge V(D + λi) R y RUD (n k) (D +λi) k k digonl, esily inverted Compleity order pk insted of p s λ Ridge Z βi( λ) zi ( λ) σ β ( λ) σ i ε ω i X y ω i X Xω i z i (λ) is i th z-sttistic ω I is i th column of Ω(X X+λI) - i y ( i i) i lim zi ( ) λ y λ λ σ σ ( i i) ε i i ε λ his is equivlent to the pointwise estimtion of z i! 7
18 /7/ ppliction to Deformtion orphometry Sptil correltion structure 8 cognitively impired nd 7 control subjects Verbl memory ssessed t bseline nd yr Bseline RI deformtion mps creted using B- spline-bsed nonliner registrtion (Studholme 4) Pointwise regression Dependent vribles were deformtion mps Independent vribles were chnge in verbl memory Convenient nd computtionlly efficient construction Ridge regression Dependent vribles were chnge in verbl memory Sptil vribles from mps were independent, p86, Deformtion ssocited with delyed memory SSO est bsolute Shrinkge nd Selection Opertor Definition It s coefficients shrunken version of the ordinry est Squre Estimte, by minimizing the Residul Sum of Squres subjecting to the constrint tht the sum of the bsolute vlue of the coefficients should be no greter thn constnt
19 /7/ Solving SSO No solution by decomposition or simple liner lgebr ust use itertive methods 9
Multivariate problems and matrix algebra
University of Ferrr Stefno Bonnini Multivrite problems nd mtrix lgebr Multivrite problems Multivrite sttisticl nlysis dels with dt contining observtions on two or more chrcteristics (vribles) ech mesured
More informationThe Algebra (al-jabr) of Matrices
Section : Mtri lgebr nd Clculus Wshkewicz College of Engineering he lgebr (l-jbr) of Mtrices lgebr s brnch of mthemtics is much broder thn elementry lgebr ll of us studied in our high school dys. In sense
More informationSCHOOL OF ENGINEERING & BUILT ENVIRONMENT. Mathematics
SCHOOL OF ENGINEERING & BUIL ENVIRONMEN Mthemtics An Introduction to Mtrices Definition of Mtri Size of Mtri Rows nd Columns of Mtri Mtri Addition Sclr Multipliction of Mtri Mtri Multipliction 7 rnspose
More informationData Assimilation. Alan O Neill Data Assimilation Research Centre University of Reading
Dt Assimiltion Aln O Neill Dt Assimiltion Reserch Centre University of Reding Contents Motivtion Univrite sclr dt ssimiltion Multivrite vector dt ssimiltion Optiml Interpoltion BLUE 3d-Vritionl Method
More informationMath 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Mth 520 Finl Exm Topic Outline Sections 1 3 (Xio/Dums/Liw) Spring 2008 The finl exm will be held on Tuesdy, My 13, 2-5pm in 117 McMilln Wht will be covered The finl exm will cover the mteril from ll of
More informationNon-Linear & Logistic Regression
Non-Liner & Logistic Regression If the sttistics re boring, then you've got the wrong numbers. Edwrd R. Tufte (Sttistics Professor, Yle University) Regression Anlyses When do we use these? PART 1: find
More informationA Matrix Algebra Primer
A Mtrix Algebr Primer Mtrices, Vectors nd Sclr Multipliction he mtrix, D, represents dt orgnized into rows nd columns where the rows represent one vrible, e.g. time, nd the columns represent second vrible,
More informationChapter 3 MATRIX. In this chapter: 3.1 MATRIX NOTATION AND TERMINOLOGY
Chpter 3 MTRIX In this chpter: Definition nd terms Specil Mtrices Mtrix Opertion: Trnspose, Equlity, Sum, Difference, Sclr Multipliction, Mtrix Multipliction, Determinnt, Inverse ppliction of Mtrix in
More informationHow do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?
XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk bout solving systems of liner equtions. These re problems tht give couple of equtions with couple of unknowns, like: 6 2 3 7 4
More informationThings to Memorize: A Partial List. January 27, 2017
Things to Memorize: A Prtil List Jnury 27, 2017 Chpter 2 Vectors - Bsic Fcts A vector hs mgnitude (lso clled size/length/norm) nd direction. It does not hve fixed position, so the sme vector cn e moved
More information1 Linear Least Squares
Lest Squres Pge 1 1 Liner Lest Squres I will try to be consistent in nottion, with n being the number of dt points, nd m < n being the number of prmeters in model function. We re interested in solving
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 2
CS434/54: Pttern Recognition Prof. Olg Veksler Lecture Outline Review of Liner Algebr vectors nd mtrices products nd norms vector spces nd liner trnsformtions eigenvlues nd eigenvectors Introduction to
More informationMatrix Eigenvalues and Eigenvectors September 13, 2017
Mtri Eigenvlues nd Eigenvectors September, 7 Mtri Eigenvlues nd Eigenvectors Lrry Cretto Mechnicl Engineering 5A Seminr in Engineering Anlysis September, 7 Outline Review lst lecture Definition of eigenvlues
More informationLINEAR ALGEBRA AND MATRICES. n ij. is called the main diagonal or principal diagonal of A. A column vector is a matrix that has only one column.
PART 1 LINEAR ALGEBRA AND MATRICES Generl Nottions Mtri (denoted by cpitl boldfce letter) A is n m n mtri. 11 1... 1 n 1... n A ij...... m1 m... mn ij denotes the component t row i nd column j of A. If
More informationSCHOOL OF ENGINEERING & BUILT ENVIRONMENT
SCHOOL OF ENGINEERING & BUIL ENVIRONMEN MARICES FOR ENGINEERING Dr Clum Mcdonld Contents Introduction Definitions Wht is mtri? Rows nd columns of mtri Order of mtri Element of mtri Equlity of mtrices Opertions
More informationElements of Matrix Algebra
Elements of Mtrix Algebr Klus Neusser Kurt Schmidheiny September 30, 2015 Contents 1 Definitions 2 2 Mtrix opertions 3 3 Rnk of Mtrix 5 4 Specil Functions of Qudrtic Mtrices 6 4.1 Trce of Mtrix.........................
More informationPartial Derivatives. Limits. For a single variable function f (x), the limit lim
Limits Prtil Derivtives For single vrible function f (x), the limit lim x f (x) exists only if the right-hnd side limit equls to the left-hnd side limit, i.e., lim f (x) = lim f (x). x x + For two vribles
More informationINTRODUCTION TO LINEAR ALGEBRA
ME Applied Mthemtics for Mechnicl Engineers INTRODUCTION TO INEAR AGEBRA Mtrices nd Vectors Prof. Dr. Bülent E. Pltin Spring Sections & / ME Applied Mthemtics for Mechnicl Engineers INTRODUCTION TO INEAR
More informationGeometric Sequences. Geometric Sequence a sequence whose consecutive terms have a common ratio.
Geometric Sequences Geometric Sequence sequence whose consecutive terms hve common rtio. Geometric Sequence A sequence is geometric if the rtios of consecutive terms re the sme. 2 3 4... 2 3 The number
More informationChapter 4 Contravariance, Covariance, and Spacetime Diagrams
Chpter 4 Contrvrince, Covrince, nd Spcetime Digrms 4. The Components of Vector in Skewed Coordintes We hve seen in Chpter 3; figure 3.9, tht in order to show inertil motion tht is consistent with the Lorentz
More informationTests for the Ratio of Two Poisson Rates
Chpter 437 Tests for the Rtio of Two Poisson Rtes Introduction The Poisson probbility lw gives the probbility distribution of the number of events occurring in specified intervl of time or spce. The Poisson
More informationMATRICES AND VECTORS SPACE
MATRICES AND VECTORS SPACE MATRICES AND MATRIX OPERATIONS SYSTEM OF LINEAR EQUATIONS DETERMINANTS VECTORS IN -SPACE AND -SPACE GENERAL VECTOR SPACES INNER PRODUCT SPACES EIGENVALUES, EIGENVECTORS LINEAR
More informationMatrix Algebra. Matrix Addition, Scalar Multiplication and Transposition. Linear Algebra I 24
Mtrix lger Mtrix ddition, Sclr Multipliction nd rnsposition Mtrix lger Section.. Mtrix ddition, Sclr Multipliction nd rnsposition rectngulr rry of numers is clled mtrix ( the plurl is mtrices ) nd the
More informationSOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS
ELM Numericl Anlysis Dr Muhrrem Mercimek SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS ELM Numericl Anlysis Some of the contents re dopted from Lurene V. Fusett, Applied Numericl Anlysis using MATLAB.
More informationLecture Solution of a System of Linear Equation
ChE Lecture Notes, Dept. of Chemicl Engineering, Univ. of TN, Knoville - D. Keffer, 5/9/98 (updted /) Lecture 8- - Solution of System of Liner Eqution 8. Why is it importnt to e le to solve system of liner
More informationElementary Linear Algebra
Elementry Liner Algebr Anton & Rorres, 1 th Edition Lecture Set 5 Chpter 4: Prt II Generl Vector Spces 163 คณ ตศาสตร ว ศวกรรม 3 สาขาว ชาว ศวกรรมคอมพ วเตอร ป การศ กษา 1/2555 163 คณตศาสตรวศวกรรม 3 สาขาวชาวศวกรรมคอมพวเตอร
More informationOperations with Polynomials
38 Chpter P Prerequisites P.4 Opertions with Polynomils Wht you should lern: How to identify the leding coefficients nd degrees of polynomils How to dd nd subtrct polynomils How to multiply polynomils
More informationNumerical Linear Algebra Assignment 008
Numericl Liner Algebr Assignment 008 Nguyen Qun B Hong Students t Fculty of Mth nd Computer Science, Ho Chi Minh University of Science, Vietnm emil. nguyenqunbhong@gmil.com blog. http://hongnguyenqunb.wordpress.com
More informationDuality # Second iteration for HW problem. Recall our LP example problem we have been working on, in equality form, is given below.
Dulity #. Second itertion for HW problem Recll our LP emple problem we hve been working on, in equlity form, is given below.,,,, 8 m F which, when written in slightly different form, is 8 F Recll tht we
More informationARITHMETIC OPERATIONS. The real numbers have the following properties: a b c ab ac
REVIEW OF ALGEBRA Here we review the bsic rules nd procedures of lgebr tht you need to know in order to be successful in clculus. ARITHMETIC OPERATIONS The rel numbers hve the following properties: b b
More informationStudent Activity 3: Single Factor ANOVA
MATH 40 Student Activity 3: Single Fctor ANOVA Some Bsic Concepts In designed experiment, two or more tretments, or combintions of tretments, is pplied to experimentl units The number of tretments, whether
More informationThe Islamic University of Gaza Faculty of Engineering Civil Engineering Department. Numerical Analysis ECIV Chapter 11
The Islmic University of Gz Fculty of Engineering Civil Engineering Deprtment Numericl Anlysis ECIV 6 Chpter Specil Mtrices nd Guss-Siedel Associte Prof Mzen Abultyef Civil Engineering Deprtment, The Islmic
More informationChapter 14. Matrix Representations of Linear Transformations
Chpter 4 Mtrix Representtions of Liner Trnsformtions When considering the Het Stte Evolution, we found tht we could describe this process using multipliction by mtrix. This ws nice becuse computers cn
More informationModule 6: LINEAR TRANSFORMATIONS
Module 6: LINEAR TRANSFORMATIONS. Trnsformtions nd mtrices Trnsformtions re generliztions of functions. A vector x in some set S n is mpped into m nother vector y T( x). A trnsformtion is liner if, for
More informationECON 331 Lecture Notes: Ch 4 and Ch 5
Mtrix Algebr ECON 33 Lecture Notes: Ch 4 nd Ch 5. Gives us shorthnd wy of writing lrge system of equtions.. Allows us to test for the existnce of solutions to simultneous systems. 3. Allows us to solve
More informationCSCI 5525 Machine Learning
CSCI 555 Mchine Lerning Some Deini*ons Qudrtic Form : nn squre mtri R n n : n vector R n the qudrtic orm: It is sclr vlue. We oten implicitly ssume tht is symmetric since / / I we write it s the elements
More informationSUMMER KNOWHOW STUDY AND LEARNING CENTRE
SUMMER KNOWHOW STUDY AND LEARNING CENTRE Indices & Logrithms 2 Contents Indices.2 Frctionl Indices.4 Logrithms 6 Exponentil equtions. Simplifying Surds 13 Opertions on Surds..16 Scientific Nottion..18
More informationMatrices and Determinants
Nme Chpter 8 Mtrices nd Determinnts Section 8.1 Mtrices nd Systems of Equtions Objective: In this lesson you lerned how to use mtrices, Gussin elimintion, nd Guss-Jordn elimintion to solve systems of liner
More informationAlgebra Of Matrices & Determinants
lgebr Of Mtrices & Determinnts Importnt erms Definitions & Formule 0 Mtrix - bsic introduction: mtrix hving m rows nd n columns is clled mtrix of order m n (red s m b n mtrix) nd mtrix of order lso in
More informationChapter 2. Vectors. 2.1 Vectors Scalars and Vectors
Chpter 2 Vectors 2.1 Vectors 2.1.1 Sclrs nd Vectors A vector is quntity hving both mgnitude nd direction. Emples of vector quntities re velocity, force nd position. One cn represent vector in n-dimensionl
More informationMath 270A: Numerical Linear Algebra
Mth 70A: Numericl Liner Algebr Instructor: Michel Holst Fll Qurter 014 Homework Assignment #3 Due Give to TA t lest few dys before finl if you wnt feedbck. Exercise 3.1. (The Bsic Liner Method for Liner
More informationHow do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?
XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk out solving systems of liner equtions. These re prolems tht give couple of equtions with couple of unknowns, like: 6= x + x 7=
More informationGoals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite
Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite
More informationHere we study square linear systems and properties of their coefficient matrices as they relate to the solution set of the linear system.
Section 24 Nonsingulr Liner Systems Here we study squre liner systems nd properties of their coefficient mtrices s they relte to the solution set of the liner system Let A be n n Then we know from previous
More informationNew Expansion and Infinite Series
Interntionl Mthemticl Forum, Vol. 9, 204, no. 22, 06-073 HIKARI Ltd, www.m-hikri.com http://dx.doi.org/0.2988/imf.204.4502 New Expnsion nd Infinite Series Diyun Zhng College of Computer Nnjing University
More informationChapter 2. Determinants
Chpter Determinnts The Determinnt Function Recll tht the X mtrix A c b d is invertible if d-bc0. The expression d-bc occurs so frequently tht it hs nme; it is clled the determinnt of the mtrix A nd is
More informationTHE DISCRIMINANT & ITS APPLICATIONS
THE DISCRIMINANT & ITS APPLICATIONS The discriminnt ( Δ ) is the epression tht is locted under the squre root sign in the qudrtic formul i.e. Δ b c. For emple: Given +, Δ () ( )() The discriminnt is used
More informationChapter 5 : Continuous Random Variables
STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll
More information5 Probability densities
5 Probbility densities 5. Continuous rndom vribles 5. The norml distribution 5.3 The norml pproimtion to the binomil distribution 5.5 The uniorm distribution 5. Joint distribution discrete nd continuous
More informationIs there an easy way to find examples of such triples? Why yes! Just look at an ordinary multiplication table to find them!
PUSHING PYTHAGORAS 009 Jmes Tnton A triple of integers ( bc,, ) is clled Pythgoren triple if exmple, some clssic triples re ( 3,4,5 ), ( 5,1,13 ), ( ) fond of ( 0,1,9 ) nd ( 119,10,169 ). + b = c. For
More informationLecture 3 Gaussian Probability Distribution
Introduction Lecture 3 Gussin Probbility Distribution Gussin probbility distribution is perhps the most used distribution in ll of science. lso clled bell shped curve or norml distribution Unlike the binomil
More informationSeptember 13 Homework Solutions
College of Engineering nd Computer Science Mechnicl Engineering Deprtment Mechnicl Engineering 5A Seminr in Engineering Anlysis Fll Ticket: 5966 Instructor: Lrry Cretto Septemer Homework Solutions. Are
More informationHW3, Math 307. CSUF. Spring 2007.
HW, Mth 7. CSUF. Spring 7. Nsser M. Abbsi Spring 7 Compiled on November 5, 8 t 8:8m public Contents Section.6, problem Section.6, problem Section.6, problem 5 Section.6, problem 7 6 5 Section.6, problem
More informationThe steps of the hypothesis test
ttisticl Methods I (EXT 7005) Pge 78 Mosquito species Time of dy A B C Mid morning 0.0088 5.4900 5.5000 Mid Afternoon.3400 0.0300 0.8700 Dusk 0.600 5.400 3.000 The Chi squre test sttistic is the sum of
More informationMatrices, Moments and Quadrature, cont d
Jim Lmbers MAT 285 Summer Session 2015-16 Lecture 2 Notes Mtrices, Moments nd Qudrture, cont d We hve described how Jcobi mtrices cn be used to compute nodes nd weights for Gussin qudrture rules for generl
More informationChapter 3 Polynomials
Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling
More informationSTURM-LIOUVILLE THEORY, VARIATIONAL APPROACH
STURM-LIOUVILLE THEORY, VARIATIONAL APPROACH XIAO-BIAO LIN. Qudrtic functionl nd the Euler-Jcobi Eqution The purpose of this note is to study the Sturm-Liouville problem. We use the vritionl problem s
More information8 Laplace s Method and Local Limit Theorems
8 Lplce s Method nd Locl Limit Theorems 8. Fourier Anlysis in Higher DImensions Most of the theorems of Fourier nlysis tht we hve proved hve nturl generliztions to higher dimensions, nd these cn be proved
More informationOperations with Matrices
Section. Equlit of Mtrices Opertions with Mtrices There re three ws to represent mtri.. A mtri cn be denoted b n uppercse letter, such s A, B, or C.. A mtri cn be denoted b representtive element enclosed
More informationIntroduction to Group Theory
Introduction to Group Theory Let G be n rbitrry set of elements, typiclly denoted s, b, c,, tht is, let G = {, b, c, }. A binry opertion in G is rule tht ssocites with ech ordered pir (,b) of elements
More informationChapter 5 Determinants
hpter 5 Determinnts 5. Introduction Every squre mtri hs ssocited with it sclr clled its determinnt. Given mtri, we use det() or to designte its determinnt. We cn lso designte the determinnt of mtri by
More informationContinuous Random Variables
STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht
More informationPre-Session Review. Part 1: Basic Algebra; Linear Functions and Graphs
Pre-Session Review Prt 1: Bsic Algebr; Liner Functions nd Grphs A. Generl Review nd Introduction to Algebr Hierrchy of Arithmetic Opertions Opertions in ny expression re performed in the following order:
More informationMatrices. Elementary Matrix Theory. Definition of a Matrix. Matrix Elements:
Mtrices Elementry Mtrix Theory It is often desirble to use mtrix nottion to simplify complex mthemticl expressions. The simplifying mtrix nottion usully mkes the equtions much esier to hndle nd mnipulte.
More informationCHAPTER 2d. MATRICES
CHPTER d. MTRICES University of Bhrin Deprtment of Civil nd rch. Engineering CEG -Numericl Methods in Civil Engineering Deprtment of Civil Engineering University of Bhrin Every squre mtrix hs ssocited
More informationContents. Outline. Structured Rank Matrices Lecture 2: The theorem Proofs Examples related to structured ranks References. Structure Transport
Contents Structured Rnk Mtrices Lecture 2: Mrc Vn Brel nd Rf Vndebril Dept. of Computer Science, K.U.Leuven, Belgium Chemnitz, Germny, 26-30 September 2011 1 Exmples relted to structured rnks 2 2 / 26
More informationEE263 homework 8 solutions
EE263 Prof S Boyd EE263 homework 8 solutions 37 FIR filter with smll feedbck Consider cscde of 00 one-smple delys: u z z y () Express this s liner dynmicl system x(t + ) = Ax(t) + Bu(t), y(t) = Cx(t) +
More informationfractions Let s Learn to
5 simple lgebric frctions corne lens pupil retin Norml vision light focused on the retin concve lens Shortsightedness (myopi) light focused in front of the retin Corrected myopi light focused on the retin
More informationLecture Note 9: Orthogonal Reduction
MATH : Computtionl Methods of Liner Algebr 1 The Row Echelon Form Lecture Note 9: Orthogonl Reduction Our trget is to solve the norml eution: Xinyi Zeng Deprtment of Mthemticl Sciences, UTEP A t Ax = A
More informationTheoretical foundations of Gaussian quadrature
Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of
More informationSTUDY GUIDE FOR BASIC EXAM
STUDY GUIDE FOR BASIC EXAM BRYON ARAGAM This is prtil list of theorems tht frequently show up on the bsic exm. In mny cses, you my be sked to directly prove one of these theorems or these vrints. There
More information1.9 C 2 inner variations
46 CHAPTER 1. INDIRECT METHODS 1.9 C 2 inner vritions So fr, we hve restricted ttention to liner vritions. These re vritions of the form vx; ǫ = ux + ǫφx where φ is in some liner perturbtion clss P, for
More informationEstimation on Monotone Partial Functional Linear Regression
A^VÇÚO 1 33 ò 1 4 Ï 217 c 8 Chinese Journl of Applied Probbility nd Sttistics Aug., 217, Vol. 33, No. 4, pp. 433-44 doi: 1.3969/j.issn.11-4268.217.4.8 Estimtion on Monotone Prtil Functionl Liner Regression
More informationReinforcement learning II
CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic
More informationVisual motion. Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys
Visul motion Mn slides dpted from S. Seitz, R. Szeliski, M. Pollefes Outline Applictions of segmenttion to video Motion nd perceptul orgniztion Motion field Opticl flow Motion segmenttion with lers Video
More information13: Diffusion in 2 Energy Groups
3: Diffusion in Energy Groups B. Rouben McMster University Course EP 4D3/6D3 Nucler Rector Anlysis (Rector Physics) 5 Sept.-Dec. 5 September Contents We study the diffusion eqution in two energy groups
More informationNew data structures to reduce data size and search time
New dt structures to reduce dt size nd serch time Tsuneo Kuwbr Deprtment of Informtion Sciences, Fculty of Science, Kngw University, Hirtsuk-shi, Jpn FIT2018 1D-1, No2, pp1-4 Copyright (c)2018 by The Institute
More informationIntroduction to Determinants. Remarks. Remarks. The determinant applies in the case of square matrices
Introduction to Determinnts Remrks The determinnt pplies in the cse of squre mtrices squre mtrix is nonsingulr if nd only if its determinnt not zero, hence the term determinnt Nonsingulr mtrices re sometimes
More informationNumerical integration
2 Numericl integrtion This is pge i Printer: Opque this 2. Introduction Numericl integrtion is problem tht is prt of mny problems in the economics nd econometrics literture. The orgniztion of this chpter
More informationPartial Differential Equations
Prtil Differentil Equtions Notes by Robert Piché, Tmpere University of Technology reen s Functions. reen s Function for One-Dimensionl Eqution The reen s function provides complete solution to boundry
More informationMatching patterns of line segments by eigenvector decomposition
Title Mtching ptterns of line segments y eigenvector decomposition Author(s) Chn, BHB; Hung, YS Cittion The 5th IEEE Southwest Symposium on Imge Anlysis nd Interprettion Proceedings, Snte Fe, NM., 7-9
More informationLinearity, linear operators, and self adjoint eigenvalue problems
Linerity, liner opertors, nd self djoint eigenvlue problems 1 Elements of liner lgebr The study of liner prtil differentil equtions utilizes, unsurprisingly, mny concepts from liner lgebr nd liner ordinry
More informationOptimization Lecture 1 Review of Differential Calculus for Functions of Single Variable.
Optimiztion Lecture 1 Review of Differentil Clculus for Functions of Single Vrible http://users.encs.concordi.c/~luisrod, Jnury 14 Outline Optimiztion Problems Rel Numbers nd Rel Vectors Open, Closed nd
More informationGeneralized Fano and non-fano networks
Generlized Fno nd non-fno networks Nildri Ds nd Brijesh Kumr Ri Deprtment of Electronics nd Electricl Engineering Indin Institute of Technology Guwhti, Guwhti, Assm, Indi Emil: {d.nildri, bkri}@iitg.ernet.in
More informationLecture 2e Orthogonal Complement (pages )
Lecture 2e Orthogonl Complement (pges -) We hve now seen tht n orthonorml sis is nice wy to descrie suspce, ut knowing tht we wnt n orthonorml sis doesn t mke one fll into our lp. In theory, the process
More information11-755/ Machine Learning for Signal Processing. Algebra. Class August Instructor: Bhiksha Raj
-755/8-797 Mchine Lerning for Signl Processing Fundmentls of Liner Alger Clss 6 August 9 Instructor: Bhiksh Rj 6 Aug -755/8-797 Administrivi Registrtion: Anyone on witlist still? Our TA is here Sourish
More informationA-Level Mathematics Transition Task (compulsory for all maths students and all further maths student)
A-Level Mthemtics Trnsition Tsk (compulsory for ll mths students nd ll further mths student) Due: st Lesson of the yer. Length: - hours work (depending on prior knowledge) This trnsition tsk provides revision
More informationapproaches as n becomes larger and larger. Since e > 1, the graph of the natural exponential function is as below
. Eponentil nd rithmic functions.1 Eponentil Functions A function of the form f() =, > 0, 1 is clled n eponentil function. Its domin is the set of ll rel f ( 1) numbers. For n eponentil function f we hve.
More informationQuantum Physics II (8.05) Fall 2013 Assignment 2
Quntum Physics II (8.05) Fll 2013 Assignment 2 Msschusetts Institute of Technology Physics Deprtment Due Fridy September 20, 2013 September 13, 2013 3:00 pm Suggested Reding Continued from lst week: 1.
More informationMATHEMATICS AND STATISTICS 1.2
MATHEMATICS AND STATISTICS. Apply lgebric procedures in solving problems Eternlly ssessed 4 credits Electronic technology, such s clcultors or computers, re not permitted in the ssessment of this stndr
More informationA matrix is a set of numbers or symbols arranged in a square or rectangular array of m rows and n columns as
RMI University ENDIX MRIX GEBR INRDUCIN Mtrix lgebr is powerful mthemticl tool, which is extremely useful in modern computtionl techniques pplicble to sptil informtion science. It is neither new nor difficult,
More information4.5 JACOBI ITERATION FOR FINDING EIGENVALUES OF A REAL SYMMETRIC MATRIX. be a real symmetric matrix. ; (where we choose θ π for.
4.5 JACOBI ITERATION FOR FINDING EIGENVALUES OF A REAL SYMMETRIC MATRIX Some reliminries: Let A be rel symmetric mtrix. Let Cos θ ; (where we choose θ π for Cos θ 4 purposes of convergence of the scheme)
More informationThe Regulated and Riemann Integrals
Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue
More informationIn Section 5.3 we considered initial value problems for the linear second order equation. y.a/ C ˇy 0.a/ D k 1 (13.1.4)
678 Chpter 13 Boundry Vlue Problems for Second Order Ordinry Differentil Equtions 13.1 TWO-POINT BOUNDARY VALUE PROBLEMS In Section 5.3 we considered initil vlue problems for the liner second order eqution
More informationState space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies
Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response
More informationSTRAND B: NUMBER THEORY
Mthemtics SKE, Strnd B UNIT B Indices nd Fctors: Tet STRAND B: NUMBER THEORY B Indices nd Fctors Tet Contents Section B. Squres, Cubes, Squre Roots nd Cube Roots B. Inde Nottion B. Fctors B. Prime Fctors,
More informationEnergy Bands Energy Bands and Band Gap. Phys463.nb Phenomenon
Phys463.nb 49 7 Energy Bnds Ref: textbook, Chpter 7 Q: Why re there insultors nd conductors? Q: Wht will hppen when n electron moves in crystl? In the previous chpter, we discussed free electron gses,
More informationConsequently, the temperature must be the same at each point in the cross section at x. Let:
HW 2 Comments: L1-3. Derive the het eqution for n inhomogeneous rod where the therml coefficients used in the derivtion of the het eqution for homogeneous rod now become functions of position x in the
More informationLINEAR ALGEBRA APPLIED
5.5 Applictions of Inner Product Spces 5.5 Applictions of Inner Product Spces 7 Find the cross product of two vectors in R. Find the liner or qudrtic lest squres pproimtion of function. Find the nth-order
More informationM344 - ADVANCED ENGINEERING MATHEMATICS
M3 - ADVANCED ENGINEERING MATHEMATICS Lecture 18: Lplce s Eqution, Anltic nd Numericl Solution Our emple of n elliptic prtil differentil eqution is Lplce s eqution, lso clled the Diffusion Eqution. If
More information