Tensors for matrix differentiation
|
|
- Philip Lang
- 6 years ago
- Views:
Transcription
1 1 Tensors for matrx dfferentaton Rchard Turner Here are some notes on how to use tensors to fnd matrx dervatves, and the reaton to the. (Hadamard), vec, (Kronecer), vec-transpose and reshape operators. I wrote these notes for mysef, and I apoogse for any mstaes and confusons. Two sectons are currenty unfnshed: I hope to compete them soon. 1 A tensor notaton Let s setup one usefu form of tensor notaton, whch ncorporates the matrx and nner product, the outer product, the Hadamard (MATLAB.* or ) product dag and dag 1. These w be denoted usng dfferent combnatons of pars of up-stars and down-stars ndces. If we have ony 2 nd order tensors (and ower) we want to be abe to easy convert the resut nto matrx representaton. We have a free choce for the horzonta orderng of ndces therefore ths can be used to denote transposes and the order of mutpcaton. a j b j = j a j b j (1) = (AB) (2) A j = (A T ) j (3) a b j = (ab T ) j (4) a = a (5) = tr A (6) A j B j = H jm n A B m n (7) = (A B) j (8) H jm n = δ δ j δ m δn j (9) A, = dag(a) (10) A j δ j = (dag 1 dag A) j (11) The Kronecer deta δ j s 1 ff = j and zero for j. From the matrx perspectve, the frst ndces ndex the rows and the second the coumns. A second order tensor must have one upstars
2 2 and one downstars ndex. The ony way of movng downstars ndces upstars, s to fp ALL ndces, and ths does not affect anythng. Summatons occur between one down-stars ndex and one up-stars ndex, (the Ensten conventon). Repeated down-stars or up-stars ndces mpy a Hadamard or entry-wse product. As an exampe, the Hadamard product between two vectors (of the same sze) s: a b = [a 1, a 2,..., a I ] T. [b 1, b 2,..., b I ] T = [a 1 b 1, a 2 b 2,..., a I b I ] T If we have a bunch of second and/or frst order tensors (eg. S j = B j W A ), we can convert them nto matrx/vector notaton usng the order of the ndces from eft to rght, and the rue for the transpose. 1. Mae the frst and the ast ndces match the LHS (S j (W T ) A B j ) = W A B j = 2. Transpose the centra objects so the ndces run consecutvey (S j = (W T ) (A T ) B j 3. Repace wth matrx notaton (S = W T A T B) 2 Basc dervatves ) To convert dervatves found usng the suffx notaton nto matrx dervatves, we need to be aware of one more conventon. Imagne dfferentatng a vector x by another vector y. The resut s a matrx, but we must choose whch way round we want the rows and coumns. Conventonay the choce s: x y j = j (12) So the chan rue s easy to appy (and ntutve) by a rght hand mutpcaton: x z = x y j y j z (13) = j j (14) We mght aso want to dfferentate a matrx by a matrx. Athough the resutng forth order tensor cannot be represented a matrx, the object coud be requred n appyng the chan rue (see the exampe n the foowng secton where we dfferentate an object e tr(w W T ) wth
3 3 respect to W ) and so we need rues for assgnng the ndces. Lucy we have enough conventons to unambguousy specfy ths: A j B = j,, (15) Where the orderng wthng the upstars and downstars sots s arbtrary. Note that ths reaton defnes a the dervatves you can form wth 2 nd order tensors and ower (a subset of whch are the three types of dervatves that can be represented as matrces) Some exampes Here are some expct exampes: Fnd: Souton: f dw j tr(aw CW T B) dw = f dw (16) = A W m C n m(w T ) p n B p (17) = AW m CmW n p nbp (18) = Aδ δj m CmW n p nbp (19) +AW m Cmδ n p δ jn Bp (20) = AC j n W p nbp (21) +AW m CmB j (22) = Cj n (W T ) n p BpA (23) +(C T ) j m(w T ) m (A T ) (B T ) (24) = (CW T BA + C T W T A T B T ) j (25) Note the dscrepancy wth some texts (for exampe the wdey used the matrx coo boo): ther resuts dffer by a transpose as ther defntons for x/ M and M/ x are not consstent wth regard to ther use n the chan rue. Fnd: da 1 (26)
4 4 Souton: δ di j d(a 1 ) j d(a 1 ) j = da (A 1 ) j 0 = da (A 1 ) j + A = (A 1 ) = da [ 1 da A A 1 3 Dfferentatng determnants compete ths secton d(a 1 ) j (27) (28) (A 1 ) j (29) ] j (30) 4 Dfferentatng structured matrces Imagne we want to dfferentate a matrx wth some structure, for exampe a symmetrc matrx. To form the dervatves we can use the matrx-sef-dervatve: So that: Γ j,, = A j A (31) f A = f A j A j A (32) When formng dfferentas we have to be carefu to ony sum over unque entres n A: df = f A = unque, da (33) f A da (34) One ubqutous form of structured matrces are the symmetrc matrces. For ths cass the matrx-sef-dervatve s:
5 5 S j S = δ δ j + δ, δ j, δ j δ δ (35) The frst two terms mae sure we count the off dagona eements twce, and the second term avods over countng of the dagona and nvoves some entry-wse products. 4.1 A sermon about symmetrc matrces Let s do a famy of dervatves correcty that most peope muc up. where S s symmetrc. Fnd: df(s) ds f(s) S ds j = df(s) ds j ds [ = f (S) j δ δ j + δ, δ j, δ j δ ] δ [ = f (S) j δ δ j + δ, δ j, δ j δ ] δ = f (S) + f (S) = f (S) + f (S) = f (S) + f (S) (36) (37) (38) f (S) jδ j δ δ (39) f (S) δ δ (40) f (S) δ (41) = [f (S) + f (S) T f (S) I] (42) = [2f (S) f (S) I] (43) If we want the dfferenta, we must sum over a the unque eements (and no more): df(s) = [2f (S) f (S) I] jds j (44),j Here s a concrete exampe where peope forget the above: magne fndng the ML covarance matrx of a mutvarate Gaussan dstrbuton, gven some data. The ehood s gven by: P ({x n } N n=1) = n [ det(2πσ) 1/2 exp 1 ] 2 (x n µ) T Σ 1 (x n µ) (45) og P ({x n } N n=1) = N 2 [ D og(2π) og det(σ 1 ) + tr ( Σ 1 X )] (46) Where we have ntroduced X = (x n µ)(x n µ) T. Now we dfferentate ths wrt. Σ 1 :
6 6 og P ({x n } N n=1) dσ 1 = N 2 [ og det(σ 1 ) + tr ] (Σ 1 X) dσ 1 dσ 1 (47) = N 2 [2Σ Σ I + 2X X I] (48) These are the correct dervatves. As 2Σ Σ I + 2X X I Σ X peope can recover Σ = X wthout usng the symmetrsed expressons. However, f we wanted to do gradent ascent here, then we shoud use the foowng updates: (Σ 1 n+1) j = (Σ 1 n+1) j η [2Σ Σ I + 2X X I] j (49) Where Σ s an upper tranguar matrx. Most peope woud use: (Σ 1 n+1) j = (Σ 1 n+1) j η[σ X] j (50) If we ntase wth symmetrc Σ 1 0 then the ncorrect proceedure w wa us aong the manfod of symmetrc matrces towards the ML souton. You mght thn numerca errors coud, n prncpe, step us off the manfod of symmetrc matrces: Aftera, (Σ 1 ) j X j s nvarant so ong as (Σ 1 ) j +(Σ 1 ) j = const. There appears to be no pressure to eep Σ 1 symmetrc. Thngs actuay turn out to be worse than ths. 45 s not actuay a correct expresson for a Gaussan dstrbuton f Σ 1 s not symmetrc. Specfcay the normasng constant shoud be repaced by det( 1 2 [Σ+ΣT ]). If you don t use ths symmetrsed form, the manfod of symmetrc matrces es aong a mnmum and the ML souton s a sadde pont (see Fg. 1). For ths reason, t seems prudent to use the correct gradents when dong gradent ascent, and to remember to normase correcty. 5 Reaton to the vec and ronecer product operators The vec, Kronecer product, vector transpose, and reshape operators shuffe tensors so they can be represented n arrays of dfferent dmensonates (and szes). The are most ntutvey defned by vsua exampes, but usefu resuts can be proved usng a tensor representaton that aways nvoves a tensor product between the object we are transformng and an ndcator tensor. In ths secton our tensor agebra does not need to dea wth entrywse products etc.. Therefore, to ad the carty of the presentaton we
7 Fgure 1: a. The expected cost functon s symmetrc, and the maxmum s a rdge. b. The cost functon wth the ncorrect normaser s not symmetrc and the MLparameter vaues correspond to a sadde pont. 7
8 8 use the more usua suffx notaton. The resuts presented here are easy to generase usng the prevous framewor (as s needed to reate the entry-wse and Kronecer products, and dag, for exampe), but the resut s ess aesthetc. 5.1 Vec The vec operator ets you represent a matrx as a vector, by stacng the coumns. For exampe: [( )] x11 x vec 12 x 13 x 21 x 22 x 23 = x 11 x 21 x 12 x 22 x 12 x 23 (51) The tensor representaton of ths operator s: 5.2 Kronecer Product x = V ab X ab (52) V ab = δ,a+(b 1)A (53) The Kronecer product operator ( ) ets you represent the outer product of two matrces (a 4th order tensor) as a matrx. ( ) x11 x 12 Y = x 21 x 22 ( x11 Y ) x 12 Y x 21 Y x 22 Y (54) Aternatvey, wrtten as a tensor, we have: Z j = K jbd ac Xa b Yd c (55) K jabcd = δ,c+(a 1)C δ j,d+(b 1)D (56) Exampes The mportant resut reatng the vec and Kronecer products s: (C T A) vec(b) = vec(abc) (57) whch can be proved usng the defntons above:
9 9 [(C T A) vec(b)] j = K jabcd C ba A cd V jef B ef (58) = δ,c+(a 1)C δ j,d+(b 1)D C ba A cd δ j,e+(f 1)E B ef (59) = δ,c+(a 1)C δ d,e δ bf C ba A cd B ef (60) = δ,c+(a 1)C C ba A cd B bd (61) = V ca A cd B bd C ba (62) = vec(acb) (63) 5.3 vec-transpose Ths s not the same as reshape n MATLAB despte what Tom Mna cams. compete ths secton 5.4 reshape Reshape generases the vec operator. It aows us to hande hgh dmensona objects easy. For exampe, by remappng tensors nto matrces we can form tensor nverses. We want to be abe to map an array of sze A B C... wth n tota A B C... = N eements nto another array of I J K... = N eements. To do ths we need to specfy where each of the eements n the od array appears n the new array. There are ots of choces for how we ay down the eements n the new array. It woud be usefu to do ths systematcay. One way to do ths s to come up wth a systematc method for numberng each eement n an array, and then we coud ay down eements n the new array such that they w be assgned the same number. A systematc numberng system can be constructed as foows. In a norma countng system, we choose a base B (eg. 10, 2). We can unquey form nteger numbers as a sum of upto B 1 1s, Bs, B 2 s and so on. eg. B = 10, 954 = eg. B = 2, 13 = The coeffcents are the representaton of the number. Let s defne a new countng system whch has a non-constant base. The ast number w agan represent the number of ones and w tae a vaues from 0 to I 1, the second number j represents the number of Is and taes vaues from 0 to J 1, the thrd number represents the number of I Js and taes vaues from 0 to K 1, and so on. eg. Usng {I, J, K} = {2, 3, 4} the number 21 s and we can represent I J K = 24 numbers ths way. Now f we assocate + 1, j + 1, + 1,... wth the poston n
10 10 an N dmensona array, we have assgned a ponts n that array wth a unque nteger number that forms a sequence. T,j,,... = R I,J,K,...;A,B,C,...,j,,...,a,b,c,... S a,b,c,... (64) = reshape(s, [I, J, K,...]) (65) R I,J,K,...;A,B,C,...,j,,...,a,b,c,... = δ a 1+(b 1)A+(c 1)AB+..., 1+(j 1)I+( 1)IJ+... (66) Exampes: Here are some exampes and usefu resuts: From ths t s smpe to show that reshapng a tensor, and then reshapng t bac to ts orgna sze returns a the entres to ther orgna postons: T a,b,c,... = R A,B,C,...;I,J,K,... a,b,c,...,,j,,... R I,J,K,...;A,B,C,...,j,,...,a,b,c,... S a,b,c,... (67) = δ a 1+(b 1)A+(c 1)AB+..., 1+(j 1)I+( 1)IJ+... δ a 1+(b 1)A+(c 1)AB+..., 1+(j 1)I+( 1)IJ+... S a,b,c,... (68) = δ a 1+(b 1)A+(c 1)AB+...,a 1+(b 1)A+(c 1)AB+...S a,b,c,... (69) = δ a,aδ b,bδ c,cs a,b,c,... (70) = S a,b,c,... (71) Ths resut shoud be obvous. The way we ntroduced the reshape operator was va a numberng system that was unque for a arrays of a gven shape. Ths means f we reshape an array and reshape t agan, the resut must be equvaent to reshapng drecty: ntermedate reshapngs cannot effect anythng. A probem where the reshape operator s usefu arses n mut-near modes. There we have to sove: α d,,j,... = g d,a,b,... β a,b,...,,j,... (72) where we want g d,a,b,... and we now α d,,j,... and β a,b,...,,j,.... The dmensonates are I = A, J = B.... The souton amounts to fndng the nverse of β a,b,...,,j,.... Reshapng the eft and rght hand sdes nto [D, Q = I J...] matrces we have: R D,Q;D,I,J,... e,q,d,,j,... α d,,j,... = R D,Q;I,J,... e,q,d,,j,... g d,a,b,...β a,b,...,,j,... (73) = δ e+(q 1)D,d+( 1)D+(j 1)DI+... g d,a,b,... β a,b,...,,j,... (74) = δ e,d δ q,+(j 1)I+... g d,a,b,... β a,b,...,,j,... (75) = g e,a,b,... δ q,+(j 1)I+... β a,b,...,,j,... (76)
11 11 Lettng X = reshape(α, [D, Q]), we repace the tensor product on the RHS wth a matrx product usng the reshape operator agan: X e,q = δ a,aδ b,b... g e,a,b,... δ q,+(j 1)I+... β a,b,...,,j,... (77) = R A,B,...;Q a,b,...,g RQ;A,B,... g,a,b,... g e,a,b,... δ q,+(j 1)I+... β a,b,...,,j,... (78) = R Q;A,B,... g,a,b,... g e,a,b,... δ q,+(j 1)I+... R A,B,...;Q a,b,...,f,g β a,b,...,,j,... (79) = R D,Q;D,A,B,... e,g,e,a,b,... g e,a,b,... R Q,Q;A,B,...,A,B,... g,q,a,b,...,,j... β a,b,...,,j,... (80) = reshape(g, [D, Q]) e,g reshape(β, [Q, Q]) g,q (81) Lettng Y = reshape(β, [Q, Q]) the souton s: g d,a,b,... = reshape(xy 1, [D, A, B,...]) d,a,b,... (82)
Quantum Runge-Lenz Vector and the Hydrogen Atom, the hidden SO(4) symmetry
Quantum Runge-Lenz ector and the Hydrogen Atom, the hdden SO(4) symmetry Pasca Szrftgser and Edgardo S. Cheb-Terrab () Laboratore PhLAM, UMR CNRS 85, Unversté Le, F-59655, France () Mapesoft Let's consder
More informationNote 2. Ling fong Li. 1 Klein Gordon Equation Probablity interpretation Solutions to Klein-Gordon Equation... 2
Note 2 Lng fong L Contents Ken Gordon Equaton. Probabty nterpretaton......................................2 Soutons to Ken-Gordon Equaton............................... 2 2 Drac Equaton 3 2. Probabty nterpretaton.....................................
More informationChapter 6. Rotations and Tensors
Vector Spaces n Physcs 8/6/5 Chapter 6. Rotatons and ensors here s a speca knd of near transformaton whch s used to transforms coordnates from one set of axes to another set of axes (wth the same orgn).
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationFrom Biot-Savart Law to Divergence of B (1)
From Bot-Savart Law to Dvergence of B (1) Let s prove that Bot-Savart gves us B (r ) = 0 for an arbtrary current densty. Frst take the dvergence of both sdes of Bot-Savart. The dervatve s wth respect to
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the non-lnear case, and also
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More information7. Products and matrix elements
7. Products and matrx elements 1 7. Products and matrx elements Based on the propertes of group representatons, a number of useful results can be derved. Consder a vector space V wth an nner product ψ
More informationAssociative Memories
Assocatve Memores We consder now modes for unsupervsed earnng probems, caed auto-assocaton probems. Assocaton s the task of mappng patterns to patterns. In an assocatve memory the stmuus of an ncompete
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationSupplementary Material: Learning Structured Weight Uncertainty in Bayesian Neural Networks
Shengyang Sun, Changyou Chen, Lawrence Carn Suppementary Matera: Learnng Structured Weght Uncertanty n Bayesan Neura Networks Shengyang Sun Changyou Chen Lawrence Carn Tsnghua Unversty Duke Unversty Duke
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationCyclic Codes BCH Codes
Cycc Codes BCH Codes Gaos Feds GF m A Gaos fed of m eements can be obtaned usng the symbos 0,, á, and the eements beng 0,, á, á, á 3 m,... so that fed F* s cosed under mutpcaton wth m eements. The operator
More information3. Stress-strain relationships of a composite layer
OM PO I O U P U N I V I Y O F W N ompostes ourse 8-9 Unversty of wente ng. &ech... tress-stran reatonshps of a composte ayer - Laurent Warnet & emo Aerman.. tress-stran reatonshps of a composte ayer Introducton
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationMARKOV CHAIN AND HIDDEN MARKOV MODEL
MARKOV CHAIN AND HIDDEN MARKOV MODEL JIAN ZHANG JIANZHAN@STAT.PURDUE.EDU Markov chan and hdden Markov mode are probaby the smpest modes whch can be used to mode sequenta data,.e. data sampes whch are not
More information9 Characteristic classes
THEODORE VORONOV DIFFERENTIAL GEOMETRY. Sprng 2009 [under constructon] 9 Characterstc classes 9.1 The frst Chern class of a lne bundle Consder a complex vector bundle E B of rank p. We shall construct
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationSupervised Learning. Neural Networks and Back-Propagation Learning. Credit Assignment Problem. Feedforward Network. Adaptive System.
Part 7: Neura Networ & earnng /2/05 Superved earnng Neura Networ and Bac-Propagaton earnng Produce dered output for tranng nput Generaze reaonaby & appropratey to other nput Good exampe: pattern recognton
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More information( ) r! t. Equation (1.1) is the result of the following two definitions. First, the bracket is by definition a scalar product.
Chapter. Quantum Mechancs Notes: Most of the matera presented n ths chapter s taken from Cohen-Tannoudj, Du, and Laoë, Chap. 3, and from Bunker and Jensen 5), Chap... The Postuates of Quantum Mechancs..
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationCOMPLEX NUMBERS AND QUADRATIC EQUATIONS
COMPLEX NUMBERS AND QUADRATIC EQUATIONS INTRODUCTION We know that x 0 for all x R e the square of a real number (whether postve, negatve or ero) s non-negatve Hence the equatons x, x, x + 7 0 etc are not
More informationBezier curves. Michael S. Floater. August 25, These notes provide an introduction to Bezier curves. i=0
Bezer curves Mchael S. Floater August 25, 211 These notes provde an ntroducton to Bezer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of the
More informationWeek 2. This week, we covered operations on sets and cardinality.
Week 2 Ths week, we covered operatons on sets and cardnalty. Defnton 0.1 (Correspondence). A correspondence between two sets A and B s a set S contaned n A B = {(a, b) a A, b B}. A correspondence from
More informationEinstein Summation Convention
Ensten Suaton Conventon Ths s a ethod to wrte equaton nvovng severa suatons n a uncuttered for Exape:. δ where δ or 0 Suaton runs over to snce we are denson No ndces appear ore than two tes n the equaton
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationPrimer on High-Order Moment Estimators
Prmer on Hgh-Order Moment Estmators Ton M. Whted July 2007 The Errors-n-Varables Model We wll start wth the classcal EIV for one msmeasured regressor. The general case s n Erckson and Whted Econometrc
More informationLECTURE 21 Mohr s Method for Calculation of General Displacements. 1 The Reciprocal Theorem
V. DEMENKO MECHANICS OF MATERIALS 05 LECTURE Mohr s Method for Cacuaton of Genera Dspacements The Recproca Theorem The recproca theorem s one of the genera theorems of strength of materas. It foows drect
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationCrystal Interpretation of Kerov Kirillov Reshetikhin Bijection II
arxv:math/6697v [math.qa] Jun 7 Crysta Interpretaton of Kerov Krov Reshethn Bjecton II Proof for sn Case Reho Saamoto Department of Physcs, Graduate Schoo of Scence, Unversty of Toyo, Hongo, Bunyo-u, Toyo,
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More informationA Quantum Gauss-Bonnet Theorem
A Quantum Gauss-Bonnet Theorem Tyler Fresen November 13, 2014 Curvature n the plane Let Γ be a smooth curve wth orentaton n R 2, parametrzed by arc length. The curvature k of Γ s ± Γ, where the sgn s postve
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationKinematics of Fluids. Lecture 16. (Refer the text book CONTINUUM MECHANICS by GEORGE E. MASE, Schaum s Outlines) 17/02/2017
17/0/017 Lecture 16 (Refer the text boo CONTINUUM MECHANICS by GEORGE E. MASE, Schaum s Outlnes) Knematcs of Fluds Last class, we started dscussng about the nematcs of fluds. Recall the Lagrangan and Euleran
More informationSociété de Calcul Mathématique SA
Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc
More informationPhysics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1
P. Guterrez Physcs 5153 Classcal Mechancs D Alembert s Prncple and The Lagrangan 1 Introducton The prncple of vrtual work provdes a method of solvng problems of statc equlbrum wthout havng to consder the
More informationSome results on a cross-section in the tensor bundle
Hacettepe Journa of Matematcs and Statstcs Voume 43 3 214, 391 397 Some resuts on a cross-secton n te tensor bunde ydın Gezer and Murat tunbas bstract Te present paper s devoted to some resuts concernng
More informationImage Classification Using EM And JE algorithms
Machne earnng project report Fa, 2 Xaojn Sh, jennfer@soe Image Cassfcaton Usng EM And JE agorthms Xaojn Sh Department of Computer Engneerng, Unversty of Caforna, Santa Cruz, CA, 9564 jennfer@soe.ucsc.edu
More information22.51 Quantum Theory of Radiation Interactions
.51 Quantum Theory of Radaton Interactons Fna Exam - Soutons Tuesday December 15, 009 Probem 1 Harmonc oscator 0 ponts Consder an harmonc oscator descrbed by the Hamtonan H = ω(nˆ + ). Cacuate the evouton
More informationBézier curves. Michael S. Floater. September 10, These notes provide an introduction to Bézier curves. i=0
Bézer curves Mchael S. Floater September 1, 215 These notes provde an ntroducton to Bézer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationare called the contravariant components of the vector a and the a i are called the covariant components of the vector a.
Non-Cartesan Coordnates The poston of an arbtrary pont P n space may be expressed n terms of the three curvlnear coordnates u 1,u,u 3. If r(u 1,u,u 3 ) s the poston vector of the pont P, at every such
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationGaussian Conditional Random Field Network for Semantic Segmentation - Supplementary Material
Gaussan Condtonal Random Feld Networ for Semantc Segmentaton - Supplementary Materal Ravtea Vemulapall, Oncel Tuzel *, Mng-Yu Lu *, and Rama Chellappa Center for Automaton Research, UMIACS, Unversty of
More informationProblem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?
Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2
More informationBallot Paths Avoiding Depth Zero Patterns
Ballot Paths Avodng Depth Zero Patterns Henrch Nederhausen and Shaun Sullvan Florda Atlantc Unversty, Boca Raton, Florda nederha@fauedu, ssull21@fauedu 1 Introducton In a paper by Sapounaks, Tasoulas,
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationLaboratory 3: Method of Least Squares
Laboratory 3: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly they are correlated wth
More informationA finite difference method for heat equation in the unbounded domain
Internatona Conerence on Advanced ectronc Scence and Technoogy (AST 6) A nte derence method or heat equaton n the unbounded doman a Quan Zheng and Xn Zhao Coege o Scence North Chna nversty o Technoogy
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More informationLagrangian Field Theory
Lagrangan Feld Theory Adam Lott PHY 391 Aprl 6, 017 1 Introducton Ths paper s a summary of Chapter of Mandl and Shaw s Quantum Feld Theory [1]. The frst thng to do s to fx the notaton. For the most part,
More informationQUARTERLY OF APPLIED MATHEMATICS
QUARTERLY OF APPLIED MATHEMATICS Voume XLI October 983 Number 3 DIAKOPTICS OR TEARING-A MATHEMATICAL APPROACH* By P. W. AITCHISON Unversty of Mantoba Abstract. The method of dakoptcs or tearng was ntroduced
More informationAndre Schneider P622
Andre Schneder P6 Probem Set #0 March, 00 Srednc 7. Suppose that we have a theory wth Negectng the hgher order terms, show that Souton Knowng β(α and γ m (α we can wrte β(α =b α O(α 3 (. γ m (α =c α O(α
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationσ τ τ τ σ τ τ τ σ Review Chapter Four States of Stress Part Three Review Review
Chapter Four States of Stress Part Three When makng your choce n lfe, do not neglect to lve. Samuel Johnson Revew When we use matrx notaton to show the stresses on an element The rows represent the axs
More informationGrover s Algorithm + Quantum Zeno Effect + Vaidman
Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the
More informationLaboratory 1c: Method of Least Squares
Lab 1c, Least Squares Laboratory 1c: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationRadar Trackers. Study Guide. All chapters, problems, examples and page numbers refer to Applied Optimal Estimation, A. Gelb, Ed.
Radar rackers Study Gude All chapters, problems, examples and page numbers refer to Appled Optmal Estmaton, A. Gelb, Ed. Chapter Example.0- Problem Statement wo sensors Each has a sngle nose measurement
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationDeriving the Dual. Prof. Bennett Math of Data Science 1/13/06
Dervng the Dua Prof. Bennett Math of Data Scence /3/06 Outne Ntty Grtty for SVM Revew Rdge Regresson LS-SVM=KRR Dua Dervaton Bas Issue Summary Ntty Grtty Need Dua of w, b, z w 2 2 mn st. ( x w ) = C z
More informationCHAPTER 4. Vector Spaces
man 2007/2/16 page 234 CHAPTER 4 Vector Spaces To crtcze mathematcs for ts abstracton s to mss the pont entrel. Abstracton s what makes mathematcs work. Ian Stewart The man am of ths tet s to stud lnear
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationCHALMERS, GÖTEBORGS UNIVERSITET. SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
CHALMERS, GÖTEBORGS UNIVERSITET SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 35, FIM 72 GU, PhD Tme: Place: Teachers: Allowed materal: Not allowed: January 2, 28, at 8 3 2 3 SB
More informationEffects of Ignoring Correlations When Computing Sample Chi-Square. John W. Fowler February 26, 2012
Effects of Ignorng Correlatons When Computng Sample Ch-Square John W. Fowler February 6, 0 It can happen that ch-square must be computed for a sample whose elements are correlated to an unknown extent.
More informationNeural network-based athletics performance prediction optimization model applied research
Avaabe onne www.jocpr.com Journa of Chemca and Pharmaceutca Research, 04, 6(6):8-5 Research Artce ISSN : 0975-784 CODEN(USA) : JCPRC5 Neura networ-based athetcs performance predcton optmzaton mode apped
More informationA how to guide to second quantization method.
Phys. 67 (Graduate Quantum Mechancs Sprng 2009 Prof. Pu K. Lam. Verson 3 (4/3/2009 A how to gude to second quantzaton method. -> Second quantzaton s a mathematcal notaton desgned to handle dentcal partcle
More informationSrednicki Chapter 34
Srednck Chapter 3 QFT Problems & Solutons A. George January 0, 203 Srednck 3.. Verfy that equaton 3.6 follows from equaton 3.. We take Λ = + δω: U + δω ψu + δω = + δωψ[ + δω] x Next we use equaton 3.3,
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationThree views of mechanics
Three vews of mechancs John Hubbard, n L. Gross s course February 1, 211 1 Introducton A mechancal system s manfold wth a Remannan metrc K : T M R called knetc energy and a functon V : M R called potental
More informationMonica Purcaru and Nicoleta Aldea. Abstract
FILOMAT (Nš) 16 (22), 7 17 GENERAL CONFORMAL ALMOST SYMPLECTIC N-LINEAR CONNECTIONS IN THE BUNDLE OF ACCELERATIONS Monca Purcaru and Ncoeta Adea Abstract The am of ths paper 1 s to fnd the transformaton
More informationCIVL 8/7117 Chapter 10 - Isoparametric Formulation 42/56
CIVL 8/77 Chapter 0 - Isoparametrc Formulaton 4/56 Newton-Cotes Example Usng the Newton-Cotes method wth = ntervals (n = 3 samplng ponts), evaluate the ntegrals: x x cos dx 3 x x dx 3 x x dx 4.3333333
More informationCOXREG. Estimation (1)
COXREG Cox (972) frst suggested the modes n whch factors reated to fetme have a mutpcatve effect on the hazard functon. These modes are caed proportona hazards (PH) modes. Under the proportona hazards
More informationErrata to Invariant Theory with Applications January 28, 2017
Invarant Theory wth Applcatons Jan Drasma and Don Gjswjt http: //www.wn.tue.nl/~jdrasma/teachng/nvtheory0910/lecturenotes12.pdf verson of 7 December 2009 Errata and addenda by Darj Grnberg The followng
More informationcorresponding to those of Heegaard diagrams by the band moves
Agebra transformatons of the fundamenta groups correspondng to those of Heegaard dagrams by the band moves By Shun HORIGUCHI Abstract. Ths paper gves the basc resut of [1](1997),.e., a hande sdng and a
More informationa b a In case b 0, a being divisible by b is the same as to say that
Secton 6.2 Dvsblty among the ntegers An nteger a ε s dvsble by b ε f there s an nteger c ε such that a = bc. Note that s dvsble by any nteger b, snce = b. On the other hand, a s dvsble by only f a = :
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationMoments of Inertia. and reminds us of the analogous equation for linear momentum p= mv, which is of the form. The kinetic energy of the body is.
Moments of Inerta Suppose a body s movng on a crcular path wth constant speed Let s consder two quanttes: the body s angular momentum L about the center of the crcle, and ts knetc energy T How are these
More informationLecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.
prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove
More informationOn the Power Function of the Likelihood Ratio Test for MANOVA
Journa of Mutvarate Anayss 8, 416 41 (00) do:10.1006/jmva.001.036 On the Power Functon of the Lkehood Rato Test for MANOVA Dua Kumar Bhaumk Unversty of South Aabama and Unversty of Inos at Chcago and Sanat
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationOn the formulation of the laws of Nature with five homogeneous coordinates
Über de Formuerung der Naturgesetze mt fünf homogenen Koordnaten Te I: Kasssche Theore Ann d Phys 18 (1933) 305-336 On the formuaton of the aws of Nature wth fve homogeneous coordnates Part I: cassca theory
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part IX The EM agorithm In the previous set of notes, we taked about the EM agorithm as appied to fitting a mixture of Gaussians. In this set of notes, we give a broader view
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationL-Edge Chromatic Number Of A Graph
IJISET - Internatona Journa of Innovatve Scence Engneerng & Technoogy Vo. 3 Issue 3 March 06. ISSN 348 7968 L-Edge Chromatc Number Of A Graph Dr.R.B.Gnana Joth Assocate Professor of Mathematcs V.V.Vannaperuma
More information