Solutions Homework 4 March 5, 2018
|
|
- Arthur Hunt
- 5 years ago
- Views:
Transcription
1 1 Solutons Homework 4 March 5, 018 Soluton to Exercse 5.1.8: Let a IR be a translaton and c > 0 be a re-scalng. ˆb1 (cx + a) cx n + a (cx 1 + a) c x n x 1 cˆb 1 (x), whch shows ˆb 1 s locaton nvarant and scale equvarant. It s not clear why anyone would want to compute ths. Turnng to the others: ˆb (cx + a) (cx (n) + a) (cx (1) + a) c(x (n) x (1) ) cˆb (x). Note that snce c > 0, f we defne y cx+a then the order statstcs (sorted values) of y satsfy y () cx () + a, a fact that was used n the frst step of the computaton above. Note that the statstc analyzed here s the sample range, whch s used often for varous reasons. ˆb3 (cx + a) 1 n(n 1) cˆb 3 (x). ˆb4 (cx + a) n c (cx () + a) 1 n n c c x () + a c 1 1 n c c x () 1 cˆb 4 (x). (cx + b) c(x j + b) j In the last verfcaton, we used the comment about the relaton between the order statstcs of y and x at the frst step, and the assumpton that c 0 n the second to last step.
2 Soluton to Exercse 5..9: These are all exponental famles, so we wll apply Proposton For part (a), f θ (x) 1 x! θx e θ exp[x log θ θ]h(x). Cleary η(θ) log θ s dfferentable and the dervatve 1/θ has rank 1 for all θ (.e., s nonzero). The formula for the Fsher Informaton n equaton (5.40) s I(θ) (1/θ) Var θ (X) 1/θ. Note as an asde that X s an unbased estmator of θ and Var θ (X) θ 1/I(θ), so X s UMVUE for θ. Turnng to part (b), n f θ (x) θ x (1 θ) n x exp[x log(θ/(1 θ)) + n log(1 θ)]h(x). x The dervatve of the natural parameter functon s η (θ) d dθ log(θ/(1 θ)) 1 θ θ 1 θ(1 θ). As ths clearly exsts for 0 < θ < 1, we conclude that Proposton 5..3 apples. Now, the Fsher Informaton s I(θ) η (θ) Var θ [X] nθ(1 θ) [θ(1 θ)] n θ(1 θ). Agan, as an asde, we can easly see that n 1 X s an unbased estmator of θ whose varance equals the lower bound, so t s UMVUE. Fnally, for part (c), the pmf s m + x 1 f θ (x) θ m (1 θ) x exp[x log(1 θ) + m log(θ)]h(x). m 1 Clearly, η (θ) d 1 log(1 θ) dθ 1 θ exsts. One can use Proposton.3.1(b) (on the moments of the suffcent statstc n an exponental famly; we have to substute n θ 1 e η nto m log(θ)) to derve that Var θ (X) m(1 θ) θ.
3 3 Therefore, I(θ) 1 m(1 θ) 1 θ θ m θ (1 θ). Soluton to Exercse 5..16: If f( x) f(x), then ψ(x) f (x)/f(x) satsfes ψ( x) ψ(x) (snce f ( x) f (x)), so ψ ( x) ψ (x), and therefore I 1 (a, b) 1 a xψ (x)f(x) dx 0, snce x s an odd functon and ψ (x)f(x) s an even functon. nformaton matrx s dagonal: I(a, b) 1 a [ I11 (1, 0) 0 0 I (1, 0) ], Thus, the and thus I 1 (a, b) a [ I 1 11 (1, 0) 0 0 I 1 (1, 0) Now t s clear that a I 11 (1, 0) (respectvely a I (1, 0)) are the Informatons for locaton estmaton when scale s known (respectvely, scale estmaton when locaton s known) for a sngle sample, so ths shows parts (a) and (b). For part (c), smply note that all the p.d.f. s mentoned there are symmetrc, so the result apples to them. Soluton to Exercse 5.3.: For part (a), the p.d.f. w.r.t. countng measure on IN {0, 1,,...} s whch s an exponental famly wth f µ (x) exp [x log µ µ] 1 x!, ]. T (x) x, η(µ) log(µ). Note that x s not a.s. constant (.e., does not satsfy a lnear constrant n one dmenson) and η ranges over IR as µ ranges over (0, ), so the famly s full rank, and X s complete and suffcent for a sngle sample, and T X s complete and suffcent for n..d. observatons. We know from e.g. a m.g.f. argument that T s P osson(nµ).
4 4 g(µ) s U-estmable f and only f there s a δ : IN IR such that g(µ) k0 δ(k) (nµ)k e nµ, µ > 0. k! Multplyng through by e nµ we see that e nµ g(µ) has a power seres that converges for all µ > 0, so t must n fact converge for all real µ (ndeed, for all complex µ), and I hope that you at least learned n calculus that f a power seres centered at 0 (a.k.a. a McLaurn seres) converges for some x, then t converges for any y wth y < x. Furthermore, such power seres representatons are unque and are gven by the classcal Taylor formula: e nµ g(µ) δ(k)n k 1 k0 k! µk µ f and only f k dk δ(k) n dµ k [enµ g(µ)] µ0. In concluson, g(µ) s U-estmable f and only f t has a power seres expanson (centered at 0) vald for all real numbers (otherwse sad, s an entre analytc functon), and then the UMVUE s δ(t ) where δ s gven by formula above. For part (c), t s sometmes easer to use ad hoc methods rather than the formula above to fnd the UMVUE. () g(µ) µ k s already gven as a power seres expanson, and t s easy to see that power seres multply, so the power seres we want s e nµ µ k µ k n j µ j. n k 1 j!! n µ! ( k)! j0 k Matchng the coeffcents, t s clear that the UMVUE s δ(t ) where In partcular, δ(t) { 0 f t < k, n k t!/(t k)! f t k. UMVUE for µ n 1 T X
5 5 and UMVUE for µ { 0 f T 0 or T 1 T (T 1)/n f T. T (T 1)/n Note that f we wrte t!/(t k)! k 1 0 (t ), then the UMVUE for µ k may be wrtten more smply as k 1 δ(t) n k (t ). 0 () Of course, Thus, g(µ) P µ [X 1 k] µk k! e µ. e nµ g(µ) µk k! e(n 1)µ µk k! k j0 1 j! (n 1)j µ j! ( k)!k! n (n 1) k n 1! µ, and we see that the desred UMVUE s t δ(t) n t (n 1) t k. k Note that the bnomal coeffcent our UMVUE of P µ [X 1 0] s ( t k δ(t) n t (n 1) t (1 1/n) t. ) s 0 unless k t. In partcular,
6 6 () Note that g(µ) log µ does not have the requred Taylor seres expanson (n partcular, t doesn t have a fnte value and dervates at µ 0). No unbased estmate exsts. (v) Pluggng n our formula, (I am changng the estmand to g(µ) e aµ ) e nµ e aµ e (n+a)µ n + a k n k 1 k0 n k! µk, from whch we can read off the UMVUE as δ(t) n + a t (1 + a/n) t. n (v) Pluggng n agan, e nµ e µ + µ k0(nµ ) k 1 k! 1 k k n j µ j+(k j) k! j k0 k0 k j0 (by the bnomal formula) 1 k k n j µ k j k0 k! j j0 k 1 k n k µ k! k 0 k /! ( k)!(k!) n(k ) 1! n µ. In the above, / denotes the smallest nteger /,.e. / f s even and ( + 1)/ f s odd. From ths, we can read off the UMVUE as T T! δ(t ) (T k)!(k T!) n(k T ) k T/ (v) Ths one s easy: e µ s at µ 0, so t can t have the requste Taylor seres expanson.
7 Soluton to Exercse 5.3.6: We have from the fact that σ n 1 (X X) has a χ n 1 dstrbuton that E[ˆσ (a)] (n 1)aσ Var[ˆσ (a)] (n 1)a σ 4. Usng the formula MSE Bas + Varance, we obtan σ 4 MSE[ˆσ (a)] [a(n 1) 1] + (n 1)a. Thus, wth a lttle algebra we obtan σ 4 { MSE[ˆσ (a)] MSE[ˆσ (1/(n 1))] } (n 1)a (n 1)a + 1 /(n 1) (n 1)[a 1/(n 1)][a 1/(n + 3)]. Note that the r.h.s. s a quadratc functon of a whose graph s an upward openng parabola wth roots at a 1/(n 1) and a 1/(n+3). Thus, ˆσ (a) has smaller MSE than the UMVUE ˆσ (1/(n 1)) for (n+3) 1 < a < (n 1) 1. Soluton to Exercse 5.3.8: (a) The lkelhood s [ f(y; a, b, σ ) (πσ ) n/ exp 1 ] n (y σ (ax + b)) 1 [ (π) n/ exp 1 y σ + a x σ y + b σ 1 (ax σ + b) n ] log σ As long as we have at least two dstnct values of the x, then the famly wll be dentfable, as dfferent values of a and b wll gve dfferent means. Ths assumpton was apparently forgotten. Thus, the suffcent statstc vector T ( x Y, Y, Y ) does not satsfy any lnear constrants. As (a, b, σ ) range over IR IR (0, ), the natural parameter vector ranges over (, 0) IR IR, whch s an open set, so has nonempty nteror. Thus, the famly s full rank, hence T s complete and suffcent. (b) Clearly Ȳ n 1 T s a functon of T. Also, y 7 n x Ỹ 1 x Y x Y Ȳ x
8 8 s a functon of T. Thus, we only need to check that â s unbased for a. Note that E[Ȳ ] a x + b. Thus, E (a,b,σ )[â] 1 x x E[Ỹ] 1 x x [(ax + b) (a x + b)] a. 1 x a x (x x ) Thus, â s a functon of the complete and suffcent statstcs whch s unbased for a, so t s the UMVUE of a. We see mmedately that ˆb s also a functon of T (snce Ȳ and â are), so we need only check that t s expectaton s always b. Thus, (c) Clearly ˆσ E (a,b,σ )[ˆb] E[Ȳ ] E[â] x a x + b a x b. 1 [ Y n + (âx + ˆb) â x Y ˆb ] Y, whch s a functon of T. Now we want to check that ts expectaton of σ. It s almost always easer to do these types of calculatons f one subtracts and adds E[Y ] n the squared quantty: (n )E[ˆσ ] [ (Y E E[Y ] + E[Y ] âx ˆb ) ] [ (ɛ E + (a â)x + (b ˆb) ) ] (snce Y E[Y ] + ɛ and E[Y ] ax + b) â a Ỹ x x a Ỹ Y Ȳ ax + b + ɛ (a x + b + ɛ) a x + ɛ (where ɛ ɛ ɛ)
9 9 â a (a x + ɛ ) x x a Put ɛ x x ˆb b Ȳ â x b a x + b + ɛ â x b (a â) x + ɛ (â a)x + (ˆb b) (â a) x + ɛ j ɛ j x j x j x + ɛ j Note that w x ( j x j ) 1/. w 1, w 0, where the latter follows because x (x x) 0. Thus, we have (n )E[ˆσ ] E ɛ w w j ɛ j ɛ j E ɛ w w j ɛ j j E ɛ ɛ w w j ɛ j + w w j ɛ j j j E ɛ w j ɛ j. j Now [ ] E ɛ [ ] E (ɛ ɛ) (n 1)σ.
10 10 Also, E j w j ɛ j E j E j w j (ɛ j ɛ) w j ɛ j (snce wj 0) σ j w j σ. Thus, we get n the end that E[ˆσ ] σ, as desred.
MATH 281A: Homework #6
MATH 28A: Homework #6 Jongha Ryu Due date: November 8, 206 Problem. (Problem 2..2. Soluton. If X,..., X n Bern(p, then T = X s a complete suffcent statstc. Our target s g(p = p, and the nave guess suggested
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationMaximum Likelihood Estimation
Maxmum Lkelhood Estmaton INFO-2301: Quanttatve Reasonng 2 Mchael Paul and Jordan Boyd-Graber MARCH 7, 2017 INFO-2301: Quanttatve Reasonng 2 Paul and Boyd-Graber Maxmum Lkelhood Estmaton 1 of 9 Why MLE?
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationBernoulli Numbers and Polynomials
Bernoull Numbers and Polynomals T. Muthukumar tmk@tk.ac.n 17 Jun 2014 The sum of frst n natural numbers 1, 2, 3,..., n s n n(n + 1 S 1 (n := m = = n2 2 2 + n 2. Ths formula can be derved by notng that
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationLinear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space.
Lnear, affne, and convex sets and hulls In the sequel, unless otherwse specfed, X wll denote a real vector space. Lnes and segments. Gven two ponts x, y X, we defne xy = {x + t(y x) : t R} = {(1 t)x +
More information6 More about likelihood
6 More about lkelhood 61 Invarance property of mle s ( θ) Theorem If θ s an mle of θ and f g s a functon, then g s an mle of g(θ) Proof If g s one-to-one, then L(θ) =L ( g 1 (g(θ)) ) are both maxmsed by
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the non-lnear case, and also
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationErrata to Invariant Theory with Applications January 28, 2017
Invarant Theory wth Applcatons Jan Drasma and Don Gjswjt http: //www.wn.tue.nl/~jdrasma/teachng/nvtheory0910/lecturenotes12.pdf verson of 7 December 2009 Errata and addenda by Darj Grnberg The followng
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationChapter 7 Generalized and Weighted Least Squares Estimation. In this method, the deviation between the observed and expected values of
Chapter 7 Generalzed and Weghted Least Squares Estmaton The usual lnear regresson model assumes that all the random error components are dentcally and ndependently dstrbuted wth constant varance. When
More informationψ = i c i u i c i a i b i u i = i b 0 0 b 0 0
Quantum Mechancs, Advanced Course FMFN/FYSN7 Solutons Sheet Soluton. Lets denote the two operators by  and ˆB, the set of egenstates by { u }, and the egenvalues as  u = a u and ˆB u = b u. Snce the
More informationParametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationF71SM1 STATISTICAL METHODS TUTORIAL ON 7 ESTIMATION SOLUTIONS
F7SM STATISTICAL METHODS TUTORIAL ON 7 ESTIMATION SOLUTIONS RJG. (a) E[X] = 0 xf(x)dx = θ 0 y e y dy = θγ(3) = θ [or note X gamma(, /θ) wth mean θ (from Yellow Book)] Settng X = θ MME θ = X/ (b) L(θ) =
More informationSL n (F ) Equals its Own Derived Group
Internatonal Journal of Algebra, Vol. 2, 2008, no. 12, 585-594 SL n (F ) Equals ts Own Derved Group Jorge Macel BMCC-The Cty Unversty of New York, CUNY 199 Chambers street, New York, NY 10007, USA macel@cms.nyu.edu
More informationSELECTED PROOFS. DeMorgan s formulas: The first one is clear from Venn diagram, or the following truth table:
SELECTED PROOFS DeMorgan s formulas: The frst one s clear from Venn dagram, or the followng truth table: A B A B A B Ā B Ā B T T T F F F F T F T F F T F F T T F T F F F F F T T T T The second one can be
More information4 Analysis of Variance (ANOVA) 5 ANOVA. 5.1 Introduction. 5.2 Fixed Effects ANOVA
4 Analyss of Varance (ANOVA) 5 ANOVA 51 Introducton ANOVA ANOVA s a way to estmate and test the means of multple populatons We wll start wth one-way ANOVA If the populatons ncluded n the study are selected
More informationNOTES ON SIMPLIFICATION OF MATRICES
NOTES ON SIMPLIFICATION OF MATRICES JONATHAN LUK These notes dscuss how to smplfy an (n n) matrx In partcular, we expand on some of the materal from the textbook (wth some repetton) Part of the exposton
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationFirst day August 1, Problems and Solutions
FOURTH INTERNATIONAL COMPETITION FOR UNIVERSITY STUDENTS IN MATHEMATICS July 30 August 4, 997, Plovdv, BULGARIA Frst day August, 997 Problems and Solutons Problem. Let {ε n } n= be a sequence of postve
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationSolution 1 for USTC class Physics of Quantum Information
Soluton 1 for 018 019 USTC class Physcs of Quantum Informaton Shua Zhao, Xn-Yu Xu and Ka Chen Natonal Laboratory for Physcal Scences at Mcroscale and Department of Modern Physcs, Unversty of Scence and
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors
Stat60: Bayesan Modelng and Inference Lecture Date: February, 00 Reference Prors Lecturer: Mchael I. Jordan Scrbe: Steven Troxler and Wayne Lee In ths lecture, we assume that θ R; n hgher-dmensons, reference
More informationBOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS
BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationMATH 5707 HOMEWORK 4 SOLUTIONS 2. 2 i 2p i E(X i ) + E(Xi 2 ) ä i=1. i=1
MATH 5707 HOMEWORK 4 SOLUTIONS CİHAN BAHRAN 1. Let v 1,..., v n R m, all lengths v are not larger than 1. Let p 1,..., p n [0, 1] be arbtrary and set w = p 1 v 1 + + p n v n. Then there exst ε 1,..., ε
More informationExpectation propagation
Expectaton propagaton Lloyd Ellott May 17, 2011 Suppose p(x) s a pdf and we have a factorzaton p(x) = 1 Z n f (x). (1) =1 Expectaton propagaton s an nference algorthm desgned to approxmate the factors
More informationLecture 6/7 (February 10/12, 2014) DIRAC EQUATION. The non-relativistic Schrödinger equation was obtained by noting that the Hamiltonian 2
P470 Lecture 6/7 (February 10/1, 014) DIRAC EQUATION The non-relatvstc Schrödnger equaton was obtaned by notng that the Hamltonan H = P (1) m can be transformed nto an operator form wth the substtutons
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More informationStrong Markov property: Same assertion holds for stopping times τ.
Brownan moton Let X ={X t : t R + } be a real-valued stochastc process: a famlty of real random varables all defned on the same probablty space. Defne F t = nformaton avalable by observng the process up
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationFoundations of Arithmetic
Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More informationThe EM Algorithm (Dempster, Laird, Rubin 1977) The missing data or incomplete data setting: ODL(φ;Y ) = [Y;φ] = [Y X,φ][X φ] = X
The EM Algorthm (Dempster, Lard, Rubn 1977 The mssng data or ncomplete data settng: An Observed Data Lkelhood (ODL that s a mxture or ntegral of Complete Data Lkelhoods (CDL. (1a ODL(;Y = [Y;] = [Y,][
More informationExplaining the Stein Paradox
Explanng the Sten Paradox Kwong Hu Yung 1999/06/10 Abstract Ths report offers several ratonale for the Sten paradox. Sectons 1 and defnes the multvarate normal mean estmaton problem and ntroduces Sten
More informationMath 702 Midterm Exam Solutions
Math 702 Mdterm xam Solutons The terms measurable, measure, ntegrable, and almost everywhere (a.e.) n a ucldean space always refer to Lebesgue measure m. Problem. [6 pts] In each case, prove the statement
More informationFINITELY-GENERATED MODULES OVER A PRINCIPAL IDEAL DOMAIN
FINITELY-GENERTED MODULES OVER PRINCIPL IDEL DOMIN EMMNUEL KOWLSKI Throughout ths note, s a prncpal deal doman. We recall the classfcaton theorem: Theorem 1. Let M be a fntely-generated -module. (1) There
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationPrimer on High-Order Moment Estimators
Prmer on Hgh-Order Moment Estmators Ton M. Whted July 2007 The Errors-n-Varables Model We wll start wth the classcal EIV for one msmeasured regressor. The general case s n Erckson and Whted Econometrc
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More informationDigital Signal Processing
Dgtal Sgnal Processng Dscrete-tme System Analyss Manar Mohasen Offce: F8 Emal: manar.subh@ut.ac.r School of IT Engneerng Revew of Precedent Class Contnuous Sgnal The value of the sgnal s avalable over
More informationCausal Diamonds. M. Aghili, L. Bombelli, B. Pilgrim
Causal Damonds M. Aghl, L. Bombell, B. Plgrm Introducton The correcton to volume of a causal nterval due to curvature of spacetme has been done by Myrhem [] and recently by Gbbons & Solodukhn [] and later
More information5 The Rational Canonical Form
5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces
More informationBallot Paths Avoiding Depth Zero Patterns
Ballot Paths Avodng Depth Zero Patterns Henrch Nederhausen and Shaun Sullvan Florda Atlantc Unversty, Boca Raton, Florda nederha@fauedu, ssull21@fauedu 1 Introducton In a paper by Sapounaks, Tasoulas,
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More information3.1 ML and Empirical Distribution
67577 Intro. to Machne Learnng Fall semester, 2008/9 Lecture 3: Maxmum Lkelhood/ Maxmum Entropy Dualty Lecturer: Amnon Shashua Scrbe: Amnon Shashua 1 In the prevous lecture we defned the prncple of Maxmum
More informationModelli Clamfim Equazione del Calore Lezione ottobre 2014
CLAMFIM Bologna Modell 1 @ Clamfm Equazone del Calore Lezone 17 15 ottobre 2014 professor Danele Rtell danele.rtell@unbo.t 1/24? Convoluton The convoluton of two functons g(t) and f(t) s the functon (g
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationMath 217 Fall 2013 Homework 2 Solutions
Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationSIO 224. m(r) =(ρ(r),k s (r),µ(r))
SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationGeorgia Tech PHYS 6124 Mathematical Methods of Physics I
Georga Tech PHYS 624 Mathematcal Methods of Physcs I Instructor: Predrag Cvtanovć Fall semester 202 Homework Set #7 due October 30 202 == show all your work for maxmum credt == put labels ttle legends
More informationSee Book Chapter 11 2 nd Edition (Chapter 10 1 st Edition)
Count Data Models See Book Chapter 11 2 nd Edton (Chapter 10 1 st Edton) Count data consst of non-negatve nteger values Examples: number of drver route changes per week, the number of trp departure changes
More informationTopic 5: Non-Linear Regression
Topc 5: Non-Lnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually non-lnear n the parameters.
More informationThe Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor
Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T
More informationSTAT 3008 Applied Regression Analysis
STAT 3008 Appled Regresson Analyss Tutoral : Smple Lnear Regresson LAI Chun He Department of Statstcs, The Chnese Unversty of Hong Kong 1 Model Assumpton To quantfy the relatonshp between two factors,
More information1 Generating functions, continued
Generatng functons, contnued. Exponental generatng functons and set-parttons At ths pont, we ve come up wth good generatng-functon dscussons based on 3 of the 4 rows of our twelvefold way. Wll our nteger-partton
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationComplex Numbers. x = B B 2 4AC 2A. or x = x = 2 ± 4 4 (1) (5) 2 (1)
Complex Numbers If you have not yet encountered complex numbers, you wll soon do so n the process of solvng quadratc equatons. The general quadratc equaton Ax + Bx + C 0 has solutons x B + B 4AC A For
More informationP exp(tx) = 1 + t 2k M 2k. k N
1. Subgaussan tals Defnton. Say that a random varable X has a subgaussan dstrbuton wth scale factor σ< f P exp(tx) exp(σ 2 t 2 /2) for all real t. For example, f X s dstrbuted N(,σ 2 ) then t s subgaussan.
More informationk t+1 + c t A t k t, t=0
Macro II (UC3M, MA/PhD Econ) Professor: Matthas Kredler Fnal Exam 6 May 208 You have 50 mnutes to complete the exam There are 80 ponts n total The exam has 4 pages If somethng n the queston s unclear,
More informationConjugacy and the Exponential Family
CS281B/Stat241B: Advanced Topcs n Learnng & Decson Makng Conjugacy and the Exponental Famly Lecturer: Mchael I. Jordan Scrbes: Bran Mlch 1 Conjugacy In the prevous lecture, we saw conjugate prors for the
More informationSystems of Equations (SUR, GMM, and 3SLS)
Lecture otes on Advanced Econometrcs Takash Yamano Fall Semester 4 Lecture 4: Sstems of Equatons (SUR, MM, and 3SLS) Seemngl Unrelated Regresson (SUR) Model Consder a set of lnear equatons: $ + ɛ $ + ɛ
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationChat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980
MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More information1 (1 + ( )) = 1 8 ( ) = (c) Carrying out the Taylor expansion, in this case, the series truncates at second order:
68A Solutons to Exercses March 05 (a) Usng a Taylor expanson, and notng that n 0 for all n >, ( + ) ( + ( ) + ) We can t nvert / because there s no Taylor expanson around 0 Lets try to calculate the nverse
More informationSupplementary material: Margin based PU Learning. Matrix Concentration Inequalities
Supplementary materal: Margn based PU Learnng We gve the complete proofs of Theorem and n Secton We frst ntroduce the well-known concentraton nequalty, so the covarance estmator can be bounded Then we
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationComplex Numbers Alpha, Round 1 Test #123
Complex Numbers Alpha, Round Test #3. Wrte your 6-dgt ID# n the I.D. NUMBER grd, left-justfed, and bubble. Check that each column has only one number darkened.. In the EXAM NO. grd, wrte the 3-dgt Test
More informationRandom Partitions of Samples
Random Parttons of Samples Klaus Th. Hess Insttut für Mathematsche Stochastk Technsche Unverstät Dresden Abstract In the present paper we construct a decomposton of a sample nto a fnte number of subsamples
More informationCOMPLEX NUMBERS AND QUADRATIC EQUATIONS
COMPLEX NUMBERS AND QUADRATIC EQUATIONS INTRODUCTION We know that x 0 for all x R e the square of a real number (whether postve, negatve or ero) s non-negatve Hence the equatons x, x, x + 7 0 etc are not
More information