DISCUSSION: LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION. By Zhao Ren and Harrison H. Zhou Yale University
|
|
- Godwin Summers
- 5 years ago
- Views:
Transcription
1 Submitted to the Aals of Statistics DISCUSSION: LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION By Zhao Re ad Harriso H. Zhou Yale Uiversity 1. Itroductio. We would like to cogratulate the authors for their refreshig cotributio to this high-dimesioal latet variables grahical model selectio roblem. The roblem of covariace ad cocetratio matrices is fudametally imortat i several classical statistical methodologies ad may alicatios. Recetly, sarse cocetratio matrices estimatio had received cosiderable attetio, artly due to its coectio to sarse structure learig for Gaussia grahical models. See, for examle, Meishause ad Bühlma (2006) ad Ravikumar et al. (2008). Cai, Liu & Zhou (2012) cosidered rate-otimal estimatio. The authors exteded the curret scoe to iclude latet variables. They assume that the fully observed Gaussia grahical model has a aturally sarse deedece grah. However, there are oly artial observatios available for which the grah is usually o loger sarse. Let X be ( + r) variate Gaussia with a sarse cocetratio matrix S(O,H). We oly observe X O, out of the whole + r variables, ad deote its covariace matrix by Σ O. I this case, usually the cocetratio matrix (Σ O ) 1 are ot sarse. Let S be the cocetratio matrix of observed variables coditioed o latet variables, which is a submatrix of S(O,H) ad hece has a sarse structure, ad let L be the summary of the margializatio over the latet variables ad its rak corresods to the umber of latet variables r for which we usually assume it is small. The authors observed (Σ O ) 1 ca be decomosed as the differece of the sarse matrix S ad the rak r matrix L, i.e., (Σ O ) 1 = S L. The followig traditioal wisdoms the authors aturally roosed a regularized maximum likelihood aroach to estimate both the sarse structure S ad the low rak art L, mi tr ((S L) (S,L):S L 0, L 0 Σ O) log det (S L) + χ (γ S 1 + tr (L)) where Σ O is the samle covariace matrix, S 1 = i,j s ij, ad γ ad χ are regularizatio tuig arameters. Here tr (L) is the trace of L. The The research was suorted i art by NSF Career Award DMS ad NSF FRG Grat DMS
2 2 otatio A 0 meas A is ositive defiite, ad A 0 deotes that A is o-egative. There is a obvious idetifiability roblem if we wat to estimate both the sarse ad low rak comoets. A matrix ca be both sarse ad low rak. By exlorig the geometric roerties of the taget saces for sarse ad low rak comoets, the authors gave a beautiful sufficiet coditio for idetifiability, ad the rovided very much ivolved theoretical justificatios based o the sufficiet coditio, which is beyod our ability to digest them i a short eriod of time i the sese that we do t fully uderstad why those techical assumtios were eeded i the aalysis of their aroach. Thus we decided to look at a relatively simle but otetially ractical model, with the hoe to still cature the essece of the roblem, ad see how well their regularized rocedure works. Let 1 1 deotes the matrix l 1 orm, i.e., S 1 1 = max 1 i j=1 s ij. We assume that S is i the followig uiformity class, (1) U (s 0 (), M ) = S = (s ij) : S 0, S 1 1 M, max 1 {s ij 0} s 0 () 1 i, where we allow s 0 () ad M to grow as ad icrease. This uiformity class was cosidered i Ravikumar et al. (2008) ad Cai, Liu ad Luo (2011). For the low rak matrix L, we assume that the effect of margializatio over the latet variables sreads out, i.e. the low rak matrix L has row/colum saces that are ot closely aliged with the coordiate axes to resolve the idetifiability roblem. Let the eige-decomositio of L be as follows (2) L = r 0 () i=1 λ i u i u T i, where r 0 () is the rak of L. We assume that there exists a uiversal costat c 0 such that u i c0 for all i, ad L 1 1 is bouded by M which ca be show to be bouded by c 0 r 0. A similar icoherece assumtio o u i was used i Cadès ad Recht (2008). We further assume that (3) λ max (Σ O) M, ad λ mi (Σ O) 1/M for some uiversal costat M. As discussed i the aer, the goals i latet variable model selectio are to obtai the sig cosistecy for the sarse matrix S as well as the rak cosistecy for the low rak semi-ositive defiite matrix L. j=1
3 Deote the miimum magitude of ozero etries of S by θ, i.e., θ = mi i,j s ij 1 {s ij 0}, ad the miimum ozero eigevalue of L by σ, i.e., σ = mi 1 i r0 λ i. To obtai theoretical guaratees of cosistecy results for the model described i (1), (2) ad (3), i additio to the strog irreresetability coditio which seems to be difficult to check i ractice, the authors require the followig assumtios (by a traslatio of the coditios i the aer to this model) for θ, σ ad : (1) θ /, which is eeded eve whe s 0 () is costat; (2) σ s 3 0 () / uder the additioal strog assumtios o the Fisher iformatio matrix Σ O Σ O (see the footote for Corollary 4.2); (3) s 4 0 () /. However, for sarse grahical model selectio without latet variables, either l 1 -regularized maximum likelihood aroach (see Ravikumar et al. (2008)) or CLIME (see Cai, Liu ad Luo (2011)) ca be show to be sig cosistet if the miimum magitude ozero etry of cocetratio matrix θ is at the order of (log ) / whe M is bouded, which isires us to study rate-otimalites for this latet variables grahical model selectio roblem. I this discussio, we roose a rocedure to obtai a algebraically cosistet estimate of the latet variable Gaussia grahical model uder much weaker coditio o both θ ad σ. For examle, for a wide rage of s 0 (), we oly require θ is at the order of (log ) / ad σ is at the order of / to cosistetly estimate the suort of S ad the rak of L. That meas the regularized maximum likelihood aroach could be far from beig otimal, but we do t kow yet whether the sub-otimality is due to the rocedure or their theoretical aalysis. 2. Latet Variable Model Selectio Cosistecy. I this sectio, we roose a rocedure to obtai a algebraically cosistet estimate of the latet variable Gaussia grahical model. The coditio o θ to recover the suort of S is reduced to that i Cai, Liu ad Luo (2011) which studied sarse grahical model selectio without latet variables, ad the coditio o σ is just at a order of /, which is smaller tha s 3 0 () / assumed i the aer whe s 0 (). Whe M is bouded, our results ca be show to be rate-otimal by lower bouds stated i Remarks 2 ad 4 for which we are ot givig roofs due to the limitatio of the sace Sig Cosistecy Procedure of S. We roose a CLIME-like estimator of S by solvig the followig liear otimizatio roblem, mi S 1 subject to Σ OS I τ, S R, 3
4 4 where Σ O = ( σ ij) is the samle covariace matrix. The tuig arameter log τ is chose as τ = C 1 M for some large costat C 1. Let Ŝ1 = ŝ 1 ij be the solutio. The CLIME-like estimator Ŝ = (ŝ ij) is obtaied by symmetrizig Ŝ1 as follows, ŝ ij = ŝ ji = ŝ 1 ij1 { ŝ 1 ij ŝ 1 } ji +ŝ 1 ji 1 { ŝ 1 ij > ŝ 1 ji}. I other words, we take the oe with smaller magitude betwee ŝ 1 ij ad ŝ1 ji. We defie a thresholdig estimator S = ( s ij ) with (4) s ij = s ij 1 { s ij > 9M τ } to estimate the suort of S. Theorem 1 Suose that S U (s 0 (), M ), (5) (log )/ = o(1), ad L M τ. With robability greater tha 1 C s 6 for some costat C s deedig o M oly, we have Ŝ S 9M τ. Hece if the miimum magitude of ozero etries θ > 18M τ, we obtai the sig cosistecy sig S = sig (S ). I articular, if M is i the costat level, the to cosistetly recover the suort of S, we oly eed that θ (log )/. Proof. The roof is similar to the Theorem 7 i Cai, Liu ad Luo (2011). The subgaussia coditio with sectral orm uer boud M imlies that each emirical covariace σ ij satisfies the followig large deviatio result ( P ( σ ij σ ij > t) C s ex 8 ) C2 2 t 2, for t ϕ, where C s, C 2 ad ϕ oly deeds o M. See, for examle, Bickel ad Levia (2008). I articular for t = C 2 (log ) / which is less tha ϕ by our assumtio we have (6) P (Σ O Σ O > t) i,j P ( σ ij σ ij > t) 2 C s 8. Let A = {Σ O Σ O C 2 (log )/ }.
5 5 Equatio (6) imlies P (A) 1 C s 6. O evet A, we will show (7) (S L ) Ŝ1 8M τ, which immediately yield S Ŝ (S L ) Ŝ1 + L 8M τ + M τ = 9M τ. Now we establish Equatio (7). O evet A, for some large costat C 1 2C 2, the choice of τ yields (8) 2M Σ O Σ O τ. By the matrix l 1 orm assumtio, we could obtai that (9) (Σ O) S L 1 1 2M. From (8) ad (9) we have Σ O (S L ) I = (Σ O Σ O) (Σ O) 1 Σ O Σ (Σ O O ) τ, which imlies (10) Σ O (S L ) Σ OŜ1 Σ O (S L ) I + Σ OŜ1 I 2τ. From the defiitio of Ŝ1 we obtai that (11) Ŝ1 S L 1 1 2M, 1 1 which, together with Equatios (8) ad (10), imlies ( Ŝ1) ( ) Σ O (S L ) Σ O (S L ) Ŝ1 + (Σ O Σ O) (S L ) Ŝ1 2τ + Σ O Σ (S O L ) Ŝ1 2τ + 4M Σ O Σ O 4τ. 1 1 Thus we have (S L ) Ŝ1 (Σ O) 1 Σ 1 1 O ((S Ŝ1) L ) 8M τ.
6 6 Remark 1 By the choice of our τ ad the eige-decomositio of L, the coditio L M τ holds whe r 0 ()C 0 / C 1 M 2 (log ) /, i.e., 2 log r0 2 ()M 4. If M is slowly icreasig (for istace 1/4 τ for ay small τ > 0), the miimum requiremet θ M 2 (log ) / is weaker tha θ / required i Corollary 4.2. Furthermore, it ca be show that the otimal rate of miimum magitude of ozero etries for sig cosistecy is θ M (log )/as i the Cai, Liu ad Zhou (2012). Remark 2 Cai, Liu ad Zhou (2012) showed the miimum requiremet for θ, θ M (log )/ is ecessary for sig cosistecy for sarse cocetratio matrices. Let U S (c) deote the class of cocetratio matrices defied i (1) ad (2), satisfyig assumtio (5) ad θ > cm (log )/. We ca show that there exists some costat c 1 > 0 such that for all 0 < c < c 1, ) lim if P sig (Ŝ sig (S ) > 0 su (Ŝ, ˆL) U S (c) similar to Cai, Liu ad Zhou (2012) Rak Cosistecy Procedure of L. I this sectio we roose a rocedure to estimate L ad its rak. We ote that with high robability Σ O is ivertible, the defie ˆL = (Σ O ) 1 S, where S is defied i (4). Deote the eige-decomositio of ˆL by { } i=1 λ i(ˆl)υ i υi T, ad let λ i( L) = λ i (ˆL)1 λ i (ˆL) > C 3 where costat C 3 will be secified later. Defie L = i=1 λ i( L)υ i υi T. The followig theorem shows that estimator L is a cosistet estimator of ) L uder the sectral orm ad with high robability rak (L ) = rak ( L. Theorem 2 Uder the coditios i Theorem 1, we assume that (12) M, ad M 2 2 s 0 () log. The there exists some costat C 3 such that ˆL L C3 with robability greater tha 1 2e C s 6. Hece if σ > 2C 3 ), we have rak (L ) = rak ( L with high robability.
7 Proof. From the Corollary 5.5 of the aer ad our assumtio o the samle size, we have ( P Σ O Σ O ) 128M 2 ex ( ). Note that λ mi (Σ O ) 1/M, ad 128M 1/ (2M) uder the assumtio (12), the λ mi (Σ O ) 1/ (2M) with high robability, which yields the same rate of covergece for the cocetratio matrix, sice (13) (Σ O) 1 (Σ O) 1 (Σ O ) 1 (Σ O ) 1 Σ O Σ O 2M 2 128M = 16 2M 3. From Theorem 1 we kow sig S = sig (S ), ad S S 9M τ with robability greater tha 1 C s 6. Sice B B 1 1 for ay symmetric matrix B, we the have (14) S S S S log s 0 () 9M τ = 9C 1 M 2 s 0 () 1 1. Equatios (13) ad (14), together the assumtio M 2 s 0 () 7 log, imly ˆL L (Σ O ) 1 (Σ O) 1 + S S 16 log 2M 3 +9C 1M 2 s 0 () C 3 with robability greater tha 1 2e C s 6. Remark 3 We should emhasize the fact that i order to cosistetly esti- mate the rak of L we eed oly that σ > 2C 3, which is smaller tha s 3 0 () required i the aer (see the footote for Corollary 4.2), as log as M 2 s 0 () log. I articular, we do t exlicitly costrai the rak r 0 (). Oe secial case is that M is costat ad s 0 () 1/2 τ for some small τ > 0, for which our requiremet is aer is at a order of 3(1/2 τ). but the assumtio i the
8 8 Remark 4 Let U L (c) deote the class of cocetratio matrices defied i (1), (2) ad (3), satisfyig assumtios (12), (5) ad σ > c. We ca show that there exists some costat c 2 > 0 such that for all 0 < c < c 2, ) lim if P rak (ˆL rak (L ) > 0. su (Ŝ, ˆL) U L (c) The roof of this lower boud is based o a modificatio of a lower boud argumet i a ersoal commuicatio of T. Toy Cai (2011). 3. Cocludig Remarks ad Further Questios. I this discussio we attemt to uderstad otimalities of results i the reset aer by studyig a relatively simle model. Our relimiary aalysis seems to idicate that their results i this aer are sub-otimal. I articular we ted to coclude that assumtios o θ ad σ i the aer ca be otetially very much weakeed. However it is ot clear to us whether the sub-otimality is due to the methodology or just its theoretical aalysis. We wat to emhasize that the relimiary results i this discussio ca be stregtheed, but for the urose of simlicity of the discussio we choose to reset weaker but simler results to hoefully shed some lights o uderstadig otimalities i estimatio. REFERENCES [1] Bickel, P. J. ad Levia, E. (2008). Regularized estimatio of large covariace matrices. A. Statist [2] Cai, T. T., Liu, W. ad Luo, X. (2011). A costraied l 1 miimizatio aroach to sarse recisio matrix estimatio. J. Amer. Statist. Assoc [3] Cai, T. T., Liu, W. ad Zhou, H. H. (2012). Otimal estimatio of large sarse recisio matrices. Mauscrit. [4] Cai, T. T. (2011). Persoal commuicatio. [5] Cadès, E. J. ad Recht, B. (2009). Exact matrix comletio via covex otimizatio. Foud. of Comut. Math [6] Meishause, N. ad Bühlma, P. (2006). High dimesioal grahs ad variable selectio with the Lasso. A. Statist [7] Ravikumar, P., Waiwright, M. J., Raskutti, G., ad Yu, B. (2008). High-dimesioal covariace estimatio by miimizig l 1 ealized log-determiat divergece. Prerit. Deartmet of Statistics, Yale Uiversity New Have, CT USA zhao.re@yale.edu huibi.zhou@yale.edu
SYMMETRIC POSITIVE SEMI-DEFINITE SOLUTIONS OF AX = B AND XC = D
Joural of Pure ad Alied Mathematics: Advaces ad Alicatios olume, Number, 009, Pages 99-07 SYMMERIC POSIIE SEMI-DEFINIE SOLUIONS OF AX B AND XC D School of Mathematics ad Physics Jiagsu Uiversity of Sciece
More informationSummary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector
Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short
More informationSUPPLEMENT TO GEOMETRIC INFERENCE FOR GENERAL HIGH-DIMENSIONAL LINEAR INVERSE PROBLEMS
Submitted to the Aals of Statistics arxiv: arxiv:0000.0000 SUPPLEMENT TO GEOMETRIC INFERENCE FOR GENERAL HIGH-DIMENSIONAL LINEAR INVERSE PROBLEMS By T. Toy Cai, Tegyua Liag ad Alexader Rakhli The Wharto
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationECE534, Spring 2018: Solutions for Problem Set #2
ECE534, Srig 08: s for roblem Set #. Rademacher Radom Variables ad Symmetrizatio a) Let X be a Rademacher radom variable, i.e., X = ±) = /. Show that E e λx e λ /. E e λx = e λ + e λ = + k= k=0 λ k k k!
More informationA Central Limit Theorem for Belief Functions
A Cetral Limit Theorem for Belief Fuctios Larry G. Estei Kyougwo Seo November 7, 2. CLT for Belief Fuctios The urose of this Note is to rove a form of CLT (Theorem.4) that is used i Estei ad Seo (2). More
More informationHybridized Heredity In Support Vector Machine
Hybridized Heredity I Suort Vector Machie May 2015 Hybridized Heredity I Suort Vector Machie Timothy Idowu Yougmi Park Uiversity of Wiscosi-Madiso idowu@stat.wisc.edu yougmi@stat.wisc.edu May 2015 Abstract
More informationUnit 5. Hypersurfaces
Uit 5. Hyersurfaces ================================================================= -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
More informationJohn H. J. Einmahl Tilburg University, NL. Juan Juan Cai Tilburg University, NL
Estimatio of the margial exected shortfall Laures de Haa, Poitiers, 202 Estimatio of the margial exected shortfall Jua Jua Cai Tilburg iversity, NL Laures de Haa Erasmus iversity Rotterdam, NL iversity
More informationLecture 12: February 28
10-716: Advaced Machie Learig Sprig 2019 Lecture 12: February 28 Lecturer: Pradeep Ravikumar Scribes: Jacob Tyo, Rishub Jai, Ojash Neopae Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationConfidence Intervals
Cofidece Itervals Berli Che Deartmet of Comuter Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Referece: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chater 5 & Teachig Material Itroductio
More informationIterative Techniques for Solving Ax b -(3.8). Assume that the system has a unique solution. Let x be the solution. Then x A 1 b.
Iterative Techiques for Solvig Ax b -(8) Cosider solvig liear systems of them form: Ax b where A a ij, x x i, b b i Assume that the system has a uique solutio Let x be the solutio The x A b Jacobi ad Gauss-Seidel
More informationBounds for the Extreme Eigenvalues Using the Trace and Determinant
ISSN 746-7659, Eglad, UK Joural of Iformatio ad Computig Sciece Vol 4, No, 9, pp 49-55 Bouds for the Etreme Eigevalues Usig the Trace ad Determiat Qi Zhog, +, Tig-Zhu Huag School of pplied Mathematics,
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationON SOME NEW SEQUENCE SPACES OF NON-ABSOLUTE TYPE RELATED TO THE SPACES l p AND l I. M. Mursaleen and Abdullah K. Noman
Faculty of Scieces ad Mathematics, Uiversity of Niš, Serbia Available at: htt://www.mf.i.ac.rs/filomat Filomat 25:2 20, 33 5 DOI: 0.2298/FIL02033M ON SOME NEW SEQUENCE SPACES OF NON-ABSOLUTE TYPE RELATED
More informationLecture 24: Variable selection in linear models
Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet
More informationOn Nonsingularity of Saddle Point Matrices. with Vectors of Ones
Iteratioal Joural of Algebra, Vol. 2, 2008, o. 4, 197-204 O Nosigularity of Saddle Poit Matrices with Vectors of Oes Tadeusz Ostrowski Istitute of Maagemet The State Vocatioal Uiversity -400 Gorzów, Polad
More informationA REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS. S. S. Dragomir 1. INTRODUCTION
TAIWANESE JOURNAL OF MATHEMATICS Vol. 14, No. 1,. 153-164, February 2010 This aer is available olie at htt://www.tjm.sysu.edu.tw/ A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS FOR f-divergence
More informationUNIFORM RATES OF ESTIMATION IN THE SEMIPARAMETRIC WEIBULL MIXTURE MODEL. BY HEMANT ISHWARAN University of Ottawa
The Aals of Statistics 1996, Vol. 4, No. 4, 1571585 UNIFORM RATES OF ESTIMATION IN THE SEMIPARAMETRIC WEIBULL MIXTURE MODEL BY HEMANT ISHWARAN Uiversity of Ottawa This aer resets a uiform estimator for
More informationStochastic Matrices in a Finite Field
Stochastic Matrices i a Fiite Field Abstract: I this project we will explore the properties of stochastic matrices i both the real ad the fiite fields. We first explore what properties 2 2 stochastic matrices
More informationarxiv: v4 [cs.it] 13 Feb 2015
Icoherece-Otimal Matrix Comletio Yudog Che Deartmet of Electrical Egieerig ad Comuter Scieces The Uiversity of Califoria, Berkeley yudogche@eecsberkeleyedu arxiv:30054v4 [csit] 3 eb 05 Abstract This aer
More informationA Note on Sums of Independent Random Variables
Cotemorary Mathematics Volume 00 XXXX A Note o Sums of Ideedet Radom Variables Pawe l Hitczeko ad Stehe Motgomery-Smith Abstract I this ote a two sided boud o the tail robability of sums of ideedet ad
More informationEstimation Theory Chapter 3
stimatio Theory Chater 3 Likelihood Fuctio Higher deedece of data PDF o ukow arameter results i higher estimatio accuracy amle : If ˆ If large, W, Choose  P  small,  W POOR GOOD i Oly data samle Data
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationSpreading Processes and Large Components in Ordered, Directed Random Graphs
Sreadig Processes ad Large Comoets i Ordered, Directed Radom Grahs Paul Hor Malik Magdo-Ismail Setember 2, 202 Abstract Order the vertices of a directed radom grah v,..., v ; edge v i, v j for i < j exists
More informationComposite Quantile Generalized Quasi-Likelihood Ratio Tests for Varying Coefficient Regression Models Jin-ju XU 1 and Zhong-hua LUO 2,*
07 d Iteratioal Coferece o Iformatio Techology ad Maagemet Egieerig (ITME 07) ISBN: 978--60595-45-8 Comosite Quatile Geeralized Quasi-Likelihood Ratio Tests for Varyig Coefficiet Regressio Models Ji-u
More informationSupplemental Material: Proofs
Proof to Theorem Supplemetal Material: Proofs Proof. Let be the miimal umber of traiig items to esure a uique solutio θ. First cosider the case. It happes if ad oly if θ ad Rak(A) d, which is a special
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationTHE INTEGRAL TEST AND ESTIMATES OF SUMS
THE INTEGRAL TEST AND ESTIMATES OF SUMS. Itroductio Determiig the exact sum of a series is i geeral ot a easy task. I the case of the geometric series ad the telescoig series it was ossible to fid a simle
More informationDimension of a Maximum Volume
Dimesio of a Maximum Volume Robert Kreczer Deartmet of Mathematics ad Comutig Uiversity of Wiscosi-Steves Poit Steves Poit, WI 54481 Phoe: (715) 346-3754 Email: rkrecze@uwsmail.uws.edu 1. INTRODUCTION.
More informationA Hadamard-type lower bound for symmetric diagonally dominant positive matrices
A Hadamard-type lower boud for symmetric diagoally domiat positive matrices Christopher J. Hillar, Adre Wibisoo Uiversity of Califoria, Berkeley Jauary 7, 205 Abstract We prove a ew lower-boud form of
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationA Note on Bilharz s Example Regarding Nonexistence of Natural Density
Iteratioal Mathematical Forum, Vol. 7, 0, o. 38, 877-884 A Note o Bilharz s Examle Regardig Noexistece of Natural Desity Cherg-tiao Perg Deartmet of Mathematics Norfolk State Uiversity 700 Park Aveue,
More informationSolutions to Problem Set 7
8.78 Solutios to Problem Set 7. If the umber is i S, we re doe sice it s relatively rime to everythig. So suose S. Break u the remaiig elemets ito airs {, }, {4, 5},..., {, + }. By the Pigeohole Pricile,
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationRound-off Errors and Computer Arithmetic - (1.2)
Roud-off Errors ad Comuter Arithmetic - (1.) 1. Roud-off Errors: Roud-off errors is roduced whe a calculator or comuter is used to erform real umber calculatios. That is because the arithmetic erformed
More informationRoberto s Notes on Series Chapter 2: Convergence tests Section 7. Alternating series
Roberto s Notes o Series Chapter 2: Covergece tests Sectio 7 Alteratig series What you eed to kow already: All basic covergece tests for evetually positive series. What you ca lear here: A test for series
More informationFinal Solutions. 1. (25pts) Define the following terms. Be as precise as you can.
Mathematics H104 A. Ogus Fall, 004 Fial Solutios 1. (5ts) Defie the followig terms. Be as recise as you ca. (a) (3ts) A ucoutable set. A ucoutable set is a set which ca ot be ut ito bijectio with a fiite
More informationarxiv: v2 [stat.ml] 5 Jun 2018
Grah-based regularizatio for regressio roblems with highly-correlated desigs Yua Li, Bejami Mark, Garvesh Raskutti ad Rebecca Willett arxiv:803.07658v [stat.ml] 5 Ju 08. Itroductio Abstract: Sarse models
More informationPAijpam.eu ON TENSOR PRODUCT DECOMPOSITION
Iteratioal Joural of Pure ad Applied Mathematics Volume 103 No 3 2015, 537-545 ISSN: 1311-8080 (prited versio); ISSN: 1314-3395 (o-lie versio) url: http://wwwijpameu doi: http://dxdoiorg/1012732/ijpamv103i314
More informationMINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER l 1 -NORM
Statistica Siica 22 (202), 39-378 doi:http://dx.doi.org/0.5705/ss.200.253 MINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER l -NORM T. Toy Cai ad Harriso H. Zhou Uiversity of Pesylvaia ad Yale Uiversity
More informationarxiv: v1 [math.st] 15 Jan 2014
Marcheko-Pastur Law for Tyler s ad Maroa s M-estimators arxiv:1401.3424v1 [math.st] 15 Ja 2014 Teg Zhag The Program i Alied ad Comutatioal Mathematics PACM, Priceto Uiversity Priceto, New Jersey 08544,
More informationConfidence intervals for proportions
Cofidece itervals for roortios Studet Activity 7 8 9 0 2 TI-Nsire Ivestigatio Studet 60 mi Itroductio From revious activity This activity assumes kowledge of the material covered i the activity Distributio
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationProposition 2.1. There are an infinite number of primes of the form p = 4n 1. Proof. Suppose there are only a finite number of such primes, say
Chater 2 Euclid s Theorem Theorem 2.. There are a ifiity of rimes. This is sometimes called Euclid s Secod Theorem, what we have called Euclid s Lemma beig kow as Euclid s First Theorem. Proof. Suose to
More informationRobust Lasso with missing and grossly corrupted observations
Robust Lasso with missig ad grossly corrupted observatios Nam H. Nguye Johs Hopkis Uiversity am@jhu.edu Nasser M. Nasrabadi U.S. Army Research Lab asser.m.asrabadi.civ@mail.mil Trac D. Tra Johs Hopkis
More informationWeak and Strong Convergence Theorems of New Iterations with Errors for Nonexpansive Nonself-Mappings
doi:.36/scieceasia53-874.6.3.67 ScieceAsia 3 (6: 67-7 Weak ad Strog Covergece Theorems of New Iteratios with Errors for Noexasive Noself-Maigs Sorsak Thiawa * ad Suthe Suatai ** Deartmet of Mathematics
More information10.6 ALTERNATING SERIES
0.6 Alteratig Series Cotemporary Calculus 0.6 ALTERNATING SERIES I the last two sectios we cosidered tests for the covergece of series whose terms were all positive. I this sectio we examie series whose
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationSieve Estimators: Consistency and Rates of Convergence
EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes
More informationA unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
A uified framework for high-dimesioal aalysis of M-estimators with decomposable regularizers Sahad Negahba, UC Berkeley Pradeep Ravikumar, UT Austi Marti Waiwright, UC Berkeley Bi Yu, UC Berkeley NIPS
More informationAn operator equality involving a continuous field of operators and its norm inequalities
Available olie at www.sciecedirect.com Liear Algebra ad its Alicatios 49 (008) 59 67 www.elsevier.com/locate/laa A oerator equality ivolvig a cotiuous field of oerators ad its orm iequalities Mohammad
More informationSYSTEMS ANALYSIS. I. V. Sergienko, E. F. Galba, and V. S. Deineka UDC :
Cyberetics ad Systems Aalysis, Vol. 44, No. 3, 008 SYSEMS ANALYSIS REPRESENAIONS AND EXPANSIONS OF WEIGHED PSEUDOINVERSE MARICES, IERAIVE MEHODS, AND PROBLEM REGULARIZAION. II. SINGULAR WEIGHS I. V. Sergieo,
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationNew Definition of Density on Knapsack Cryptosystems
Africacryt008@Casablaca 008.06.1 New Defiitio of Desity o Kasac Crytosystems Noboru Kuihiro The Uiversity of Toyo, Jaa 1/31 Kasac Scheme rough idea Public Key: asac: a={a 1, a,, a } Ecrytio: message m=m
More informationNonlinear Gronwall Bellman Type Inequalities and Their Applications
Article Noliear Growall Bellma Tye Iequalities ad Their Alicatios Weimi Wag 1, Yuqiag Feg 2, ad Yuayua Wag 1 1 School of Sciece, Wuha Uiversity of Sciece ad Techology, Wuha 4365, Chia; wugogre@163.com
More informationChapter Vectors
Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More information13.1 Shannon lower bound
ECE598: Iformatio-theoretic methods i high-dimesioal statistics Srig 016 Lecture 13: Shao lower boud, Fao s method Lecturer: Yihog Wu Scribe: Daewo Seo, Mar 8, 016 [Ed Mar 11] I the last class, we leared
More informationMIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS
MIDTERM 3 CALCULUS MATH 300 FALL 08 Moday, December 3, 08 5:5 PM to 6:45 PM Name PRACTICE EXAM S Please aswer all of the questios, ad show your work. You must explai your aswers to get credit. You will
More informationPROBLEM SET I (Suggested Solutions)
Eco3-Fall3 PROBLE SET I (Suggested Solutios). a) Cosider the followig: x x = x The quadratic form = T x x is the required oe i matrix form. Similarly, for the followig parts: x 5 b) x = = x c) x x x x
More informationClassification of DT signals
Comlex exoetial A discrete time sigal may be comlex valued I digital commuicatios comlex sigals arise aturally A comlex sigal may be rereseted i two forms: jarg { z( ) } { } z ( ) = Re { z ( )} + jim {
More informationHypothesis Testing. H 0 : θ 1 1. H a : θ 1 1 (but > 0... required in distribution) Simple Hypothesis - only checks 1 value
Hyothesis estig ME's are oit estimates of arameters/coefficiets really have a distributio Basic Cocet - develo regio i which we accet the hyothesis ad oe where we reject it H - reresets all ossible values
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationYALE UNIVERSITY DEPARTMENT OF COMPUTER SCIENCE
YALE UNIVERSITY DEPARTMENT OF COMPUTER SCIENCE CPSC 467a: Crytograhy ad Comuter Security Notes 16 (rev. 1 Professor M. J. Fischer November 3, 2008 68 Legedre Symbol Lecture Notes 16 ( Let be a odd rime,
More informationtests 17.1 Simple versus compound
PAS204: Lecture 17. tests UMP ad asymtotic I this lecture, we will idetify UMP tests, wherever they exist, for comarig a simle ull hyothesis with a comoud alterative. We also look at costructig tests based
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationSeptember 2012 C1 Note. C1 Notes (Edexcel) Copyright - For AS, A2 notes and IGCSE / GCSE worksheets 1
September 0 s (Edecel) Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright www.pgmaths.co.uk - For AS, A otes ad IGCSE / GCSE worksheets September 0 Copyright
More informationChapter 2. Periodic points of toral. automorphisms. 2.1 General introduction
Chapter 2 Periodic poits of toral automorphisms 2.1 Geeral itroductio The automorphisms of the two-dimesioal torus are rich mathematical objects possessig iterestig geometric, algebraic, topological ad
More informationA Note on Matrix Rigidity
A Note o Matrix Rigidity Joel Friedma Departmet of Computer Sciece Priceto Uiversity Priceto, NJ 08544 Jue 25, 1990 Revised October 25, 1991 Abstract I this paper we give a explicit costructio of matrices
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationarxiv: v1 [math.pr] 13 Oct 2011
A tail iequality for quadratic forms of subgaussia radom vectors Daiel Hsu, Sham M. Kakade,, ad Tog Zhag 3 arxiv:0.84v math.pr] 3 Oct 0 Microsoft Research New Eglad Departmet of Statistics, Wharto School,
More informationLecture 8: October 20, Applications of SVD: least squares approximation
Mathematical Toolkit Autum 2016 Lecturer: Madhur Tulsiai Lecture 8: October 20, 2016 1 Applicatios of SVD: least squares approximatio We discuss aother applicatio of sigular value decompositio (SVD) of
More informationNYU Center for Data Science: DS-GA 1003 Machine Learning and Computational Statistics (Spring 2018)
NYU Ceter for Data Sciece: DS-GA 003 Machie Learig ad Computatioal Statistics (Sprig 208) Brett Berstei, David Roseberg, Be Jakubowski Jauary 20, 208 Istructios: Followig most lab ad lecture sectios, we
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationEstimation with Overidentifying Inequality Moment Conditions Technical Appendix
Estimatio with Overidetifyig Iequality Momet Coditios Techical Aedix Hyugsik Roger Moo Uiversity of Souther Califoria Frak Schorfheide Uiversity of Pesylvaia, CEPR ad NBER August 27, 2008 Not iteded to
More informationA unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
A uified framework for high-dimesioal aalysis of M-estimators with decomposable regularizers Sahad Negahba Departmet of EECS UC Berkeley sahad @eecs.berkeley.edu Marti J. Waiwright Departmet of Statistics
More informationON SUPERSINGULAR ELLIPTIC CURVES AND HYPERGEOMETRIC FUNCTIONS
ON SUPERSINGULAR ELLIPTIC CURVES AND HYPERGEOMETRIC FUNCTIONS KEENAN MONKS Abstract The Legedre Family of ellitic curves has the remarkable roerty that both its eriods ad its suersigular locus have descritios
More informationSongklanakarin Journal of Science and Technology SJST R1 Teerapabolarn
Soglaaari Joural of Sciece ad Techology SJST--.R Teeraabolar A No-uiform Boud o Biomial Aroimatio to the Beta Biomial Cumulative Distributio Fuctio Joural: Soglaaari Joural of Sciece ad Techology For Review
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationKinetics of Complex Reactions
Kietics of Complex Reactios by Flick Colema Departmet of Chemistry Wellesley College Wellesley MA 28 wcolema@wellesley.edu Copyright Flick Colema 996. All rights reserved. You are welcome to use this documet
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMath 116 Second Midterm November 13, 2017
Math 6 Secod Midterm November 3, 7 EXAM SOLUTIONS. Do ot ope this exam util you are told to do so.. Do ot write your ame aywhere o this exam. 3. This exam has pages icludig this cover. There are problems.
More informationDifferentiable Convex Functions
Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More informationRead carefully the instructions on the answer book and make sure that the particulars required are entered on each answer book.
THE UNIVERSITY OF WARWICK FIRST YEAR EXAMINATION: Jauary 2009 Aalysis I Time Allowed:.5 hours Read carefully the istructios o the aswer book ad make sure that the particulars required are etered o each
More informationMatrices and vectors
Oe Matrices ad vectors This book takes for grated that readers have some previous kowledge of the calculus of real fuctios of oe real variable It would be helpful to also have some kowledge of liear algebra
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More informationSolutions to Tutorial 3 (Week 4)
The Uiversity of Sydey School of Mathematics ad Statistics Solutios to Tutorial Week 4 MATH2962: Real ad Complex Aalysis Advaced Semester 1, 2017 Web Page: http://www.maths.usyd.edu.au/u/ug/im/math2962/
More informationSection 5.5. Infinite Series: The Ratio Test
Differece Equatios to Differetial Equatios Sectio 5.5 Ifiite Series: The Ratio Test I the last sectio we saw that we could demostrate the covergece of a series a, where a 0 for all, by showig that a approaches
More information