SUPPLEMENT TO GEOMETRIC INFERENCE FOR GENERAL HIGH-DIMENSIONAL LINEAR INVERSE PROBLEMS
|
|
- Ira Lane
- 5 years ago
- Views:
Transcription
1 Submitted to the Aals of Statistics arxiv: arxiv: SUPPLEMENT TO GEOMETRIC INFERENCE FOR GENERAL HIGH-DIMENSIONAL LINEAR INVERSE PROBLEMS By T. Toy Cai, Tegyua Liag ad Alexader Rakhli The Wharto School at Uiversity of Pesylvaia APPENDIX A: ADDITIONAL PROOFS Proof of Lemma. The roof uses cocetratio of Lischitz fuctios o Gaussia sace, which is illustrated i the followig lemma take from equatio 1.6 i [3]. Lemma A.1 Gaussia cocetratio iequality for Lischitz fuctios. Let g R be a Gaussia vector with i.i.d mea zero ad variace oe elemets ad let F : R R be a Lischitz fuctio with Lischitz costat L i.e. F x F y L x y for ay x, y R, with Euclidea metric o R. The for ay λ > 0, P F g E g F g λ ex λ L. We would like to uer boud X Z A with high robability, where Z N0, σ I. We have X Z A = su X Z, v = su Z, X v. Fixig X, we ca thik of su, X v : R R as a fuctio o the Gaussia sace g N0, I satisfyig the Lischitz costat K A X := su X v l su g 1, X v su g, X v KX A g 1 g l. I fact, first fixig a u 1 = arg su g 1, X v, the su g 1, X v su g, X v g 1 g, X u 1 X u 1 l g 1 g l. The other side uses the same trick, fixig u = arg su g, X v su g 1, X v su g, X v g 1 g, X u X u l g 1 g l. 1
2 CAI, LIANG & RAKHLIN Thus we roved the Lischitz costat is uer bouded by KX A. Now we ca aly the cocetratio of Lischitz fuctio o Gaussia sace ad get A.1 P X Z A E X Z A + λ ex λ σ KX A. Thus we have with robability at least 1 ex δ /, X Z σ { ] } A. E g [su g, X v + δ su X v l. Proof of Lemma 3. The roof uses Gordo s method []. The lower boud side art of this lemma is a modified versio of the key lemma i [1]. First let s itroduce a imortat lemma i Gordo s aalysis. Lemma A.. Corollary 1. i [] Let Ω be a closed subset of S 1. Let Φ : R R be a radom ma with i.i.d. zero-mea Gaussia etries havig variace oe. The λ wω E [ mi z Ω Φz l ] [ ] E max Φz l λ + wω z Ω where λ = Γ +1 /Γ satisfies / + 1 < λ <. Use the same ste as i Lemma : for ay closed subset Ω S 1, the fuctios Φ mi z Ω Φz l ad Φ max z Ω Φz l both are Lischitz mas o Gaussia sace Φ with Lichitz costat 1: mi z Ω Φ 1z l mi z Ω Φ z l Φ 1 Φ F, Thus usig the Lichitz cocetratio i Gaussia sace, we have P mi X z l E[mi X z l ] t ex t / z Ω z Ω P max X z l E[max X z l ] + t ex t / z Ω z Ω max z Ω Φ 1z l max z Ω Φ z l Φ 1 Φ F. where X is a Gaussia esemble desig. Ad we have P mi X z l 1 c ex λ wω 1 c / z Ω P max X z l 1 + c ex 1 + c λ wω /. z Ω
3 GEOMETRIC INFERENCE LINEAR INVERSE PROBLEMS 3 Thus uder the coditio we have 4[wΩ + δ] c A c λ wω 1 + c wω [wω + δ] wω δ ad λ wω 1 c 1 c wω δ + δ c wω + δ + δ c + + δ δ. Thus λ wω 1 c δ > 0, I fact, we roved a stroger result P mi X z l 1 c 1 ex δ /, z Ω 1 c 1 + c λ wω δ > 0. P max X z l 1 + c 1 ex δ /. z Ω Now aly our lemma to local taget coe T A M, observe wb T AM = ws 1 T A M. Now the lemma holds by luggig i the taget coe. Proof of Lemma 4. The roof requires a observatio The A.4 A.5 A.6 wa = E g su g, v = E g g A γ A MwA = E g g A = E g [ E g su h T A M su h T A M h A = E g h l ] g A h A h l defiitio of dual orm. [ g A g, h su = wb h T A M h T AM. l su h T A M The last ste requires the Cauchy Schwartz relatioshi.3. h A h l ]
4 4 CAI, LIANG & RAKHLIN Proof of Theorem 6 with Sudakov Etroy Estimate. The key techical tool i rovig Theorem 6 is the followig well-kow Fao s iformatio Lemma A.3. This versio is from [4], similar versios are rovided i [6, 7, 5], ad the basic ideas are essetially the same. Lemma A.3 Fao s Lemma. Let Θ, d, be a seudo metric sace ad {P θ : θ Θ} be a collectio of robability measures. Let r be a iteger ad let S T Θ. Deote by MS, ɛ, d the ɛ ackig set as well as the ackig umber of T with resect to metric d, i.e. if dθ, θ,θ MS,ɛ,d θ ɛ. Suose β := su θ,θ MS,ɛ,d D KL P θ P θ > 0. The if ˆθ su E θ d ˆθ, θ θ T ɛ su S T,ɛ>0 4 1 β + log. log MS, ɛ, d We use Sudakov estimate for the lower boud. Recall the model Y = X M + Z, where Z N0, σ I. Without loss of geerality, we ca assume σ = 1. The Kullback-Leiber divergece betwee stadardized liear iverse models with differet arameters uder the Gaussia oise is D KL M M = X M X M l. Recall Sudakov Mioratio i Lemma 1, ad deote the critical radius ɛb T := arg max ɛ ɛ log N B T, ɛ Cosider the coe itersected with l ball with radius δ, Kδ := B δ T R, where δ will be secified later. As before, defie ψ = su v B T X v l su D KL M M X M l + X M l M,M Kδ δ ψ. The ackig umber is lower bouded by the coverig umber as the last equality holds because we ca scale both the set ad coverig ball by δ MKδ, ɛ N Kδ, ɛ = N K1, ɛ δ Alyig the Fao s lemma, we have if su E X M l ɛ su δ>0,0<ɛ<δ 4 1 δ ψ + log log N K1, ɛ δ.
5 GEOMETRIC INFERENCE LINEAR INVERSE PROBLEMS 5 Because K1 = B T, set δ = 1 log N B ψ T, ɛb T, ɛ = δ ɛb T The we have if su E X M l c 0 ψ with some uiversal costat c 0. Thus if ɛb T log N B T, ɛb T. su E X M l c 0σ ψ eb T. Proof of Theorem 6 with Volume Ratio. For the lower boud usig volume ratio. Recall the stadardized liear iverse model Y = X M + Z, where Z N0, σ I. Without loss of geerality, we ca assume σ = 1. The Kullback-Leiber divergece betwee stadardized liear iverse models with differet arameters uder the Gaussia oise is D KL M M = X M X M l. Cosider the itersectio of a coe T with l ball of radius δ, Kδ := B δ T R, where δ will be secified later. Defiig ψ = su v B T X v l, su D KL M M X M l + X M l M,M Kδ δ ψ. We have the ackig umber lower bouded by coverig umber as follows: MKδ, ɛ N Kδ, ɛ volkδ δ volb ɛ = volb A.7 T ɛ volb. Alyig Fao s iequality of Lemma A.3, we have if su E X M l ɛ su δ>0,0<ɛ<δ 4 1 δ ψ + log [ log δ ɛ volb T 1 volb ].
6 6 CAI, LIANG & RAKHLIN If for a > 0, 0 < b < 1 we choose the we have if su E X M l δ = 1 a volb ψ, ɛ = δ b volb ψ T volb T volb 1 su a>0,0<b<1 As show i [4, equatio 9], there is a uiversal costat c 0 > 0 such that ab 4 1 a + log log 1. b A.8 Thus if if su E X M l c 0 volb ψ T volb su E X M l c 0σ ψ vb T.. APPENDIX B: PROOF OF COROLLARIES Proof of Corollary 1. Defie Φ = X Σ 1, the Φ satisfies Gaussia esemble desig. Recall the Proof of Theorem 1, the oly lace we require the covariace of the desig matrix to be orthogoal is rovig local isometry costat LIC is bouded o the local taget coe. Via Lemma 3, max X z = max X z T A M z T A M Σ 1 1 Σ z = max Φz z Σ 1 T A M where Σ 1 T A M deotes the image of T A M uder liear trasform Σ 1. Here exact same calculatio holds for mi as max. Thus as log as wb Σ 1 TA M Theorem 1 still holds for. Similar to Theorem, let s first rove with the choice of η, rogram 3.6 is feasible o-emty for Ω with high robability. I articular, if we lug i Ω = Σ 1, ote X X Ω i Σ 1 ei A = su Φ Φe i e i, Σ 1 w.h. v wφσ 1 A wx A =
7 GEOMETRIC INFERENCE LINEAR INVERSE PROBLEMS 7 where the last iequality followigs from 6.3 i roof of Theorem. Hece followig decomositio holds for the de-biased estimator M M M = Σ 1 + σ Σ 1 ΩX W where W N0, I is the stadard Gaussia vector, ad = ΩX X Σ 1 M has l cotrol Note we have l max X X Ω i Σ 1 ei A M A i γam λη σ γ A Mw X A. Σ 1 v, M M = v, + σ v, ΩX W ad v, v l1 l ρσ γ A Mw X A ΩX W N0, ΩX X Ω. The rest of the roof follows exactly as i Theorem. 0, I the followig, we deote as the solutio to the rogram.1 ad the estimatio error to be H = M. We refer exlicit calculatios of Gaussia width for various local taget coe to Sectio 3.4 roositios i [1] for simlicity of our aer. Proof of Corollary. Let s calculate the rate for sarse vector recovery. We will treat the geometric terms γ A M, φ A M, X, λ A X, σ, searately. For γ A M: We kow that H lives i the taget coe T A M. Decomose H = H 0 + H c accordig to the suort of M, where H 0 l0 = s ad share the same suort as M. We have M l1 + H c l1 H 0 l1 = M +H c l1 H 0 l1 M +H 0 +H c l1 M l1 which meas H l1 H 0 l1, H 0 l H l. Thus we have the followig relatios H l1 H 0 l1 s H 0 l s H l
8 8 CAI, LIANG & RAKHLIN Therefore, H l 1 H l s ad thus γ A M s. As for φ A M, X : By the taget coe calculatio, we ca rove φ A M, X 1 c with high robability if 4[wB T AM + δ] c 1 c s log s The last boud is from Gaussia width uer boud for local taget coe for s sarse vector. Lastly, for λ A X, σ, : We kow the oerator X is orm reservig i the sese that su X v l 1 + c. ad wx A is the Gaussia width of discrete oits o Euclidea ball, which is at most log due to the log behavior of maximum of Gaussia variables. Thus we ca rove λ σ with some roer costat is eough with high robability. The corollary the follows from Theorem 1. Proof of Corollary 3. Let s calculate the rate for low rak matrix recovery. We will boud the geometric terms γ A M, φ A M, X, λ A X, σ, searately. For γ A M: Note H lives i the taget coe T A M. We ca write H = H 0 + H c accordig to the sa of M that is, M = UDV T, H 0 is saed by either U as the row sace or V as the colum sace, ad H c is saed by U as the row sace ad V as the colum sace with the followig roerties M + H c H 0 = M + H c H 0 M + H M. Thus we have rakh 0 r ad H H 0, H 0 F H F. Thus we have the followig relatios H H 0 r H 0 F r H F We the have H H F r ad thus γ A M r. As for φ A M, X : By the taget coe calculatio for rak-r matrix, we ca rove φ A M, X 1 c with high robability if 4[wB T AM + δ] c 1 c r + q r At last, for λ A X, σ, : Rak oe matrix maifold is a subsace with dimesio + q. Thus wx A ca be bouded by + q because the
9 GEOMETRIC INFERENCE LINEAR INVERSE PROBLEMS 9 Gaussia width of the + q-dimesioal subsace is + q ad the liear trasformatio caot elarge the dimesio. The rak oe matrices are of uit Frobeius orm ad the X is orm reservig i the sese that +q su X v l 1 + c. Puttig together, we ca rove λ σ with some roer costat is eough with high robability. The corollary follows by uttig together the geometric terms ad alyig the Theorem 1. Proof of Corollary 4. As usual, we will boud the geometric terms oe at a time. For γ A M, it is clear that H l / H l 1 ad so γ A M 1. As for φ A M, X, by the taget coe calculatio for sig vector, we ca rove φ A M, X 1 c with high robability if 4[wB T AM + δ] c 1 c. Fially, for λ, µ: wx A is the Gaussia width of l ball, which is of order ad wx B is the Gaussia width of dimesio Euclidea ball, which is. Thus we ca rove λ σ ad µ σ with some roer costat i frot of the order is eough with high robability. We kow from Theorem 1, Thus φ AM, X M l X M l µ M l, M l γ A M M l. M l, M l, X M l C σ. Proof of Corollary 5. We boud searately the three geometric terms γ A M, φ A M, X, ad λ A X, σ,. For γ A M, it is clear that H F / H 1 ad thus γ A M 1. As for φ A M, X, by the taget coe calculatio for orthogoal matrix, it is easy to show that φ A M, X 1 c with high robability if 4[wB T AM + δ] c 1 c mm 1.
10 10 CAI, LIANG & RAKHLIN At last, for λ, µ: Orthogoal matrix maifold is a subsace with dimesio mm 1, thus wx A is uer bouded by mm 1 m because the Gaussia width of the mm 1 -dimesioal maifold itersects Euclidea ball is mm 1 ad liear trasformatio caot elarge the dimesio. wx B ca be bouded by m m. Thus we ca show λ σ 3 ad µ σ m with some roer costat i frot of the order is eough with high robability. Recall Theorem 1 Hece φ AM, X M l X M l µ M l, M γ A M M l. M F, M, X m M l C σ. REFERENCES [1] Chadrasekara, V., Recht, B., Parrilo, P. A., ad Willsky, A. S. 01. The covex geometry of liear iverse roblems. Foudatios of Comutatioal Mathematics, 16: [] Gordo, Y O Milma s iequality ad radom subsaces which escae through a mesh i R. Sriger. [3] Ledoux, M. ad Talagrad, M Probability i Baach Saces: isoerimetry ad rocesses, volume 3. Sriger. [4] Ma, Z. ad Wu, Y Volume ratio, sarsity, ad miimaxity uder uitarily ivariat orms. arxiv rerit arxiv: [5] Tsybakov, A. B Itroductio to oarametric estimatio, volume 11. Sriger. [6] Yag, Y. ad Barro, A Iformatio-theoretic determiatio of miimax rates of covergece. The Aals of Statistics, 7: [7] Yu, B Assouad, fao, ad le cam. I Festschrift for Lucie Le Cam, ages Sriger. Deartmet of Statistics The Wharto School Uiversity of Pesylvaia Philadelhia, PA USA tcai@wharto.ue.edu tegyua@wharto.ue.edu rakhli@wharto.ue.edu
ECE534, Spring 2018: Solutions for Problem Set #2
ECE534, Srig 08: s for roblem Set #. Rademacher Radom Variables ad Symmetrizatio a) Let X be a Rademacher radom variable, i.e., X = ±) = /. Show that E e λx e λ /. E e λx = e λ + e λ = + k= k=0 λ k k k!
More information13.1 Shannon lower bound
ECE598: Iformatio-theoretic methods i high-dimesioal statistics Srig 016 Lecture 13: Shao lower boud, Fao s method Lecturer: Yihog Wu Scribe: Daewo Seo, Mar 8, 016 [Ed Mar 11] I the last class, we leared
More informationA Central Limit Theorem for Belief Functions
A Cetral Limit Theorem for Belief Fuctios Larry G. Estei Kyougwo Seo November 7, 2. CLT for Belief Fuctios The urose of this Note is to rove a form of CLT (Theorem.4) that is used i Estei ad Seo (2). More
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationarxiv: v1 [math.pr] 4 Dec 2013
Squared-Norm Empirical Process i Baach Space arxiv:32005v [mathpr] 4 Dec 203 Vicet Q Vu Departmet of Statistics The Ohio State Uiversity Columbus, OH vqv@statosuedu Abstract Jig Lei Departmet of Statistics
More informationA REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS. S. S. Dragomir 1. INTRODUCTION
TAIWANESE JOURNAL OF MATHEMATICS Vol. 14, No. 1,. 153-164, February 2010 This aer is available olie at htt://www.tjm.sysu.edu.tw/ A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS FOR f-divergence
More informationOn Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities
O Equivalece of Martigale Tail Bouds ad Determiistic Regret Iequalities Sasha Rakhli Departmet of Statistics, The Wharto School Uiversity of Pesylvaia Dec 16, 2015 Joit work with K. Sridhara arxiv:1510.03925
More informationThe random version of Dvoretzky s theorem in l n
The radom versio of Dvoretzky s theorem i l Gideo Schechtma Abstract We show that with high probability a sectio of the l ball of dimesio k cε log c > 0 a uiversal costat) is ε close to a multiple of the
More informationUnit 5. Hypersurfaces
Uit 5. Hyersurfaces ================================================================= -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
More informationLecture 13: Maximum Likelihood Estimation
ECE90 Sprig 007 Statistical Learig Theory Istructor: R. Nowak Lecture 3: Maximum Likelihood Estimatio Summary of Lecture I the last lecture we derived a risk (MSE) boud for regressio problems; i.e., select
More informationarxiv: v3 [math.st] 16 Jun 2015
Geometric Iferece for Geeral High-Dimesioal Liear Iverse Problems T. Toy Cai, Tegyua Liag ad Alexader Rakhli arxiv:1404.4408v3 [math.st] 16 Ju 2015 Departmet of Statistics The Wharto School Uiversity of
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More information5.1 A mutual information bound based on metric entropy
Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local
More informationSYMMETRIC POSITIVE SEMI-DEFINITE SOLUTIONS OF AX = B AND XC = D
Joural of Pure ad Alied Mathematics: Advaces ad Alicatios olume, Number, 009, Pages 99-07 SYMMERIC POSIIE SEMI-DEFINIE SOLUIONS OF AX B AND XC D School of Mathematics ad Physics Jiagsu Uiversity of Sciece
More informationON SOME NEW SEQUENCE SPACES OF NON-ABSOLUTE TYPE RELATED TO THE SPACES l p AND l I. M. Mursaleen and Abdullah K. Noman
Faculty of Scieces ad Mathematics, Uiversity of Niš, Serbia Available at: htt://www.mf.i.ac.rs/filomat Filomat 25:2 20, 33 5 DOI: 0.2298/FIL02033M ON SOME NEW SEQUENCE SPACES OF NON-ABSOLUTE TYPE RELATED
More informationDISCUSSION: LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION. By Zhao Ren and Harrison H. Zhou Yale University
Submitted to the Aals of Statistics DISCUSSION: LATENT VARIABLE GRAPHICAL MODEL SELECTION VIA CONVEX OPTIMIZATION By Zhao Re ad Harriso H. Zhou Yale Uiversity 1. Itroductio. We would like to cogratulate
More information6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition
6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.
More informationFinal Solutions. 1. (25pts) Define the following terms. Be as precise as you can.
Mathematics H104 A. Ogus Fall, 004 Fial Solutios 1. (5ts) Defie the followig terms. Be as recise as you ca. (a) (3ts) A ucoutable set. A ucoutable set is a set which ca ot be ut ito bijectio with a fiite
More information5.1 Review of Singular Value Decomposition (SVD)
MGMT 69000: Topics i High-dimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of
More informationEquations and Inequalities Involving v p (n!)
Equatios ad Iequalities Ivolvig v (!) Mehdi Hassai Deartmet of Mathematics Istitute for Advaced Studies i Basic Scieces Zaja, Ira mhassai@iasbs.ac.ir Abstract I this aer we study v (!), the greatest ower
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMaximum Likelihood Estimation and Complexity Regularization
ECE90 Sprig 004 Statistical Regularizatio ad Learig Theory Lecture: 4 Maximum Likelihood Estimatio ad Complexity Regularizatio Lecturer: Rob Nowak Scribe: Pam Limpiti Review : Maximum Likelihood Estimatio
More informationA Note on Sums of Independent Random Variables
Cotemorary Mathematics Volume 00 XXXX A Note o Sums of Ideedet Radom Variables Pawe l Hitczeko ad Stehe Motgomery-Smith Abstract I this ote a two sided boud o the tail robability of sums of ideedet ad
More informationNew Definition of Density on Knapsack Cryptosystems
Africacryt008@Casablaca 008.06.1 New Defiitio of Desity o Kasac Crytosystems Noboru Kuihiro The Uiversity of Toyo, Jaa 1/31 Kasac Scheme rough idea Public Key: asac: a={a 1, a,, a } Ecrytio: message m=m
More informationFunctional Analysis: Assignment Set # 10 Spring Professor: Fengbo Hang April 22, 2009
Eduardo Coroa Fuctioal Aalysis: Assigmet Set # 0 Srig 2009 Professor: Fegbo Hag Aril 22, 2009 Theorem Let S be a comact Hausdor sace. Suose fgg is a collectio of comlex-valued fuctios o S satisfyig (i)
More informationEstimation Theory Chapter 3
stimatio Theory Chater 3 Likelihood Fuctio Higher deedece of data PDF o ukow arameter results i higher estimatio accuracy amle : If ˆ If large, W, Choose  P  small,  W POOR GOOD i Oly data samle Data
More informationLecture 12: February 28
10-716: Advaced Machie Learig Sprig 2019 Lecture 12: February 28 Lecturer: Pradeep Ravikumar Scribes: Jacob Tyo, Rishub Jai, Ojash Neopae Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationA survey on penalized empirical risk minimization Sara A. van de Geer
A survey o pealized empirical risk miimizatio Sara A. va de Geer We address the questio how to choose the pealty i empirical risk miimizatio. Roughly speakig, this pealty should be a good boud for the
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationNon-Archimedian Fields. Topological Properties of Z p, Q p (p-adics Numbers)
BULETINUL Uiversităţii Petrol Gaze di Ploieşti Vol. LVIII No. 2/2006 43-48 Seria Matematică - Iformatică - Fizică No-Archimedia Fields. Toological Proerties of Z, Q (-adics Numbers) Mureşa Alexe Căli Uiversitatea
More informationAn operator equality involving a continuous field of operators and its norm inequalities
Available olie at www.sciecedirect.com Liear Algebra ad its Alicatios 49 (008) 59 67 www.elsevier.com/locate/laa A oerator equality ivolvig a cotiuous field of oerators ad its orm iequalities Mohammad
More informationSupplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate
Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationSupplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting
Supplemetary Materials for Statistical-Computatioal Phase Trasitios i Plated Models: The High-Dimesioal Settig Yudog Che The Uiversity of Califoria, Berkeley yudog.che@eecs.berkeley.edu Jiamig Xu Uiversity
More informationAdaptive estimation of linear functionals under different performance measures
Beroulli 11(2), 2005, 341 358 Adative estimatio of liear fuctioals uder differet erformace measures T. TO N Y C A I * ad MARK G. LOW** Deartmet of Statistics, The Wharto School, Uiversity of Pesylvaia,
More informationSelections of set-valued functions satisfying the general linear inclusion
J. Fixed Poit Theory Al. 18 (2016 133 145 DOI 10.1007/s11784-015-0265-9 Published olie October 17, 2015 Joural of Fixed Poit Theory 2015 The Author(s ad Alicatios This article is ublished with oe access
More informationLecture 15: Learning Theory: Concentration Inequalities
STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 5: Learig Theory: Cocetratio Iequalities Istructor: Ye-Chi Che 5. Itroductio Recall that i the lecture o classificatio, we have see that
More informationSignal Processing. Lecture 02: Discrete Time Signals and Systems. Ahmet Taha Koru, Ph. D. Yildiz Technical University.
Sigal Processig Lecture 02: Discrete Time Sigals ad Systems Ahmet Taha Koru, Ph. D. Yildiz Techical Uiversity 2017-2018 Fall ATK (YTU) Sigal Processig 2017-2018 Fall 1 / 51 Discrete Time Sigals Discrete
More informationA remark on p-summing norms of operators
A remark o p-summig orms of operators Artem Zvavitch Abstract. I this paper we improve a result of W. B. Johso ad G. Schechtma by provig that the p-summig orm of ay operator with -dimesioal domai ca be
More informationDimension of a Maximum Volume
Dimesio of a Maximum Volume Robert Kreczer Deartmet of Mathematics ad Comutig Uiversity of Wiscosi-Steves Poit Steves Poit, WI 54481 Phoe: (715) 346-3754 Email: rkrecze@uwsmail.uws.edu 1. INTRODUCTION.
More informationarxiv: v1 [math.pr] 13 Oct 2011
A tail iequality for quadratic forms of subgaussia radom vectors Daiel Hsu, Sham M. Kakade,, ad Tog Zhag 3 arxiv:0.84v math.pr] 3 Oct 0 Microsoft Research New Eglad Departmet of Statistics, Wharto School,
More informationJohn H. J. Einmahl Tilburg University, NL. Juan Juan Cai Tilburg University, NL
Estimatio of the margial exected shortfall Laures de Haa, Poitiers, 202 Estimatio of the margial exected shortfall Jua Jua Cai Tilburg iversity, NL Laures de Haa Erasmus iversity Rotterdam, NL iversity
More informationLECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)
LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More informationConcentration on the l n p ball
Cocetratio o the l p ball Gideo chechtma Joel Zi Abstract We prove a cocetratio iequality for fuctios, Lipschitz with respect to the Euclidea metric, o the ball of l p,1 p
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationHybridized Heredity In Support Vector Machine
Hybridized Heredity I Suort Vector Machie May 2015 Hybridized Heredity I Suort Vector Machie Timothy Idowu Yougmi Park Uiversity of Wiscosi-Madiso idowu@stat.wisc.edu yougmi@stat.wisc.edu May 2015 Abstract
More informationMATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1
MATH 2: HOMEWORK 6 SOLUTIONS CA PRO JIRADILOK Problem. If s = 2, ad Problem : Rudi, Chapter 3, Problem 3. s + = 2 + s ( =, 2, 3,... ), prove that {s } coverges, ad that s < 2 for =, 2, 3,.... Proof. The
More informationBoundaries and the James theorem
Boudaries ad the James theorem L. Vesely 1. Itroductio The followig theorem is importat ad well kow. All spaces cosidered here are real ormed or Baach spaces. Give a ormed space X, we deote by B X ad S
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationarxiv: v1 [math.dg] 27 Jul 2012
ESTIMATES FOR EIGENVALUES OF THE PANEITZ OPERATOR* arxiv:107.650v1 [math.dg] 7 Jul 01 QING-MING CHENG Abstract. For a -dimesioal compact submaifold i the Euclidea space R N, we study estimates for eigevalues
More informationEstimation of the Mean and the ACVF
Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators
More informationSummary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector
Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationABOUT CHAOS AND SENSITIVITY IN TOPOLOGICAL DYNAMICS
ABOUT CHAOS AND SENSITIVITY IN TOPOLOGICAL DYNAMICS EDUARD KONTOROVICH Abstract. I this work we uify ad geeralize some results about chaos ad sesitivity. Date: March 1, 005. 1 1. Symbolic Dyamics Defiitio
More informationSongklanakarin Journal of Science and Technology SJST R1 Teerapabolarn
Soglaaari Joural of Sciece ad Techology SJST--.R Teeraabolar A No-uiform Boud o Biomial Aroimatio to the Beta Biomial Cumulative Distributio Fuctio Joural: Soglaaari Joural of Sciece ad Techology For Review
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More informationWeak and Strong Convergence Theorems of New Iterations with Errors for Nonexpansive Nonself-Mappings
doi:.36/scieceasia53-874.6.3.67 ScieceAsia 3 (6: 67-7 Weak ad Strog Covergece Theorems of New Iteratios with Errors for Noexasive Noself-Maigs Sorsak Thiawa * ad Suthe Suatai ** Deartmet of Mathematics
More informationNotes for Lecture 11
U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with
More informationOn Cesáro means for Fox-Wright functions
Joural of Mathematics ad Statistics: 4(3: 56-6, 8 ISSN: 549-3644 8 Sciece Publicatios O Cesáro meas for Fox-Wright fuctios Maslia Darus ad Rabha W. Ibrahim School of Mathematical Scieces, Faculty of Sciece
More informationMinimal surface area position of a convex body is not always an M-position
Miimal surface area positio of a covex body is ot always a M-positio Christos Saroglou Abstract Milma proved that there exists a absolute costat C > 0 such that, for every covex body i R there exists a
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationThe value of Banach limits on a certain sequence of all rational numbers in the interval (0,1) Bao Qi Feng
The value of Baach limits o a certai sequece of all ratioal umbers i the iterval 0, Bao Qi Feg Departmet of Mathematical Scieces, Ket State Uiversity, Tuscarawas, 330 Uiversity Dr. NE, New Philadelphia,
More informationarxiv: v1 [math.st] 15 Jan 2014
Marcheko-Pastur Law for Tyler s ad Maroa s M-estimators arxiv:1401.3424v1 [math.st] 15 Ja 2014 Teg Zhag The Program i Alied ad Comutatioal Mathematics PACM, Priceto Uiversity Priceto, New Jersey 08544,
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationA unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
A uified framework for high-dimesioal aalysis of M-estimators with decomposable regularizers Sahad Negahba, UC Berkeley Pradeep Ravikumar, UT Austi Marti Waiwright, UC Berkeley Bi Yu, UC Berkeley NIPS
More informationThe Asymptotic Expansions of Certain Sums Involving Inverse of Binomial Coefficient 1
Iteratioal Mathematical Forum, 5, 2, o. 6, 76-768 The Asymtotic Easios of Certai Sums Ivolvig Iverse of Biomial Coefficiet Ji-Hua Yag Deartmet of Mathematics Zhoukou Normal Uiversity, Zhoukou 466, P.R.
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationSOME SEQUENCE SPACES DEFINED BY ORLICZ FUNCTIONS
ARCHIVU ATHEATICU BRNO Tomus 40 2004, 33 40 SOE SEQUENCE SPACES DEFINED BY ORLICZ FUNCTIONS E. SAVAŞ AND R. SAVAŞ Abstract. I this paper we itroduce a ew cocept of λ-strog covergece with respect to a Orlicz
More informationNonlinear Gronwall Bellman Type Inequalities and Their Applications
Article Noliear Growall Bellma Tye Iequalities ad Their Alicatios Weimi Wag 1, Yuqiag Feg 2, ad Yuayua Wag 1 1 School of Sciece, Wuha Uiversity of Sciece ad Techology, Wuha 4365, Chia; wugogre@163.com
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationConfidence Intervals
Cofidece Itervals Berli Che Deartmet of Comuter Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Referece: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chater 5 & Teachig Material Itroductio
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationPositive Schatten-Herz class Toeplitz operators on the ball
Positive Schatte-Herz class Toelitz oerators o the ball Boo Rim Choe, Hyugwoo Koo ad Youg Joo Lee Abstract. O the harmoic Bergma sace of the ball, we give characterizatios for a arbitrary ositive Toelitz
More informationOn equivalent strictly G-convex renormings of Banach spaces
Cet. Eur. J. Math. 8(5) 200 87-877 DOI: 0.2478/s533-00-0050-3 Cetral Europea Joural of Mathematics O equivalet strictly G-covex reormigs of Baach spaces Research Article Nataliia V. Boyko Departmet of
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationSupplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate
Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We
More informationLecture 24: Variable selection in linear models
Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet
More informationCov(aX, cy ) Var(X) Var(Y ) It is completely invariant to affine transformations: for any a, b, c, d R, ρ(ax + b, cy + d) = a.s. X i. as n.
CS 189 Itroductio to Machie Learig Sprig 218 Note 11 1 Caoical Correlatio Aalysis The Pearso Correlatio Coefficiet ρ(x, Y ) is a way to measure how liearly related (i other words, how well a liear model
More informationSupplemental Material: Proofs
Proof to Theorem Supplemetal Material: Proofs Proof. Let be the miimal umber of traiig items to esure a uique solutio θ. First cosider the case. It happes if ad oly if θ ad Rak(A) d, which is a special
More informationA Note on the Symmetric Powers of the Standard Representation of S n
A Note o the Symmetric Powers of the Stadard Represetatio of S David Savitt 1 Departmet of Mathematics, Harvard Uiversity Cambridge, MA 0138, USA dsavitt@mathharvardedu Richard P Staley Departmet of Mathematics,
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More information18.S096: Homework Problem Set 1 (revised)
8.S096: Homework Problem Set (revised) Topics i Mathematics of Data Sciece (Fall 05) Afoso S. Badeira Due o October 6, 05 Exteded to: October 8, 05 This homework problem set is due o October 6, at the
More information3.1. Introduction Assumptions.
Sectio 3. Proofs 3.1. Itroductio. A roof is a carefully reasoed argumet which establishes that a give statemet is true. Logic is a tool for the aalysis of roofs. Each statemet withi a roof is a assumtio,
More informationSolutions to Problem Set 7
8.78 Solutios to Problem Set 7. If the umber is i S, we re doe sice it s relatively rime to everythig. So suose S. Break u the remaiig elemets ito airs {, }, {4, 5},..., {, + }. By the Pigeohole Pricile,
More informationEstimation with Overidentifying Inequality Moment Conditions Technical Appendix
Estimatio with Overidetifyig Iequality Momet Coditios Techical Aedix Hyugsik Roger Moo Uiversity of Souther Califoria Frak Schorfheide Uiversity of Pesylvaia, CEPR ad NBER August 27, 2008 Not iteded to
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationThe Growth of Functions. Theoretical Supplement
The Growth of Fuctios Theoretical Supplemet The Triagle Iequality The triagle iequality is a algebraic tool that is ofte useful i maipulatig absolute values of fuctios. The triagle iequality says that
More information