Stein s Method for Matrix Concentration
|
|
- Gabriel Harper
- 5 years ago
- Views:
Transcription
1 Stein s Method for Matrix Concentration Lester Mackey Collaborators: Michael I. Jordan, Richard Y. Chen, Brendan Farrell, and Joel A. Tropp University of California, Berkeley California Institute of Technology BEARS 2012 Mackey (UC Berkeley) Stein s Method BEARS / 20
2 Motivation Concentration Inequalities Matrix concentration P{ X EX t} δ P{λ max (X EX) t} δ Non-asymptotic control of random matrices with complex distributions Applications Matrix estimation from sparse random measurements (Gross, 2011; Recht, 2009; Mackey, Talwalkar, and Jordan, 2011) Randomized matrix multiplication and factorization (Drineas, Mahoney, and Muthukrishnan, 2008; Hsu, Kakade, and Zhang, 2011b) Convex relaxation of robust or chance-constrained optimization (Nemirovski, 2007; So, 2011; Cheung, So, and Wang, 2011) Random graph analysis (Christofides and Markström, 2008; Oliveira, 2009) Mackey (UC Berkeley) Stein s Method BEARS / 20
3 Motivation Concentration Inequalities Matrix concentration P{λ max (X EX) t} δ Difficulty: Matrix multiplication is not commutative Past approaches (Oliveira, 2009; Tropp, 2011; Hsu, Kakade, and Zhang, 2011a) Deep results from matrix analysis Sums of independent matrices and matrix martingales This work Stein s method of exchangeable pairs (1972), as advanced by Chatterjee (2007) for scalar concentration Improved exponential tail inequalities (Hoeffding, Bernstein) Polynomial moment inequalities (Khintchine, Rosenthal) Dependent sums and more general matrix functionals Mackey (UC Berkeley) Stein s Method BEARS / 20
4 Roadmap Motivation 1 Motivation 2 Stein s Method Background and Notation 3 Exponential Tail Inequalities 4 Polynomial Moment Inequalities 5 Extensions Mackey (UC Berkeley) Stein s Method BEARS / 20
5 Notation Background Hermitian matrices: H d = {A C d d : A = A } All matrices in this talk are Hermitian. Maximum eigenvalue: λ max ( ) Trace: trb, the sum of the diagonal entries of B Spectral norm: B, the maximum singular value of B Schatten p-norm: B p := ( tr B p) 1/p for p 1 Mackey (UC Berkeley) Stein s Method BEARS / 20
6 Matrix Stein Pair Background Definition (Exchangeable Pair) (Z,Z ) is an exchangeable pair if (Z,Z ) d = (Z,Z). Definition (Matrix Stein Pair) Let (Z,Z ) be an auxiliary exchangeable pair, and let Ψ : Z H d be a measurable function. Define the random matrices X := Ψ(Z) and X := Ψ(Z ). (X,X ) is a matrix Stein pair with scale factor α (0,1] if E[X Z] = (1 α)x. Matrix Stein pairs are exchangeable pairs Matrix Stein pairs always have zero mean Mackey (UC Berkeley) Stein s Method BEARS / 20
7 Background The Conditional Variance Definition (Conditional Variance) Suppose that (X,X ) is a matrix Stein pair with scale factor α, constructed from the exchangeable pair (Z,Z ). The conditional variance is the random matrix X := X (Z) := 1 2α E[ (X X ) 2 Z ]. X is a stochastic estimate for the variance, EX 2 Control over X yields control over λ max (X) Mackey (UC Berkeley) Stein s Method BEARS / 20
8 Exponential Tail Inequalities Exponential Concentration for Random Matrices Theorem (Mackey, Jordan, Chen, Farrell, and Tropp, 2012) Let (X,X ) be a matrix Stein pair with X H d. Suppose that X cx +vi almost surely for c,v 0. Then, for all t 0, { } t 2 P{λ max (X) t} d exp. 2v +2ct Control over the conditional variance X yields Gaussian tail for λ max (X) for small t, Poisson tail for large t When d = 1, reduces to scalar result of Chatterjee (2007) The dimensional factor d cannot be removed Mackey (UC Berkeley) Stein s Method BEARS / 20
9 Exponential Tail Inequalities Application: Matrix Hoeffding Inequality Corollary (Mackey, Jordan, Chen, Farrell, and Tropp, 2012) Let (Y k ) k 1 be independent matrices in H d satisfying EY k = 0 and Yk 2 A 2 k for deterministic matrices (A k ) k 1. Define the variance parameter σ 2 := 1 ( ) A 2 2 k +EYk 2. k Then, for all t 0, ( P {λ max k Y k ) } t d e t2 /2σ 2. Improves upon the matrix Hoeffding inequality of Tropp (2011) Optimal constant 1/2 in the exponent Variance parameter σ 2 smaller than the bound k A2 k Tighter than classical Hoeffding inequality (1963) when d = 1 Mackey (UC Berkeley) Stein s Method BEARS / 20
10 Exponential Tail Inequalities Exponential Concentration: Proof Sketch 1. Matrix Laplace transform method (Ahlswede & Winter, 2002) Relate tail probability to the trace of the mgf of X where m(θ) := Etre θx How to bound the trace mgf? P{λ max (X) t} inf θ>0 e θt m(θ) Past approaches: Golden-Thompson, Lieb s concavity theorem Chatterjee s strategy for scalar concentration Control mgf growth by bounding derivative m (θ) = EtrXe θx for θ R. Rewrite using exchangeable pairs Mackey (UC Berkeley) Stein s Method BEARS / 20
11 Exponential Tail Inequalities Method of Exchangeable Pairs Lemma Suppose that (X,X ) is a matrix Stein pair with scale factor α. Let F : H d H d be a measurable function satisfying Then Intuition E (X X )F(X) <. E[X F(X)] = 1 2α E[(X X )(F(X) F(X ))]. (1) Can characterize the distribution of a random matrix by integrating it against a class of test functions F Eq. 1 allows us to estimate this integral using the smoothness properties of F and the discrepancy X X Mackey (UC Berkeley) Stein s Method BEARS / 20
12 Exponential Tail Inequalities Exponential Concentration: Proof Sketch 2. Method of Exchangeable Pairs Rewrite the derivative of the trace mgf m (θ) = EtrXe θx = 1 2α Etr[ (X X ) ( e θx e θx )]. Goal: Use the smoothness of F(X) = e θx to bound the derivative Mackey (UC Berkeley) Stein s Method BEARS / 20
13 Exponential Tail Inequalities Mean Value Trace Inequality Lemma (Mackey, Jordan, Chen, Farrell, and Tropp, 2012) Suppose that g : R R is a weakly increasing function and that h : R R is a function whose derivative h is convex. For all matrices A,B H d, it holds that tr[(g(a) g(b)) (h(a) h(b))] 1 2 tr[(g(a) g(b)) (A B) (h (A)+h (B))]. Standard matrix functions: If g : R R, then g(λ 1 ) 1 g(a) := Q... Q when A := Q λ... g(λd) λ d Q Inequality does not hold without the trace For exponential concentration we let g(a) = A and h(b) = e θb Mackey (UC Berkeley) Stein s Method BEARS / 20
14 Exponential Tail Inequalities Exponential Concentration: Proof Sketch 3. Mean Value Trace Inequality Bound the derivative of the trace mgf m (θ) = 1 2α Etr[ (X X ) ( e θx e θx )] θ 4α Etr[ (X X ) 2 (e θx +e θx )] = θ Etr [ X e θx]. 4. Conditional Variance Bound: X cx +vi Yields differential inequality m (θ) cθ m (θ)+vθ m(θ). Solve to bound m(θ) and thereby bound P{λ max (X) t} Mackey (UC Berkeley) Stein s Method BEARS / 20
15 Polynomial Moment Inequalities Polynomial Moments for Random Matrices Theorem (Mackey, Jordan, Chen, Farrell, and Tropp, 2012) Let p = 1 or p 1.5. Suppose that (X,X ) is a matrix Stein pair where E X 2p 2p <. Then ( E X 1/2p 2p) 2p 1 (E X p 1/2p p). Moral: The conditional variance controls the moments of X Generalizes Chatterjee s version (2007) of the scalar Burkholder-Davis-Gundy inequality (Burkholder, 1973) See also Pisier & Xu (1997); Junge & Xu (2003, 2008) Proof techniques mirror those for exponential concentration Also holds for infinite dimensional Schatten-class operators Mackey (UC Berkeley) Stein s Method BEARS / 20
16 Polynomial Moment Inequalities Application: Matrix Khintchine Inequality Corollary (Mackey, Jordan, Chen, Farrell, and Tropp, 2012) Let (ε k ) k 1 be an independent sequence of Rademacher random variables and (A k ) k 1 be a deterministic sequence of Hermitian matrices. Then if p = 1 or p 1.5, ( E k ) εka 2p 1/2p k ( 1/2 2p 1 k) k A2. 2p 2p Noncommutative Khintchine inequality (Lust-Piquard, 1986; Lust-Piquard and Pisier, 1991) is a dominant tool in applied matrix analysis e.g., Used in analysis of column sampling and projection for approximate SVD (Rudelson and Vershynin, 2007) Stein s method offers an unusually concise proof The constant 2p 1 is within e of optimal Mackey (UC Berkeley) Stein s Method BEARS / 20
17 Extensions Extensions Refined Exponential Concentration Relate trace mgf of conditional variance to trace mgf of X Yields matrix generalization of classical Bernstein inequality Offers tool for unbounded random matrices General Complex Matrices Map any matrix B C d 1 d 2 [ to a Hermitian ] matrix via dilation 0 B D(B) := B H d 1+d 2. 0 Preserves spectral information: λ max (D(B)) = B Dependent Sequences Sums of conditionally zero-mean random matrices Combinatorial matrix statistics (e.g., sampling w/o replacement) Matrix-valued functions satisfying a self-reproducing property Yields a dependent bounded differences inequality for matrices Mackey (UC Berkeley) Stein s Method BEARS / 20
18 The End Extensions Thanks! Mackey (UC Berkeley) Stein s Method BEARS / 20
19 References I Extensions Ahlswede, R. and Winter, A. Strong converse for identification via quantum channels. IEEE Trans. Inform. Theory, 48(3): , Mar Burkholder, D. L. Distribution function inequalities for martingales. Ann. Probab., 1:19 42, doi: /aop/ Chatterjee, S. Stein s method for concentration inequalities. Probab. Theory Related Fields, 138: , Cheung, S.-S., So, A. Man-Cho, and Wang, K. Chance-constrained linear matrix inequalities with dependent perturbations: A safe tractable approximation approach. Available at Christofides, D. and Markström, K. Expansion properties of random cayley graphs and vertex transitive graphs via matrix martingales. Random Struct. Algorithms, 32(1):88 100, Drineas, P., Mahoney, M. W., and Muthukrishnan, S. Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30: , Gross, D. Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory, 57(3): , Mar Hoeffding, W. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13 30, Hsu, D., Kakade, S. M., and Zhang, T. Dimension-free tail inequalities for sums of random matrices. Available at arxiv: , 2011a. Hsu, D., Kakade, S. M., and Zhang, T. Dimension-free tail inequalities for sums of random matrices. arxiv: v3[math.pr], 2011b. Junge, M. and Xu, Q. Noncommutative Burkholder/Rosenthal inequalities. Ann. Probab., 31(2): , Junge, M. and Xu, Q. Noncommutative Burkholder/Rosenthal inequalities II: Applications. Israel J. Math., 167: , Lust-Piquard, F. Inégalités de Khintchine dans C p (1 < p < ). C. R. Math. Acad. Sci. Paris, 303(7): , Lust-Piquard, F. and Pisier, G. Noncommutative Khintchine and Paley inequalities. Ark. Mat., 29(2): , Mackey, L., Talwalkar, A., and Jordan, M. I. Divide-and-conquer matrix factorization. In Advances in Neural Information Processing Systems Mackey (UC Berkeley) Stein s Method BEARS / 20
20 References II Extensions Mackey, L., Jordan, M. I., Chen, R. Y., Farrell, B., and Tropp, J. A. Matrix concentration inequalities via the method of exchangeable pairs. Available at arxiv, Jan Nemirovski, A. Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Math. Program., 109: , January ISSN doi: /s URL Oliveira, R. I. Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges. Available at arxiv: , Nov Pisier, G. and Xu, Q. Non-commutative martingale inequalities. Comm. Math. Phys., 189(3): , Recht, B. A simpler approach to matrix completion. arxiv: v2[cs.it], Rudelson, M. and Vershynin, R. Sampling from large matrices: An approach through geometric functional analysis. J. Assoc. Comput. Mach., 54(4):Article 21, 19 pp., Jul (electronic). So, A. Man-Cho. Moment inequalities for sums of random matrices and their applications in optimization. Math. Program., 130 (1): , Stein, C. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proc. 6th Berkeley Symp. Math. Statist. Probab., Berkeley, Univ. California Press. Tropp, J. A. User-friendly tail bounds for sums of random matrices. Found. Comput. Math., August Mackey (UC Berkeley) Stein s Method BEARS / 20
Matrix Completion and Matrix Concentration
Matrix Completion and Matrix Concentration Lester Mackey Collaborators: Ameet Talwalkar, Michael I. Jordan, Richard Y. Chen, Brendan Farrell, Joel A. Tropp, and Daniel Paulin Stanford University UCLA UC
More informationMATRIX CONCENTRATION INEQUALITIES VIA THE METHOD OF EXCHANGEABLE PAIRS 1
The Annals of Probability 014, Vol. 4, No. 3, 906 945 DOI: 10.114/13-AOP89 Institute of Mathematical Statistics, 014 MATRIX CONCENTRATION INEQUALITIES VIA THE METHOD OF EXCHANGEABLE PAIRS 1 BY LESTER MACKEY,MICHAEL
More informationUsing Friendly Tail Bounds for Sums of Random Matrices
Using Friendly Tail Bounds for Sums of Random Matrices Joel A. Tropp Computing + Mathematical Sciences California Institute of Technology jtropp@cms.caltech.edu Research supported in part by NSF, DARPA,
More informationMatrix Concentration. Nick Harvey University of British Columbia
Matrix Concentration Nick Harvey University of British Columbia The Problem Given any random nxn, symmetric matrices Y 1,,Y k. Show that i Y i is probably close to E[ i Y i ]. Why? A matrix generalization
More informationarxiv: v2 [math.pr] 27 Mar 2014
The Annals of Probability 2014, Vol. 42, No. 3, 906 945 DOI: 10.1214/13-AOP892 c Institute of Mathematical Statistics, 2014 arxiv:1201.6002v2 [math.pr] 27 Mar 2014 MATRIX CONCENTRATION INEQUALITIES VIA
More informationarxiv: v1 [math.pr] 11 Feb 2019
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm arxiv:190.03736v1 math.pr] 11 Feb 019 Chi Jin University of California, Berkeley chijin@cs.berkeley.edu Rong Ge Duke
More informationA Matrix Expander Chernoff Bound
A Matrix Expander Chernoff Bound Ankit Garg, Yin Tat Lee, Zhao Song, Nikhil Srivastava UC Berkeley Vanilla Chernoff Bound Thm [Hoeffding, Chernoff]. If X 1,, X k are independent mean zero random variables
More informationMatrix concentration inequalities
ELE 538B: Mathematics of High-Dimensional Data Matrix concentration inequalities Yuxin Chen Princeton University, Fall 2018 Recap: matrix Bernstein inequality Consider a sequence of independent random
More informationA NOTE ON SUMS OF INDEPENDENT RANDOM MATRICES AFTER AHLSWEDE-WINTER
A NOTE ON SUMS OF INDEPENDENT RANDOM MATRICES AFTER AHLSWEDE-WINTER 1. The method Ashwelde and Winter [1] proposed a new approach to deviation inequalities for sums of independent random matrices. The
More informationDivide-and-Conquer Matrix Factorization
Divide-and-Conquer Matrix Factorization Lester Mackey Collaborators: Ameet Talwalkar Michael I. Jordan Stanford University UCLA UC Berkeley December 14, 2015 Mackey (Stanford) Divide-and-Conquer Matrix
More informationSparsification by Effective Resistance Sampling
Spectral raph Theory Lecture 17 Sparsification by Effective Resistance Sampling Daniel A. Spielman November 2, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened
More informationDERIVING MATRIX CONCENTRATION INEQUALITIES FROM KERNEL COUPLINGS. Daniel Paulin Lester Mackey Joel A. Tropp
DERIVING MATRIX CONCENTRATION INEQUALITIES FROM KERNEL COUPLINGS By Daniel Paulin Lester Mackey Joel A. Tropp Technical Report No. 014-10 August 014 Department of Statistics STANFORD UNIVERSITY Stanford,
More informationOn Some Extensions of Bernstein s Inequality for Self-Adjoint Operators
On Some Extensions of Bernstein s Inequality for Self-Adjoint Operators Stanislav Minsker e-mail: sminsker@math.duke.edu Abstract: We present some extensions of Bernstein s inequality for random self-adjoint
More informationUser-Friendly Tail Bounds for Sums of Random Matrices
DOI 10.1007/s10208-011-9099-z User-Friendly Tail Bounds for Sums of Random Matrices Joel A. Tropp Received: 16 January 2011 / Accepted: 13 June 2011 The Authors 2011. This article is published with open
More informationOn the Spectra of General Random Graphs
On the Spectra of General Random Graphs Fan Chung Mary Radcliffe University of California, San Diego La Jolla, CA 92093 Abstract We consider random graphs such that each edge is determined by an independent
More informationStein Couplings for Concentration of Measure
Stein Couplings for Concentration of Measure Jay Bartroff, Subhankar Ghosh, Larry Goldstein and Ümit Işlak University of Southern California [arxiv:0906.3886] [arxiv:1304.5001] [arxiv:1402.6769] Borchard
More informationStein s Method: Distributional Approximation and Concentration of Measure
Stein s Method: Distributional Approximation and Concentration of Measure Larry Goldstein University of Southern California 36 th Midwest Probability Colloquium, 2014 Concentration of Measure Distributional
More informationConcentration of Measures by Bounded Couplings
Concentration of Measures by Bounded Couplings Subhankar Ghosh, Larry Goldstein and Ümit Işlak University of Southern California [arxiv:0906.3886] [arxiv:1304.5001] May 2013 Concentration of Measure Distributional
More informationEfron Stein Inequalities for Random Matrices
Efron Stein Inequalities for Random Matrices Daniel Paulin, Lester Mackey and Joel A. Tropp D. Paulin Department of Statistics and Applied Probability National University of Singapore 6 Science Drive,
More informationLecture 3. Random Fourier measurements
Lecture 3. Random Fourier measurements 1 Sampling from Fourier matrices 2 Law of Large Numbers and its operator-valued versions 3 Frames. Rudelson s Selection Theorem Sampling from Fourier matrices Our
More informationUser-Friendly Tools for Random Matrices
User-Friendly Tools for Random Matrices Joel A. Tropp Computing + Mathematical Sciences California Institute of Technology jtropp@cms.caltech.edu Research supported by ONR, AFOSR, NSF, DARPA, Sloan, and
More informationAn iterative hard thresholding estimator for low rank matrix recovery
An iterative hard thresholding estimator for low rank matrix recovery Alexandra Carpentier - based on a joint work with Arlene K.Y. Kim Statistical Laboratory, Department of Pure Mathematics and Mathematical
More informationUser-Friendly Tools for Random Matrices
User-Friendly Tools for Random Matrices Joel A. Tropp Computing + Mathematical Sciences California Institute of Technology jtropp@cms.caltech.edu Research supported by ONR, AFOSR, NSF, DARPA, Sloan, and
More informationTechnical Report No January 2011
USER-FRIENDLY TAIL BOUNDS FOR MATRIX MARTINGALES JOEL A. TRO! Technical Report No. 2011-01 January 2011 A L I E D & C O M U T A T I O N A L M A T H E M A T I C S C A L I F O R N I A I N S T I T U T E O
More informationConcentration Inequalities for Random Matrices
Concentration Inequalities for Random Matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify the asymptotic
More informationUpper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1
Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1 Feng Wei 2 University of Michigan July 29, 2016 1 This presentation is based a project under the supervision of M. Rudelson.
More informationRandom regular digraphs: singularity and spectrum
Random regular digraphs: singularity and spectrum Nick Cook, UCLA Probability Seminar, Stanford University November 2, 2015 Universality Circular law Singularity probability Talk outline 1 Universality
More informationarxiv: v7 [math.pr] 15 Jun 2011
USER-FRIENDLY TAIL BOUNDS FOR SUMS OF RANDOM MATRICES JOEL A. TRO arxiv:1004.4389v7 [math.r] 15 Jun 2011 Abstract. This paper presents new probability inequalities for sums of independent, random, selfadjoint
More informationMultivariate trace inequalities. David Sutter, Mario Berta, Marco Tomamichel
Multivariate trace inequalities David Sutter, Mario Berta, Marco Tomamichel What are trace inequalities and why we should care. Main difference between classical and quantum world are complementarity and
More informationEFRON STEIN INEQUALITIES FOR RANDOM MATRICES
The Annals of Probability 016, Vol. 44, No. 5, 3431 3473 DOI: 10.114/15-AOP1054 Institute of Mathematical Statistics, 016 EFRON STEIN INEQUALITIES FOR RANDOM MATRICES BY DANIEL PAULIN,1,LESTER MACKEY AND
More informationarxiv: v1 [math.pr] 22 May 2008
THE LEAST SINGULAR VALUE OF A RANDOM SQUARE MATRIX IS O(n 1/2 ) arxiv:0805.3407v1 [math.pr] 22 May 2008 MARK RUDELSON AND ROMAN VERSHYNIN Abstract. Let A be a matrix whose entries are real i.i.d. centered
More informationOXPORD UNIVERSITY PRESS
Concentration Inequalities A Nonasymptotic Theory of Independence STEPHANE BOUCHERON GABOR LUGOSI PASCAL MASS ART OXPORD UNIVERSITY PRESS CONTENTS 1 Introduction 1 1.1 Sums of Independent Random Variables
More informationHandout 6: Some Applications of Conic Linear Programming
ENGG 550: Foundations of Optimization 08 9 First Term Handout 6: Some Applications of Conic Linear Programming Instructor: Anthony Man Cho So November, 08 Introduction Conic linear programming CLP, and
More informationarxiv: v6 [math.pr] 16 Jan 2011
USER-FRIENDLY TAIL BOUNDS FOR SUMS OF RANDOM MATRICES JOEL A. TRO arxiv:1004.4389v6 [math.r] 16 Jan 2011 Abstract. This paper presents new probability inequalities for sums of independent, random, selfadjoint
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationOPTIMAL PERTURBATION OF UNCERTAIN SYSTEMS
Stochastics and Dynamics, Vol. 2, No. 3 (22 395 42 c World Scientific Publishing Company OPTIMAL PERTURBATION OF UNCERTAIN SYSTEMS Stoch. Dyn. 22.2:395-42. Downloaded from www.worldscientific.com by HARVARD
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationSpectral k-support Norm Regularization
Spectral k-support Norm Regularization Andrew McDonald Department of Computer Science, UCL (Joint work with Massimiliano Pontil and Dimitris Stamos) 25 March, 2015 1 / 19 Problem: Matrix Completion Goal:
More informationA note on the growth rate in the Fazekas Klesov general law of large numbers and on the weak law of large numbers for tail series
Publ. Math. Debrecen 73/1-2 2008), 1 10 A note on the growth rate in the Fazekas Klesov general law of large numbers and on the weak law of large numbers for tail series By SOO HAK SUNG Taejon), TIEN-CHUNG
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationConcentration of Measures by Bounded Size Bias Couplings
Concentration of Measures by Bounded Size Bias Couplings Subhankar Ghosh, Larry Goldstein University of Southern California [arxiv:0906.3886] January 10 th, 2013 Concentration of Measure Distributional
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationConcentration inequalities for non-lipschitz functions
Concentration inequalities for non-lipschitz functions University of Warsaw Berkeley, October 1, 2013 joint work with Radosław Adamczak (University of Warsaw) Gaussian concentration (Sudakov-Tsirelson,
More informationLecture 9: Matrix approximation continued
0368-348-01-Algorithms in Data Mining Fall 013 Lecturer: Edo Liberty Lecture 9: Matrix approximation continued Warning: This note may contain typos and other inaccuracies which are usually discussed during
More informationIntrinsic Volumes of Convex Cones Theory and Applications
Intrinsic Volumes of Convex Cones Theory and Applications M\cr NA Manchester Numerical Analysis Martin Lotz School of Mathematics The University of Manchester with the collaboration of Dennis Amelunxen,
More informationSparse and Low Rank Recovery via Null Space Properties
Sparse and Low Rank Recovery via Null Space Properties Holger Rauhut Lehrstuhl C für Mathematik (Analysis), RWTH Aachen Convexity, probability and discrete structures, a geometric viewpoint Marne-la-Vallée,
More informationA Fast Algorithm For Computing The A-optimal Sampling Distributions In A Big Data Linear Regression
A Fast Algorithm For Computing The A-optimal Sampling Distributions In A Big Data Linear Regression Hanxiang Peng and Fei Tan Indiana University Purdue University Indianapolis Department of Mathematical
More informationCOR-OPT Seminar Reading List Sp 18
COR-OPT Seminar Reading List Sp 18 Damek Davis January 28, 2018 References [1] S. Tu, R. Boczar, M. Simchowitz, M. Soltanolkotabi, and B. Recht. Low-rank Solutions of Linear Matrix Equations via Procrustes
More informationOn a class of stochastic differential equations in a financial network model
1 On a class of stochastic differential equations in a financial network model Tomoyuki Ichiba Department of Statistics & Applied Probability, Center for Financial Mathematics and Actuarial Research, University
More informationConvergence Rates of Kernel Quadrature Rules
Convergence Rates of Kernel Quadrature Rules Francis Bach INRIA - Ecole Normale Supérieure, Paris, France ÉCOLE NORMALE SUPÉRIEURE NIPS workshop on probabilistic integration - Dec. 2015 Outline Introduction
More informationBounds for all eigenvalues of sums of Hermitian random matrices
20 Chapter 2 Bounds for all eigenvalues of sums of Hermitian random matrices 2.1 Introduction The classical tools of nonasymptotic random matrix theory can sometimes give quite sharp estimates of the extreme
More informationHandout 8: Dealing with Data Uncertainty
MFE 5100: Optimization 2015 16 First Term Handout 8: Dealing with Data Uncertainty Instructor: Anthony Man Cho So December 1, 2015 1 Introduction Conic linear programming CLP, and in particular, semidefinite
More informationFundamentals of Matrices
Maschinelles Lernen II Fundamentals of Matrices Christoph Sawade/Niels Landwehr/Blaine Nelson Tobias Scheffer Matrix Examples Recap: Data Linear Model: f i x = w i T x Let X = x x n be the data matrix
More informationAdaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas Department of Mathematics Department of Statistical Science Cornell University London, January 7, 2016 Joint work
More informationOn the exponential convergence of. the Kaczmarz algorithm
On the exponential convergence of the Kaczmarz algorithm Liang Dai and Thomas B. Schön Department of Information Technology, Uppsala University, arxiv:4.407v [cs.sy] 0 Mar 05 75 05 Uppsala, Sweden. E-mail:
More informationPreface to the Second Edition...vii Preface to the First Edition... ix
Contents Preface to the Second Edition...vii Preface to the First Edition........... ix 1 Introduction.............................................. 1 1.1 Large Dimensional Data Analysis.........................
More informationOperator-valued extensions of matrix-norm inequalities
Operator-valued extensions of matrix-norm inequalities G.J.O. Jameson 1. INTRODUCTION. Let A = (a j,k ) be a matrix (finite or infinite) of complex numbers. Let A denote the usual operator norm of A as
More informationDifferent quantum f-divergences and the reversibility of quantum operations
Different quantum f-divergences and the reversibility of quantum operations Fumio Hiai Tohoku University 2016, September (at Budapest) Joint work with Milán Mosonyi 1 1 F.H. and M. Mosonyi, Different quantum
More informationInvertibility of random matrices
University of Michigan February 2011, Princeton University Origins of Random Matrix Theory Statistics (Wishart matrices) PCA of a multivariate Gaussian distribution. [Gaël Varoquaux s blog gael-varoquaux.info]
More informationNotes on matrix arithmetic geometric mean inequalities
Linear Algebra and its Applications 308 (000) 03 11 www.elsevier.com/locate/laa Notes on matrix arithmetic geometric mean inequalities Rajendra Bhatia a,, Fuad Kittaneh b a Indian Statistical Institute,
More informationThe Moment Method; Convex Duality; and Large/Medium/Small Deviations
Stat 928: Statistical Learning Theory Lecture: 5 The Moment Method; Convex Duality; and Large/Medium/Small Deviations Instructor: Sham Kakade The Exponential Inequality and Convex Duality The exponential
More informationPreliminary Examination, Numerical Analysis, August 2016
Preliminary Examination, Numerical Analysis, August 2016 Instructions: This exam is closed books and notes. The time allowed is three hours and you need to work on any three out of questions 1-4 and any
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationItô isomorphisms for L p -valued Poisson stochastic integrals
L -valued Rosenthal inequalities Itô isomorhisms in L -saces One-sided estimates in martingale tye/cotye saces Itô isomorhisms for L -valued Poisson stochastic integrals Sjoerd Dirksen - artly joint work
More information8. Diagonalization.
8. Diagonalization 8.1. Matrix Representations of Linear Transformations Matrix of A Linear Operator with Respect to A Basis We know that every linear transformation T: R n R m has an associated standard
More informationA Generalized Uncertainty Principle and Sparse Representation in Pairs of Bases
2558 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 9, SEPTEMBER 2002 A Generalized Uncertainty Principle Sparse Representation in Pairs of Bases Michael Elad Alfred M Bruckstein Abstract An elementary
More informationStein s Method: Distributional Approximation and Concentration of Measure
Stein s Method: Distributional Approximation and Concentration of Measure Larry Goldstein University of Southern California 36 th Midwest Probability Colloquium, 2014 Stein s method for Distributional
More informationAsymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½
University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 1998 Asymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½ Lawrence D. Brown University
More informationarxiv: v1 [math.na] 1 Sep 2018
On the perturbation of an L -orthogonal projection Xuefeng Xu arxiv:18090000v1 [mathna] 1 Sep 018 September 5 018 Abstract The L -orthogonal projection is an important mathematical tool in scientific computing
More informationSpectral Partitiong in a Stochastic Block Model
Spectral Graph Theory Lecture 21 Spectral Partitiong in a Stochastic Block Model Daniel A. Spielman November 16, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened
More informationDissertation Defense
Clustering Algorithms for Random and Pseudo-random Structures Dissertation Defense Pradipta Mitra 1 1 Department of Computer Science Yale University April 23, 2008 Mitra (Yale University) Dissertation
More informationThe circular law. Lewis Memorial Lecture / DIMACS minicourse March 19, Terence Tao (UCLA)
The circular law Lewis Memorial Lecture / DIMACS minicourse March 19, 2008 Terence Tao (UCLA) 1 Eigenvalue distributions Let M = (a ij ) 1 i n;1 j n be a square matrix. Then one has n (generalised) eigenvalues
More informationOPTIMAL PERTURBATION OF UNCERTAIN SYSTEMS
Stochastics and Dynamics c World Scientific Publishing Company OPTIMAL PERTURBATION OF UNCERTAIN SYSTEMS BRIAN F. FARRELL Division of Engineering and Applied Sciences, Harvard University Pierce Hall, 29
More informationLecture 24: Element-wise Sampling of Graphs and Linear Equation Solving. 22 Element-wise Sampling of Graphs and Linear Equation Solving
Stat260/CS294: Randomized Algorithms for Matrices and Data Lecture 24-12/02/2013 Lecture 24: Element-wise Sampling of Graphs and Linear Equation Solving Lecturer: Michael Mahoney Scribe: Michael Mahoney
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationTHE PAVING PROPERTY FOR UNIFORMLY BOUNDED MATRICES
THE PAVING PROPERTY FOR UNIFORMLY BOUNDED MATRICES JOEL A TROPP Abstract This note presents a new proof of an important result due to Bourgain and Tzafriri that provides a partial solution to the Kadison
More informationAnalysis of Robust PCA via Local Incoherence
Analysis of Robust PCA via Local Incoherence Huishuai Zhang Department of EECS Syracuse University Syracuse, NY 3244 hzhan23@syr.edu Yi Zhou Department of EECS Syracuse University Syracuse, NY 3244 yzhou35@syr.edu
More informationCompressed Sensing and Robust Recovery of Low Rank Matrices
Compressed Sensing and Robust Recovery of Low Rank Matrices M. Fazel, E. Candès, B. Recht, P. Parrilo Electrical Engineering, University of Washington Applied and Computational Mathematics Dept., Caltech
More informationError Bound for Compound Wishart Matrices
Electronic Journal of Statistics Vol. 0 (2006) 1 8 ISSN: 1935-7524 Error Bound for Compound Wishart Matrices Ilya Soloveychik, The Rachel and Selim Benin School of Computer Science and Engineering, The
More informationThe Hilbert Space of Random Variables
The Hilbert Space of Random Variables Electrical Engineering 126 (UC Berkeley) Spring 2018 1 Outline Fix a probability space and consider the set H := {X : X is a real-valued random variable with E[X 2
More informationSubset Selection. Deterministic vs. Randomized. Ilse Ipsen. North Carolina State University. Joint work with: Stan Eisenstat, Yale
Subset Selection Deterministic vs. Randomized Ilse Ipsen North Carolina State University Joint work with: Stan Eisenstat, Yale Mary Beth Broadbent, Martin Brown, Kevin Penner Subset Selection Given: real
More informationSpectral inequalities and equalities involving products of matrices
Spectral inequalities and equalities involving products of matrices Chi-Kwong Li 1 Department of Mathematics, College of William & Mary, Williamsburg, Virginia 23187 (ckli@math.wm.edu) Yiu-Tung Poon Department
More informationFast Angular Synchronization for Phase Retrieval via Incomplete Information
Fast Angular Synchronization for Phase Retrieval via Incomplete Information Aditya Viswanathan a and Mark Iwen b a Department of Mathematics, Michigan State University; b Department of Mathematics & Department
More informationNon commutative Khintchine inequalities and Grothendieck s theo
Non commutative Khintchine inequalities and Grothendieck s theorem Nankai, 2007 Plan Non-commutative Khintchine inequalities 1 Non-commutative Khintchine inequalities 2 µ = Uniform probability on the set
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini April 27, 2018 1 / 80 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationExtreme eigenvalues of Erdős-Rényi random graphs
Extreme eigenvalues of Erdős-Rényi random graphs Florent Benaych-Georges j.w.w. Charles Bordenave and Antti Knowles MAP5, Université Paris Descartes May 18, 2018 IPAM UCLA Inhomogeneous Erdős-Rényi random
More information18.S096: Concentration Inequalities, Scalar and Matrix Versions
18S096: Concentration Inequalities, Scalar and Matrix Versions Topics in Mathematics of Data Science Fall 015 Afonso S Bandeira bandeira@mitedu http://mathmitedu/~bandeira October 15, 015 These are lecture
More informationOverparametrization for Landscape Design in Non-convex Optimization
Overparametrization for Landscape Design in Non-convex Optimization Jason D. Lee University of Southern California September 19, 2018 The State of Non-Convex Optimization Practical observation: Empirically,
More informationLecture 1 and 2: Random Spanning Trees
Recent Advances in Approximation Algorithms Spring 2015 Lecture 1 and 2: Random Spanning Trees Lecturer: Shayan Oveis Gharan March 31st Disclaimer: These notes have not been subjected to the usual scrutiny
More informationWEAK TYPE ESTIMATES ASSOCIATED TO BURKHOLDER S MARTINGALE INEQUALITY
WEAK TYPE ESTIMATES ASSOCIATED TO BURKHOLDER S MARTINGALE INEQUALITY JAVIER PARCET Abstract Given a probability space, A, µ), let A 1, A 2, be a filtration of σ-subalgebras of A and let E 1, E 2, denote
More informationAssessing the dependence of high-dimensional time series via sample autocovariances and correlations
Assessing the dependence of high-dimensional time series via sample autocovariances and correlations Johannes Heiny University of Aarhus Joint work with Thomas Mikosch (Copenhagen), Richard Davis (Columbia),
More informationRandom Matrices: Invertibility, Structure, and Applications
Random Matrices: Invertibility, Structure, and Applications Roman Vershynin University of Michigan Colloquium, October 11, 2011 Roman Vershynin (University of Michigan) Random Matrices Colloquium 1 / 37
More informationELE 538B: Mathematics of High-Dimensional Data. Spectral methods. Yuxin Chen Princeton University, Fall 2018
ELE 538B: Mathematics of High-Dimensional Data Spectral methods Yuxin Chen Princeton University, Fall 2018 Outline A motivating application: graph clustering Distance and angles between two subspaces Eigen-space
More informationMultivariate Trace Inequalities
Multivariate Trace Inequalities Mario Berta arxiv:1604.03023 with Sutter and Tomamichel (to appear in CMP) arxiv:1512.02615 with Fawzi and Tomamichel QMath13 - October 8, 2016 Mario Berta (Caltech) Multivariate
More informationRandNLA: Randomized Numerical Linear Algebra
RandNLA: Randomized Numerical Linear Algebra Petros Drineas Rensselaer Polytechnic Institute Computer Science Department To access my web page: drineas RandNLA: sketch a matrix by row/ column sampling
More informationMahler Conjecture and Reverse Santalo
Mini-Courses Radoslaw Adamczak Random matrices with independent log-concave rows/columns. I will present some recent results concerning geometric properties of random matrices with independent rows/columns
More informationConvexity of the Joint Numerical Range
Convexity of the Joint Numerical Range Chi-Kwong Li and Yiu-Tung Poon October 26, 2004 Dedicated to Professor Yik-Hoi Au-Yeung on the occasion of his retirement. Abstract Let A = (A 1,..., A m ) be an
More informationStructured Random Matrices
Structured Random Matrices Ramon van Handel Abstract Random matrix theory is a well-developed area of probability theory that has numerous connections with other areas of mathematics and its applications.
More informationExtremal properties of the variance and the quantum Fisher information; Phys. Rev. A 87, (2013).
1 / 24 Extremal properties of the variance and the quantum Fisher information; Phys. Rev. A 87, 032324 (2013). G. Tóth 1,2,3 and D. Petz 4,5 1 Theoretical Physics, University of the Basque Country UPV/EHU,
More informationThe Knaster problem and the geometry of high-dimensional cubes
The Knaster problem and the geometry of high-dimensional cubes B. S. Kashin (Moscow) S. J. Szarek (Paris & Cleveland) Abstract We study questions of the following type: Given positive semi-definite matrix
More information