OXPORD UNIVERSITY PRESS

Transcription

1 Concentration Inequalities A Nonasymptotic Theory of Independence STEPHANE BOUCHERON GABOR LUGOSI PASCAL MASS ART OXPORD UNIVERSITY PRESS

2 CONTENTS 1 Introduction Sums of Independent Random Variables and the Martingale Method The Concentration-of-Measure Phenomenon The Entropy Method The Transportation Method Reading Guide Acknowledgments 16 2 Basic Inequalities From Moments to Tails The Cramer-Chernoff Method Sub-Gaussian Random Variables Sub-Gamma Random Variables A Maximal Inequality Hoeffding's Inequality Bennett's Inequality Bernstein's Inequality Random Proj ections and the Johnson-Lindenstrauss Lemma Association Inequalities Minkowski's Inequality Bibliographical Remarks Exercises 46 3 Bounding the Variance The Efron-Stein Inequality Functions with Bounded Differences Self-Bounding Functions More Examples and Applications A Convex Poincare Inequality Exponential Tail Bounds via the Efron-Stein Inequality The Gaussian Poincare Inequality A Proof of the Efron-Stein Inequality Based on Duality Bibliographical Remarks Exercises 78 4 Basic Information Inequalities Shannon Entropy and Relative Entropy Entropy on Product Spaces and the Chain Rule Han's Inequality 86

3 VÜi I CONTENTS 4.4 Edge Isoperimetric Inequality on the Binary Hypercube Combinatorial Entropies Han's Inequality for Relative Entropies Sub-Additivity of the Entropy Entropy of General Random Variables Duality and Variational Formulas A Transportation Lemma Pinsker's Inequality Birg^'s Inequality Sub-Additivity of Entropy: The General Case The Brunn-Minkowski Inequality Bibliographical Remarks Exercises Logarithmic Sobolev Inequalities Symmetric Bernoulli Distributions Herbst's Argument: Concentration on the Hypercube A Gaussian Logarithmic Sobolev Inequality Gaussian Concentration: The Tsirelson-Ibragimov-Sudakov Inequality A Concentration Inequality for Suprema of Gaussian Processes Gaussian Random Projections A Performance Bound for the Lasso Hypercontractivity: The Bonami-Beckner Inequality Gaussian Hypercontractivity The Largest Eigenvalue of Random Matrices Bibliographical Remarks Exercises The Entropy Method The Bounded Differences Inequality More on Bounded Differences Modified Logarithmic Sobolev Inequalities Beyond Bounded Differences Inequalities for the Lower Tail Concentration of Convex Lipschitz Functions Exponential Inequalities for Self-Bounding Functions Symmetrized Modified Logarithmic Sobolev Inequalities Exponential Efron-Stein Inequalities A Modified Logarithmic Sobolev Inequality for the Poisson Distribution Weakly Self-Bounding Functions Proof of Lemma Some Variations Janson's Inequality Bibliographical Remarks Exercises 209

4 CONTENTS I ix 7 Concentration and Isoperimetry Levy's Inequalities The Classical Isoperimetric Theorem Vertex Isoperimetric Inequality in the Hypercube Convex Distance Inequality Convex Lipschitz Functions Revisited Bin Packing Bibliographical Remarks Exercises The Transportation Method The Bounded Differences Inequality Revisited Bounded Differences in Quadratic Mean Applications of Marton's Conditional Transportation Inequality The Convex Distance Inequality Revisited Talagrand's Gaussian Transportation Inequality Appendix: A General Induction Lemma Bibliographical Remarks Exercises Influences and Threshold Phenomena Influences Some Fundamental Inequalities for Influences Local Concentration Discrete Fourier Analysis and a Variance Inequality Monotone Sets Threshold Phenomena Bibliographical Remarks Exercises Isoperimetry on the Hypercube and Gaussian Spaces Bobkov's Inequality for Functions on the Hypercube An Isoperimetric Inequality on the Binary Hypercube Asymmetric Bernoulli Distributions and Threshold Phenomena The Gaussian Isoperimetric Theorem Lipschitz Functions of Gaussian Random Variables Bibliographical Remarks Exercises The Variance ofsuprema of Empirical Processes General Upper Bounds for the Variance Nemirovski's Inequality The Symmetrization and Contraction Principles Weak and Wimpy Variances Unbounded Summands Bibliographical Remarks Exercises 336

5 X CONTENTS 12 Suprema of Empirical Processes: Exponential Inequalities An Extension of Hoeffding's Inequality A Bernstein-Type Inequality for Bounded Processes A Symmetrization Argument Bousquet's Inequality for Suprema of Empirical Processes Non-Identically Distributed Summands and Left-Tail Inequalities Chi-Square Statistics and Quadratic Forms Bibliographical Remarks Exercises The Expected Value of Suprema of Empirical Processes Classical Chaining Lower Bounds for Gaussian Processes Chaining and VC-Classes Gaussian and Rademacher Averages of Symmetric Matrices Variations of Nemirovski's Inequality Random Projections of Sparse and Large Sets Normalized Processes: Slicing and Reweighting Relative Deviations for L 2 Distances Risk Bounds in Classification Bibliographical Remarks Exercises O-Entropies O-EntropyanditsSub-Additivity From <t>-entropies to <J>-Sobolev Inequalities < -Sobolev Inequalities for Bernoulli Random Variables Bibliographical Remarks Exercises Moment Inequalities Generalized Efron-Stein Inequalities Moments of Functions of Independent Random Variables Some Variants and Corollaries Sums of Random Variables Suprema of Empirical Processes Conditional Rademacher Averages Bibliographical Remarks Exercises 449 References 451 Author Index 473 Subject Index 477