SpringerBriefs in Mathematics Series Editors Nicola Bellomo Michele Benzi Palle E.T. Jorgensen Tatsien Li Roderick Melnik Otmar Scherzer Benjamin Steinberg Lothar Reichel Yuri Tschinkel G. George Yin Ping Zhang SpringerBriefs in Mathematics showcases expositions in all areas of mathematics and applied mathematics. Manuscripts presenting new results or a single new result in a classical field, new field, or an emerging topic, applications, or bridges between new results and already published works, are encouraged. The series is intended for mathematicians and applied mathematicians. More information about this series at http://www.springer.com/series/10030
Bernard Bercu Bernard Delyon Emmanuel Rio Concentration Inequalities for Sums and Martingales 123
Bernard Bercu Institut de Mathématiques de Bordeaux Université de Bordeaux Talence, France Bernard Delyon Institut de Recherche Mathématique de Rennes Université de Rennes Rennes, France Emmanuel Rio Laboratoire de Mathématiques de Versailles Université de Versailles St. Quentin en Yvelines Versailles, France ISSN 2191-8198 ISSN 2191-8201 (electronic) SpringerBriefs in Mathematics ISBN 978-3-319-22098-7 ISBN 978-3-319-22099-4 (ebook) DOI 10.1007/978-3-319-22099-4 Library of Congress Control Number: 2015945946 Mathematics Subject Classification (2010): 60-01, 60E15, 60F10, 60G42, 60G50 Springer Cham Heidelberg New York Dordrecht London The Authors 2015 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. Printed on acid-free paper Springer International Publishing AG Switzerland is part of Springer Science+Business Media (www. springer.com)
In memory of our friend Abderrahmen Touati
Preface Over the last two decades, there has been a renewed interest in the area of concentration inequalities. The starting point of this short book was a project on exponential inequalities for martingales with a view toward applications in probability and statistics. During the preparation of this book, we realized that the classical exponential inequalities for sums of independent random variables were not well reported in the literature. This motivated us to write a chapter entirely devoted to sums of independent random variables, which includes the classical deviation inequalities of Bernstein, Bennett, and Hoeffding as well as less-recognized inequalities and new results. Some of these inequalities are extended to martingales in the third chapter, which deals with concentration inequalities for martingales and self-normalized martingales. We end this book with a brief chapter devoted to a few applications in probability and statistics, which shows the striking efficiency of martingales techniques on some examples. We wish to emphasize that this short book does not provide a complete overview of martingale exponential inequalities and their applications. More sophisticated results can be found in the literature. We hope that researchers interested in concentration inequalities for sums and martingales will find in this book useful tools for their future research. Talence, France Rennes, France Versailles, France June 2015 Bernard Bercu Bernard Delyon Emmanuel Rio vii
Contents 1 Classical results... 1 1.1 Sums of independent random variables... 1 1.1.1 Stronglawoflargenumbersforsums... 1 1.1.2 Central limit theorem for sums.... 2 1.1.3 Largedeviations... 2 1.2 Martingales... 6 1.2.1 Stronglawoflargenumbersformartingales... 8 1.2.2 Central limit theorem for martingales......... 9 References... 10 2 Concentration inequalities for sums... 11 2.1 Bernstein s inequalities...... 11 2.1.1 One-sided inequalities.... 11 2.1.2 Two-sided inequalities.... 17 2.1.3 About the second term in Bernstein s inequality........ 18 2.2 Hoeffding s inequality...... 21 2.3 Binomialratefunctions... 26 2.4 Bennett s inequality........ 31 2.5 SubGaussian inequalities.... 36 2.5.1 Random variables bounded from above........ 37 2.5.2 Nonnegative random variables.... 40 2.5.3 Symmetric conditions for bounded random variables... 42 2.5.4 Asymmetric conditions for bounded random variables..... 43 2.6 Always a little further on weighted sums...... 45 2.7 Sums of Gamma random variables....... 48 2.8 McDiarmid s inequality..... 52 2.9 ComplementsandExercises... 57 References... 59 ix
x Contents 3 Concentration inequalities for martingales... 61 3.1 Azuma-Hoeffding inequalities.... 61 3.1.1 Martingales with differences bounded from above...... 62 3.1.2 Symmetric conditions for bounded difference martingales.. 65 3.1.3 Asymmetric conditions for bounded difference martingales. 66 3.2 Freedman and Fan-Grama-Liu inequalities.... 68 3.3 Bernstein s inequality....... 71 3.4 De la Pena s inequalities..... 74 3.4.1 Conditionallysymmetricmartingales... 74 3.4.2 Themissingfactors... 75 3.5 Gaussianmartingales... 78 3.6 Always a little further on martingales..... 80 3.7 Martingalesheavyonleftorright... 85 3.8 ComplementsandExercises... 90 References... 97 4 Applications in probability and statistics... 99 4.1 Autoregressive process...... 99 4.2 Random permutations.......102 4.3 Empirical periodogram...... 108 4.4 Random matrices...111 4.4.1 Independent entries with mixture matrix....... 113 4.4.2 Independent columns with dependent coordinates......114 4.4.3 Proofs...115 References...119