Canadian Mathematical Society Société mathématique du Canada

Transcription

1 Canadian Mathematical Society Société mathématique du Canada Editors-in-Chief Rédacteurs-en-chef K. Dilcher K. Taylor Department of Mathematics and Statistics Dalhousie University Halifax, Nova Scotia B3H 4R2 Canada Advisory Board Comité consultatif G. Bluman P. Borwein R. Kane For other titles published in this series, go to

2

3 Heinz H. Bauschke Patrick L. Combettes Convex Analysis and Monotone Operator Theory in Hilbert Spaces ABC

4 Heinz H. Bauschke Mathematics Irving K. Barber School University of British Columbia Kelowna, B.C. V1V 1V7 Canada Patrick L. Combettes Laboratoire Jacques-Louis Lions Université Pierre et Marie Curie - Paris 6 4, Place Jussieu Paris France plc@math.jussieu.fr ISSN ISBN e-isbn DOI / Springer New York Dordrecht Heidelberg London Library of Congress Control Number: Mathematics Subject Classification (2010): Primary: 41A50, 46-01, 46-02, 46Cxx, 46C05, 47-01, 47-02, 47H05, 47H09, 47H10, 90-01, Secondary: 26A51, 26B25, 46N10, 47-H04, 47N10, 52A05, 52A41, 65K05, 65K10, 90C25, 90C30 Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (

5 Für Steffi, Andrea & Kati À ma famille

6

7 Foreword This self-contained book offers a modern unifying presentation of three basic areas of nonlinear analysis, namely convex analysis, monotone operator theory, and the fixed point theory of nonexpansive mappings. This turns out to be a judicious choice. Showing the rich connections and interplay between these topics gives a strong coherence to the book. Moreover, these particular topics are at the core of modern optimization and its applications. Choosing to work in Hilbert spaces offers a wide range of applications, while keeping the mathematics accessible to a large audience. Each topic is developed in a self-contained fashion, and the presentation often draws on recent advances. The organization of the book makes it accessible to a large audience. Each chapter is illustrated by several exercises, which makes the monograph an excellent textbook. In addition, it offers deep insights into algorithmic aspects of optimization, especially splitting algorithms, which are important in theory and applications. Let us point out the high quality of the writing and presentation. The authors combine an uncompromising demand for rigorous mathematical statements and a deep concern for applications, which makes this book remarkably accomplished. Montpellier (France), October 2010 Hédy Attouch vii

8

9 Preface Three important areas of nonlinear analysis emerged in the early 1960s: convex analysis, monotone operator theory, and the theory of nonexpansive mappings. Over the past four decades, these areas have reached a high level of maturity, and an increasing number of connections have been identified between them. At the same time, they have found applications in a wide array of disciplines, including mechanics, economics, partial differential equations, information theory, approximation theory, signal and image processing, game theory, optimal transport theory, probability and statistics, and machine learning. The purpose of this book is to present a largely self-contained account of the main results of convex analysis, monotone operator theory, and the theory of nonexpansive operators in the context of Hilbert spaces. Authoritative monographs are already available on each of these topics individually. A novelty of this book, and indeed, its central theme, is the tight interplay among the key notions of convexity, monotonicity, and nonexpansiveness. We aim at making the presentation accessible to a broad audience, and to reach out in particular to the applied sciences and engineering communities, where these tools have become indispensable. We chose to cast our exposition in the Hilbert space setting. This allows us to cover many applications of interest to practitioners in infinite-dimensional spaces and yet to avoid the technical difficulties pertaining to general Banach space theory that would exclude a large portion of our intended audience. We have also made an attempt to draw on recent developments and modern tools to simplify the proofs of key results, exploiting for instance heavily the concept of a Fitzpatrick function in our exposition of monotone operators, the notion of Fejér monotonicity to unify the convergence proofs of several algorithms, and that of a proximity operator throughout the second half of the book. The book in organized in 29 chapters. Chapters 1 and 2 provide background material. Chapters 3 to 7 cover set convexity and nonexpansive operators. Various aspects of the theory of convex functions are discussed in Chapters 8 to 19. Chapters 20 to 25 are dedicated to monotone operator theix

10 x Preface ory. In addition to these basic building blocks, we also address certain themes from different angles in several places. Thus, optimization theory is discussed in Chapters 11, 19, 26, and 27. Best approximation problems are discussed in Chapters 3, 19, 27, 28, and 29. Algorithms are also present in various parts of the book: fixed point and convex feasibility algorithms in Chapter 5, proximal-point algorithms in Chapter 23, monotone operator splitting algorithms in Chapter 25, optimization algorithms in Chapter 27, and best approximation algorithms in Chapters 27 and 29. More than 400 exercises are distributed throughout the book, at the end of each chapter. Preliminary drafts of this book have been used in courses in our institutions and we have benefited from the input of postdoctoral fellows and many students. To all of them, many thanks. In particular, HHB thanks Liangjin Yao for his helpful comments. We are grateful to Hédy Attouch, Jon Borwein, Stephen Simons, Jon Vanderwerff, Shawn Wang, and Isao Yamada for helpful discussions and pertinent comments. PLC also thanks Oscar Wesler. Finally, we thank the Natural Sciences and Engineering Research Council of Canada, the Canada Research Chair Program, and France s Agence Nationale de la Recherche for their support. Kelowna (Canada) Paris (France) October 2010 Heinz H. Bauschke Patrick L. Combettes

11 Contents 1 Background Sets in Vector Spaces Operators Order Nets The Extended Real Line Functions Topological Spaces Two Point Compactification of the Real Line Continuity Lower Semicontinuity Sequential Topological Notions Metric Spaces Exercises Hilbert Spaces Notation and Examples Basic Identities and Inequalities Linear Operators and Functionals Strong and Weak Topologies Weak Convergence of Sequences Differentiability Exercises Convex Sets Definition and Examples Best Approximation Properties Topological Properties Separation Exercises xi

12 xii Contents 4 Convexity and Nonexpansiveness Nonexpansive Operators Projectors onto Convex Sets Fixed Points of Nonexpansive Operators Averaged Nonexpansive Operators Common Fixed Points Exercises Fejér Monotonicity and Fixed Point Iterations Fejér Monotone Sequences Krasnosel skiĭ Mann Iteration Iterating Compositions of Averaged Operators Exercises Convex Cones and Generalized Interiors Convex Cones Generalized Interiors Polar and Dual Cones Tangent and Normal Cones Recession and Barrier Cones Exercises Support Functions and Polar Sets Support Points Support Functions Polar Sets Exercises Convex Functions Definition and Examples Convexity Preserving Operations Topological Properties Exercises Lower Semicontinuous Convex Functions Lower Semicontinuous Convex Functions Proper Lower Semicontinuous Convex Functions Affine Minorization Construction of Functions in Γ 0 (H) Exercises Convex Functions: Variants Between Linearity and Convexity Uniform and Strong Convexity Quasiconvexity Exercises

13 Contents xiii 11 Convex Variational Problems Infima and Suprema Minimizers Uniqueness of Minimizers Existence of Minimizers Minimizing Sequences Exercises Infimal Convolution Definition and Basic Facts Infimal Convolution of Convex Functions Pasch Hausdorff Envelope Moreau Envelope Infimal Postcomposition Exercises Conjugation Definition and Examples Basic Properties The Fenchel Moreau Theorem Exercises Further Conjugation Results Moreau s Decomposition Proximal Average Positively Homogeneous Functions Coercivity The Conjugate of the Difference Exercises Fenchel Rockafellar Duality The Attouch Brézis Theorem Fenchel Duality Fenchel Rockafellar Duality A Conjugation Result Applications Exercises Subdifferentiability Basic Properties Convex Functions Lower Semicontinuous Convex Functions Subdifferential Calculus Exercises

14 xiv Contents 17 Differentiability of Convex Functions Directional Derivatives Characterizations of Convexity Characterizations of Strict Convexity Directional Derivatives and Subgradients Gâteaux and Fréchet Differentiability Differentiability and Continuity Exercises Further Differentiability Results The Ekeland Lebourg Theorem The Subdifferential of a Maximum Differentiability of Infimal Convolutions Differentiability and Strict Convexity Stronger Notions of Differentiability Differentiability of the Distance to a Set Exercises Duality in Convex Optimization Primal Solutions via Dual Solutions Parametric Duality Minimization under Equality Constraints Minimization under Inequality Constraints Exercises Monotone Operators Monotone Operators Maximally Monotone Operators Bivariate Functions and Maximal Monotonicity The Fitzpatrick Function Exercises Finer Properties of Monotone Operators Minty s Theorem The Debrunner Flor Theorem Domain and Range Local Boundedness and Surjectivity Kenderov s Theorem and Fréchet Differentiability Exercises Stronger Notions of Monotonicity Para, Strict, Uniform, and Strong Monotonicity Cyclic Monotonicity Rockafellar s Cyclic Monotonicity Theorem Monotone Operators on R Exercises

15 Contents xv 23 Resolvents of Monotone Operators Definition and Basic Identities Monotonicity and Firm Nonexpansiveness Resolvent Calculus Zeros of Monotone Operators Asymptotic Behavior Exercises Sums of Monotone Operators Maximal Monotonicity of a Sum Monotone Operators The Brézis Haraux Theorem Parallel Sum Exercises Zeros of Sums of Monotone Operators Characterizations Douglas Rachford Splitting Forward Backward Splitting Tseng s Splitting Algorithm Variational Inequalities Exercises Fermat s Rule in Convex Optimization General Characterizations of Minimizers Abstract Constrained Minimization Problems Affine Constraints Cone Constraints Convex Inequality Constraints Regularization of Minimization Problems Exercises Proximal Minimization The Proximal-Point Algorithm Douglas Rachford Algorithm Forward Backward Algorithm Tseng s Splitting Algorithm A Primal Dual Algorithm Exercises Projection Operators Basic Properties Projections onto Affine Subspaces Projections onto Special Polyhedra Projections Involving Convex Cones Projections onto Epigraphs and Lower Level Sets

16 xvi Contents Exercises Best Approximation Algorithms Dykstra s Algorithm Haugazeau s Algorithm Exercises Bibliographical Pointers Symbols and Notation References Index