Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradient
|
|
- Benjamin Floyd
- 5 years ago
- Views:
Transcription
1 Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradient Xingyu Zhou The Ohio State University December 5, 2017 Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
2 Main Result Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
3 Main Result Main Result Theorem (i) If f is closed and strong convex with parameter µ, then f has a Lipschitz continuous gradient with parameter 1 µ. (ii) If f is convex and has a Lipschitz continuous gradient with parameter L, then f is strong convex with parameter 1 L. We provide a very simple proof for this theorem (two-line). To this end, we first present equivalent conditions for strong convexity and Lipschitz continuous gradient. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
4 Review of Convex Function Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
5 Review of Convex Function Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
6 Review of Convex Function (Convexity) Suppose f : R n R with the extended-value extension. Then, f is said to be a convex function if for any α [0, 1], for any x and y. f (αx + (1 α)y) αf (x) + (1 α)f (y) Note: This definition does not require the differentiability of f. We are familiar that this definition is equivalent to the first-order condition and monotonicity condition assuming f is differentiable. In the following, we will first show that this equivalence still holds for a non-smooth function. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
7 Review of Convex Function Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
8 Review of Convex Function for Convexity Lemma (Equivalence for Convexity) Suppose f : R n R with the extended-value extension. Then, the following statements are equivalent: (i) (Jensen s inequality): f is convex. (ii) (First-order): f (y) f (x) + s T x (y x) for all x, y and any s x f (x). (iii) (Monotonicity of subgradient): (s y s x ) T (y x) 0 for all x, y and any s x f (x), s y f (y). Note: The lemma is a non-trivial extension as we will see later in the proof. Moreover, this lemma will be extremely useful as we later deal with strong convexity. Specifically, we will prove this lemma by showing (i) = (ii) = (iii) = (i). Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
9 Review of Convex Function Proof (i) = (ii): Since f is convex, by definition, we have for any 0 < α 1 f (x + α(y x)) αf (y) + (1 α)f (x) f (x + α(y x)) f (x) f (y) f (x) α (a) = f (x, y x) f (y) f (x) (b) = s T x (y x) f (y) f (x) s x f (x) (a) is obtained by letting α 0 and the definition of directional derivate. Note that the limit always exists for a convex f by the monotonicity of difference quotient. (b) follows from the fact that f (x, v) = sup s f (x) s T v for a convex f. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
10 Review of Convex Function Proof (Cont d) (ii) = (iii): From (ii), we have Adding together directly yields (iii). f (y) f (x) + s T x (y x) f (x) f (y) + s T y (x y) Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
11 Review of Convex Function Proof (Cont d) (iii) = (i): To this end, we will show (iii) = (ii) and (ii) = (i). First, (iii) = (ii): Let φ(α) := f (x + α(y x)) and x α := x + α(y x), then f (y) f (x) = φ(1) φ(0) = 1 0 s T α (y x)dα (1) where s α f (x α ) for t [0, 1]. Then by (iii), for any s x f (x), we have s T α (y x) s T x (y x) which combined with inequality above directly implies (ii). Second, (ii) = (i): By (ii), we have f (y) f (x α ) + s T t (y x α ) (2) f (x) f (x α ) + s T t (x x α ) (3) Multiplying (2) by α and (3) by 1 α, and adding together yields (i). Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
12 Strong Convexity Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
13 Strong Convexity Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
14 Strong Convexity Strongly Convex Function (Strong Convexity) Suppose f : R n R with the extended-value extension. Then, f is said to be strongly convex with parameter µ if is convex. g(x) = f (x) µ 2 x T x Note: As before, this definition does not require the differentiability of f. In the following, by applying the previous equivalent conditions of convexity to g, we can easily establish equivalence between strong convexity. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
15 Strong Convexity Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
16 Strong Convexity for Strong Convexity Lemma (Equivalence for Strong Convexity) Suppose f : R n R with the extended-value extension. Then, the following statements are equivalent: (i) f is strongly convex with parameter µ. (ii) f (αx + (1 α)y) αf (x) + (1 α)f (y) µ 2 α(1 α) y x 2 for any x, y. (iii) f (y) f (x) + s T x (y x) + µ 2 y x 2 for all x, y and any s x f (x). (iv) (s y s x ) T (y x) µ y x 2 for all x, y and any s x f (x), s y f (y). Proof: (i) (ii): It follows from the definition of convexity for g(x) = f (x) µ 2 x T x. (i) (iii): It follows from the equivalent first-order condition of convexity for g. (i) (iv): It follows from the equivalent monotonicity of (sub)-gradient of convexity for g. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
17 Strong Convexity Implications of Strong Convexity Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
18 Strong Convexity Implications of Strong Convexity Conditions Implied by Strong Convexity (SC) Lemma (Implications of SC) Suppose f : R n R with the extended-value extension. The following conditions are all implied by strong convexity with parameter µ: (i) 1 2 s x 2 µ(f (x) f ), x and s x f (x). (ii) s y s x µ y x x, y and any s x f (x), s y f (y). (iii) f (y) f (x) + s T x (y x) + 1 2µ s y s x 2 x, y and any s x f (x), s y f (y). (iv) (s y s x ) T (y x) 1 µ s y s x 2 x, y and any s x f (x), s y f (y). Note: In the case of smooth function f, (i) reduced to Polyak-Lojasiewicz (PL) inequality, which is more general than SC and is extremely useful for linear convergence. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
19 Strong Convexity Implications of Strong Convexity Proof (i) 1 2 s x 2 µ(f (x) f ) (SC) = (i): By equivalence of strong convexity in the previous lemma, we have f (y) f (x) + sx T (y x) + µ y x 2 2 Taking minimization with respect to y on both sides, yields Re-arranging it yields (i). f f (x) 1 2µ s x 2 Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
20 Strong Convexity Implications of Strong Convexity Proof (Cont d) (ii) s y s x µ y x SC = (ii): By equivalence of strong convexity in the previous lemma, we have (s y s x ) T (y x) µ y x 2 Applying Cauchy-Schwartz inequality to the left-hand-side, yields (ii). Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
21 Strong Convexity Implications of Strong Convexity Proof (Cont d) (iii) f (y) f (x) + s T x (y x) + 1 2µ s y s x 2 SC = (iii): For any given s x f (x), let φ x (z) := f (z) s T x z. First, since (s φ z 1 s φ z 2 )(z 1 z 2 ) = (s f z 1 s f z 2 )(z 1 z 2 ) µ z 1 z 2 2 which implies that φ x (z) is also strong convexity with parameter µ. Then, applying PL-inequality to φ x (z), yields Re-arranging it yields (iii). φ x = f (x) sx T x φ x (y) 1 sy φ 2 2µ = f (y) sx T y 1 2µ s y s x 2. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
22 Strong Convexity Implications of Strong Convexity Proof (Cont d) (iv) (s y s x ) T (y x) 1 µ s y s x 2 SC = (iv): From previous result, we have Adding together yields (iv). f (y) f (x) + s T x (y x) + 1 2µ s y s x 2 f (x) f (y) + s T y (x y) + 1 2µ s x s y 2 Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
23 Lipschitz Continuous Gradient Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
24 Lipschitz Continuous Gradient Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
25 Lipschitz Continuous Gradient A differentiable function f is said to have an L-Lipschitz continuous gradient if for some L > 0 f (x) f (y) L x y, x, y. Note: The definition does not require the convexity of f. In the following, we will see different conditions that are equivalent or implied by Lipschitz continuous gradient condition. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
26 Lipschitz Continuous Gradient Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
27 Lipschitz Continuous Gradient Relationship [0] f (x) f (y) L x y, x, y. [1] g(x) = L 2 x T x f (x) is convex, x [2] f (y) f (x) + f (x) T (y x) + L 2 y x 2, x, y. [3] ( f (x) f (y) T (x y) L x y 2, x, y. α(1 α)l [4] f (αx + (1 α)y) αf (x) + (1 α)f (y) x y 2, x, y and α [ 2 [5] f (y) f (x) + f (x) T (y x) + 1 2L f (y) f (x) 2, x, y. [6] ( f (x) f (y) T (x y) 1 L f (x) f (y) 2, x, y. [7] f (αx + (1 α)y) αf (x) + (1 α)f (y) α(1 α) f (x) f (y) 2, x, y 2L Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
28 Lipschitz Continuous Gradient Relationship (Cont d) Lemma For a function f with a Lipschitz continuous gradient over R n, the following relations hold: [5] [7] = [6] = [0] = [1] [2] [3] [4] (4) If the function f is convex, then all the conditions [0]-[7] are equivalent. Note: From (5), we can see that conditions [0]-[7] can be grouped into four classes. Class A: [1]-[4] Class B: [5],[7] Class C: [6] Class D: [0] Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
29 Lipschitz Continuous Gradient Proof Class A [1] g(x) = L 2 x T x f (x) is convex [2] f (y) f (x) + f (x) T (y x) + L y x 2 2 [3] ( f (x) f (y) T (x y) L x y 2 [4] f (αx + (1 α)y) αf (x) + (1 α)f (y) α(1 α)l x y 2 2 [1] [2]: It follows from the first-order equivalence of convexity. [1] [3]: It follows from the monotonicity of gradient equivalence of convexity. [1] [4]: It follows from the definition of convexity. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
30 Lipschitz Continuous Gradient Proof (Cont d) Class B [5] f (y) f (x) + f (x) T (y x) + 1 2L f (y) f (x) 2. [7] f (αx + (1 α)y) αf (x) + (1 α)f (y) [5] = [7]: Let x α := αx + (1 α)y, then α(1 α) f (x) f (y) 2. 2L f (x) f (x α ) + f (x α ) T (x x α ) + 1 2L f (x) f (x α) 2 f (y) f (x α ) + f (z) T (y x α ) + 1 2L f (y) f (x α) 2 Multiplying the first inequality with α and second inequality with 1 α, and adding them together and using α x 2 + (1 α) y 2 α(1 α) x y 2, yields [7]. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
31 Lipschitz Continuous Gradient Proof (Cont d) Class B [5] f (y) f (x) + f (x) T (y x) + 1 2L f (y) f (x) 2. [7] f (αx + (1 α)y) αf (x) + (1 α)f (y) [7] = [5]: Interchanging x and y in [7] and re-writing it as f (y) f (x) + Letting α 0, yields [5]. f (x + α(y x)) f (x) α α(1 α) f (x) f (y) 2. 2L + 1 α f (x) f (y) 2 2L Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
32 Lipschitz Continuous Gradient Relationship (Cont d) Lemma For a function f with a Lipschitz continuous gradient over R n, the following relations hold: [5] [7] = [6] = [0] = [1] [2] [3] [4] (5) If the function f is convex, then all the conditions [0]-[7] are equivalent. Note: From (5), we can see that conditions [0]-[7] can be grouped into four classes. Class A: [1]-[4] Class B: [5],[7] Class C: [6] Class D: [0] Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
33 Lipschitz Continuous Gradient Proof (Cont d) Class B ([5]) = Class C ([6]) = Class D ([0]) = Class A ([3]) [5] f (y) f (x) + f (x) T (y x) + 1 2L f (y) f (x) 2. [6] ( f (x) f (y)) T (x y) 1 f (x) f (y) 2 L [0] f (x) f (y) L x y [3] ( f (x) f (y)) T (x y) L x y 2 [5] = [6]: Interchanging x and y and adding together. [6] = [0]: It follows directly from Cauchy-Schwartz inequality. [0] = [3]: It following directly from Cauchy-Schwartz inequality. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
34 Lipschitz Continuous Gradient Proof (Cont d) Class A ([3]) = Class B ([5]) for convex f [3] ( f (x) f (y)) T (x y) L x y 2. [5] f (y) f (x) + f (x) T (y x) + 1 2L f (y) f (x) 2. [3] = [5]: Let φ x (z) := f (z) f (x) T z. Note that since f is convex, φ x (z) attains its minimum at z = x. By [3], we have which implies that ( φ x (z 1 ) φ x (z 2 ) T (z 1 z 2 ) L z 1 z 2 2 φ x (z) φ x (y) + φ x (y) T (z y) + L 2 z y 2. Taking minimization over z, yields [5]. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
35 Conjugate Function Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
36 Conjugate Function Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
37 Conjugate Function (Conjugate) The conjugate of a function f is f (s) = sup (s T x f (x)) x domf Note: f is convex and closed (even f is not). (Subgradient) s is a subgradient of f at x if Note: f need not be convex. f (y) f (x) + s T (y x) for all y Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
38 Conjugate Function Useful Results Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
39 Conjugate Function Useful Results Useful Results Lemma (Fenchel-Young s Equality) Consider the following conditions for a general f : [1] f (s) = s T x f (x) [2] s f (x) [3] x f (s) Then, we have [1] [2] = [3] Further, if f is closed and convex, then all these conditions are equivalent. Lemma (Differentiability) For a closed and strictly convex f, f (s) = argmax x (s T x f (x)). Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
40 Conjugate Function Useful Results Proof [1] f (s) = s T x f (x) [2] s f (x) [1] [2]: s f (x) f (y) f (x) + s T (y x) y s T x f (x) s T y f (y) y s T x f (x) f (s) Since f (s) s T x f (x) always holds, [1] [2]. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
41 Conjugate Function Useful Results Proof (Cont d) [2] s f (x) [3] x f (s) [2] = [3]: If [2] holds, then by previous result we have f (s) = s T x f (x). Thus, f (z) = sup(z T u f (u)) u z T x f (x) = s T x f (x) + x T (z s) = f (s) + x T (z s) This holds for all z, which implies x f (s). For a closed and convex function, we have [3] = [2]: This follows from f = f. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
42 Conjugate Function Useful Results Proof (Differentiability) Suppose x 1 f (s) and x 2 f (s). By closeness and convexity, we have [3] = [2], which gives s f x 1 and s f x 2 By previous result ([2] = [1]), we have x 1 = argmax(s T x f (x)) and x 2 = argmax(s T x f (x)) By strictly convexity, s T x f (x) has a unique maximizer for every s. Thus, x := x 1 = x 2 Thus f (s) = {x}, which implies f is differentiable at s and f (s) = argmax x (s T x f (x)). Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
43 Proof of Main Result Outline 1 Main Result 2 Review of Convex Function 3 Strong Convexity Implications of Strong Convexity 4 Lipschitz Continuous Gradient 5 Conjugate Function Useful Results 6 Proof of Main Result Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
44 Proof of Main Result Proof of Main Result Theorem (i) If f is closed and strong convex with parameter µ, then f has a Lipschitz continuous gradient with parameter 1 µ. (ii) If f is convex and has a Lipschitz continuous gradient with parameter L, then f is strong convex with parameter 1 L. Proof: (1): By implication of strong convexity, we have which implies s x s y µ x y s x f (x), s y f (y) s x s y µ f (s x ) f (s y ) Hence, f has a Lipschitz continuous gradient with 1 µ. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
45 Proof of Main Result Proof (Cont d) (2): By implication of Lipschitz continuous gradient for convex f, we have which implies ( f (x) f (y) T (x y) 1 f (x) f (y) 2 L (s x s y ) T (x y) 1 L (s x s y ) x f (s x ), y f (s y ) Hence, f is strongly convex with parameter 1 L. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
46 Proof of Main Result Summary A very simple proof of Fenchel duality between strong convexity and Lipschitz continuous gradient. A summary of conditions for strong convexity. A summary of conditions for Lipschitz continuous gradient Common tricks: Taking limit (α 0). Taking minimization over both sides. Interchanging x and y, and adding together. Applying mean-value theorem. Applying Cauchy-Schwartz inequality. Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
47 Proof of Main Result Thank you! Q & A Xingyu Zhou (OSU) Fenchel Duality December 5, / 47
Optimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More information1. Gradient method. gradient method, first-order methods. quadratic bounds on convex functions. analysis of gradient method
L. Vandenberghe EE236C (Spring 2016) 1. Gradient method gradient method, first-order methods quadratic bounds on convex functions analysis of gradient method 1-1 Approximate course outline First-order
More informationLocal strong convexity and local Lipschitz continuity of the gradient of convex functions
Local strong convexity and local Lipschitz continuity of the gradient of convex functions R. Goebel and R.T. Rockafellar May 23, 2007 Abstract. Given a pair of convex conjugate functions f and f, we investigate
More informationMath 273a: Optimization Subgradients of convex functions
Math 273a: Optimization Subgradients of convex functions Made by: Damek Davis Edited by Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com 1 / 42 Subgradients Assumptions
More informationBASICS OF CONVEX ANALYSIS
BASICS OF CONVEX ANALYSIS MARKUS GRASMAIR 1. Main Definitions We start with providing the central definitions of convex functions and convex sets. Definition 1. A function f : R n R + } is called convex,
More informationIt follows from the above inequalities that for c C 1
3 Spaces L p 1. In this part we fix a measure space (, A, µ) (we may assume that µ is complete), and consider the A -measurable functions on it. 2. For f L 1 (, µ) set f 1 = f L 1 = f L 1 (,µ) = f dµ.
More informationConvex Optimization Notes
Convex Optimization Notes Jonathan Siegel January 2017 1 Convex Analysis This section is devoted to the study of convex functions f : B R {+ } and convex sets U B, for B a Banach space. The case of B =
More informationON GAP FUNCTIONS OF VARIATIONAL INEQUALITY IN A BANACH SPACE. Sangho Kum and Gue Myung Lee. 1. Introduction
J. Korean Math. Soc. 38 (2001), No. 3, pp. 683 695 ON GAP FUNCTIONS OF VARIATIONAL INEQUALITY IN A BANACH SPACE Sangho Kum and Gue Myung Lee Abstract. In this paper we are concerned with theoretical properties
More informationMath 273a: Optimization Subgradient Methods
Math 273a: Optimization Subgradient Methods Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com Nonsmooth convex function Recall: For ˉx R n, f(ˉx) := {g R
More informationLecture 2: Convex Sets and Functions
Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are
More informationDual Decomposition.
1/34 Dual Decomposition http://bicmr.pku.edu.cn/~wenzw/opt-2017-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/34 1 Conjugate function 2 introduction:
More informationL p Spaces and Convexity
L p Spaces and Convexity These notes largely follow the treatments in Royden, Real Analysis, and Rudin, Real & Complex Analysis. 1. Convex functions Let I R be an interval. For I open, we say a function
More informationMath 273a: Optimization Subgradients of convex functions
Math 273a: Optimization Subgradients of convex functions Made by: Damek Davis Edited by Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com 1 / 20 Subgradients Assumptions
More informationLecture 4 Lebesgue spaces and inequalities
Lecture 4: Lebesgue spaces and inequalities 1 of 10 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 4 Lebesgue spaces and inequalities Lebesgue spaces We have seen how
More informationIntroduction to Convex Analysis Microeconomics II - Tutoring Class
Introduction to Convex Analysis Microeconomics II - Tutoring Class Professor: V. Filipe Martins-da-Rocha TA: Cinthia Konichi April 2010 1 Basic Concepts and Results This is a first glance on basic convex
More informationOnline Convex Optimization
Advanced Course in Machine Learning Spring 2010 Online Convex Optimization Handouts are jointly prepared by Shie Mannor and Shai Shalev-Shwartz A convex repeated game is a two players game that is performed
More informationIt follows from the above inequalities that for c C 1
3 Spaces L p 1. Appearance of normed spaces. In this part we fix a measure space (, A, µ) (we may assume that µ is complete), and consider the A - measurable functions on it. 2. For f L 1 (, µ) set f 1
More information6. Proximal gradient method
L. Vandenberghe EE236C (Spring 2016) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationMath 273a: Optimization Convex Conjugacy
Math 273a: Optimization Convex Conjugacy Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com Convex conjugate (the Legendre transform) Let f be a closed proper
More informationSolving Dual Problems
Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem
More informationConvex Optimization Theory. Chapter 4 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 4 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationLecture: Smoothing.
Lecture: Smoothing http://bicmr.pku.edu.cn/~wenzw/opt-2018-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Smoothing 2/26 introduction smoothing via conjugate
More informationReproducing Kernel Hilbert Spaces Class 03, 15 February 2006 Andrea Caponnetto
Reproducing Kernel Hilbert Spaces 9.520 Class 03, 15 February 2006 Andrea Caponnetto About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More information8. Conjugate functions
L. Vandenberghe EE236C (Spring 2013-14) 8. Conjugate functions closed functions conjugate function 8-1 Closed set a set C is closed if it contains its boundary: x k C, x k x = x C operations that preserve
More informationConvex Functions and Optimization
Chapter 5 Convex Functions and Optimization 5.1 Convex Functions Our next topic is that of convex functions. Again, we will concentrate on the context of a map f : R n R although the situation can be generalized
More informationConvex Optimization Conjugate, Subdifferential, Proximation
1 Lecture Notes, HCI, 3.11.211 Chapter 6 Convex Optimization Conjugate, Subdifferential, Proximation Bastian Goldlücke Computer Vision Group Technical University of Munich 2 Bastian Goldlücke Overview
More informationProblem set 5, Real Analysis I, Spring, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, 1 x (log 1/ x ) 2 dx 1
Problem set 5, Real Analysis I, Spring, 25. (5) Consider the function on R defined by f(x) { x (log / x ) 2 if x /2, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, R f /2
More informationConvex Analysis and Economic Theory AY Elementary properties of convex functions
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory AY 2018 2019 Topic 6: Convex functions I 6.1 Elementary properties of convex functions We may occasionally
More informationOn the interior of the simplex, we have the Hessian of d(x), Hd(x) is diagonal with ith. µd(w) + w T c. minimize. subject to w T 1 = 1,
Math 30 Winter 05 Solution to Homework 3. Recognizing the convexity of g(x) := x log x, from Jensen s inequality we get d(x) n x + + x n n log x + + x n n where the equality is attained only at x = (/n,...,
More informationLecture 16: FTRL and Online Mirror Descent
Lecture 6: FTRL and Online Mirror Descent Akshay Krishnamurthy akshay@cs.umass.edu November, 07 Recap Last time we saw two online learning algorithms. First we saw the Weighted Majority algorithm, which
More informationIterative Convex Optimization Algorithms; Part One: Using the Baillon Haddad Theorem
Iterative Convex Optimization Algorithms; Part One: Using the Baillon Haddad Theorem Charles Byrne (Charles Byrne@uml.edu) http://faculty.uml.edu/cbyrne/cbyrne.html Department of Mathematical Sciences
More informationRandomized Coordinate Descent Methods on Optimization Problems with Linearly Coupled Constraints
Randomized Coordinate Descent Methods on Optimization Problems with Linearly Coupled Constraints By I. Necoara, Y. Nesterov, and F. Glineur Lijun Xu Optimization Group Meeting November 27, 2012 Outline
More informationFixed point theory for nonlinear mappings in Banach spaces and applications
Kangtunyakarn Fixed Point Theory and Applications 014, 014:108 http://www.fixedpointtheoryandapplications.com/content/014/1/108 R E S E A R C H Open Access Fixed point theory for nonlinear mappings in
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationSubdifferential representation of convex functions: refinements and applications
Subdifferential representation of convex functions: refinements and applications Joël Benoist & Aris Daniilidis Abstract Every lower semicontinuous convex function can be represented through its subdifferential
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More informationALGORITHMS FOR MINIMIZING DIFFERENCES OF CONVEX FUNCTIONS AND APPLICATIONS
ALGORITHMS FOR MINIMIZING DIFFERENCES OF CONVEX FUNCTIONS AND APPLICATIONS Mau Nam Nguyen (joint work with D. Giles and R. B. Rector) Fariborz Maseeh Department of Mathematics and Statistics Portland State
More informationConvex Sets and Functions (part III)
Convex Sets and Functions (part III) Prof. Dan A. Simovici UMB 1 / 14 Outline 1 The epigraph and hypograph of functions 2 Constructing Convex Functions 2 / 14 The epigraph and hypograph of functions Definition
More informationSome Properties of the Augmented Lagrangian in Cone Constrained Optimization
MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364-765X eissn 1526-5471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented
More informationChapter 1 Preliminaries
Chapter 1 Preliminaries 1.1 Conventions and Notations Throughout the book we use the following notations for standard sets of numbers: N the set {1, 2,...} of natural numbers Z the set of integers Q the
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationEpiconvergence and ε-subgradients of Convex Functions
Journal of Convex Analysis Volume 1 (1994), No.1, 87 100 Epiconvergence and ε-subgradients of Convex Functions Andrei Verona Department of Mathematics, California State University Los Angeles, CA 90032,
More informationLECTURE 9 LECTURE OUTLINE. Min Common/Max Crossing for Min-Max
Min-Max Problems Saddle Points LECTURE 9 LECTURE OUTLINE Min Common/Max Crossing for Min-Max Given φ : X Z R, where X R n, Z R m consider minimize sup φ(x, z) subject to x X and maximize subject to z Z.
More informationLecture 8 Plus properties, merit functions and gap functions. September 28, 2008
Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plus-properties and F-uniqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FP-I book:
More informationCONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS
CONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS A Dissertation Submitted For The Award of the Degree of Master of Philosophy in Mathematics Neelam Patel School of Mathematics
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationConvex Analysis and Optimization Chapter 4 Solutions
Convex Analysis and Optimization Chapter 4 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationSolution of the 8 th Homework
Solution of the 8 th Homework Sangchul Lee December 8, 2014 1 Preinary 1.1 A simple remark on continuity The following is a very simple and trivial observation. But still this saves a lot of words in actual
More informationEcon Slides from Lecture 1
Econ 205 Sobel Econ 205 - Slides from Lecture 1 Joel Sobel August 23, 2010 Warning I can t start without assuming that something is common knowledge. You can find basic definitions of Sets and Set Operations
More informationAnalysis II - few selective results
Analysis II - few selective results Michael Ruzhansky December 15, 2008 1 Analysis on the real line 1.1 Chapter: Functions continuous on a closed interval 1.1.1 Intermediate Value Theorem (IVT) Theorem
More informationFinite Dimensional Optimization Part III: Convex Optimization 1
John Nachbar Washington University March 21, 2017 Finite Dimensional Optimization Part III: Convex Optimization 1 1 Saddle points and KKT. These notes cover another important approach to optimization,
More informationConvergent Iterative Algorithms in the 2-inner Product Space R n
Int. J. Open Problems Compt. Math., Vol. 6, No. 4, December 2013 ISSN 1998-6262; Copyright c ICSRS Publication, 2013 www.i-csrs.org Convergent Iterative Algorithms in the 2-inner Product Space R n Iqbal
More informationConditional Gradient (Frank-Wolfe) Method
Conditional Gradient (Frank-Wolfe) Method Lecturer: Aarti Singh Co-instructor: Pradeep Ravikumar Convex Optimization 10-725/36-725 1 Outline Today: Conditional gradient method Convergence analysis Properties
More informationElements of Convex Optimization Theory
Elements of Convex Optimization Theory Costis Skiadas August 2015 This is a revised and extended version of Appendix A of Skiadas (2009), providing a self-contained overview of elements of convex optimization
More information212a1214Daniell s integration theory.
212a1214 Daniell s integration theory. October 30, 2014 Daniell s idea was to take the axiomatic properties of the integral as the starting point and develop integration for broader and broader classes
More informationConvex Functions. Pontus Giselsson
Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum
More informationLECTURE 12 LECTURE OUTLINE. Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions
LECTURE 12 LECTURE OUTLINE Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions Reading: Section 5.4 All figures are courtesy of Athena
More informationPROPERTIES OF A CLASS OF APPROXIMATELY SHRINKING OPERATORS AND THEIR APPLICATIONS
Fixed Point Theory, 15(2014), No. 2, 399-426 http://www.math.ubbcluj.ro/ nodeacj/sfptcj.html PROPERTIES OF A CLASS OF APPROXIMATELY SHRINKING OPERATORS AND THEIR APPLICATIONS ANDRZEJ CEGIELSKI AND RAFA
More information6. Proximal gradient method
L. Vandenberghe EE236C (Spring 2013-14) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping
More informationLecture 6: September 17
10-725/36-725: Convex Optimization Fall 2015 Lecturer: Ryan Tibshirani Lecture 6: September 17 Scribes: Scribes: Wenjun Wang, Satwik Kottur, Zhiding Yu Note: LaTeX template courtesy of UC Berkeley EECS
More informationDual methods and ADMM. Barnabas Poczos & Ryan Tibshirani Convex Optimization /36-725
Dual methods and ADMM Barnabas Poczos & Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given f : R n R, the function is called its conjugate Recall conjugate functions f (y) = max x R n yt x f(x)
More informationLogarithmic Regret Algorithms for Strongly Convex Repeated Games
Logarithmic Regret Algorithms for Strongly Convex Repeated Games Shai Shalev-Shwartz 1 and Yoram Singer 1,2 1 School of Computer Sci & Eng, The Hebrew University, Jerusalem 91904, Israel 2 Google Inc 1600
More informationGARP and Afriat s Theorem Production
GARP and Afriat s Theorem Production Econ 2100 Fall 2017 Lecture 8, September 21 Outline 1 Generalized Axiom of Revealed Preferences 2 Afriat s Theorem 3 Production Sets and Production Functions 4 Profits
More informationAD for PS Functions and Relations to Generalized Derivative Concepts
AD for PS Functions and Relations to Generalized Derivative Concepts Andrea Walther 1 and Andreas Griewank 2 1 Institut für Mathematik, Universität Paderborn 2 School of Math. Sciences and Information
More informationDual and primal-dual methods
ELE 538B: Large-Scale Optimization for Data Science Dual and primal-dual methods Yuxin Chen Princeton University, Spring 2018 Outline Dual proximal gradient method Primal-dual proximal gradient method
More informationQUADRATIC MAJORIZATION 1. INTRODUCTION
QUADRATIC MAJORIZATION JAN DE LEEUW 1. INTRODUCTION Majorization methods are used extensively to solve complicated multivariate optimizaton problems. We refer to de Leeuw [1994]; Heiser [1995]; Lange et
More informationM2PM1 Analysis II (2008) Dr M Ruzhansky List of definitions, statements and examples Preliminary version
M2PM1 Analysis II (2008) Dr M Ruzhansky List of definitions, statements and examples Preliminary version Chapter 0: Some revision of M1P1: Limits and continuity This chapter is mostly the revision of Chapter
More informationMonotone Operator Splitting Methods in Signal and Image Recovery
Monotone Operator Splitting Methods in Signal and Image Recovery P.L. Combettes 1, J.-C. Pesquet 2, and N. Pustelnik 3 2 Univ. Pierre et Marie Curie, Paris 6 LJLL CNRS UMR 7598 2 Univ. Paris-Est LIGM CNRS
More informationFrank-Wolfe Method. Ryan Tibshirani Convex Optimization
Frank-Wolfe Method Ryan Tibshirani Convex Optimization 10-725 Last time: ADMM For the problem min x,z f(x) + g(z) subject to Ax + Bz = c we form augmented Lagrangian (scaled form): L ρ (x, z, w) = f(x)
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationMaster 2 MathBigData. 3 novembre CMAP - Ecole Polytechnique
Master 2 MathBigData S. Gaïffas 1 3 novembre 2014 1 CMAP - Ecole Polytechnique 1 Supervised learning recap Introduction Loss functions, linearity 2 Penalization Introduction Ridge Sparsity Lasso 3 Some
More informationEE 546, Univ of Washington, Spring Proximal mapping. introduction. review of conjugate functions. proximal mapping. Proximal mapping 6 1
EE 546, Univ of Washington, Spring 2012 6. Proximal mapping introduction review of conjugate functions proximal mapping Proximal mapping 6 1 Proximal mapping the proximal mapping (prox-operator) of a convex
More informationMTH 503: Functional Analysis
MTH 53: Functional Analysis Semester 1, 215-216 Dr. Prahlad Vaidyanathan Contents I. Normed Linear Spaces 4 1. Review of Linear Algebra........................... 4 2. Definition and Examples...........................
More informationA. Derivation of regularized ERM duality
A. Derivation of regularized ERM dualit For completeness, in this section we derive the dual 5 to the problem of computing proximal operator for the ERM objective 3. We can rewrite the primal problem as
More information1/37. Convexity theory. Victor Kitov
1/37 Convexity theory Victor Kitov 2/37 Table of Contents 1 2 Strictly convex functions 3 Concave & strictly concave functions 4 Kullback-Leibler divergence 3/37 Convex sets Denition 1 Set X is convex
More informationLimited Memory Kelley s Method Converges for Composite Convex and Submodular Objectives
Limited Memory Kelley s Method Converges for Composite Convex and Submodular Objectives Madeleine Udell Operations Research and Information Engineering Cornell University Based on joint work with Song
More information14 Lecture 14 Local Extrema of Function
14 Lecture 14 Local Extrema of Function 14.1 Taylor s Formula with Lagrangian Remainder Term Theorem 14.1. Let n N {0} and f : (a,b) R. We assume that there exists f (n+1) (x) for all x (a,b). Then for
More informationCHAPTER III THE PROOF OF INEQUALITIES
CHAPTER III THE PROOF OF INEQUALITIES In this Chapter, the main purpose is to prove four theorems about Hardy-Littlewood- Pólya Inequality and then gives some examples of their application. We will begin
More informationGauge optimization and duality
1 / 54 Gauge optimization and duality Junfeng Yang Department of Mathematics Nanjing University Joint with Shiqian Ma, CUHK September, 2015 2 / 54 Outline Introduction Duality Lagrange duality Fenchel
More information9. Dual decomposition and dual algorithms
EE 546, Univ of Washington, Spring 2016 9. Dual decomposition and dual algorithms dual gradient ascent example: network rate control dual decomposition and the proximal gradient method examples with simple
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationNonlinear Functional Analysis and its Applications
Eberhard Zeidler Nonlinear Functional Analysis and its Applications III: Variational Methods and Optimization Translated by Leo F. Boron With 111 Illustrations Ш Springer-Verlag New York Berlin Heidelberg
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationSubgradient Method. Ryan Tibshirani Convex Optimization
Subgradient Method Ryan Tibshirani Convex Optimization 10-725 Consider the problem Last last time: gradient descent min x f(x) for f convex and differentiable, dom(f) = R n. Gradient descent: choose initial
More informationEcon Lecture 3. Outline. 1. Metric Spaces and Normed Spaces 2. Convergence of Sequences in Metric Spaces 3. Sequences in R and R n
Econ 204 2011 Lecture 3 Outline 1. Metric Spaces and Normed Spaces 2. Convergence of Sequences in Metric Spaces 3. Sequences in R and R n 1 Metric Spaces and Metrics Generalize distance and length notions
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationSPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS
SPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS TSOGTGEREL GANTUMUR Abstract. After establishing discrete spectra for a large class of elliptic operators, we present some fundamental spectral properties
More informationRadial Subgradient Descent
Radial Subgradient Descent Benja Grimmer Abstract We present a subgradient method for imizing non-smooth, non-lipschitz convex optimization problems. The only structure assumed is that a strictly feasible
More information4. We accept without proofs that the following functions are differentiable: (e x ) = e x, sin x = cos x, cos x = sin x, log (x) = 1 sin x
4 We accept without proofs that the following functions are differentiable: (e x ) = e x, sin x = cos x, cos x = sin x, log (x) = 1 sin x x, x > 0 Since tan x = cos x, from the quotient rule, tan x = sin
More informationLecture 3. Econ August 12
Lecture 3 Econ 2001 2015 August 12 Lecture 3 Outline 1 Metric and Metric Spaces 2 Norm and Normed Spaces 3 Sequences and Subsequences 4 Convergence 5 Monotone and Bounded Sequences Announcements: - Friday
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationConvergence Rates in Regularization for Nonlinear Ill-Posed Equations Involving m-accretive Mappings in Banach Spaces
Applied Mathematical Sciences, Vol. 6, 212, no. 63, 319-3117 Convergence Rates in Regularization for Nonlinear Ill-Posed Equations Involving m-accretive Mappings in Banach Spaces Nguyen Buong Vietnamese
More informationUses of duality. Geoff Gordon & Ryan Tibshirani Optimization /
Uses of duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Remember conjugate functions Given f : R n R, the function is called its conjugate f (y) = max x R n yt x f(x) Conjugates appear
More informationMajorization Minimization - the Technique of Surrogate
Majorization Minimization - the Technique of Surrogate Andersen Ang Mathématique et de Recherche opérationnelle Faculté polytechnique de Mons UMONS Mons, Belgium email: manshun.ang@umons.ac.be homepage:
More informationarxiv: v1 [math.oc] 7 Dec 2018
arxiv:1812.02878v1 [math.oc] 7 Dec 2018 Solving Non-Convex Non-Concave Min-Max Games Under Polyak- Lojasiewicz Condition Maziar Sanjabi, Meisam Razaviyayn, Jason D. Lee University of Southern California
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More information