A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function

Transcription

1 A forwar-backwar ynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function Rau Ioan Boţ Ernö Robert Csetnek February 28, 2017 Abstract. We aress the minimization of the sum of a proper, convex an lower semicontinuous with a (possibly nonconvex) smooth function from the perspective of an implicit ynamical system of forwar-backwar type. The latter is formulate by means of the graient of the smooth function an of the proximal point operator of the nonsmooth one. The trajectory generate by the ynamical system is prove to asymptotically converge to a critical point of the objective, provie a regularization of the latter satisfies the Kuryka- Lojasiewicz property. Convergence rates for the trajectory in terms of the Lojasiewicz exponent of the regularize objective function are also provie. Key Wors. ynamical systems, continuous forwar-backwar metho, nonsmooth optimization, limiting subifferential, Kuryka- Lojasiewicz property AMS subject classification. 34G25, 47J25, 47H05, 90C26, 90C30, 65K10 1 Introuction In this paper we approach the solving of the optimization problem inf x Rn[f(x) + g(x)], (1) where f : R n R {+ } is a proper, convex, lower semicontinuous function an g : R n R a (possibly nonconvex) Fréchet ifferentiable function with β-lipschitz continuous graient for β 0, i.e., g(x) g(y) β x y x, y R n, by associating to it the implicit ynamical system { ( ) ẋ(t) + x(t) = prox ηf x(t) η g(x(t)) (2) x(0) = x 0, where η > 0, x 0 R n is chosen arbitrary an prox ηf : R n R n, efine by { prox ηf (y) = argmin f(u) + 1 } u y 2 u R n 2η University of Vienna, Faculty of Mathematics, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria, rau.bot@univie.ac.at. University of Vienna, Faculty of Mathematics, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria, ernoe.robert.csetnek@univie.ac.at. Research supporte by FWF (Austrian Science Fun), Lise Meitner Programme, project M 1682-N25. 1

2 is the proximal point operator of ηf. Due to the Lipschitz property of the proximal point operator an of the graient of the ifferentiable function, the existence an uniqueness of strong global solutions of the ynamical system (2) is ensure in the framework of the Cauchy-Lipschitz Theorem. The asymptotic analysis of the trajectories is carrie out in the setting of functions satisfying the Kuryka- Lojasiewicz property (so-calle KL functions). To this large class belong functions with ifferent analytic features. The convergence analysis relies on methos an techniques of real algebraic geometry introuce by Lojasiewicz [38] an Kuryka [37] an evelope recently in the nonsmooth setting by Attouch, Bolte an Svaiter [13] an Bolte, Sabach an Teboulle [24]. The approach for proving asymptotic convergence for the trajectories generate by (2) towars a critical point of the objective function of (1), expresse as a zero of the limiting (Morukhovich) subifferential, use three main ingreients (see [6] for the continuous case an also [13, 24] for a similar approach in the iscrete setting). Namely, we show a sufficient ecrease property along the trajectories of a regularization of the objective function, the existence of a subgraient lower boun for the trajectories an, finally, we obtain convergence by making use of the Kuryka- Lojasiewicz property of the objective function. The case when the objective function is semi-algebraic follows as particular case of our analysis. We close our investigations by establishing convergence rates for the trajectories expresse in terms of the Lojasiewicz exponent of the regularize objective function. In the context of minimizing a (nonconvex) smooth function (which correspons to the case when in (1) f is taken equal to zero), several first- an secon-orer graient type ynamical systems have been investigate by Lojasiewicz [38], Simon [42], Haraux an Jenoubi [35], Alvarez, Attouch, Bolte an Reont [6, Section 4], Bolte, Daniiliis an Lewis [21, Section 4], etc. In the aforementione papers, the convergence of the trajectories is obtaine in the framework of KL functions. In what concerns implicit ynamical systems of the same type as (2), let us mention that Bolte has stuie in [20] the asymptotic convergence of the trajectories of { ( ) ẋ(t) + x(t) = projc x(t) η g(x(t)) (3) x(0) = x 0. where g : R n R n is convex an ifferentiable with Lipschitz continuous graient an proj C enotes the projection operator on the nonempty, close an convex set C R n, towars a minimizer of g over C. This correspons to the case when in (1) f is the inicator function of the set C, namely, f(x) = 0, for x C an +, otherwise. We refer also to the work of Antipin [7] for more statements an results concerning the ynamical system (3). The approach of (1) by means of (2), state as a generalization of (3), has been recently consiere by Abbas an Attouch in [1, Section 5.2] in the full convex setting. Implicit ynamical systems relate to both optimization problems an monotone inclusions have been consiere in the literature also by Attouch an Svaiter in [16], Attouch, Abbas an Svaiter in [2] an Attouch, Alvarez an Svaiter in [10]. These investigations have been continue an extene in [17, 26 29]. A further argument in favour of implicit-type ynamical systems of type (2) is that its time iscretization leas to the relaxe forwar-backwar iterative algorithm x k+1 = (1 λ k )x k + λ k prox ηf (x k η g(x k )) k 0, (4) 2

3 where the starting point x 0 R n is arbitrarily chosen an (λ k ) k 0 is the sequence of relaxation parameters. Relaxation algorithms are important for applications, since the parameter involving the relaxation offers the opportunity to accelerate the convergence of the iterates. While in the convex setting this is a well-known an unerstoo fact (see for example [18]), in the nonconvex setting the convergence analysis of relaxation versions is less stuie. The results presente below let us hope that one can evelope a convergence analysis for the relaxe forwar-backwar metho (4) in the nonconvex setting, too, an issue which is left as an open problem an which can be subject of future investigations. The non-relaxe variant of (4) (when λ k = 1 for all k 0) has been investigate also in the nonconvex setting for KL functions. We refer the reaer to [11 13,24,25,30,32,33,36,40] for ifferent treatments of iterative schemes of type (4) in the nonconvex setting. Among the concrete applications of optimization problems involving analytic features, we mention here the use of the sparsity measure in compressive sensing, constraine feasibility problems involving semi-algebraic sets [13], sparse matrix factorization problems [24], restoration of noisy blurre images by means of the Stuent-t istribution [30, 40] an phase retrieval [19]. 2 Preliminaries In this section we recall some notions an results which are neee throughout the paper. Let N = {0, 1, 2,...} be the set of nonnegative integers. For n 1, the Eucliean scalar prouct an the inuce norm on R n are enote by, an, respectively. Notice that all the finite-imensional spaces consiere in the manuscript are enowe with the topology inuce by the Eucliean norm. The omain of the function f : R n R {+ } is efine by om f = {x R n : f(x) < + }. We say that f is proper if om f. For the following generalize subifferential notions an their basic properties we refer to [39,41]. Let f : R n R {+ } be a proper an lower semicontinuous function. If x om f, we consier the Fréchet (viscosity) subifferential of f at x as the set { } ˆ f(x) = v R n f(y) f(x) v, y x : lim inf 0. y x y x For x / om f we set ˆ f(x) :=. The limiting (Morukhovich) subifferential is efine at x om f by f(x) = {v R n : x k x, f(x k ) f(x) an v k ˆ f(x k ), v k v as k + }, while for x / om f, one takes f(x) :=. Therefore ˆ f(x) f(x) for each x R n. Notice that in case f is convex, these subifferential notions coincie with the convex subifferential, thus ˆ f(x) = f(x) = {v R n : f(y) f(x) + v, y x y R n } for all x R n. The graph of the limiting subifferential fulfills the following closeness criterion: if (x k ) k N an (v k ) k N are sequences in R n such that v k f(x k ) for all k N, (x k, v k ) (x, v) an f(x k ) f(x) as k +, then v f(x). The Fermat rule reas in this nonsmooth setting as follows: if x R n is a local minimizer of f, then 0 f(x). We enote by crit(f) = {x R n : 0 f(x)} 3

4 the set of (limiting)-critical points of f. When f is continuously ifferentiable aroun x R n we have f(x) = { f(x)}. We will make use of the following subifferential sum rule: if f : R n R {+ } is proper an lower semicontinuous an h : R n R is a continuously ifferentiable function, then (f + h)(x) = f(x) + h(x) for all x R m. We turn now our attention to functions satisfying the Kuryka- Lojasiewicz property. This class of functions will play a crucial role in the asymptotic analysis of the ynamical system (2). For η (0, + ], we enote by Θ η the class of concave an continuous functions ϕ : [0, η) [0, + ) such that ϕ(0) = 0, ϕ is continuously ifferentiable on (0, η), continuous at 0 an ϕ (s) > 0 for all s (0, η). In the following efinition (see [12,24]) we use also the istance function to a set, efine for A R n as ist(x, A) = inf y A x y for all x R n. Definition 1 (Kuryka- Lojasiewicz property) Let f : R n R {+ } be a proper an lower semicontinuous function. We say that f satisfies the Kuryka- Lojasiewicz (KL) property at x om f = {x R n : f(x) }, if there exist η (0, + ], a neighborhoo U of x an a function ϕ Θ η such that for all x in the intersection the following inequality hols U {x R n : f(x) < f(x) < f(x) + η} ϕ (f(x) f(x)) ist(0, f(x)) 1. If f satisfies the KL property at each point in om f, then f is calle KL function. The origins of this notion go back to the pioneering work of Lojasiewicz [38], where it is prove that for a real-analytic function f : R n R an a critical point x R n (that is f(x) = 0), there exists θ [1/2, 1) such that the function f f(x) θ f 1 is boune aroun x. This correspons to the situation when ϕ(s) = Cs 1 θ, where C > 0. The result of Lojasiewicz allows the interpretation of the KL property as a reparametrization of the function values in orer to avoi flatness aroun the critical points. Kuryka [37] extene this property to ifferentiable functions efinable in o-minimal structures. Further extensions to the nonsmooth setting can be foun in [12, 21 23]. One of the remarkable properties of the KL functions is their ubiquity in applications (see [24]). To the class of KL functions belong semi-algebraic, real sub-analytic, semiconvex, uniformly convex an convex functions satisfying a growth conition. We refer the reaer to [11 13,21 24] an the references therein for more on KL functions an illustrating examples. An important role in our convergence analysis will be playe by the following uniform KL property given in [24, Lemma 6]. Lemma 1 Let Ω R n be a compact set an let f : R n R {+ } be a proper an lower semicontinuous function. Assume that f is constant on Ω an that it satisfies the KL property at each point of Ω. Then there exist ε, η > 0 an ϕ Θ η such that for all x Ω an all x in the intersection {x R n : ist(x, Ω) < ε} {x R n : f(x) < f(x) < f(x) + η} (5) 4

5 the inequality hols. ϕ (f(x) f(x)) ist(0, f(x)) 1. (6) In the following we recall the notion of locally absolutely continuous function an state two of its basic properties. Definition 2 (see, for instance, [2, 16]) A function x : [0, + ) R n is sai to be locally absolutely continuous, if it absolutely continuous on every interval [0, T ], where T > 0, which means that one of the following equivalent properties hols: (i) there exists an integrable function y : [0, T ] R n such that x(t) = x(0) + t 0 y(s)s t [0, T ]; (ii) x is continuous an its istributional erivative is Lebesgue integrable on [0, T ]; (iii) for every ε > 0, there exists η > 0 such that for any finite family of intervals I k = (a k, b k ) [0, T ] we have the implication ( ) I k I j = an k b k a k < η = k x(b k ) x(a k ) < ε. Remark 2 (a) It follows from the efinition that an absolutely continuous function is ifferentiable almost everywhere, its erivative coincies with its istributional erivative almost everywhere an one can recover the function from its erivative ẋ = y by the integration formula (i). (b) If x : [0, T ] R n is absolutely continuous for T > 0 an B : R n R n is L-Lipschitz continuous for L 0, then the function z = B x is absolutely continuous, too. This can be easily seen by using the characterization of absolute continuity in Definition 2(iii). Moreover, z is ifferentiable almost everywhere on [0, T ] an the inequality ż(t) L ẋ(t) hols for almost every t [0, T ]. The following two results, which can be interprete as continuous versions of the quasi- Fejér monotonicity for sequences, will play an important role in the asymptotic analysis of the trajectories of the ynamical system investigate in this paper. For their proofs we refer the reaer to [2, Lemma 5.1] an [2, Lemma 5.2], respectively. Lemma 3 Suppose that F : [0, + ) R is locally absolutely continuous an boune from below an that there exists G L 1 ([0, + )) such that for almost every t [0, + ) Then there exists lim t F (t) R. F (t) G(t). t 5

6 Lemma 4 If 1 p <, 1 r, F : [0, + ) [0, + ) is locally absolutely continuous, F L p ([0, + )), G : [0, + ) R, G L r ([0, + )) an for almost every t [0, + ) F (t) G(t), t then lim t + F (t) = 0. Further we recall a ifferentiability result involving the composition of convex functions with absolutely continuous trajectories which is ue to Brézis ( [31, Lemme 3.3, p. 73]; see also [14, Lemma 3.2]). Lemma 5 Let f : R n R {+ } be a proper, convex an lower semicontinuous function. Let x L 2 ([0, T ], R n ) be absolutely continuous such that ẋ L 2 ([0, T ], R n ) an x(t) om f for almost every t [0, T ]. Assume that there exists ξ L 2 ([0, T ], R n ) such that ξ(t) f(x(t)) for almost every t [0, T ]. Then the function t f(x(t)) is absolutely continuous an for almost every t such that x(t) om f we have f(x(t)) = ẋ(t), h h f(x(t)). t We close this section by recalling the following characterization of the proximal point operator of a proper, convex an lower semincontinuous function f : R n R {+ }. For every η > 0 it hols (see for example [18]) p = prox ηf (x) if an only if x p + η f(p), (7) where f enotes the convex subifferential of f. 3 Asymptotic analysis Before starting with the convergence analysis for the ynamical system (2), we woul like to point out that this can be written as { ( ) ẋ(t) = (prox (I η g) I) x(t), (8) x(0) = x 0, where prox (I η g) I is a (2 + ηβ)-lipschitz continuous operator. This follows from the fact that the proximal point operator of a proper, convex an lower semicontinuous function is nonexpansive, i.e., 1-Lipschitz continuous (see for example [18]). Accoring to the global version of the Cauchy-Lipschitz Theorem (see for instance [9, Theorem (b)]), there exists a unique global solution x C 1 ([0, + ), R n ) of the above ynamical system. 3.1 Convergence of the trajectories Lemma 6 Suppose that f + g is boune from below an η > 0 fulfills the inequality ηβ(3 + ηβ) < 1. (9) For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2). following statements hol: Then the 6

7 (a) ẋ L 2 ([0, + ); R n ) an lim t + ẋ(t) = 0; (b) lim t + (f + g) ( ẋ(t) + x(t) ) R. Proof. Let us start by noticing that in the light of the the reformulation in (8) an of Remark 2(b), ẋ is locally absolutely continuous, hence ẍ exists an for almost every t [0, + ) one has ẍ(t) (2 + ηβ) ẋ(t). (10) We fix an arbitrary T > 0. Due to the continuity properties of the trajectory on [0, T ], (10) an the Lipschitz continuity of g, one has x, ẋ, ẍ, g(x) L 2 ([0, T ]; R n ). Further, from the characterization (7) of the proximal point operator we have 1 ẋ(t) g(x(t)) f(ẋ(t) + x(t)) t [0, + ). (11) η Applying Lemma 5 we obtain that the function t f ( ẋ(t)+x(t) ) is absolutely continuous an t f( ẋ(t) + x(t) ) = 1η ẋ(t) g(x(t)), ẍ(t) + ẋ(t) for almost every t [0, T ]. Moreover, it hols t g( ẋ(t) + x(t) ) = g ( ẋ(t) + x(t) ), ẍ(t) + ẋ(t) for almost every t [0, T ]. Summing up the last two equalities, we obtain t (f + g)( ẋ(t) + x(t) ) = 1 η ẋ(t) g(x(t)) + g( ẋ(t) + x(t) ), ẍ(t) + ẋ(t) = 1 ( ẋ(t) 2 ) 1 2η t η ẋ(t) 2 + g ( ẋ(t) + x(t) ) g(x(t)), ẍ(t) + ẋ(t) 1 2η 1 2η = 1 2η ( ẋ(t) 2 ) 1 t η ẋ(t) 2 + β ẋ(t) ẍ(t) + ẋ(t) (12) ( ẋ(t) 2 ) 1 t η ẋ(t) 2 + β(3 + ηβ) ẋ(t) 2 (13) ( ẋ(t) 2 ) [ ] 1 β(3 + ηβ) ẋ(t) 2 t η for almost every t [0, T ], where in (12) we use the Lipschitz continuity of g an in (13) the inequality (10). Altogether, we conclue that for almost every t [0, T ] we have [ (f + g) ( ẋ(t) + x(t) ) + 1 ] [ ] 1 t 2η ẋ(t) 2 + β(3 + ηβ) ẋ(t) 2 0 (14) η an by integration we get (f + g) ( ẋ(t ) + x(t ) ) + 1 2η ẋ(t ) 2 + [ ] 1 T β(3 + ηβ) ẋ(t) 2 t η 0 (f + g) ( ẋ(0) + x 0 ) + 1 2η ẋ(0) 2. (15) 7

8 By using (9) an the fact that f + g is boune from below an by taking into account that T > 0 has been arbitrarily chosen, we obtain Due to (10), this further implies Furthermore, for almost every t [0, + ) we have ẋ L 2 ([0, + ); R n ). (16) ẍ L 2 ([0, + ); R n ). (17) ( ẋ(t) 2 ) = 2 ẋ(t), ẍ(t) ẋ(t) 2 + ẍ(t) 2. t By applying Lemma 4, it follows that lim t + ẋ(t) = 0 an the proof of (a) is complete. From (14), (9) an by using that T > 0 has been arbitrarily chosen, we get [ (f + g) ( ẋ(t) + x(t) ) + 1 ] t 2η ẋ(t) 2 0 for almost every t [0, + ). From Lemma 3 it follows that [ lim (f + g) ( ẋ(t) + x(t) ) + 1 ] t + 2η ẋ(t) 2 exists an it is a real number, hence from lim t + ẋ(t) = 0 the conclusion follows. We efine the limit set of x as ω(x) = {x R n : t k + such that x(t k ) x as k + }. Lemma 7 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2). Then Proof. have ω(x) crit(f + g). Let x ω(x) an t k + be such that x(t k ) x as k +. From (11) we 1 η ẋ(t k) g(x(t k )) + g ( ẋ(t k ) + x(t k ) ) f ( ẋ(t k ) + x(t k ) ) + g ( ẋ(t k ) + x(t k ) ) Lemma 6(a) an the Lipschitz continuity of g ensure that = (f + g) ( ẋ(t k ) + x(t k ) ) k N. (18) 1 η ẋ(t k) g(x(t k )) + g ( ẋ(t k ) + x(t k ) ) 0 as k + (19) an We claim that ẋ(t k ) + x(t k ) x as k +. (20) lim (f + g)( ẋ(t k ) + x(t k ) ) = (f + g)(x). (21) k + 8

9 Due to the lower semicontinuity of f it hols Further, since ẋ(t k ) + x(t k ) = argmin u R n we have the inequality = argmin u R n lim inf k + f( ẋ(t k ) + x(t k ) ) f(x). (22) [ f(u) + 1 2η ( u x(tk ) η g(x(t k )) ) ] 2 [ f(u) + 1 2η u x(t k) 2 + u x(t k ), g(x(t k )) f ( ẋ(t k ) + x(t k ) ) + 1 2η ẋ(t k) 2 + ẋ(t k ), g(x(t k )) f(x) + 1 2η x x(t k) 2 + x x(t k ), g(x(t k )) k N. Taking the limit as k + we erive by using again Lemma 6(a) that which combine with (22) implies lim sup f ( ẋ(t k ) + x(t k ) ) f(x), k + lim k + f( ẋ(t k ) + x(t k ) ) = f(x). By using (20) an the continuity of g we conclue that (21) is true. Altogether, from (18), (19), (20), (21) an the closeness criteria of the limiting subifferential we obtain 0 (f + g)(x) an the proof is complete. Lemma 8 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2) an consier the function H : R n R n R {+ }, H(u, v) = (f + g)(u) + 1 2η u v 2. Then the following statements are true: (H 1 ) for almost every t [0, + ) it hols t H( ẋ(t) + x(t), x(t) ) [ ] 1 (3 + ηβ)β ẋ(t) 2 0 η an lim t + H( ẋ(t) + x(t), x(t) ) R; (H 2 ) for every t [0, + ) it hols z(t) := ( g(x(t)) + g ( ẋ(t) + x(t) ), 1η ) ẋ(t) H ( ẋ(t) + x(t), x(t) ) ] an z(t) ( β + 1 ) ẋ(t) ; η 9

10 (H 3 ) for x ω(x) an t k + such that x(t k ) x as k +, we have H ( ẋ(t k ) + x(t k ), x(t k ) ) H(x, x) as k +. Proof. (H1) follows from Lemma 6. The first statement in (H2) is a consequence of (11) an the relation H(u, v) = ( (f + g)(u) + η 1 (u v) ) {η 1 (v u)} (u, v) R n R n, (23) while the secon one is a consequence of the Lipschitz continuity of g. Finally, (H3) has been shown as intermeiate step in the proof of Lemma 7. Lemma 9 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2) an consier the function H : R n R n R {+ }, H(u, v) = (f + g)(u) + 1 2η u v 2. Suppose that x is boune. Then the following statements are true: (a) ω(ẋ + x, x) crit(h) = {(u, u) R n R n : u crit(f + g)}; (b) lim t + ist ( (ẋ(t) + x(t), x(t) ), ω (ẋ + x, x ) ) = 0; (c) ω ( ẋ + x, x ) is nonempty, compact an connecte; () H is finite an constant on ω ( ẋ + x, x ). Proof. (a), (b) an () are irect consequences Lemma 6, Lemma 7 an Lemma 8. Finally, (c) is a classical result from [34]. We also refer the reaer to the proof of Theorem 4.1 in [6], where it is shown that the properties of ω(x) of being nonempty, compact an connecte are generic for boune trajectories fulfilling lim t + ẋ(t) = 0). Remark 10 Suppose that η > 0 fulfills the inequality (9) an f + g is coercive, that is lim (f + g)(u) = +. u + For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2). Then f + g is boune from below an x is boune. Inee, since f + g is a proper, lower semicontinuous an coercive function, it follows that inf u R n[f(u) + g(u)] is finite an the infimum is attaine. Hence f + g is boune from below. On the other han, from (15) it follows (f + g) ( ẋ(t ) + x(t ) ) (f + g) ( ẋ(t ) + x(t ) ) + 1 ẋ(t ) 2 2η (f + g) ( ẋ(0) + x 0 ) ) + 1 2η ẋ(0) 2 T 0. Since the lower level sets of f +g are boune, the above inequality yiels the bouneness of ẋ + x, which combine with lim t + ẋ(t) = 0 elivers the bouneness of x. 10

11 We come now to the main result of the paper. Theorem 11 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2) an consier the function H : R n R n R {+ }, H(u, v) = (f + g)(u) + 1 2η u v 2. Suppose that x is boune an H is a KL function. Then the following statements are true: (a) ẋ L 1 ([0, + ); R n ); (b) there exists x crit(f + g) such that lim t + x(t) = x. Proof. Accoring to Lemma 9, we can choose an element x crit(f + g) such that (x, x) ω(ẋ + x, x). Accoring to Lemma 8, it follows that lim H( ẋ(t) + x(t), x(t) ) = H(x, x). t + We treat the following two cases separately. I. There exists t 0 such that Since from Lemma 8(H1) we have we obtain for every t t that H ( ẋ(t) + x(t), x(t) ) = H(x, x). t H( ẋ(t) + x(t), x(t) ) 0 t [0, + ), H ( ẋ(t) + x(t), x(t) ) H ( ẋ(t) + x(t), x(t) ) = H(x, x). Thus H ( ẋ(t) + x(t), x(t) ) = H(x, x) for every t t. This yiels by Lemma 8(H1) that ẋ(t) = 0 for almost every t [t, + ), hence x is constant on [t, + ) an the conclusion follows. II. For every t 0 it hols H ( ẋ(t) + x(t), x(t) ) > H(x, x). Take Ω = ω(ẋ + x, x). In virtue of Lemma 9(c) an () an since H is a KL function, by Lemma 1, there exist positive numbers ɛ an η an a concave function ϕ Θ η such that for all one has (x, y) {(u, v) R n R n : ist((u, v), Ω) < ɛ} {(u, v) R n R n : H(x, x) < H(u, v) < H(x, x) + η} (24) ϕ (H(x, y) H(x, x)) ist((0, 0), H(x, y)) 1. (25) Let t 1 0 be such that H ( ẋ(t) + x(t), x(t) ) ( < H(x, x) + η for all t t 1. Since (ẋ(t) ) ) ( (ẋ(t) lim t + ist + x(t), x(t), Ω = 0, there exists t 2 0 such that ist + 11

12 x(t), x(t) ) ), Ω < ɛ for all t t 2. Hence for all t T := max{t 1, t 2 }, ( ẋ(t) + x(t), x(t) ) belongs to the intersection in (24). Thus, accoring to (25), for every t T we have ϕ ( H ( ẋ(t) + x(t), x(t) ) ) ( H(x, x) ist (0, 0), H ( ẋ(t) + x(t), x(t) )) 1. (26) By applying Lemma 8(H2) we obtain for almost every t [T, + ) (β + η 1 ) ẋ(t) ϕ ( H ( ẋ(t) + x(t), x(t) ) ) H(x, x) 1. (27) From here, by using Lemma 8(H1) an that ϕ > 0 an ( t ϕ H ( ẋ(t) + x(t), x(t) ) ) H(x, x) = ϕ ( H ( ẋ(t) + x(t), x(t) ) H(x, x)) t H( ẋ(t) + x(t), x(t) ), we euce that for almost every t [T, + ) it hols ( t ϕ H ( ẋ(t) + x(t), x(t) ) ) H(x, x) ( β + η 1) 1 [ ] 1 (3 + ηβ)β ẋ(t). (28) η Since ϕ is boune from below, by taking into account (9), it follows ẋ L 1 ([0, + ); R n ). From here we obtain that lim t + x(t) exists an this closes the proof. Since the class of semi-algebraic functions is close uner aition (see for example [24]) an (u, v) c u v 2 is semi-algebraic for c > 0, we can stat the following irect consequence of the previous theorem. Corollary 12 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2). Suppose that x is boune an f + g is semi-algebraic. Then the following statements are true: (a) ẋ L 1 ([0, + ); R n ); (b) there exists x crit(f + g) such that lim t + x(t) = x. Remark 13 The construction of the function H, which we use in the above arguments in orer to erive a escent property, has been inspire by the ecrease property obtaine in (14). Similar regularizations of the objective function of (1) have been consiere also in [30, 40], in the context of the investigation of non-relaxe forwar-backwar methos involving inertial an memory effects in the nonconvex setting. Remark 14 Some comments concerning conition (9) that involves the step size an the Lipschitz constant of the smooth function are in orer. In the full convex setting, it has been prove by Abbas an Attouch in [1, Theorem 5.2(2)] that in orer to obtain weak convergence of the trajectory to an optimal solution of (1) no restriction on the step size is neee. This surprising fact which is closely relate to the continuous case, as for obtaining convergence for the time-iscretize version (4) one usually has to assume that η is in the interval (0, 2β). Working in the nonconvex setting, one has to even strengthen this assumption an it will be a question of future research to fin out if one can relax the restriction (9). Another question of future interest will be to properly choose the step size when the Lipschitz constant of the graient is not known, eventually by making use of backtracking strategies, as is was alreay one in the convex setting. 12

13 3.2 Convergence rates In this subsection we investigate the convergence rates of the trajectories generate by the ynamical system (2). When solving optimization problems involving KL functions, convergence rates have been prove to epen on the so-calle Lojasiewicz exponent (see [11,21,33,38]). The main result of this subsection refers to the KL functions which satisfy Definition 1 for ϕ(s) = Cs 1 θ, where C > 0 an θ (0, 1). We recall the following efinition consiere in [11]. Definition 3 Let f : R n R {+ } be a proper an lower semicontinuous function. The function f is sai to have the Lojasiewicz property, if for every x crit f there exist C, ε > 0 an θ (0, 1) such that f(x) f(x) θ C x for every x fulfilling x x < ε an every x f(x). (29) Accoring to [12, Lemma 2.1 an Remark 3.2(b)], the KL property is automatically satisfie at any noncritical point, fact which motivates the restriction to critical points in the above efinition. The real number θ in the above efinition is calle Lojasiewicz exponent of the function f at the critical point x. Theorem 15 Suppose that f + g is boune from below an η > 0 fulfills the inequality (9). For x 0 R n, let x C 1 ([0, + ), R n ) be the unique global solution of (2) an consier the function H : R n R n R {+ }, H(u, v) = (f + g)(u) + 1 2η u v 2. Suppose that x is boune an H satisfies Definition 1 for ϕ(s) = Cs 1 θ, where C > 0 an θ (0, 1). Then there exists x crit(f + g) such that lim t + x(t) = x. Let θ be the Lojasiewicz exponent of H at (x, x) crit H, accoring to the Definition 3. Then there exist a, b, c, > 0 an t 0 0 such that for every t t 0 the following statements are true: (a) if θ (0, 1 2 ), then x converges in finite time; (b) if θ = 1 2, then x(t) x a exp( bt); (c) if θ ( 1 1 θ 2, 1), then x(t) x (ct + ) ( 2θ 1). Proof. We efine for every t 0 (see also [21]) It is immeiate that σ(t) = + Inee, this follows by noticing that for T t t ẋ(s) s for all t 0. x(t) x σ(t) t 0. (30) x(t) x = x(t ) x x(t ) x + T t T t ẋ(s)s ẋ(s) s, 13

14 an by letting afterwars T +. We assume that for every t 0 we have H ( ẋ(t) + x(t), x(t) ) > H(x, x). As seen in the proof of Theorem 11, in the other case the conclusion follows automatically. Furthermore, by invoking again the proof of above-name result, there exist t 0 0 an M > 0 such that for almost every t t 0 (see (28)) an M ẋ(t) + t [( H ( ẋ(t) + x(t), x(t) ) H(x, x))] 1 θ 0 ( ẋ(t) + x(t), x(t) ) (x, x) < ε. We erive by integration (for T t t 0 ) M T t [( ẋ(s) s + H ( ẋ(t ) + x(t ), x(t ) ) )] 1 θ H(x, x) [( H ( ẋ(t) + x(t), x(t) ) H(x, x))] 1 θ, hence Mσ(t) [( H ( ẋ(t) + x(t), x(t) ) H(x, x))] 1 θ t t0. (31) Since θ is the Lojasiewicz exponent of H at (x, x), we have H ( ẋ(t) + x(t), x(t) ) H(x, x) θ C x x H ( ẋ(t) + x(t), x(t) ) for every t t 0. Accoring to Lemma 8(H2), we can fin a constant N > 0 an x (t) H ( ẋ(t) + x(t), x(t) ) such that for every t [0, + ) x (t) N ẋ(t). From the above two inequalities we erive for almost every t [t 0, + ) which combine with (31) yiels Since H ( ẋ(t) + x(t), x(t) ) H(x, x) θ C N ẋ(t), Mσ(t) (C N ẋ(t) ) 1 θ θ. (32) σ(t) = ẋ(t) (33) we conclue that there exists α > 0 such that for almost every t [t 0, + ) If θ = 1 2, then σ(t) α ( σ(t) ) θ 1 θ. (34) σ(t) ασ(t) for almost every t [t 0, + ). By multiplying with exp(αt) an integrating afterwars from t 0 to t, it follows that there exist a, b > 0 such that σ(t) a exp( bt) t t 0 14

15 an the conclusion of (b) is immeiate from (30). Assume that 0 < θ < 1 2. We obtain from (34) for almost every t [t 0, + ). By integration we get t ) (σ 1 2θ 1 θ α 1 2θ 1 θ ( σ(t) ) 1 2θ 1 θ αt + β t t 0, where α > 0. Thus there exists T 0 such that σ(t ) 0 t T, which implies that x is constant on [T, + ). Finally, suppose that 1 2 < θ < 1. We obtain from (34) By integration one erives t ) (σ 1 2θ 1 θ α 2θ 1 1 θ. 1 θ σ(t) (ct + ) ( 2θ 1) t t0, where c, > 0, an (c) follows from (30). Acknowlegement We are grateful to the anonymous referees for suggestions which improve the presentation of the manuscript. References [1] B. Abbas, H. Attouch, Dynamical systems an forwar-backwar algorithms associate with the sum of a convex subifferential an a monotone cocoercive operator, Optimization 64(10), , 2015 [2] B. Abbas, H. Attouch, B.F. Svaiter, Newton-like ynamics an forwar-backwar methos for structure monotone inclusions in Hilbert spaces, Journal of Optimization Theory an its Applications 161(2), , 2014 [3] F. Alvarez, On the minimizing property of a secon orer issipative system in Hilbert spaces, SIAM Journal on Control an Optimization 38(4), , 2000 [4] F. Alvarez, Weak convergence of a relaxe an inertial hybri projection-proximal point algorithm for maximal monotone operators in Hilbert space, SIAM Journal on Optimization 14(3), , 2004 [5] F. Alvarez, H. Attouch, An inertial proximal metho for maximal monotone operators via iscretization of a nonlinear oscillator with amping, Set-Value Analysis 9(1-2), 311,

16 [6] F. Alvarez, H. Attouch, J. Bolte, P. Reont, A secon-orer graient-like issipative ynamical system with Hessian-riven amping. Application to optimization an mechanics, Journal e Mathématiques Pures et Appliquées (9) 81(8), , 2002 [7] A.S. Antipin, Minimization of convex functions on convex sets by means of ifferential equations, (Russian) Differentsial nye Uravneniya 30(9), , 1994; translation in Differential Equations 30(9), , 1994 [8] H. Attouch, F. Alvarez, The heavy ball with friction ynamical system for convex constraine minimization problems, in: Optimization (Namur, 1998), 25 35, in: Lecture Notes in Economics an Mathematical Systems 481, Springer, Berlin, 2000 [9] H. Attouch, G. Buttazzo, G. Michaille, Variational Analysis in Sobolev an BV Spaces: Applications to PDEs an Optimization, Secon Eition, MOS-SIAM Series on Optimization, Philaelphia, 2014 [10] H. Attouch, M. Marques Alves, B.F. Svaiter, A ynamic approach to a proximal- Newton metho for monotone inclusions in Hilbert spaces, with complexity O(1/n 2 ), Journal of Convex Analyis 23(1), , 2016 [11] H. Attouch, J. Bolte, On the convergence of the proximal algorithm for nonsmooth functions involving analytic features, Mathematical Programming 116(1-2) Series B, 5 16, 2009 [12] H. Attouch, J. Bolte, P. Reont, A. Soubeyran, Proximal alternating minimization an projection methos for nonconvex problems: an approach base on the Kuryka- Lojasiewicz inequality, Mathematics of Operations Research 35(2), , 2010 [13] H. Attouch, J. Bolte, B.F. Svaiter, Convergence of escent methos for semi-algebraic an tame problems: proximal algorithms, forwar-backwar splitting, an regularize Gauss-Seiel methos, Mathematical Programming 137(1-2) Series A, , 2013 [14] H. Attouch, M.-O. Czarnecki, Asymptotic behavior of couple ynamical systems with multiscale aspects, Journal of Differential Equations 248(6), , 2010 [15] H. Attouch, X. Gouou, P. Reont, The heavy ball with friction metho. I. The continuous ynamical system: global exploration of the local minima of a real-value function by asymptotic analysis of a issipative ynamical system, Communications in Contemporary Mathematics 2(1), 1 34, 2000 [16] H. Attouch, B.F. Svaiter, A continuous ynamical Newton-like approach to solving monotone inclusions, SIAM Journal on Control an Optimization 49(2), , 2011 [17] S. Banert, R.I. Boţ, A forwar-backwar-forwar ifferential equation an its asymptotic properties, Journal of Convex Analysis 25(2), 2018 [18] H.H. Bauschke, P.L. Combettes, Convex Analysis an Monotone Operator Theory in Hilbert Spaces, CMS Books in Mathematics, Springer, New York,

17 [19] H.H. Bauschke, P.L. Combettes, D.R. Luke, Phase retrieval, error reuction algorithm, an Fienup variants: a view from convex optimization, Journal of the Optical Society of America A. Optics, Image Science, an Vision 19(7), , 2002 [20] J. Bolte, Continuous graient projection metho in Hilbert spaces, Journal of Optimization Theory an its Applications 119(2), , 2003 [21] J. Bolte, A. Daniiliis, A. Lewis, The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgraient ynamical systems, SIAM Journal on Optimization 17(4), , 2006 [22] J. Bolte, A. Daniiliis, A. Lewis, M. Shota, Clarke subgraients of stratifiable functions, SIAM Journal on Optimization 18(2), , 2007 [23] J. Bolte, A. Daniiliis, O. Ley, L. Mazet, Characterizations of Lojasiewicz inequalities: subgraient flows, talweg, convexity, Transactions of the American Mathematical Society 362(6), , 2010 [24] J. Bolte, S. Sabach, M. Teboulle, Proximal alternating linearize minimization for nonconvex an nonsmooth problems, Mathematical Programming Series A (146)(1 2), , 2014 [25] R.I. Boţ, E.R. Csetnek, An inertial Tseng s type proximal algorithm for nonsmooth an nonconvex optimization problems, Journal of Optimization Theory an Applications 171(2), , 2016 [26] R.I. Boţ, E.R. Csetnek, A ynamical system associate with the fixe points set of a nonexpansive operator, Journal of Dynamics an Differential Equations 29(1), , 2017 [27] R.I. Boţ, E.R. Csetnek, Approaching the solving of constraine variational inequalities via penalty term-base ynamical systems, Journal of Mathematical Analysis an Applications 435(2), , 2016 [28] R.I. Boţ, E.R. Csetnek, Secon orer forwar-backwar ynamical systems for monotone inclusion problems, SIAM Journal on Control an Optimization 54(3), , 2016 [29] R.I. Boţ, E.R. Csetnek, Convergence rates for forwar-backwar ynamical systems associate with strongly monotone inclusions, Journal of Mathematical Analysis an Applications, DOI: /j.jmaa , 2016 [30] R.I. Boţ, E.R. Csetnek, S. László, An inertial forwar-backwar algorithm for the minimization of the sum of two nonconvex functions, EURO Journal on Computational Optimization 4, 3 25, 2016 [31] H. Brézis, Opérateurs maximaux monotones et semi-groupes e contractions ans les espaces e Hilbert, North-Hollan Mathematics Stuies No. 5, Notas e Matemática (50), North-Hollan/Elsevier, New York,

18 [32] E. Chouzenoux, J.-C. Pesquet, A. Repetti, Variable metric forwar-backwar algorithm for minimizing the sum of a ifferentiable function an a convex function, Journal of Optimization Theory an its Applications 162(1), , 2014 [33] P. Frankel, G. Garrigos, J. Peypouquet, Splitting methos with variable metric for Kuryka- Lojasiewicz functions an general convergence rates, Journal of Optimization Theory an its Applications 165(3), , 2015 [34] A. Haraux, Systèmes Dynamiques Dissipatifs et Applications, Recherches en Mathématiques Appliquées 17, Masson, Paris, 1991 [35] A. Haraux, M. Jenoubi, Convergence of solutions of secon-orer graient-like systems with analytic nonlinearities, Journal of Differential Equations 144(2), , 1998 [36] R. Hesse, D.R. Luke, S. Sabach, M.K. Tam, Proximal heterogeneous block input-output metho an application to blin ptychographic iffraction imaging, SIAM Journal on Imaging Sciences 8(1), , 2015 [37] K. Kuryka, On graients of functions efinable in o-minimal structures, Annales e l institut Fourier (Grenoble) 48(3), , 1998 [38] S. Lojasiewicz, Une propriété topologique es sous-ensembles analytiques réels, Les Équations aux Dérivées Partielles, Éitions u Centre National e la Recherche Scientifique Paris, 87 89, 1963 [39] B. Morukhovich, Variational Analysis an Generalize Differentiation, I: Basic Theory, II: Applications, Springer-Verlag, Berlin, 2006 [40] P. Ochs, Y. Chen, T. Brox, T. Pock, ipiano: Inertial proximal algorithm for nonconvex optimization, SIAM Journal on Imaging Sciences 7(2), , 2014 [41] R.T. Rockafellar, R.J.-B. Wets, Variational Analysis, Funamental Principles of Mathematical Sciences 317, Springer-Verlag, Berlin, 1998 [42] L. Simon, Asymptotics for a class of nonlinear evolution equations, with applications to geometric problems, Annals of Mathematics (2) 118, ,