Adaptive discretization and first-order methods for nonsmooth inverse problems for PDEs

Transcription

1 Adaptive discretization and first-order methods for nonsmooth inverse problems for PDEs Christian Clason Faculty of Mathematics, Universität Duisburg-Essen joint work with Barbara Kaltenbacher, Tuomo Valkonen, Daniel Wachsmuth Turing/LMS Worshop Inverse Problems and Data Science Edinburgh, May 9, / 31

2 Motivation Tikhonov regularization min F(S(u) u U yδ ) + G α (u) F : Y R discrepancy term, y δ noisy data G α : X R regularization term, α > 0 S : U X Y forward operator, involves solution to PDE Adaptive regularization and discretization: regularization: u δ α(δ) u with S(u ) = y for δ 0 discretization: FE approximation u δ α,h uδ α for h 0 goal: combined choice (α, h)(δ) for nonsmooth functionals 2 / 31

3 Motivation Primal-dual extragradient method: Here: first-order algorithm for nonsmooth convex problems with linear operators [Chambolle/Pock 2011] very popular in imaging (TV denoising, deblurring,...) acceleration (Nesterov, O(1/k 2 ) convergence) version for nonlinear operators [Valkonen 2014] application to parameter identification for PDEs function space algorithm Difficulty: convergence proof requires set-valued analysis in infinite-dimensional spaces 3 / 31

4 Model problems L 1 -fitting min u S(u) y δ L 1 + α 2 u 2 L 2 L -fitting 1 min u 2 u 2 L s. t. S(u)(x) y δ (x) δ a. e. in Ω 2 S : U L 2 (Ω) L 2 (Ω), S(u) =: y satisfies { y + uy = f ν y = 0 4 / 31

5 1 Overview 2 Adaptive discretization 3 Algorithm 4 Set-valued analysis Metric regularity Stability of saddle points 5 Application to parameter identification L 1 fitting L fitting 6 Numerical examples 5 / 31

6 Adaptive discretization Tikhonov functional for α > 0 u δ α = arg min u U J α(u) := F(S(u) y δ ) + G α (u) Discretized Tikhonov functional for α > 0, h > 0 u δ α,h = arg min u U J α,h(u) := F(S h (u) y δ ) + G α (u) S h (conforming) finite element approximation of S S h (u) S(u) for h 0 for any u U 6 / 31

7 Adaptive discretization Choose for given δ > 0: 1 α = α(δ) such that δ F(S(u δ α) y δ ) τδ 2 h = h(α, δ) such that Then, for all δ > 0: F(S(u δ α,h yδ )) F(S(u δ α) y δ ) c 1 δ J α,h (u δ α,h ) J α(u δ α) c 2 δ G α (u δ α,h ) G α(u ), F(S h (u δ α,h ) yδ ) τδ no direct control of u δ α,h uδ α X needed! convergence, rates for u δ α,h u X 0 for δ 0 (standard) 6 / 31

8 Functional error estimator Goal: a posteriori error estimator for discrepancy, functional accuracy Here: linear operator S = A 1 for elliptic differential operator A Functional error estimator: If there exist φ α, ψ with 1 λ(1 λ)ψ(y 1, y 2 ) λf(y 1 ) + (1 λ)f(y 2 ) F(λy 1 + (1 λ)y 2 ) 2 λ(1 λ)φ α (u 1, u 2 ) λg α (u 1 ) + (1 λ)g α (u 2 ) G α (λu 1 + (1 λ)u 2 ) Then, for all u U, y Y: ψ(su δ α, Su) + φ α (u δ α, u) F(Su) + F ( y) + G α (v) + G α(s y) F, G α Fenchel conjugates, S adjoint proof by strong convexity, weak duality apply for u = u δ α,h, y = Suδ α,h 7 / 31

9 Application to quadratic penalty X, Y Hilbert space: F(y) = 1 2 y yδ 2 Y G α (u) = α 2 u 2 X ψ(y 1, y 2 ) = 1 2 y 1 y 2 2 Y φ α (u 1, u 2 ) = α 2 u 1 u 2 2 X F (y) = 1 2 y 2 Y (y, y δ ) (G α ) (u) = 1 2α u 2 X Functional error estimate for ū = u δ α, ū h = u δ α,h α ū ū h X + Sū Sū h 1 α αū h + S (Sū h y δ ) 2 X + Sū Sū h 2 Y 8 / 31

10 Application to quadratic penalty Residual estimate for right-hand side for ȳ h = S h ū h, ȳ = Sū α ū ū h X + ȳ ȳ h 2 α S ρ w + ρ u 2 X + 4 Sρ y 2 Y J α (ū h ) J α (ū) 1 2α S ρ w + ρ u 2 X + (Sū h y δ, Sρ y ) Y ρ w := A w h + ȳ h y δ ρ u := αū h w h = αū h S h (ȳ h y δ ) ρ y := Aȳ h + ū h residuals of optimality conditions standard element-based a posteriori estimators for rhs 8 / 31

11 Application to norm penalty X Banach space (e.g., X = M(Ω) sparsity) G α (u) = α u X φ α (u 1, u 2 ) = 0 (G α ) (u ) = ι αbx (u ) := { 0 u x α, Functional estimate with y = κȳ h + (1 κ)y δ else Sū Sū h 2 Y 2α ū h X + 2 ū h, S (y κ(ȳ h y δ )) + Sū h κȳ h + (1 κ)y δ 2 Y for κ such that S y X α (can be estimated) 9 / 31

12 Application to norm penalty Residual estimate for right-hand side for w h = S h (y δ ȳ h ) ȳ h ȳ 2 Y 4(α ū h X ū h, w h ) + 4κ ū h, w h S (y δ ȳ h ) + 4(1 κ) ū h, w h + 4 Sρ y + (κ 1)(ȳ h y δ ) 2 Y J α (ū h ) J α (ū) α ū h X ū h, w h + κ ū h, w h S (y δ ȳ h ) + (1 κ) ū h, w h + Sρ y + (κ 1)(ȳ h g δ ) Y ȳ h y δ Y (non-)standard element-based a posteriori estimators for rhs X = C 0 (Ω) L a posteriori estimators for κ no bound on ū h ū, not needed for convergence! 9 / 31

13 1 Overview 2 Adaptive discretization 3 Algorithm 4 Set-valued analysis Metric regularity Stability of saddle points 5 Application to parameter identification L 1 fitting L fitting 6 Numerical examples 10 / 31

14 Problem statement min F(K(u)) + G(u) u X F : Y R, G : X R X, Y Hilbert spaces convex, lower semicontinuous K C 2 (X, Y) nonlinear (here: K(u) = S(u) y δ ) saddle point formulation: min sup G(u) + K(u), v F (v) u X v Y F : Y R Fenchel conjugate 11 / 31

15 Algorithm K linear: u k+1 = prox τg (u k τk v k ) ū k+1 = 2u k+1 u k v k+1 = prox σf (v k + σkū k+1 ) σ, τ step sizes, στ < K 2 prox σf (v) = arg min u 1 2σ u v 2 + F(u) proximal mapping 12 / 31

16 Algorithm K nonlinear: u k+1 = prox τg (u k τk (u k ) v k ) ū k+1 = 2u k+1 u k v k+1 = prox σf (v k + σk(ū k+1 )) σ, τ step sizes, στ < sup u BR K (u) 2 prox σf (v) = arg min u K (u) Fréchet derivative, 1 2σ u v 2 + F(u) proximal mapping K (u) adjoint 12 / 31

17 Algorithm K nonlinear, accelerated: u k+1 = prox τk G(u k τ k K (u k ) v k ) ω k = 1/ 1 + 2cτ k τ k+1 = ω k τ k σ k+1 = σ k /ω k ū k+1 = u k+1 + ω k (u k+1 u k ) v k+1 = prox σk+1 F (vk + σ k+1 K(ū k+1 )) σ, τ step sizes, σ 0 τ 0 < sup u BR K (u) 2 prox σf (v) = arg min u K (u) Fréchet derivative, c 0 acceleration parameter 1 2σ u v 2 + F(u) proximal mapping K (u) adjoint 12 / 31

18 Convergence Theorem Iterates converge locally to saddle point (ū, v) if 1 G is c G -strongly convex (here: c G = 1) 2 c = c n [0, c G ), c n = 0 for n > N N (finite acceleration) 3 metric regularity around saddle point (cf. [Valkonen 2014]) allows for inexact evaluation of K ( adaptive discretization) Difficulty: metric regularity in function spaces requires infinite-dimensional set-valued analysis here: only rough outline, no details 13 / 31

19 1 Overview 2 Adaptive discretization 3 Algorithm 4 Set-valued analysis Metric regularity Stability of saddle points 5 Application to parameter identification L 1 fitting L fitting 6 Numerical examples 14 / 31

20 Metric regularity Saddle-point problem min sup G(u) + K(u), v F (v) u X v Y Primal-dual optimality conditions { K(ū) F ( v) K (ū) v G(ū) 15 / 31

21 Metric regularity Primal-dual optimality conditions { K(ū) F ( v) K (ū) v G(ū) Set inclusion for H : L 2 (Ω) 2 L 2 (Ω) 2 0 Hū(ū, v) := ( G(ū) + K (ū) ) v F ( v) K(ū) 15 / 31

22 Metric regularity Set inclusion for H : L 2 (Ω) 2 L 2 (Ω) 2 0 Hū(ū, v) := ( G(ū) + K (ū) ) v F ( v) K(ū) Metric regularity at (ū, v) inf q (ū, v) L w for all w ρ q: w Hū(q) interpretation: small perturbation w of 0 small perturbation q of saddle point (ū, v) Lipschitz property for set-valued H 1 ū at ((ū, v), 0) 15 / 31

23 Metric regularity Metric regularity at (ū, v) inf q (ū, v) L w for all w ρ q: w Hū(q) Mordukhovich criterion { } L R = inf sup D R(q w ) q B((ū, v), t), w R(q ) B(w, t) t>0 Aubin constant L H 1 is minimal choice of L D R regular coderivative of R(= H 1 ) (cf. L = f for f C 1 ) set-valued analysis in function spaces 15 / 31

24 Set-valued analysis in function space Difficulties: Here: multiple non-equivalent concepts (regular, limiting) calculus not tight set-valued mappings from subdifferentials of pointwise functionals infinite-dimensional (regular) derivatives pointwise via nice finite-dimensional (regular, graphical) derivatives cf. pointwise Fenchel conjugates, subdifferentials [Ekeland] 16 / 31

25 Application to saddle point inclusion 0 Hū(ū, v) = ( G(ū) + K (ū) ) v F ( v) K(ū) ( D[ G](u ξ K DHū(q w)( q) = (ū) v)( u) + K (ū) ) v D[ F ](v η + K (ū)u + cū)( v) K (ū) u { T q q + V(q w) DHū(q w)( q) q V(q w) = otherwise q = (u, v), w = (ξ, η), cū = K(ū) K (ū)ū T q linear Operator (independent of w), V(q w) cone D Hū(q w) = [ DHū(q w)] + upper adjoint of convexification 17 / 31

26 Mordukhovich criterion DHū(q w)( q) = { T q q + V(q w) q V(q w) otherwise Then: Aubin constant L H c < iff sup t>0 inf ( w,z) W t (q w), w >0 T q w z w c 1 > 0 W t (q w) = { V(q w ) V(q w ) w Hū(q ), q q < t, w w < t } 18 / 31

27 1 Overview 2 Adaptive discretization 3 Algorithm 4 Set-valued analysis Metric regularity Stability of saddle points 5 Application to parameter identification L 1 fitting L fitting 6 Numerical examples 19 / 31

28 Application to PDEs Primal-dual optimality conditions { S(ū) y δ F ( v) S (ū) v = ū Metric regularity around (ū, v) if either 1 sup inf t>0 { S (ū)s (ū) z ν z (z, ν) V t F ( v y δ S(ū)), z 0 2 Moreau Yosida regularization: F F γ := F + γ finite-dimensional data: Y Y h } > 0 In case 1: S (ū) z c z for z V t F ( v y δ S(ū)) necessary! 20 / 31

29 L 1 fitting 1 min u α S(u) yδ L u 2 L 2 F(y) = α 1 y(x) dx f (z) = ι [ α 1,α 1 ](z) Ω S : U L 2 (Ω) L 2 (Ω), S(u) =: y satisfies { y + uy = f ν y = 0 21 / 31

30 L 1 fitting: metric regularity Here: z V F t (v η) if {0} v (x) = α 1 and η (x) 0 z(x) sign v (x)[0, ) v (x) = α 1 and η (x) = 0 R v (x) < α 1 and η (x) = 0 for some v v t, η η t S compact operator: S (ū) z c z only holds for z = 0 η = S(ū) y δ, α v sign η in general not satisfied! 22 / 31

31 L 1 fitting: algorithm z k+1 = S (u k ) v k u k+1 = τ k (u k τ k z k+1 ) ω k = 1/ 1 + 2cτ k τ k+1 = ω k τ k σ k+1 = σ k /ω k ū k+1 = u k+1 + ω k (u k+1 u k ) ( ) v k+1 = proj 1 [ α 1,α 1 ] 1+σ k+1 γ (vk + σ k+1 (S(ū k+1 ) y δ )) S (u k ) v k solution of adjoint equation proj C pointwise projection on convex set C R Moreau Yosida parameter γ 0 local convergence if γ > 0 or finite-dimensional 23 / 31

32 L fitting 1 min u 2 u 2 L s. t. S(u)(x) y δ (x) δ a. e. in Ω 2 F(y) = ι { y(x) δ} (y) f (z) = δ z S : U L 2 (Ω) L 2 (Ω), S(u) =: y satisfies { y + uy = f ν y = 0 for t = 0: z(x) = 0 if S(ū)(x) y δ (x) < δ estimate unlikely 24 / 31

33 L fitting: algorithm z k+1 = S (u k ) v k u k+1 1 = (u k τ k z k+1 ) 1 + τ k ω k = 1/ 1 + 2cτ k τ k+1 = ω k τ k σ k+1 = σ k /ω k ū k+1 = u k+1 + ω k (u k+1 u k ) v k+1 = v k + σ k+1 (S(ū k+1 ) y δ ) v k+1 1 = 1+σ k+1 γ ( vk+1 δσ) + sign( v k+1 ) local convergence if γ > 0 or finite-dimensional 25 / 31

34 1 Overview 2 Adaptive discretization 3 Algorithm 4 Set-valued analysis Metric regularity Stability of saddle points 5 Application to parameter identification L 1 fitting L fitting 6 Numerical examples 26 / 31

35 L 1 fitting Ω = [ 1, 1], FE (P 1 P 0 ) discretization of (y, u) random impulsive noise: { y δ y (x) + y ξ(x) with probability 0.3 (x) = y (x) else y = S(u ), ξ(x) N(0, 0.1), α = 10 2 σ 0 = L 1, τ 0 = 0.99 L 1, L = S(u 0 ) / u 0 γ = 10 12, u 0 1, v 0 = 0 (no warmstart!) compare c {0, }, N {100, 1000, 10000} 27 / 31

36 L 1 fitting: acceleration (same data) c G = 0 c G 1 Jγ(u i ) / 31

37 L 1 fitting: discretization (avg. of 10) 10 3 N = 100 N = 1000 N = Jγ(u i ) / 31

38 L fitting Ω = [ 1, 1], FE (P 1 P 0 ) discretization of (y, u) quantization noise: round y to n b = 11 equidistant values γ = 10 12, σ 0 = L 1, τ 0 = 0.99 L 1, L = S(u 0 ) / u 0 u 0 1, v 0 = 0 (no warmstart!) compare c {0, }, N {100, 1000, 10000} 29 / 31

39 L fitting: acceleration (same data) c G = 0 c G 1 Jγ(u i ) / 31

40 L fitting: discretization (same data) N = 100 N = 1000 N = Jγ(u i ) / 31

41 Conclusion Primal-dual extragradient methods in function space: can be accelerated analyzed using set-valued analysis in function space requires Moreau Yosida regularization no norm gap, continuation needed; mesh-independence Outlook: full acceleration partial stability (w.r.t. primal variable only) adaptive discretization using functional estimators Preprints/code: 31 / 31