A numerical solution of the constrained weighted energy problem and its relation to rational Lanczos iterations

A numerical solution of the constrained weighted energy problem and its relation to rational Lanczos iterations Karl Deckers, Andrey Chesnokov, and Marc Van Barel Department of Computer Science, Katholieke Universiteit Leuven, Heverlee, Belgium June 18, 2009 Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 1/40

Outline 1 Preliminaries Logarithmic potential and WEP Constrained weighted energy problem 2 Algorithm Main loop Discretization Numerical examples 3 Connection with the rational Lanczos algorithm The rational Lanczos method Characterization of converged Ritz values Numerical examples Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 2/40

Logarithmic potential and energy Definition M(E) = {µ : µ(c) = 1 and supp(µ) E}; Logarithmic potential of a measure µ: U µ 1 (z) = log z z dµ(z ); Logarithmic energy of a measure µ: 1 I(µ) = log z z dµ(z )dµ(z). In remainder: E = J j=1 [a j,b j ] R, with < a 1 < b 1 < a 2 <... < b J 1 < a J < b J <. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 4/40

Weighted energy problem WEP min I(µ ν), µ M(E) for a given positive measure ν, with compact supp(ν) (C \ E) bounded away from E, and ν(c) = s [0,1]. Theorem (equivalent formulation of the WEP) Let µ ν M(E) be a solution of the WEP. Then U µν ν (z) = C ν, z E U µν ν (z) < C ν, z / E. The solution µ ν is unique and is called the balayage-measure of the probability measure η = ν + (1 s)δ (µ ν = Bal(η,E)). Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 5/40

Example Let E = [ 1,1] and ν = 1 3 (δ α 1 + δ α2 + δ α3 ), where α 1 = 0.6 + 0.01i, α 2 = 0.4 0.2i and α 3 = 1.01..1e2 y 2 1 1 2 0 0.2 0.4 1 0.8 0.6 0.4 0.2 1. 0.2 0.4 0.6 0.8 1 x 0.6 0.8.1 1 Figure: Density of µ ν. Figure: Weighted potential U µν ν. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 6/40

Constrained weighted energy problem CWEP min I(µ ν), tµ σ, t (0,1), µ M(E) for a given σ M(E), with I(σ) <. Theorem (equivalent formulation of the CWEP) Assume U σ (z) continuous and real-valued, and let µ ν t be a solution of the CWEP. Then The solution µ ν t is unique. U µν t ν (z) = C ν t, z supp(σ tµν t ) U µν t ν (z) C ν t, z C. Note: supp(σ tµ ν t ) = the set where tµ ν t < σ. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 8/40

Key lemma to devise the algorithm Lemma Suppose for some µ M(E) (not necessarily with tµ σ): U µ ν (z) = C, z supp(σ tµ) U µ ν (z) C, z C. Then supp(σ tµ ν t ) supp((σ tµ) + ), and µ ν t µ on supp(σ tµ ν t ). Consequence tµ ν t = σ on the region where tµ σ. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 9/40

Sketch of the main Algorithm Sketch Supposing ν, σ and E are given. Solve the CWEP based on the previous lemma: 1 Find µ (1) M(E) s.t. U µ(1) ν = C (1) on S (1) = E. 2 Put µ (2) = σ/t on the region where tµ (1) σ. 3 Require that U µ(2) ν = C (2) on the remaining region S (2) S (1). 4 Repeat until µ (k) σ/t, i.e., until S (k) = S (k 1). µ t = µ (k). Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 11/40

Algorithm 1: continuous version of the CWEP algorithm begin I = supp(σ) J = µ = while µ σ/t do µ J = { 1 t σ J U µ I (z) = C U µ J(z) + U ν (z), solve µ I (C) = 1 µ J (C) I = {tµ < σ} J = {tµ σ} end return µ t = µ end z I Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 12/40

Correctness of the Algorithm Output: µ t s.t. tµ t σ and U µt ν = C t on supp(σ tµ t ) U µt ν C t outside supp(σ tµ t )? Or, equivalently, µ t = µ ν t? Theorem For every k, U µ(k) ν C (k) outside S (k) = supp(σ tµ (k) ). Will the algorithm ever stop? Not necessarily, but the discrete version always stops. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 13/40

Discretization Similar as for the C(uW)EP (Helsen, Van Barel ( 06)) {y 1,y 2,...,y N } discretization of supp(σ). Consider dµ to be piecewise linear: dµ(y) = (a j y + b j )dy for y [y j 1,y j ] Let v = [µ 1,...,µ N ] T, with µ j = µ (y j ). Let s = [σ 1,...,σ N ] T, with σ j = σ (y j ). Let m s.t. m T v = µ(c) for every µ M(E). Suppose an explicit representation κ(x) exists for U ν (x) on E. Require the (in-)equalities of the CWEP to hold only in the discretization points. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 15/40

Discretization Let f (y,x) := log x y dy and g(y,x) := log x y ydy. Then: U µ (x) = N a j (g(y j,x) g(y j 1,x)) + b j (f (y j,x) f (y j 1,x)), j=2 where the a j s and b j s are linear in the µ j s. P s.t. U µ (y j ) = (Pv) j, µ M(E) with discretization v. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 16/40

Algorithm 2: discretized version of the CWEP algorithm begin I = {1,2,...,N} J = v = e(i) while v s/t do v(j) = { 1 t s(j) P(I,I) v(i) = Ce(I) P(I,J) v(j) + k(i) solve m(i) T v(i) = 1 m(j) T v(j) I = {tv < s} J = {tv s} end return vt ν = v end Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 17/40

Complexity of Algorithm 2 Stopping On every step at least 1 point is added to J. When no point is added, the stopping criterion is fulfilled. Complexity O(N 2 ) ops to construct P, O(M 3 k ) ops to compute v(i k), where M k = lenght(i k ) < M k 1 N total number of iterations is (generally) N Total cost of order O(N 3 ) Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 18/40

Numerical examples Consider the counting measure ν = 1 m δ αk, m n, α k / E. n k=1 Logarithmic potential: U ν (x) = 1 m n log (x α k ). k=1 If E = [a,b], the density of µ ν is given by (Deckers, Bultheel ( 07)): [ { } ] m dµ ν k=1 (x) R (αk b)(α k a) α k x + (n m) = dx nπ, x [a,b], (b x)(x a) where the square root is s.t. R{ } is positive on [a,b]. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 20/40

Example (WEP) E = [ 1,1], and ν = 21 22 δ α, with α = 0.5 + 0.1i. 4 computed density 10 1 relative error 3.5 10 2 3 10 3 2.5 dµ/dx 2 1.5 10 4 10 5 1 0.5 10 6 0 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x 10 7 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x Figure: Computed density of µ ν. Figure: Relative error. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 21/40

Example (CWEP) dσ(x) = 2 1 x 2 π dx on E = [ 1,1], and ν = { 1 0.9 + 0.1i, k 50 t (0,1), and α k =. 0.5 0.1i, k > 50. 100t 100t 1 k=1 α k, with 0.7 0.6 0.5 tdµt/dx 0.4 0.3 0.2 0.1 0 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x Figure: Computed density of µ ν t for t = 0.05 + 0.15r, r = 0,...,5. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 22/40

10 0tdµt/dx 10 0tdµt/dx 10 0tdµt/dx Example (CWEP) σ = Bal(δ β,e), where β = 0.6 + 0.1i and E = [ 1,1], and ν = δ α. 10 1 10 1 10 1 10 2 10 2 10 2 10 3 x 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 x 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 Figure: Density of tµ ν t for α = β, Figure: for α = 0.5 0.1i, Figure: and for α = 0.6 + 0.01i. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 23/40

Rational Krylov subspaces Let A N Hermitian N N matrix with distinct eigenvalues in E. q N non-zero column vector in C N. A n sequence of complex poles {α 1,...,α n } C { } bounded away from ch(e), and α k 0. Define x j b j (x) = ( ) = xj, j = 1,...,n. j k=1 1 x π j (x) α k Definition The rational Krylov subspace K n+1 (A N,q N, A n ) is given by K n+1 (A N,q N, A n ) = span {q N,b 1 (A N )q N,...,b n (A N )q N }, n = 1,...,N 1. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 25/40

Rational Krylov subspaces The rational Lanczos iterations produce an orthonormal basis for K n+1 (n < N): Put v 1 = q N / q N. Compute x k = (I A N /α k ) 1 A N v k, k > 0. Orthogonalize x k against v 1,...,v k, followed by normalization k+1 x k = h j,k v j, k = 1,...,n. j=1 Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 26/40

Rational Lanczos iterations, matrix language A n = B [αn] n where h ( n+1,n Vn H A N v n+1 [0... 0 1] I n + H n [αn] α n V n (v k ) C N n, projection matrix H [αn] n (h j,k ) C n n, upper Hessenberg D n [αn] diag(1/α 1,...,1/α n ) A n Vn H A N V n B [αn] n H [αn] n ( I n + H [αn] n ) D [αn] 1 n D n [αn] ) 1, Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 27/40

Special case α n =...since the left-hand side does not depend on α n, choose α n = which gives the nice projection formula A n = B [ ] n, A n = V H n A N V n = H [ ] n ( I n + H [ ] n D [ ] n ) 1. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 28/40

Rational Lanczos minimization problem By definition, v k+1 = ϕ k (A N )q N, where ϕ k (x) = p k(x) π k (x) orthonormal rational function (ORF) with poles in A k. is an The eigenvalues of B [αn] n are zeros of the ORF ϕ n. If α n is real or infinite, they are all real and in ch(e). (Deckers, Bultheel ( 07)) Corollary The Ritz values are the zeros of an ORF ϕ n = pn(x) π n 1 (x). Let P n denote the space of monic polynomials of degree n: Rational Lanczos minimization problem p(a N ) π n 1 (A N ) q N. min p P n Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 29/40

Characterization of converged Ritz values I 1 N,n s.t. n/n t (0,1) 2 sequence of Hermitian matrices (A N ) C N N λ 1,N < λ 2,N <... < λ N,N 3 asymptotic distribution of the eigenvalues: 1 N lim δ λk,n = σ M(E) N N k=1 4 asymptotic distribution of the poles: lim N n(n) 1 n(n) lim N k=1 δ αk = ν t + (1 s t )δ = η t, 5 q N C N : q N = 1 and chosen sufficiently random: ( 1/N min 1 k N q H N k,n ) u = 1 Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 31/40

Characterization of converged Ritz values Theorem (cf. Kuijlaars ( 00), Beckermann et al. ( 09)) Under the assumptions 1 5 it holds that the asymptotic distribution of the n-th Ritz values θ 1,n < θ 2,n <... < θ n,n is given by lim N n(n) 1 n(n) k=1 δ θk,n = µ t M(E). The measure µ t satisfies tµ t σ and minimizes the weighted energy I(µ ν t ) among all µ M(E) satisfying tµ σ. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 32/40

Characterization of converged Ritz values Definition The free region is the region S t = supp(σ tµ t ). The complement of S t is called the saturated region. Theorem (cf. Kuijlaars ( 00), Beckermann et al. ( 09)) The saturated region is the region where the n-th Ritz values converged to an eigenvalue of A N. The rate of convergence is described by the weighted potential U µt νt. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 33/40

Numerical examples A R 200 200 diagonal starting vector q = [1 1... 1] T rational Lanczos method with full reorthogonalization convergence criterion: Marker Color Distance to nearest Ritz value + Red less than 0.5 10 14 Yellow between 0.5 10 14 and 0.5 10 8 Blue between 0.5 10 8 and 0.5 10 4 Green more than 0.5 10 4 Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 35/40

Preliminaries Algorithm Rational Lanczos References Equally spaced eigenvalues, one multiple pole Example 1 1 0.8 0.8 0.6 0.6 0.4 0.4 Ritz values Ritz values dσ(x) = 12 dx on [ 1, 1], and νt = 1 0.2 0 0.2 δα. 0 0.2 0.4 0.6 0.6 1 0.2 0.4 0.8 1 200t 0.8 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Figure: Convergence of Ritz values, α = 2. Karl Deckers, Andrey Chesnokov, and Marc Van Barel 1 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Figure: Convergence of Ritz values, α = 0.2 + 0.1i. A numerical solution of the CWEP 36/40

Equally spaced eigenvalues, separate intervals Example dσ(x) = 1 2 dx on [0,1] [2,3], and ν t = ( 1 1 200t ) δα. 0.7 3 0.6 2.5 0.5 2 tdµt/dx 0.4 0.3 0.2 Ritz values 1.5 1 0.1 0.5 0 0 0.5 1 1.5 2 2.5 3 x 0 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Figure: CWEP solution, t = {0.05, 0.2, 0.35, 0.5, 0.65, 0.8} Figure: Convergence of Ritz values, α = 0.7 + 0.1i. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 37/40

Preliminaries Algorithm Rational Lanczos References Equally spaced eigenvalues, two different poles Example dσ(x) = 12 dx on [ 1, 1], and two different poles α1 = 0.2 + 0.1i and α2 = 0.5 + 0.1i, each with multiplicity 100. (a) Left: poles are ordered as {α1,..., α1, α2,..., α2 }, 1 1 0.8 0.8 0.6 0.6 0.4 0.4 Ritz values Ritz values (b) Right: poles are ordered as {α1, α2, α1, α2,...}. 0.2 0 0.2 0.2 0 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Karl Deckers, Andrey Chesnokov, and Marc Van Barel 1 0 20 40 60 80 100 120 140 160 180 200 Number of iterations A numerical solution of the CWEP 38/40

Preliminaries Algorithm Rational Lanczos References Eigenvalues distributed according to a balayage-measure Example 199 200 δ1.1, [ 1, 1] σ = Bal 0.8 0.6 0.6 0.4 0.4 0.2 0 0.2 δα. 0 0.2 0.4 0.6 0.6 0.8 0.2 0.4 1 1 200t 1 0.8 Ritz values Ritz values 1, and νt = 1 0.8 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Figure: Convergence of Ritz values, α = 1.1 Karl Deckers, Andrey Chesnokov, and Marc Van Barel 1 0 20 40 60 80 100 120 140 160 180 200 Number of iterations Figure: Convergence of Ritz values, α = 2.0 A numerical solution of the CWEP 39/40

References B. Beckermann, S. Güttel, and R. Vandebril, On the convergence of rational Ritz values. Preprint, 35 pages, Apr. 2009. K. Deckers and A. Bultheel, Rational Krylov sequences and orthogonal rational functions, Technical Report TW499, K.U.Leuven, Department of Computer Science, Aug. 2007. S. Helsen and M. Van Barel, A numerical solution of the constrained energy problem, Journal of Computational and Applied Mathematics, 189 (2006), pp. 442 452. A. B. J. Kuijlaars, Convergence analysis of Krylov subspace iterations with methods from potential theory, SIAM Review, 48 (2006), pp. 3 40. Karl Deckers, Andrey Chesnokov, and Marc Van Barel A numerical solution of the CWEP 40/40

Thank you...