arxiv: v2 [math.na] 7 Sep 2016

Transcription

1 A MODIFIED IMPLEMENTATION OF MINRES TO MONITOR RESIDUAL SUBVECTOR NORMS FOR BLOCK SYSTEMS ROLAND HERZOG AND KIRK M. SOODHALTER Abstract. Saddle-oint systems, i.e., strctred linear systems with symmetric matrices are considered. A modified imlementation of reconditioned) MINRES is derived which allows to monitor the norms of the sbvectors individally. Comared to the imlementation from the textbook of [Elman, Sylvester and Wathen, Oxford University Press, 2014, or method reqires one extra vector of storage and no additional alications of the reconditioner. Nmerical exeriments are inclded. Key words. MINRES, saddle-oint roblems, strctred linear systems, reconditioning, sbvector norms arxiv: v2 [math.na 7 Se ) AMS sbject classifications. 65F08, 65F10, 15B57, 65M22, 74S05, 76M10 1. Introdction. We are solving symmetric linear systems of the form [ [ A B T = B C [ f f, where A R m m and C R are symmetric, and B R m, by alying MINRES or reconditioned MINRES [8 ation. After j ations, we have aroximation j), j) ) and residal [ [ [ [ [ r j) f A B T j) f A = B C j) = j) + B T j) B j) C j). f We denote the two arts of the residal as r j) = f A j) + B T j)) and r j) = f B j) C j)). The qestion we seek to answer is: can we monitor the individal reconditioned norms of r j) and r j) as oosed to the fll norm of r j) ) sing only qantities arising in an efficient imlementation of the reconditioned MINRES algorithm, namely that in [2, Algorithm 4.1? The answer is yes, nder certain conditions. The technical reqirements to do so are connected to the notion of so-called natral reconditioners which arise in this setting; see, e.g., [9, 12 and references contained therein. In section 2, we demonstrate that at the storage cost of one additional fll-length vector and six scalars bt no additional alications of the reconditioner, one can modify the reconditioned MINRES method to calclate these norms in a rogressive fashion. An extension to more than two residal arts is straightforward. An imlementation of or modified version of MINRES is given in algorithm 1 and is available at [6 as a Matlab file. f This version dated Setember 8, Technische Universität Chemnitz, Faclty of Mathematics, Professorshi Nmerical Mathematics Partial Differential Eqations), D Chemnitz, Germany roland.herzog@mathematik.tchemnitz.de, htts:// Transfer Gro, Johann Radon Institte for Comtational and Alied Mathematics, Altenbergerstraße 69, A 4040 Linz, Astria kirk.soodhalter@ricam.oeaw.ac.at, htt://math. soodhalter.com). 1

2 2 R. HERZOG AND K. M. SOODHALTER As a motivation for or stdy, we mention that the individal arts of the residal in 1.1) often have different hysical interretations. Monitoring them individally allows a better insight into the convergence of MINRES and it allows the formlation of refined stoing cria. We resent examles and nmerical exeriments in section How to monitor both arts of the reconditioned residal. In this section, we derive how one monitors these norms withot incrring mch extra comtational or storage exense. We will describe everything in terms of the variable names se in [2, Algorithm 4.1 with two excetions, which will be noted below. Here we describe qickly the derivation of reconditioned MINRES as a Krylov sbsace method, secifically in order to relate certain qantities from [2, Algorithm 4.1 to the common qantities arising in the Lanczos rocess. In rincile, the reconditioned MINRES algorithm is an imlementation of the minimm residal Krylov sbsace method for matrices which are symmetric. It can be eqivalently formlated as an ation for oerator eqations in which the oerator is self-adjoint, maing elements of a Hilbert sace into its dal; see [5. We resent the derivation for the finite dimensional case. For the roses [ of this discssion, let x j) = j), j) ) A B T be the jth aroximation. Let K =. We frther assme that the reconditioner P is symmetric ositive definite and has block diagonal strctre B 0 [ P P = with P R m m and P R. P This assmtion is natral in many sitations when we acknowledge that the residal sbvectors often have different hysical meaning and need to be measred in different norms; see [9, 12 and section 3 for examles. We briefly review some basic facts abot Krylov sbsaces for symmetric matrices. We begin with the nreconditioned sitation. Let A R n n be symmetric. For some starting element h R n, we define the jth Krylov sbsace generated by A and h to be K j A, h) = san { h, Ah, A 2 h,..., A j h } sing the Lanczos rocess, where if V j = [ v 1 v 2 v j R n j is the matrix with colmns that form an orthonormal basis of K j A, h), then we have the Lanczos relation 2.2) AV j = V j+1 T j where T j R j+1) j. The matrix T j is tridiagonal, and the matrix T j defined as the first j rows of T j ) is symmetric. This imlies that to generate each new Lanczos vector, we mst orthogonalize the newest vector against only the two revios Lanczos vectors. This leads to the following well-known three-term recrrence formla 2.3) Av j = γ j+1 v j+1 + δ j v j + γ j v j.

3 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 3 Using the naming conventions in [2, Algorithm 2.4, we have δ 1 γ 2. γ T j = γj. γ j δ j γ j+1 In the case that we have a symmetric ositive definite reconditioner P, we show that a Krylov sbsace can still constrcted sing the short-term Lanczos ation and that a MINRES method can be sed for solving 1.1). In, e.g., [2, reconditioned MINRES is derived by first observing that P admits the Cholesky decomosition P = HH T. Ths one cold consider solving the two-sided reconditioned eqations [ [ H KH T y = H b with y = H T f1 x where x = and b =, where we have the initial aroximation y 0) = H T x 0). The matrix H KH T is still symmetric. However, one can notice that the reconditioned residal satisfies H r k) = r k) P where P is the norm arising from the inner rodct indced by P, i.e.,, P = P,. One frther notes that we have the eqivalence K j H KH T, H r 0) ) = H K j KP, r 0) ). If reconditioned MINRES at ation j rodce a correction s j) K j H KH T, H r 0) ) sch that y j) = y 0) + s j), then we recover the aroximation for x with x j) = H T y 0) + s j)). Since x 0) = H T y 0) the correction sace for reconditioned MINRES with resect to the original variables is actally P K j KP, r 0) ). From here, one can show that the reconditioned MINRES ation is eqivalent to an ation with the sbsace K j KP, r 0) ) bt with resect to the inner rodct, P indced by the reconditioner. This is indeed a short-term recrrence ation, as it can be shown that KP is symmetric with resect to, P. Ths the derivation of the reconditioned MINRES method [2, Algorithm 4.1 can be resented as a small modification of the nreconditioned MINRES [2, Algorithm 2.4. This same fact was also shown in [5 bt by interreting the nderlying oerator as a maing from a Hilbert sace to its dal. There are many benefits to this interretation. One advantage is that even in the finite dimensional sitation, it allows the derivation of reconditioned MINRES withot the temorary introdction of Cholesky factors. Let the colmns of V j still form an orthonormal basis for a Krylov sbsace, bt this time the sace K j KP, r 0) ), and the orthonormality is with resect to, P. Let Z j = P V j have as colmns the image of those vectors nder the action of the reconditioner. Then we have the reconditioned Lanczos relation, 2.4) KP V j = KZ j = V j T j where T j has the tridiagonal strctre described earlier. What differs here is that the colmns of V j have been orthonormalized with resect to the P inner rodct, and f 2

4 4 R. HERZOG AND K. M. SOODHALTER the entries of the tridiagonal matrix T j are formed from these P inner rodcts. We have the three-term recrrence 2.5) Kz j = γ j+1 v j+1 + δ j v j + γ j v j. The reconditioned MINRES method solves the following residal minimization roblem, 2.6) x j) = x 0) + Z j y j) where y j) = argmin y R j γ1 e 1 T j y 2, where γ 1 = r 0 P. Thogh this redces the minimization of the residal to a small least-sqares roblem, this is not the most efficient way to imlement the MINRES method; and indeed, this is not what is done in, e.g., [2, Algorithm 4.1. De to 2.3), MINRES can be derived sch that only six fll-length vectors mst be stored. In the interest of not again deriving everything, we will simly rovide a few relationshis between qantities from the above exlanation of MINRES and those in [2, Algorithm 4.1. Frther imlementation details can be fond in, e.g., [4, Section 2.5. Let T j = Q j R j be a QR-factorization of T j comted with Givens rotations where Q j R j+1) j+1) is a nitary matrix constrcted from the rodct of Givens rotations, and R j R j+1) j is er trianglar. The matrix R j R j j is simly R j with the last row of zeros) deleted. We can write the matrix Q T j as the rodct of the Givens rotation sed to trianglarize T j. Since T j is tridiagonal and Hessenberg, we mst annihilate only one sbdiagonal entry er colmn. Following from [2, Algorithm 4.1, we denote s i and c i to be the Givens sine and cosine [ sed to annihilate ci s the ith entry of colmn i 1 sing the nitary matrix F i = i. Ths we can s i c i denote Q T j = G j) j+1 Gj) j G j) 2 where we define G j) i R j+1) j+1) to be the matrix alying the ith Givens rotation to rows i 1 and i to a j + 1) j + 1) matrix. Using a normal eqations formlation of 2.6), we can derive an exression for the least sqares minimizer 2.7) y j) = R j { Q T j γ 1 e 1 ) } 1:j, where { } 1:j indicates we take only the first j rows of the argment. For the rose of discssion, we make an additional modification to the variable naming sing in [2, Algorithm 4.1. We index η, which is sed to track the residal norm. On Line 4 of the algorithm, we wold have η 0 = γ 1, and at Line 18, we wold write η j = s j+1 η j. From the roof of [3, Corollary 2.5.3, one can see that at ation j, the least sqares residal can be written 2.8) γ 1 e 1 T j y j) = Q j η j e j+1 ), where e j+1 is the last colmn of the j + 1) j + 1) identity matrix, and becase Q j is nitary, it follows that η j is the norm of the residal r j) P ertaining to x j). We now derive an exression for the norms of r j) and r j) in terms of qantities arising in the Lanczos rocess and residal minimization. De to the block strctre

5 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 5 of K, we artition V j and Z j similarly, 2.9) V j = [ Vj, V j, and Z j = [ Zj, Z j, and then insert the artitioned vectors into 2.2) [ [ [ [ A B T Zj, AZj, + B = T Z j, Vj+1, = T B C Z j, BZ j, CZ j, V j = j+1, [ Vj+1, T j V j+1, T j We note that for tracking and dating the fll residal vector, this artition is artificially imosed. One cold simly track the fll residal and artition at the end to comte the sbvector norms searately. We derive everything sing this block artition becase one may wish to track the norm of only one sbvector. Frthermore, one needs the artition qantities to define the recrsions for dating the reconditioned sbvector norms. It then follows that we have Lanczos relations for the blocks 2.10) AZ j, + B T Z j, = V j+1, T j and BZ j, CZ j, = V j+1, T j. From [3, Theorem 2.5.7, we know that the jth MINRES residal satisfies the recrsion 2.11) r j) = r j) c j+1 η j m j+1) j, where the axiliary vectors can also be rogressively dated. The rogressive dating scheme for the axiliary vectors can also be seen by observing that these axiliary vectors form an orthonormal basis for the constraint sace KP K j KP, r 0) ), namely M j+1 = V j+1 Q j = [ m j+1) 1 m j+1) 2 m j+1) j m j+1) j+1. Using indction, one can show that since Q j is the rodct of Givens rotations with Q j = Q j G j)t j+1, we have that mj+1) i = m j) i for all 1 i j 1, and m j+1) j = c j+1 m j) j + s j+1 v j+1 and m j+1) j+1 = s j+1 m j) j + c j+1 v j+1, where we set m 1) 1 = v 1. This imlies that we can dro the serindices for the first j 1 colmns of M j+1 which remain nchanged, i.e., 2.12) M j+1 = From 2.8), we then have that [ m 1 m 2 m j m j+1) j If we then artition the vector m j+1) j+1 = artial residals, 2.13) r j) r j) = η j V j+1 Q j e j+1 = η j m j+1) j+1. m j+1) j+1.. [ j+1) m j+1,, we then have formlas for the m j+1) j+1, = η j m j+1) j+1, and r j) = η j m j+1) j+1,.

6 6 R. HERZOG AND K. M. SOODHALTER We can also rewrite 2.13) in order to recover 2.11). By direct comtation with some sbstittions, we see that r j) = η j m j+1) j+1 = s j+1 η j ) s j+1 m j) j + c j+1 v j+1 ) = s 2 j+1η j m j) j + c j+1 s j+1 η j v j+1 = η j m j) j Ths the residal sbvectors r j) r j) = r j) c 2 j+1η j m j) j + c j+1 s j+1 η j v j+1 = r j) c j+1 c j+1 η j m j) j + s j+1 η j v j+1 ) = r j) c j+1 η j m j+1) j. and r j) satisfy the rogressive dating formlas c j+1 η j m j+1) j, and r j) = r j) c j+1 η j m j+1) j,. However, this reresentation wold reqire storage of two fll-length vectors. We have seen that throgh recrsion formlas, we can comte one or both sbvectors of the residal vector rodced by reconditioned MINRES. However, we wish to avoid storing or re-evalating these vectors since we actally want to comte only the aroriate norm of each iece. The fll residal minimization is with resect to the P norm P. We have assmed in this aer that the reconditioner has block diagonal strctre; and as P is symmetric ositive definite, it follows that its blocks are as well. Ths we can write r j) 2 P = r j) 2 P + r j) 2 P These norms can also be comted recrsively sing information already on hand. We note that doing it as in the following allows s to avoid additional alications of the reconditioner. In order to comte the norms searately, it is necessary that we also se the same fact for the newest Lanczos vector, Let 1 = v j+1 2 P = v j+1, 2 P ψ j+1, = v j+1, 2 P ψ j+1, = v j+1, 2 P + v j+1, 2 P. = z j+1,, v j+1, and = z j+1,, v j+1,. We can store these vales so they are available for later se. We frther denote η j, = r j) P and η j, = r j) P. This allows s to concisely state the following theorem. Theorem 2.1. The ratio between the fll reconditioned residal norm and the resective reconditioned residal sbvector norms can be written rogressively as follows: ηj, η j ηj, η j ) 2 = s 2 j+1 ) 2 = s 2 j+1 ηj, η j ηj, η j ) 2 2s j+1c j+1 m j). j,) T zj+1, + c 2 j+1ψ j+1, and ) 2 2s j+1c j+1 m j) j,) T zj+1, + c 2 j+1ψ j+1,.

7 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 7 Proof. The roofs for each sbvector are identical, so we rove it only for one. We can verify the identity by direct calclation and then by showing that the qantities arising from the calclation are available by recrsion. We comte r j) 2 P = η j m j+1) ) T j+1, P ηj m j+1) ) j+1, = ηj 2 sj+1 m j) j, + c ) T j+1v j+1, P sj+1 m j) j, + c ) j+1v j+1, = ηj 2 s 2 j)) T j+1 m j, P m j) j, 2s j)) T j+1c j+1 m j, P v j+1, ) + c 2 j+1vj+1,p T v j+1, ) = ηj 2 s 2 ηj, 2 ) T j+1 2s j+1 c j+1 m j) zj+1, + c 2 j+1ψ j+1,, η 2 j j, Basic algebra finishes the roof. For comleteness, we now resent a modified version of [2, Algorithm 4.1, here called algorithm 1. Note that we omit serscrits for m l) i, and some other sbscrits where they are not needed. Frthermore, we introdce the scalars θ and θ defined by 2.14) θ = m j) j,, z j+1, and θ = m j) j,, z j+1, and the sqared residal norm fractions ) 2 ηj, 2.15) µ = and µ = η j ηj, Notice that one can work with fll-length vectors and the artitioning is only imortant for the comtation of artial inner rodcts as in 2.14). 3. Examles and Nmerical Exeriments. In the introdction, we motivated or stdy by mentioning that the residal sbvectors in saddle-oint systems 1.1) often have different hysical interretations. In this section, we discss several examles to nderline this fact, and we also resent nmerical reslts which demonstrate the correctness of algorithm 1. We do this by storing all the residal vectors throghot the ation and evalating the reconditioned sbvector norms a-osteriori. This is a debgging ste which is introdced to verify the correctness of algorithm 1. Or Matlab imlementation of algorithm 1 is available at [6. In all examles, we begin with an all-zero initial gess and we sto when the relative redction of the total residal η j r j) = P η 0 r 0) P falls below We cold easily tilize a refined stoing crion sch as η j, = r j) P η j ) 2. ε and η j, = r j) P ε to take advantage of the ability to monitor the residal sbvector norms. Concrete alications of this e.g., in otimization) are otside the scoe of this aer and will be addressed elsewhere.

8 8 R. HERZOG AND K. M. SOODHALTER Algorithm 1: The reconditioned MINRES method with monitoring of η j, = j) r and η j, = j) r P P Int : K R m+) m+), symmetric, and f R m+ as in 1.1), P = blkdiagp, P ) R m+) m+), symmetric ositive definite. Ott: x j) for some j sch that r j) P satisfies some convergence cria. 1 Set v 0 = 0, w 0 = w 1 = [ 0 2 Choose x 0) f1, set v 1 = K x 0), z 1 = P v 1 3 γ 1 = z 1, v 1, v 1 v 1 /γ 1, z 1 z 1 /γ 1 4 ψ = z 1,, v 1,, ψ = z 1,, v 1, 5 µ = ψ, µ = ψ 6 m 1 = v 1 f 2 7 Set η 0 = γ 1, η 0, = γ 1 ψ, η 0, = γ 1 ψ, s 0 = s 1 = 0, c 0 = c 1 = 1 8 for j = 1 ntil convergence do 9 δ j = Kz j, z j 10 v j+1 = Kz j δ j v j γ j v j // Lanczos 11 z j+1 = P v j+1 12 γ j+1 = z j+1, v j+1 13 v j+1 v j+1 /γ j+1 14 z j+1 z j+1 /γ j+1 15 α 0 = c j δ j c j s j γ j // Udate QR factorization 16 α 1 = α0 2 + γ2 j+1 17 α 2 = s j δ j + c j c j γ j 18 α 3 = s j γ j 19 c j+1 = α 0 /α 1, s j+1 = γ j+1 /α 1 // Givens rotations 20 θ m j,, z j+1,, θ m j,, z j+1, 21 ψ z j+1,, v j+1,, ψ z j+1,, v j+1, 22 m j+1 = s j+1 m j + c j+1 v j+1 23 w j+1 = z j α 3 w j α 2 w j )/α 1 24 x j) = x j) + c j+1 η j w j+1 25 µ s 2 j+1 µ 2s j+1 c j+1 θ + c 2 j+1 ψ 26 µ s 2 j+1 µ 2s j+1 c j+1 θ + c 2 j+1 ψ 27 η j = s j+1 η j // total residal norm 28 η j, = η j µ, η j, = η j µ // artial residal norms 29 Test for convergence Examle 3.1 Least-Norm Soltion of Underdetermined Linear System). We consider an nderdetermined and consistent linear system B = b with a matrix B R m n of fll row rank and m < n. Its least-norm soltion Minimize H sch that B = b, Rn in the sense of the inner rodct defined by the symmetric ositive definite matrix

9 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 9 H R n n, is niqely determined by the saddle-oint system [ [ [ H B T 0 =. B 0 b Notice that the first block of eqations reresents otimality, while the second block reresents feasibility of a candidate soltion, ) T. Or test case ses data created throgh the commands rng 42); n =100; m =30; B= randn m,n); b= randn m,1); H= sdiags rand n,1),0,n,n); As reconditioner we se either P 1 = blkdiagi n n, I m m ) the nreconditioned case) or P 2 = blkdiagh, I m m ). The convergence histories in Figre 3.1 show that the amont by which the two residal sbvectors contribte to their combined norm may indeed be qite different, and it deends on the reconditioner. In this examle, the feasibility residal r = b B is significantly smaller than the otimality residal r = H B T in the nreconditioned case where we sed P 1. To be more recise, the average vale of µ throghot the ation history is abot 21%. Qite the oosite is tre for the case of P 2, when µ is close to 100%. nderdetermined nreconditioned m=100, =30) nderdetermined reconditioned m=100, =30) 10 0 r r r 10 0 r r r r redicted r redicted r redicted r redicted Fig Convergence history of the residal sbvectors for Examle 3.1 nderdetermined linear system) in the nreconditioned left) and reconditioned case right). The lot, as all following convergence lots, also confirms the correctness of algorithm 1 and of or imlementation. Notice that in the right figre, the first residal norm r P is artially otside of the lot range. This reslts from or choice to maintain the same scales for both figres. Examle 3.2 Least-Sqares Soltion of Overdetermined Linear System). Here we consider an overdetermined linear system B T = b with a matrix B R m n of fll row rank and m < n. Its least-sqares soltion Minimize 1 B T b 2, 2 H R m with H R n n symmetric ositive definite as above, is niqely determined by the normal eqations, B H B T b) = 0. By defining as the reconditioned residal H B T b), we find that the least-sqares soltion is in trn eqivalent to the saddle-oint system [ [ [ H B T b =. B 0 0

10 10 R. HERZOG AND K. M. SOODHALTER The first block of eqations now reresents feasibility for the constraint defining the axiliary qantity, while the second reqires to lie in the kernel of B. Similarly as above, we derive test data throgh rng 42); n =100; m =30; B= randn m,n); b= randn n,1); H= sdiags rand n,1),0,n,n); We emloy the same reconditioners P 1 and P 2 as in Examle 3.1. Once again, the convergence behavior of the two residal sbvectors is fndamentally different Figre 3.2) for the two cases: in the nreconditioned case, the average vale of µ is abot 9%, while it is 72% in the reconditioned case overdetermined nreconditioned m=100, =30) r r 10 1 overdetermined reconditioned m=100, =30) r r 10 0 r r redicted 10 0 r r redicted r redicted r redicted Fig Convergence history of the residal sbvectors for Examle 3.2 overdetermined linear system) in the nreconditioned left) and reconditioned case right). Or remaining examles involve artial differential eqations and they will be stated in variational form, by secifying bilinear forms a, ) : V V R, b, ) : V Q R, and, where aroriate, c, ) : Q Q R. Here V and Q are real) Hilbert saces. The matrices A, B and C in 1.1) are then obtained by evalating the resective bilinear forms on a basis of an aroriate finite dimensional sbsace V h or Q h, e.g., [B ij = bϕ j, ψ i ) for basis elements ϕ j V h and ψ i Q h. Similarly, the right hand side vectors f and f are obtained from evalating roblem deendent linear forms f ) and f ) on the same basis fnctions ϕ i and ψ i, resectively. Finally, the matrices and vectors obtained in this way may need to be dated de to the incororation of essential Dirichlet) bondary conditions. All nmerical tests were condcted sing the Python interface of the finite element library FEniCS [7 version 1.6) to generate the matrices and vectors. Those were then exorted in PETSc binary format and read from Matlab throgh the heler fnctions readpetscbinmat.m and readpetscbinvec.m rovided by Samar Khatiwala on his web age. We deliberately trned off the reordering featre of FEniCS for the degrees of freedom to reserve the block strctre of 1.1) for illstration roses see Figre 3.6) and in order for the sbvectors and and the residals r and r to remain contigos in memory. However, or theory does not rely on a articlar ordering of the sbvector comonents, and or imlementation of algorithm 1 allows for arbitrary comonent ordering. It will trn ot to be sefl for the following examles to secify the hysical nits Newton: N, meters: m, seconds: s, Watt: W, and Kelvin: K) for all involved qantities in the following examles.

11 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 11 Examle 3.3 Stokes Channel Flow). We consider the variational formlation of a 3D stationary Stokes channel flow configration within the domain = 0, 10) 0, 1) 0, 1). Dirichlet conditions are imosed on the flid velocity everywhere excet at the right otflow) bondary x = 10), where do-nothing conditions hold. The Dirichlet conditions are homogeneos no-sli) excet at the left inflow) bondary x = 0), where x, y, z) = in x, y, z) = y 1 y) z 1 z), 0, 0) T m s is imosed. Aroriate fnction saces for this set are V = {v H 1 ; R 3 ) : v = 0 on Γ\ Γ right } for the velocity and Q = L 2 ) for the ressre. The relevant bilinear and linear forms associated with this roblem are a, v) = µ : v dx, b, q) = q div dx and f v) = f v dx. We se the dynamic viscosity arameter µ = N s m 2 water) and zero right hand side force f = 0, 0, 0) T N m 3. By considering nits, or by investigating the nderlying hysics, we infer that the first comonent of the residal, r = f a, ) b, ), reresents a net sm of forces, measred in N. Similarly, the second residal r = b, ) reresents the net flx of flid throgh the imermeable channel walls, measred in m 3 s. Clearly, both arts of the residal mst be zero at the converged soltion, bt their deartre from zero at intermediate ates has different hysical interretations. As reconditioner P = blkdiagp, P ), we se the block diagonal matrix indced by the bilinear forms a, v) and µ q dx, resectively, similar to [2, Section 4.2, where the constant-free roblem is considered. Notice that the inclsion of the constant µ into the ressre mass matrix renders the reconditioner comatible with the hysical nits of the roblem. For or nmerical test, we discretized the roblem sing the Taylor-Hood finite element. The homogeneos and non-homogeneos Dirichlet bondary conditions were inclded by modifying the saddle-oint comonents A, B, B T, and right hand side f in 1.1) in a symmetric way throgh the assemble_system call in FEniCS. Notice that this effectively modifies some comonents of the first residal sbvector r by exressions of the form in x, y, z) x, y, z), measred in m s. The same modifications aly to the reconditioner P. Figre 3.3 dislays the convergence behavior of the residal sbvector norms on a relatively coarse mesh and its refinement. The reslt illstrates the mesh indeendence of the reconditioned ation, and it also shows that the residal r reresenting mass conservation) lags behind the residal r over the majority of the ations. Examle 3.4 Linear Elasticity with Nearly Incomressible Material). This examle describes a tensile test with a rod of sqare cross section, which occies the domain = 0, 100) 0, 10) 0, 10). We se mm here in lace of m as or length nit. Homogeneos Dirichlet conditions for the dislacement are imosed at the left claming) bondary x = 0), while natral traction) bondary conditions are imosed elsewhere. The imosed traction ressre is zero excet at the right forcing) bondary x = 100), where a niform ressre of g = 1, 0, 0) T N mm 2 is imosed. As is cstomary for nearly incomressible material, we introdce an extra

12 12 R. HERZOG AND K. M. SOODHALTER 10 0 Stokes3D level 1 m=19683, =1025) r r 10 0 Stokes3D level 2 m=139587, =6561) r r r r redicted r r redicted r redicted r redicted Fig Convergence history of the residal sbvectors for Examle 3.3 Stokes) on a coarse grid left) and its niform refinement right). variable for the hydrostatic ressre see for instance [11) in order to overcome the ill-conditioning of a rely dislacement based formlation known as locking [1. We emloy the standard isotroic stress-strain relation, σ = 2 µ ε) + I and div = λ. Here ε) = + T )/2 denotes the symmetrized Jacobian of, while µ and λ denote the Lamé constants. We choose as material arameters Yong s modls E = 50 N mm 2 and a Poisson ratio of ν = These articlar vales describe a material like nearly incomressible rbber, and a conversion to the Lamé constants yields µ = E 2 1+ν) = N mm 2 νe and λ = 1+ν)1 2 ν) = N mm 2. The variational mixed formlation obtained in this way is described by the saces V = {v H 1 ; R 3 ) : v = 0 on Γ left } for the dislacement and Q = L 2 ) for the hydrostatic ressre. The bilinear and linear forms associated with this roblem are a, v) = 2 µ ε) : εv) dx, b, q) = q div dx, c, q) = λ q dx and f v) = g v dx. Γ right At the converged soltion, the body is in eqilibrim. At an intermediate ate, we can interret r = f a, ) b, ) as a net force acting on the body, measred in N. This force is attribted to a violation of the eqilibrim conditions div σ = f in, and σ n = 0 or σ n = g at the non-claming bondary arts at an intermediate ate, ) T. The second residal r = b, ) + c, ) admits an interretation of a volme measred in m 3 de to a violation of div = λ. As reconditioner P = blkdiagp, P ), we se the block diagonal matrix indced by the bilinear forms a, v) and c, q), resectively. Once again, we remark that this choice is comatible with the hysical nits of the roblem. Similarly as for Examle 3.3, we discretize the roblem sing the Taylor-Hood finite element, and similar modifications de to Dirichlet dislacement bondary conditions aly. Figre 3.4 illstrates the convergence behavior on sccessively refined meshes, revealing once again different orders of magnitde for the two comonents of the residal. Examle 3.5 Otimal Bondary Control). Or final examle is an otimal bondary control roblem for the stationary heat

13 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 13 Elasticity3Dressre level 1 m=6075, =369) Elasticity3Dressre level 2 m=39123, =2025) 10 1 r r 10 1 r r 10 0 r r redicted 10 0 r r redicted r redicted r redicted Fig Convergence history of the residal sbvectors for Examle 3.4 elasticity) on a coarse grid left) and its niform refinement right). eqation on the nit cbe = 0, 1) 0, 1) 0, 1) with bondary Γ. The roblem statement is Minimize s.t. α 1 2 d 2 L 2 ) + α 2 2 f 2 L 2 Γ) κ + δ = 0 in, κ = f on Γ. n As data we se α 1 = 1 m 3 K 2, d x, y, z) = x K, α 2 = m 2 W 2, heat condction coefficient κ = 1 W m K, and radiation coefficient δ = 1 W m 3 K. The necessary and sfficient otimality conditions for this roblem are standard; see, for instance, [10, Chater 2.8. When we eliminate the control fnction f from the roblem, a saddle-oint system for the state and adjoint state remains, which is described by the following data: a, v) = α 1 v dx, b, q) = κ q dx + δ q dx, c, q) = α2 q dx and f v) = α 1 d v dx. Γ The nderlying saces are V = Q = H 1 ), and we discretize both sing iecewise linear, continos finite elements. Note that the elements, v V are measred in K while the elements, q Q are measred in W. As reconditioner P = blkdiagp, P ), we se the block diagonal matrix indced by the bilinear forms, v) = α 1 κ δ v dx + α 1 v dx, q) = α 2 κ δ ) 1/2 q dx + α 2 κ δ) 1/2 q dx, resectively. As in or revios examles, this choice is comatible with the hysical nits of the roblem. Figre 3.5 illstrates the convergence behavior on sccessively refined meshes. Besides the mesh indeendence, we observe that both residal sbvector norms converge in nison in this examle.

14 14 R. HERZOG AND K. M. SOODHALTER OtimalControl3D level 2 m=729, =729) OtimalControl3D level 3 m=4913, =4913) OtimalControl3D level 4 m=35937, =35937) r r r r r r r r r r redicted r redicted r redicted r redicted r redicted r redicted Fig Convergence history of the residal sbvectors for Examle 3.5 otimal bondary control) on a coarse grid left) and its niform refinements middle and right). Fig Sarsity lots of the saddle-oint systems arising in Examle 3.1 nderdetermined linear system), Examle 3.3 Stokes), Examle 3.4 elasticity), and Examle 3.5 otimal bondary control). 4. Discssion. In this aer we develoed a modified imlementation of MIN- RES. When alied to saddle-oint systems, the new imlementation allows s to monitor the norms of the sbvectors j) r and j) r individally, while con- P P ventional imlementations kee track of only the total residal norm r j) P. It shold be obvios how algorithm 1 generalizes to systems with more than two residal sbvectors and block-diagonal reconditioners strctred accordingly. The rice to ay to monitor the sbvector norms is the storage of one additional vector for comonents m j, and m j, comared to the imlementation of MINRES given in [2, Algorithm 4.1 as well as some additional inner rodct calclations. It shold be noted that if the cost of alying the oerator and reconditioner are chea enogh, it may be referable to simly constrct the fll residal and calclate reconditioned

15 MONITORING OF RESIDUAL IN SADDLE-POINT PROBLEMS 15 sbvector norms exlicitly. However, in the case that alication of these oerators has significant costs articlarly in the case that they are available only as rocedres and never actally constrcted) the method resented in this aer has imortant advantages. While we develoed the details in finite dimensions sing matrices and vectors, or aroach directly transfers to linear saddle-oint systems in a Hilbert sace setting sing linear oerators and linear forms, as described in [5. Being able to differentiate between the contribtions to the total residal offers new oortnities for the design of ative algorithms for nonlinear roblems, which reqire inexact solves of 1.1) as their main ingredient. We envision for instance solvers for eqality constrained nonlinear otimization roblems which may now assign individal stoing cria for the residals reresenting otimality and feasibility, resectively. The design of sch an algorithm is, however, beyond the scoe of this work. Similarly, for the Stokes Examle 3.3, the ser may now assign individal stoing cria for the flfillment of the balance of forces first residal) and the conservation of mass second residal). A toic of ftre research cold be to stdy the decay roerties of the sbvector norms, rather than stdy the decay of the total residal norm; see for instance [2, Section We exect that sch an analysis will be more involved bt it may shed light on how the relative scaling of the reconditioners blocks affects the convergence of the residal sbvectors. Acknowledgment. The athors wold like to thank David Silvester and Valeria Simoncini for their comments on an earlier version of the manscrit. REFERENCES [1 I. Babška and M. Sri, Locking effects in the finite element aroximation of elasticity roblems, Nmerische Mathematik, ), , doi: /bf [2 H. C. Elman, D. J. Silvester, and A. J. Wathen, Finite elements and fast ative solvers: with alications in incomressible flid dynamics, Nmerical Mathematics and Scientific Comtation, Oxford University Press, Oxford, second ed., 2014, doi: /acrof:oso/ [3 B. Fischer, Polynomial based ation methods for symmetric linear systems, Wiley-Tebner Series Advances in Nmerical Mathematics, John Wiley & Sons Ltd., Chichester, [4 A. Greenbam, Iterative Methods for Solving Linear Systems, SIAM, Philadelhia, [5 A. Günnel, R. Herzog, and E. Sachs, A note on reconditioners and scalar rodcts in Krylov sbsace methods for self-adjoint roblems in Hilbert sace, Electronic Transactions on Nmerical Analysis, ), [6 R. Herzog and K. Soodhalter, SUBMINRES. A modified imlementation of MINRES to monitor residal sbvector norms for block systems, 2016), doi: /zenodo [7 A. Logg, K.-A. Mardal, G. N. Wells, et al., Atomated Soltion of Differential Eqations by the Finite Element Method, Sringer, 2012, doi: / [8 C. C. Paige and M. A. Sanders, Soltions of sarse indefinite systems of linear eqations, SIAM Jornal on Nmerical Analysis, ), [9 J. Pestana and A. J. Wathen, Natral reconditioning and ative methods for saddle oint systems, SIAM Rev., ), , doi: / , htt://dx.doi. org/ / [10 F. Tröltzsch, Otimal Control of Partial Differential Eqations, vol. 112 of Gradate Stdies in Mathematics, American Mathematical Society, Providence, 2010, doi: /gsm/112. [11 C. Wieners, Robst mltigrid methods for nearly incomressible elasticity, Comting. Archives for Scientific Comting, ), , doi: /s International GAMM-Worksho on Mltigrid Methods Bonn, 1998). [12 W. Zlehner, Nonstandard norms and robst estimates for saddle oint roblems, SIAM J. Matrix Anal. Al., ), , doi: / , htt://dx.doi.org/ /