Towards Mechanical Derivation of Krylov Solver Libraries

Transcription

1 Towards Mechanical Derivation of Krylov Solver Libraries Victor Eijkhout Texas Advanced Computing Center with Paolo Bientinesi and Robert van de Geijn support: National Science Foundation award # /04/05

2 Overview Problem statement Derivation of Krylov methods in FLAME Outlook for mechanical systems 20yr ICL 2010/04/05 2

3 Problem statement Krylov solvers are Hard to derive Hard to code We aim for (in order of increasing ambition) Easy experimentation Formal treatment of algorithm derivation Mechanical system for library generation Automatic reasoning about performance 20yr ICL 2010/04/05 3

4 The view from 30k feet up Algorithm implementations are an overspecification. Separation of concerns: what to compute from how to compute Mechanical generation of algorithm code from a database of algorithm properties 20yr ICL 2010/04/05 4

5 Block formulation of Krylov methods 20yr ICL 2010/04/05 5

6 Introducing one tool Updating equation: x i+1 = x i + p i δ i can be written as X (J I ) = PD using J = so J I = 1 1, writing sequence equations in block form 20yr ICL 2010/04/05 6

7 Coupled recurrences Krylov methods r i+1 = r i + Ap i δ i, p i+1 = r i+1 j i p j u ji (where p i search direction and r i residual in iteration i) On block form: APD = R(I J), P(I + U) = R Coefficients in D and U will follow from orthogonality relations 20yr ICL 2010/04/05 7

8 Idea vs execution Abstract description of iterative method: APD = R(I J), P(I +U) = R (and PD = X (I J)) Basic form of R and P update equation is given Coefficients follow: Stationary Iteration: U 0, D constant CG: R t R = Ω diagonal et cetera Actual computation of D and U is tedious (especially for BiCGstab, QMR with look-ahead) Can we leave that last step to a system? 20yr ICL 2010/04/05 8

9 Krylov methods in Flame 20yr ICL 2010/04/05 9

10 FLAME Formal Linear Algebra Methods Environment System for formal derivation of linear algebra algorithms Predicate driven: predicates on input and output lead to proved correctness of algorithm Main tool: loop with invariant predicate and termination condition (this disqualifies certain algorithms from Flame treatment) Resulting worksheets can (almost) be directly translated into code 20yr ICL 2010/04/05 10

11 Interpretation A Flame worksheet facilitates the derivation of an algorithm: predicates describing the result are needed; derivation of the details is mechanical 20yr ICL 2010/04/05 11

12 Step 1: description of the problem Description of the sequences in block form: X (I J) = PD, APD = R(I J), P(I + U) = R, R t R = Ω (diagonal), x 1, r 1 given (underline means last column omitted) Extra equation: P t AP = (I + U) t R t R(I J)D 1 = (I + U) t Ω(I J)D 1 so P t AP lower triangular 20yr ICL 2010/04/05 12

13 Interpretation Finding the problem description is the one step in the story that requires inspiration. 20yr ICL 2010/04/05 13

14 Step 2: Partitioned Matrix Expression I ` ` TL J TL 0 0 B APL D L Ap M d M AP R D R = RL r M R er t C 1 0 A, 0 e 0 I BR J BR 0 1 I ` TL + U TL u TM u TR B PL p M P 0 1 u MR >< 0 0 I BR + U 0 BR R t L rm t C A ` 0 0 B C R L r M R R 0 0 A, RR t PL t B AP L PL t Ap M PL t AP R 0 p >: M t AP L pm t Ap M pm t AP C B C R A 0 A, PR t AP L PR t Ap M PR t AP R C A = ` R L r M R R, 20yr ICL 2010/04/05 14

15 Interpretation Matrix partitioning is trivial 20yr ICL 2010/04/05 15

16 Step 3: Choice of invariant The PME is a description of the finished result, we choose a subset of the equations (the invariant ) and keep this subset satisfied as the partition progresses through the matrix r 1 given ( ITL J TL AP L D L = (R L, r M ) e t r ) ( ) ( ) ( ) I TL + U TL u TM RL r M = PL p M ( 0 1 R t L R L RL tr ) ( ) ( M 0 P t rm t R L rm t r = L AP L PL tap M M 0 pm t AP L pm t Ap M ) ( 0 = ) 20yr ICL 2010/04/05 16

17 Interpretation There are only finitely many invariants. Some may lead to interesting algorithms, some can be uncomputable. 20yr ICL 2010/04/05 17

18 Step 4: Initialization and loop guard Initial condition: A given, first column of R is given, other matrices are empty or indeterminate. The L block is null, so the invariant holds trivially for at most one column. Loop guard: the size of the R block is null, so the invariant holds for the whole of the computed result. 20yr ICL 2010/04/05 18

19 Step 5: Partition and repartition At the start of the loop body: ( RL r M R R ) ( R0 r 1 r 2 R 3 ) At the end of the loop body: ( R0 r 1 r 2 R 3 ) ( RL r M R R ) In between: steps to make the predicate hold after this shift of the partition boundaries 20yr ICL 2010/04/05 19

20 Step 6: the before equations Use ( R L r M R R ) ( R0 r 1 r 2 R 3 ) in the invariant: ( ) ( ) J 00 AP 0 D 0 = R 0 r 1 j10 t ( ) ( ) I + U 00 u ( 01 P 0 p 1 = 0 1 P before : ( ) ( ) R t ( ) 0 0 R r1 t 0 r 1 =, ( 0 P t 0 AP 0 P0 t Ap ) ( ) 1 0 p1 tap 0 p1 tap =. 1 R 0 r 1 ), 20yr ICL 2010/04/05 20

21 Step 7: the after equations Use ( R 0 r 1 r 2 R 3 ) ( RL r M R R ) : 8!! D 0 0 J 00 0 A P 0 p 1 = R 0 r 1 δ 1 j t + r I + U 00 u 01 u 02 P 0 p 1 p υ 12 C A = R 0 r 1 r 2, >< P after : R t r1 t C A R 0 r 1 r 2 = 0 0 C A, r2 t 0 0 P0 t B AP 0 P0 t Ap 1 P0 T Ap p1 >: t AP 0 p1 t Ap 1 p1 T Ap C 2 A = 0 C A. p2 t AP 0 p2 t Ap 1 p2 T Ap 2, 20yr ICL 2010/04/05 21

22 Step 8: predicate on the update Comparing the before and after predicates yields P after = P before (δ 1 Ap 1 = r 1 r 2 ) (P 0 u 02 + υ 12 p 1 + p 2 = r 2 ) (R t 0r 2 = 0) (r t 1r 2 = 0)) (P t 0Ap 2 = 0) (p T 1 Ap 2 = 0). 20yr ICL 2010/04/05 22

23 Interpretation Deriving the predicate on the update from the invariant is purely mechanical. 20yr ICL 2010/04/05 23

24 Remaining question How do we derive a, provably correct, algorithm for constructing r 2, p 2, δ 1, u 2, that makes P after satisfied 20yr ICL 2010/04/05 24

25 Hoare triples Hoare triple: {Q}S{R} with Q precondition predicate R postcondition predicate S statement(s) is true if, given Q, the statement S makes R true. Example: {χ = η}χ := χ + 1{χ = η + 1} is true. 20yr ICL 2010/04/05 25

26 Weakest preconditions Let (S, R) be the weakest precondition. Define ( x := exp, R) as R with all instances of x replaced by the expression exp. Then { ( x := exp, R)} x := exp {R} For example, { ( x := x + 1, x = y + 4)} x := x + 1 {x = y + 4} and ( x := x + 1, x = y + 4) = {(x + 1) = y + 4} = {x = y + 3}. 20yr ICL 2010/04/05 26

27 Road map for derivation of the update {P before } {Q 0 = ( δ 1 := exp 0, Q 1 )} S 0 : δ 1 = exp 0 {Q 1 = ( r 2 := exp 1, Q 2 )} S 1 : r 2 := exp 1 {Q 2 = ( u 02 := exp 2, Q 3 )} S 2 : u 02 := exp 2 {Q 3 = ( υ 12 := exp 3, Q 4 )} S 3 : υ 12 := exp 3 {Q 4 = ( p 2 := exp 4, P after )} S 4 : p 2 := exp 4 P after = P before (δ 1 Ap 1 = r 1 r 2 ) (P 0 u 02 + υ 12 p 1 + p 2 = r 2 ) (R0 t r 2 = 0) (r1 tr 2 = 0) (r2 tr 2 = ω 2 ) (P0 tap 2 = 0 (p1 tap 2 = 0) 20yr ICL 2010/04/05 27

28 Start at the end: compute p 2 Final triple and so {Q 4 = ( p 2 := exp 4, P after )} S 4 : p 2 := exp 4 {P after } Q 4 = ( p 2 := exp 4, P after ) = {P before (δ 1 Ap 1 = r 1 r 2 ) (P 0 u 02 + υ 12 p 1 + exp 4 = r 2 ) (R0 tr 2 = 0) (r1 tr 2 = 0) (P0 taexp 4 = 0) (p1 taexp 4 = 0)} exp 4 = r 2 P 0 u 02 υ 12 p 1 20yr ICL 2010/04/05 28

29 Substituting exp 4 = r 2 P 0 u 02 υ 12 p 1 : Q 4 = {P before (δ 1 Ap 1 = r 1 r 2 ) (R0 tr 2 = 0) (r1 tr 2 = 0)) (P0 tar 2 P0 tap 0u 02 = 0) (p1 tar 2 p1 tap 0u 02 υ 12 p1 tap 1 = 0)} Similarly, we can determine υ 12 := exp 3 = (p t 1 Ar 2 p t 1 AP 0u 02 )/p t 1 Ap 1, u 02 := exp 2 = (P t 0 AP 0) 1 P t 0 Ar 2 r 2 := exp 1 = r 1 δ 1 Ap 1 δ 1 := exp 0 = r t 1 r 1/r t 1 Ap 1 20yr ICL 2010/04/05 29

30 Conclusion Krylov method derivation can be made systematic, and can almost be done by a mechanical system: Human decides on the PME and invariant System derives the worksheet, in particular the update Degrees of freedom can be explored by search strategies The algorithm is proved correct by derivation; translation to code can be fully automated Research question: can all Krylov methods be derived this way? Will interesting new variants arise? Can we use Flame for other predicates, e.g. relating to performance? 20yr ICL 2010/04/05 30

31 As a famous Dutchman has said The only effective way to raise the confidence level of a program significantly is to give a convincing proof of its correctness. But one should not first make the program and then prove its correctness, because then the requirement of providing the proof would only increase the poor programmer s burden. On the contrary: the programmer should let correctness proof and program grow hand in hand. E.W. Dijkstra 20yr ICL 2010/04/05 31