Amortized Complexity Verified

Transcription

1 Amortized Complexity Verified Tobias Nipkow August 28, 2014 Abstract A framework for the analysis of the amortized complexity of (functional) data structures is formalized in Isabelle/HOL and applied to a number of standard examples and to two non-trivial ones: skew heaps and splay trees. In the same spirit we show that the move-to-front algorithm for the list update problem is 2-competitive. A preliminary paper describing this research has been presented at the Isabelle Workshop Contents 1 Amortized Complexity Binary Counter Dynamic tables: insert only Stack with multipop Queue Dynamic tables: insert and delete Amortized Analysis of Skew Heaps Amortized Analysis Splay Tree Analysis Time Wlog in tree Analysis of splay Optimal Interpretation Overall analysis List Inversion 14 5 Online Algorithm Move-to-Front 17 1

2 theory Amor imports Complex Main begin 1 Amortized Complexity For the background and most of the examples see some algorithms textbook, for example, [1]. locale amor = fixes init :: s fixes nxt :: o s s fixes inv :: s bool fixes t :: o s real fixes Φ :: s real fixes U :: o s real assumes inv init: inv init assumes inv nxt: inv s = inv(nxt f s) assumes ppos: inv s = Φ s 0 assumes p0 : Φ init = 0 assumes U : inv s = t f s + Φ(nxt f s) Φ s U f s begin fun state :: (nat o) nat s where state f 0 = init state f (Suc n) = nxt (f n) (state f n) lemma inv state: inv(state f n) definition a :: (nat o) nat real where a f i = t (f i) (state f i) + Φ(state f (i+1 )) Φ(state f i) lemma aeq: ( i<n. t (f i) (state f i)) = ( i<n. a f i) Φ(state f n) corollary ta: ( i<n. t (f i) (state f i)) ( i<n. a f i) lemma aa1 : a f i U (f i) (state f i) lemma ub: ( i<n. t (f i) (state f i)) ( i<n. U (f i) (state f i)) 2

3 end 1.1 Binary Counter fun incr where incr [] = [True] incr (False#bs) = True # bs incr (True#bs) = False # incr bs fun t incr :: bool list real where t incr [] = 1 t incr (False#bs) = 1 t incr (True#bs) = t incr bs + 1 definition p incr :: bool list real (Φ incr ) where Φ incr bs = length(filter id bs) lemma a incr: t incr bs + Φ incr (incr bs) Φ incr bs = 2 interpretation incr: amor where init = [] and nxt = %. incr and inv = λ. True and t = λ. t incr and Φ = Φ incr and U = λ. 2 thm incr.ub 1.2 Dynamic tables: insert only fun t ins :: nat nat real where t ins (n,l) = (if n<l then 1 else n+1 ) interpretation ins: amor where init = (0 ::nat,0 ::nat) and nxt = λ (n,l). (n+1, if n<l then l else if l=0 then 1 else 2 l) and inv = λ(n,l). if l=0 then n=0 else n l l < 2 n and t = λ. t ins and Φ = λ(n,l). 2 n l and U = λ. 3 locale table insert = fixes a :: real fixes c :: real assumes c1 [arith]: c > 1 3

4 assumes ac2 : a c/(c 1 ) begin lemma ac: a 1 /(c 1 ) lemma a0 [arith]: a>0 definition b = 1 /(c 1 ) lemma b0 [arith]: b > 0 fun ins :: nat nat nat nat where ins(n,l) = (n+1, if n<l then l else if l=0 then 1 else nat(ceiling(c l))) fun pins :: nat nat => real where pins(n,l) = a n b l interpretation ins: amor where init = (0,0 ) and nxt = %. ins and inv = λ(n,l). if l=0 then n=0 else n l (b/a) l n and t = λ. t ins and Φ = pins and U = λ. a + 1 thm ins.ub end 1.3 Stack with multipop datatype a op stk = Push a Pop nat fun nxt stk :: a op stk a list a list where nxt stk (Push x) xs = x # xs nxt stk (Pop n) xs = drop n xs fun t stk :: a op stk a list real where t stk (Push x) xs = 1 t stk (Pop n) xs = min n (length xs) interpretation stack: amor 4

5 where init = [] and nxt = nxt stk and inv = λ. True and t = t stk and Φ = length and U = λf. case f of Push 0 2 Pop 1.4 Queue See, for example, the book by Okasaki [3]. datatype a op q = Enq a Deq type synonym a queue = a list a list fun nxt q :: a op q a queue a queue where nxt q (Enq x) (xs,ys) = (x#xs,ys) nxt q Deq (xs,ys) = (if ys = [] then ([], tl(rev xs)) else (xs,tl ys)) fun t q :: a op q a queue real where t q (Enq x) (xs,ys) = 1 t q Deq (xs,ys) = (if ys = [] then length xs else 0 ) interpretation queue: amor where init = ([],[]) and nxt = nxt q and inv = λ. True and t = t q and Φ = λ(xs,ys). length xs and U = λf. 2 fun balance :: a queue a queue where balance(xs,ys) = (if size xs size ys then (xs,ys) else ([], rev xs)) fun nxt q2 :: a op q a queue a queue where nxt q2 (Enq a) (xs,ys) = balance (a#xs,ys) nxt q2 Deq (xs,ys) = balance (xs, tl ys) fun t q2 :: a op q a queue real where t q2 (Enq ) (xs,ys) = 1 + (if size xs + 1 size ys then 0 else size xs size ys) t q2 Deq (xs,ys) = (if size xs size ys 1 then 0 else size xs + (size ys 1 )) interpretation queue2 : amor where init = ([],[]) and nxt = nxt q2 and inv = λ(xs,ys). size xs size ys 5

6 and t = t q2 and Φ = λ(xs,ys). 2 size xs and U = λf. case f of Enq 3 Deq Dynamic tables: insert and delete datatype op tb = Ins Del fun nxt tb :: op tb nat nat nat nat where nxt tb Ins (n,l) = (n+1, if n<l then l else if l=0 then 1 else 2 l) nxt tb Del (n,l) = (n 1, if n=1 then 0 else if 4 (n 1 )<l then l div 2 else l) fun t tb :: op tb nat nat real where t tb Ins (n,l) = (if n<l then 1 else n+1 ) t tb Del (n,l) = (if n=1 then 1 else if 4 (n 1 )<l then n else 1 ) interpretation tb: amor where init = (0,0 ) and nxt = nxt tb and inv = λ(n,l). if l=0 then n=0 else n l l 4 n and t = t tb and Φ = (λ(n,l). if 2 n < l then l/2 n else 2 n l) and U = λf. case f of Ins 3 Del 2 end theory Skew Heap Analysis imports../skew Heap/Skew Heap Amor begin 2 Amortized Analysis of Skew Heaps The following proof is a simplified version of the one by Kaldewaij and Schoenmakers [2]. fun rheavy :: a heap bool where rheavy(node l r) = (size l < size r) fun rpath :: a heap a heap list where rpath Leaf = [] rpath (Node l a r) = Node l a r # rpath r fun lpath :: a heap a heap list where lpath Leaf = [] lpath (Node l a r) = Node l a r # lpath l 6

7 fun G where G h = length(filter rheavy (lpath h)) fun D where D h = length(filter (λp. rheavy p) (rpath h)) lemma Gexp: 2 ˆ G h size h + 1 corollary Glog: G h log 2 (size h + 1 ) lemma Dexp: 2 ˆ D h size h + 1 corollary Dlog: D h log 2 (size h + 1 ) function t meld :: a::linorder heap a heap nat where t meld Leaf h = 1 t meld h Leaf = 1 t meld (Node l1 a1 r1 ) (Node l2 a2 r2 ) = (if a1 a2 then t meld (Node l2 a2 r2 ) r1 else t meld (Node l1 a1 r1 ) r2 ) + 1 termination function t meld 2 :: a::linorder heap a heap nat where t meld 2 Leaf Leaf = 1 t meld 2 Leaf (Node l2 a2 r2 ) = t meld 2 r2 Leaf + 1 t meld 2 (Node l1 a1 r1 ) Leaf = t meld 2 r1 Leaf + 1 t meld 2 (Node l1 a1 r1 ) (Node l2 a2 r2 ) = (if a1 a2 then t meld 2 (Node l2 a2 r2 ) r1 else t meld 2 (Node l1 a1 r1 ) r2 ) + 1 termination 2.1 Amortized Analysis fun Φ :: a heap nat where Φ Leaf = 0 7

8 Φ (Node l r) = Φ l + Φ r + (if size l < size r then 1 else 0 ) corollary amor eq: t meld 2 t1 t2 + Φ(meld2 t1 t2 ) Φ t1 Φ t2 = G(meld2 t1 t2 ) + D t1 + D t2 + 1 lemma plus log le 2log plus: assumes [arith]: x 1 y 1 b > 1 shows log b x + log b y 2 log b (x + y) lemma amor le: t meld t1 t2 + Φ (meld t1 t2 ) Φ t1 Φ t2 G(meld t1 t2 ) + D t1 + D t2 + 1 lemma a meld ub: t meld t1 t2 + Φ(meld t1 t2 ) Φ t1 Φ t2 3 log 2 (size t1 + size t2 + 2 ) + 1 (is?l ) datatype a op pq = Insert a Delmin fun nxt pq :: a::linorder op pq a heap a heap where nxt pq (Insert a) h = Skew Heap.insert a h nxt pq Delmin h = del min h fun t pq :: a::linorder op pq a heap nat where t pq (Insert a) h = t meld (Node Leaf a Leaf ) h + 1 t pq Delmin h = (case h of Leaf 1 Node t1 a t2 t meld t1 t2 + 1 ) interpretation pq: amor where init = Leaf and nxt = nxt pq and inv = λ. True and t = t pq and Φ = Φ and U = λf h. case f of Insert 3 log 2 (size h + 3 ) + 2 Delmin 3 log 2 (size h + 3 ) + 4 end theory Splay Tree Analysis imports../splay Tree/Splay Tree 8

9 Amor /src/hol/library/sum of Squares begin 3 Splay Tree Analysis This analysis follows Schoenmakers [4]. 3.1 Time fun t splay :: a::linorder a tree nat where t splay a Leaf = 0 t splay a (Node l b r) = (if a=b then 0 else if a < b then case l of Leaf 0 Node ll c lr (if a=c then 0 else if a < c then if ll = Leaf then 0 else t splay a ll + 1 else if lr = Leaf then 0 else t splay a lr + 1 ) else case r of Leaf 0 Node rl c rr (if a=c then 0 else if a < c then if rl = Leaf then 0 else t splay a rl + 1 else if rr = Leaf then 0 else t splay a rr + 1 )) lemma t splay simps[simp]: t splay a (Node l a r) = 0 a<b = t splay a (Node Leaf b r) = 0 a<b = t splay a (Node (Node ll a lr) b r) = 0 a<b = a<c = t splay a (Node (Node ll c lr) b r) = (if ll = Leaf then 0 else t splay a ll + 1 ) a<b = c<a = t splay a (Node (Node ll c lr) b r) = (if lr = Leaf then 0 else t splay a lr + 1 ) b<a = t splay a (Node l b Leaf ) = 0 b<a = t splay a (Node l b (Node rl a rr)) = 0 b<a = a<c = t splay a (Node l b (Node rl c rr)) = (if rl=leaf then 0 else t splay a rl + 1 ) b<a = c<a = t splay a (Node l b (Node rl c rr)) = (if rr=leaf then 0 else t splay a rr + 1 ) 9

10 declare t splay.simps(2 )[simp del] 3.2 Wlog in tree lemma ex in set tree: t Leaf = bst t = a set tree t. splay a t = splay a t t splay a t = t splay a t fun t splay max :: a::linorder tree nat where t splay max Leaf = 0 t splay max (Node l b Leaf ) = 0 t splay max (Node l b (Node rl c Leaf )) = 0 t splay max (Node l b (Node rl c rr)) = t splay max rr Analysis of splay locale Splay Analysis = fixes α :: real and β :: real assumes a1 [arith]: α > 1 assumes A1 : [0 x; 0 y; 0 z ] = (2 +x+y) (2 +y+z) powr β (2 +x+y) powr β (3 +x+y+z) assumes A2 : [0 l ; 0 r ; 0 lr; 0 r ] = α (2 +l +r ) (2 +lr+r) powr β (3 +lr+r +r) powr β (2 +l +r ) powr β (3 +l +lr+r ) powr β (l +lr+r +r+4 ) assumes A3 : [0 l ; 0 r ; 0 ll; 0 r ] = α (2 +l +r ) (2 +l +ll) powr β (2 +r +r) powr β (2 +l +r ) powr β (3 +l +ll+r ) powr β (l +ll+r +r+4 ) begin lemma nl2 : [ ll 0 ; lr 0 ; r 0 ] = log α (2 + ll + lr) + β log α (2 + lr + r) β log α (2 + ll + lr) + log α (3 + ll + lr + r) definition ϕ :: a tree a tree real where ϕ t1 t2 = β log α (size t1 + size t2 + 2 ) fun Φ :: a tree real where Φ Leaf = 0 Φ (Node l r) = Φ l + Φ r + ϕ l r 10

11 definition A :: a::linorder a tree real where A a t = t splay a t + Φ(splay a t) Φ t lemma A simps[simp]: A a (Node l a r) = 0 a<b = A a (Node (Node ll a lr) b r) = ϕ lr r ϕ lr ll b<a = A a (Node l b (Node rl a rr)) = ϕ rl l ϕ rr rl lemma A simp3 : [ a<b; b<c; bst ll; a set tree ll ] = A a (Node (Node ll b lr) c r) = (case splay a ll of Node lll llr A a ll + ϕ llr (Node lr c r) + ϕ lr r ϕ ll lr ϕ lll llr + 1 ) lemma A simp3 : [ b<a; c<b; bst rr; a set tree rr ] = A a (Node l c (Node rl b rr)) = (case splay a rr of Node rrl rrr A a rr + ϕ rrl (Node l c rl) + ϕ l rl ϕ rl rr ϕ rrl rrr + 1 ) lemma A simp4 : [ b<a; a<c; bst lr; a set tree lr ] = A a (Node (Node ll b lr) c r) = (case splay a lr of Node lrl lrr A a lr + ϕ ll lrl + ϕ lrr r ϕ ll lr ϕ lrl lrr + 1 ) lemma A simp4 : [ c<a; a<b; bst rl; a set tree rl ] = A a (Node l c (Node rl b rr)) = (case splay a rl of Node rll rlr A a rl + ϕ rr rlr + ϕ rll l ϕ rl rr ϕ rll rlr + 1 ) lemma A ub: [ bst t; Node la a ra : subtrees t ] = A a t log α ((size t + 1 )/(size la + size ra + 2 )) lemma A ub2 : assumes bst t a : set tree t shows A a t log α ((size t + 1 )/2 ) lemma A ub3 : assumes bst t shows A a t log α (size t + 1 ) 11

12 definition Am :: a::linorder tree real where Am t = t splay max t + Φ(splay max t) Φ t lemma Am simp3 : [ c<b; bst rr; rr Leaf ] = Am (Node l c (Node rl b rr)) = (case splay max rr of Node rrl rrr Am rr + ϕ rrl (Node l c rl) + ϕ l rl ϕ rl rr ϕ rrl rrr + 1 ) lemma Am ub: [ bst t; t Leaf ] = Am t log α ((size t + 1 )/2 ) lemma Am ub3 : assumes bst t shows Am t log α (size t + 1 ) end 3.4 Optimal Interpretation lemma mult root eq root: n>0 = y 0 = root n x y = root n (x (y ˆ n)) lemma mult root eq root2 : n>0 = y 0 = y root n x = root n ((y ˆ n) x) lemma powr inverse numeral: 0 < x = x powr (1 / numeral n) = root (numeral n) x lemmas root simps = mult root eq root mult root eq root2 powr inverse numeral lemma nl31 : [ (l ::real) 0 ; r 0 ; lr 0 ; r 0 ] = 4 (2 + l + r ) (2 + lr + r) (l + lr + r + r + 4 )ˆ2 lemma nl32 : assumes (l ::real) 0 r 0 lr 0 r 0 shows 4 (2 + l + r ) (2 + lr + r) (3 + lr + r + r) (l + lr + r + r + 4 )ˆ3 12

13 lemma nl3 : assumes (l ::real) 0 r 0 lr 0 r 0 shows 4 (2 + l + r )ˆ2 (2 + lr + r) (3 + lr + r + r) (3 + l + lr + r ) (l + lr + r + r + 4 )ˆ3 lemma nl41 : assumes (l ::real) 0 r 0 ll 0 r 0 shows 4 (2 + l + ll) (2 + r + r) (l + ll + r + r + 4 )ˆ2 lemma nl42 : assumes (l ::real) 0 r 0 ll 0 r 0 shows 4 (2 + l + r ) (2 + l + ll) (2 + r + r) (l + ll + r + r + 4 )ˆ3 lemma nl4 : assumes (l ::real) 0 r 0 ll 0 r 0 shows 4 (2 + l + r )ˆ2 (2 + l + ll) (2 + r + r) (3 + l + ll + r ) (l + ll + r + r + 4 )ˆ3 lemma cancel: x>(0 ::real) = c x ˆ 2 y z u v = c x ˆ 3 y z x u v interpretation S34 : Splay Analysis root /3 lemma log4 log2 : log 4 x = log 2 x / 2 declare log base root[simp] real of nat Suc[simp] lemma A34 ub: assumes bst t shows S34.A a t (3 /2 ) log 2 (size t + 1 ) lemma Am34 ub: assumes bst t shows S34.Am t (3 /2 ) log 2 (size t + 1 ) 13

14 3.5 Overall analysis fun t delete :: a::linorder a tree nat where t delete a Leaf = 0 t delete a t = t splay a t + (case splay a t of Node l x r if x=a then case l of Leaf 0 t splay max l else 0 ) datatype a op st = Splay a Insert a Delete a fun nxt st :: a::linorder op st a tree a tree where nxt st (Splay a) t = splay a t nxt st (Insert a) t = Splay Tree.insert a t nxt st (Delete a) t = Splay Tree.delete a t fun t st :: a::linorder op st a tree real where t st (Splay a) t = t splay a t t st (Insert a) t = t splay a t t st (Delete a) t = t delete a t interpretation ST : amor where init = Leaf and nxt = nxt st and inv = bst and t = t st and Φ = S34.Φ and U = λf t. case f of Splay (3 /2 ) log 2 (size t + 1 ) Insert 2 log 2 (size t + 1 ) + 1 /2 Delete 3 log 2 (size t + 1 ) end theory Inversion imports../list Index/List Index begin 4 List Inversion abbreviation dist perm xs ys distinct xs distinct ys set xs = set ys definition before in :: a a a list bool (( </ / in ) [55,55,55 ] 55 ) where x < y in xs = (index xs x < index xs y y set xs) definition Inv :: a list a list ( a a) set where 14

15 Inv xs ys = {(x,y). x < y in xs y < x in ys} lemma before in setd1 : x < y in xs = x : set xs lemma before in setd2 : x < y in xs = y : set xs lemma not before in: x : set xs = y : set xs = x < y in xs y < x in xs x=y lemma no before ini [simp]: x < y in xs = ( y < x in xs) = True lemma finite Invs[simp]: finite(inv xs ys) lemma Inv id[simp]: Inv xs xs = {} lemma card Inv sym: card(inv xs ys) = card(inv ys xs) lemma Inv tri ineq: dist perm xs ys = dist perm ys zs = Inv xs zs Inv xs ys Un Inv ys zs lemma card Inv tri ineq: dist perm xs ys = dist perm ys zs = card (Inv xs zs) card(inv xs ys) + card (Inv ys zs) definition mtf x xs = (if x set xs then x # (take (index xs x) drop (index xs x + 1 ) xs else xs) lemma mtf id[simp]: x / set xs = mtf x xs = xs lemma before in mtf : assumes z set xs shows x < y in mtf z xs 15

16 (y z (if x=z then y set xs else x < y in xs)) lemma Inv mtf : set xs = set ys = z : set ys = Inv xs (mtf z ys) = Inv xs ys {(x,z) x. x < z in xs x < z in ys} {(z,x) x. z < x in xs x < z in ys} lemma set mtf [simp]: set(mtf x xs) = set xs lemma length mtf [simp]: size (mtf x xs) = size xs lemma distinct mtf [simp]: distinct (mtf x xs) = distinct xs definition swapsuc n xs = xs[n := xs!suc n, Suc n := xs!n] lemma index swapsuc distinct: distinct xs = Suc n < length xs = index (swapsuc n xs) x = (if x = xs!n then Suc n else if x = xs!suc n then n else index xs x) lemma set swapsuc[simp]: Suc n < size xs = set(swapsuc n xs) = set xs lemma before in swapsuc: dist perm xs ys = Suc n < size xs = x < y in (swapsuc n xs) x < y in xs (x = xs!n y = xs!suc n) x = xs!suc n y = xs!n lemma Inv swap: assumes dist perm xs ys Suc n < size xs shows Inv xs (swapsuc n ys) = (if ys!n < ys!suc n in xs then Inv xs ys {(ys!n, ys!suc n)} else Inv xs ys {(ys!suc n, ys!n)}) end theory Move to Front 16

17 imports Complex Main Inversion begin 5 Online Algorithm Move-to-Front It was first proved by Sleator and Tarjan [5] that the Move-to-Front algorithm is 2-competitive. lemma sum telescope: fixes f :: nat a::linordered ab group add shows ( i=0..<k. f (Suc i) f i) = f k f 0 lemma potential: fixes t :: nat a::linordered ab group add and p :: nat a assumes p0 : p 0 = 0 and ppos: n. p n 0 and ub: n. t n + p(suc n) p n u n shows ( i<n. t i) ( i<n. u i) abbreviation before x xs {y. y < x in xs} abbreviation after x xs {y. x < y in xs} lemma finite before[simp]: finite (before x xs) lemma finite after[simp]: finite (after x xs) lemma before conv take: x : set xs = before x xs = set(take (index xs x) xs) lemma card before: distinct xs = x : set xs = card (before x xs) = index xs x lemma card subset before: distinct xs = x : set xs = A before x xs = card(a) index xs x lemma before Un: set xs = set ys = x : set xs = before x ys = before x xs before x ys Un after x xs before x ys 17

18 lemma before disj : dist perm xs ys = x : set xs = before x xs before x ys (after x xs before x ys) = {} lemma phi diff aux: card (Inv xs ys {(y, x) y. y < x in xs y < x in ys} {(x, y) y. x < y in xs y < x in ys}) = card(inv xs ys) + card(before x xs before x ys) int(card(after x xs before x ys)) (is card(?i?b?a) = card?i + card?b int(card?a)) lemma foldr swap inv: i set sws. Suc i < length xs = set (foldr swapsuc sws xs) = set xs size(foldr swapsuc sws xs) = size xs distinct(foldr swapsuc sws xs) = distinct xs lemma card Inv foldr swapsuc: i set sws. Suc i < length xs = distinct xs = card (Inv xs (foldr swapsuc sws xs)) length sws locale MTF Off = fixes q :: nat a fixes init :: a list fixes sw off :: nat nat list assumes dist init[simp]: distinct init assumes sw off size: i set(sw off n). Suc i < size init begin fun s off :: nat a list where s off 0 = init s off (Suc n) = foldr swapsuc (sw off n) (s off n) lemma length s off [simp]: length(s off n) = length init lemma dist s off [simp]: distinct(s off n) 18

19 lemma set s off [simp]: set(s off n) = set init fun s mtf :: nat a list where s mtf 0 = init s mtf (Suc n) = mtf (q n) (s mtf n) definition t mtf :: nat int where t mtf n = index (s mtf n) (q n) + 1 definition t off :: nat int where t off n = index (s off n) (q n) size(sw off n) lemma length s mtf [simp]: length(s mtf n) = length init lemma dist s mtf [simp]: distinct(s mtf n) lemma set s mtf [simp]: set (s mtf n) = set init lemma dperm inv: dist perm (s off n) (s mtf n) definition Phi :: nat int (Φ) where Phi n = card(inv (s off n) (s mtf n)) lemma phi0 : Phi 0 = 0 lemma phi pos: Phi n 0 lemma mtf ub: t mtf n + Phi (Suc n) Phi n 2 t off n theorem t mtf t off : ( i<n. t mtf i) 2 ( i<n. t off i) end 19

20 end References [1] T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. MIT Press, [2] A. Kaldewaij and B. Schoenmakers. The derivation of a tighter bound for top-down skew heaps. Information Processing Letters, 37: , [3] C. Okasaki. Purely Functional Data Structures. Cambridge University Press, [4] B. Schoenmakers. A systematic analysis of splaying. Information Processing Letters, 45:41 50, [5] D. D. Sleator and R. E. Tarjan. Amortized efficiency of list update and paging rules. Comm. ACM, 28(2): ,