DATA STRUCTURES I, II, III, AND IV

Transcription

1 Data structures DATA STRUCTURES I, II, III, AND IV I. Amortized Aalysis II. Biary ad Biomial Heaps III. Fiboacci Heaps IV. Uio Fid Static problems. Give a iput, produce a output. Ex. Sortig, FFT, edit distace, shortest paths, MST, max-flow,... Dyamic problems. Give a sequece of operatios (give oe at a time), produce a sequece of outputs. Ex. Stack, queue, priority queue, symbol table, uio fid,. Algorithm. Step-by-step procedure to solve a problem. Data structure. Way to store ad orgaize data. Ex. Array, liked list, biary heap, biary search tree, hash table, Lecture slides by Kevi Waye Last updated o 2/2/18 5:50 AM Appetizer Goal. Desig a data structure to support all operatios i O(1) time. INIT(): create ad retur a iitialized array (all zero) of legth. READ(A, i): retur elemet i i array. WRITE(A, i, value): set elemet i i array to value. true i C or C++, but ot Java Assumptios. Ca MALLOC a uiitialized array of legth i O(1) time. Give a array, ca read or write elemet i i O(1) time. Remark. A array does INIT i Θ() time ad READ ad WRITE i Θ(1) time. Appetizer Data structure. Three arrays A[1.. ], B[1.. ], ad C[1.. ], ad a iteger k. A[i] stores the curret value for READ (if iitialized). k = umber of iitialized etries. C[j] = idex of j th iitialized elemet for j = 1,, k. If C[j] = i, the B[i] = j for j = 1,, k. Theorem. A[i] is iitialized iff both 1 B[i] k ad C[B[i]] = i. Ahead. A[ ] ? ? 33?? B[ ]? 3 4 1? 2?? C[ ] ???? 3 k = 4 A[4]=99, A[6]=33, A[2]=22, ad A[3]=55 iitialized i that order 4

2 Appetizer Appetizer Theorem. A[i] is iitialized iff both 1 B[i] k ad C[B[i]] = i. INIT (A, ) k 0. A MALLOC(). READ (A, i) IF (IS-INITIALIZED (A[i])) RETURN A[i]. WRITE (A, i, value) IF (IS-INITIALIZED (A[i])) A[i] value. Suppose A[i] is the j th etry to be iitialized. The C[j] = i ad B[i] = j. Thus, C[B[i]] = i. B MALLOC(). ELSE ELSE C MALLOC(). RETURN 0. k k + 1. A[i] value. B[i] k. A[ ] ? ? 33?? IS-INITIALIZED (A, i) IF (1 B[i] k) ad (C[B[i]] = i) C[k] i. B[ ]? 3 4 1? 2?? RETURN true. ELSE C[ ] ???? RETURN false. k = 4 5 A[4]=99, A[6]=33, A[2]=22, ad A[3]=55 iitialized i that order 6 Appetizer Theorem. A[i] is iitialized iff both 1 B[i] k ad C[B[i]] = i. Suppose A[i] is uiitialized. If B[i] < 1 or B[i] > k, the A[i] clearly uiitialized. If 1 B[i] k by coicidece, the we still ca t have C[B[i]] = i because oe of the etries C[1.. k] ca equal i. AMORTIZED ANALYSIS biary couter multi-pop stack dyamic table A[ ]? ? 33?? B[ ] C[ ]? 3 4 1? 2?? ???? k = 4 Lecture slides by Kevi Waye A[4]=99, A[6]=33, A[2]=22, ad A[3]=55 iitialized i that order 7 Last updated o 2/2/18 5:50 AM

3 Amortized aalysis Worst-case aalysis. Determie worst-case ruig time of a data structure operatio as fuctio of the iput size. Amortized aalysis. Determie worst-case ruig time of a sequece of data structure operatios. ca be too pessimistic if the oly way to ecouter a expesive operatio is whe there were lots of previous cheap operatios Ex. Startig from a empty stack implemeted with a dyamic table, ay sequece of push ad pop operatios takes O() time i the worst case. Amortized aalysis: applicatios Splay trees. Dyamic table. Fiboacci heaps. Garbage collectio. Move-to-frot list updatig. Push relabel algorithm for max flow. Path compressio for disjoit-set uio. Structural modificatios to red black trees. Security, databases, distributed computig,... SIAM J. ALG. DISC. METH. Vol. 6, No. 2, April Society for Idustrial ad Applied Mathematics 016 AMORTIZED COMPUTATIONAL COMPLEXITY* ROBERT ENDRE TARJANt Abstract. A powerful techique i the complexity aalysis of data structures is amortizatio, or averagig over time. Amortized ruig time is a realistic but robust complexity measure for which we ca obtai surprisigly tight upper ad lower bouds o a variety of algorithms. By followig the priciple of desigig algorithms whose amortized complexity is low, we obtai "self-adjustig" data structures that are simple, flexible ad efficiet. This paper surveys recet work by several researchers o amortized complexity. ASM(MOS) subject classificatios. 68C25, 68E05 9 Biary couter CHAPTER 17 AMORTIZED ANALYSIS biary couter multi-pop stack dyamic table Goal. Icremet a k-bit biary couter (mod 2 k ). Represetatio. a j = j th least sigificat bit of couter. Couter value A[7] A[6] A[5] A[4] Cost model. Number of bits flipped. A[3] A[2] A[1] A[0] 12

4 Biary couter Aggregate method (brute force) Goal. Icremet a k-bit biary couter (mod 2 k ). Represetatio. aj = j th least sigificat bit of couter. Couter value Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O( k) bits. A[5] A[4] A[3] A[2] A[1] A[0] At most k bits flipped per icremet. A[7] A[6] overly pessimistic upper boud Aggregate method. Aalyze cost of a sequece of operatios. Couter value A[7] A[6] A[5] A[4] A[3] A[2] A[1] A[0] Total cost Biary couter: aggregate method Startig from the zero couter, i a sequece of INCREMENT operatios: Bit 0 flips times. Bit 1 flips / 2 times. Bit 2 flips / 4 times. Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. Bit j flips / 2 j times. The total umber of bits flipped is k 1 j=0 2 j < = j j=0 Accoutig method (baker s method) Assig (potetially) differet charges to each operatio. D i = data structure after i th operatio. c i = actual cost of i th operatio. ĉ i = amortized cost of i th operatio = amout we charge operatio i. Whe ĉ i > c i, we store credits i data structure D i to pay for future ops; whe ĉ i < c i, we cosume credits i data structure D i. Iitial data structure D 0 starts with 0 credits. Credit ivariat. The total umber of credits i the data structure 0. ĉ i c i 0 ca be more or less tha actual cost our job is to choose suitable amortized costs so that this ivariat holds Remark. Theorem may be false if iitial couter is ot zero

5 Accoutig method (baker s method) Biary couter: accoutig method Assig (potetially) differet charges to each operatio. Di = data structure after i th operatio. ci = actual cost of i th operatio. ĉ i = amortized cost of i th operatio = amout we charge operatio i. Whe ĉ i > c i, we store credits i data structure D i to pay for future ops; whe ĉ i < c i, we cosume credits i data structure D i. Iitial data structure D 0 starts with 0 credits. Credit ivariat. The total umber of credits i the data structure 0. ĉ i c i 0 Theorem. Startig from the iitial data structure D 0, the total actual cost of ay sequece of operatios is at most the sum of the amortized costs. The amortized cost of the sequece of operatios is: ca be more or less tha actual cost ĉ i Ituitio. Measure ruig time i terms of credits (time = moey). c i. credit ivariat 17 Credits. Oe credit pays for a bit flip. Ivariat. Each 1 bit has oe credit; each 0 bit has zero credits. Accoutig. Flip bit j from 0 to 1: charge 2 credits (use oe ad save oe i bit j). icremet Biary couter: accoutig method Biary couter: accoutig method Credits. Oe credit pays for a bit flip. Ivariat. Each 1 bit has oe credit; each 0 bit has zero credits. Accoutig. Flip bit j from 0 to 1: charge 2 credits (use oe ad save oe i bit j). Flip bit j from 1 to 0: pay for it with the 1 credit saved i bit j. Credits. Oe credit pays for a bit flip. Ivariat. Each 1 bit has oe credit; each 0 bit has zero credits. Accoutig. Flip bit j from 0 to 1: charge 2 credits (use oe ad save oe i bit j). Flip bit j from 1 to 0: pay for it with the 1 credit saved i bit j. icremet

6 Biary couter: accoutig method Credits. Oe credit pays for a bit flip. Ivariat. Each 1 bit has oe credit; each 0 bit has zero credits. Accoutig. Flip bit j from 0 to 1: charge 2 credits (use oe ad save oe i bit j). Flip bit j from 1 to 0: pay for it with the 1 credit saved i bit j. Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. the rightmost 0 bit (uless couter overflows) Each INCREMENT operatio flips at most oe 0 bit to a 1 bit, so the amortized cost per INCREMENT 2. Ivariat umber of credits i data structure 0. Total actual cost of operatios sum of amortized costs 2. Potetial method (physicist s method) Potetial fuctio. Φ(Di) maps each data structure Di to a real umber s.t.: Φ(D0) = 0. Φ(Di) 0 for each data structure Di. Actual ad amortized costs. c i = actual cost of i th operatio. ĉ i = c i + Φ(D i) Φ(D i 1) = amortized cost of i th operatio. accoutig method theorem Potetial method (physicist s method) Biary couter: potetial method Potetial fuctio. Φ(D i) maps each data structure D i to a real umber s.t.: Φ(D i) 0 for each data structure D i. Potetial fuctio. Let Φ(D) = umber of 1 bits i the biary couter D. Actual ad amortized costs. c i = actual cost of i th operatio. ĉ i = c i + Φ(D i) Φ(D i 1) = amortized cost of i th operatio. Theorem. Startig from the iitial data structure D 0, the total actual cost of ay sequece of operatios is at most the sum of the amortized costs. The amortized cost of the sequece of operatios is: icremet ĉ i = = c i (c i + (D i ) (D i 1 )) c i + (D ) (D 0 ) 23 24

7 Biary couter: potetial method Potetial fuctio. Let Φ(D) = umber of 1 bits i the biary couter D. Φ(D0) = 0. Φ(Di) 0 for each Di. Biary couter: potetial method Potetial fuctio. Let Φ(D) = umber of 1 bits i the biary couter D. Φ(D0) = 0. Φ(Di) 0 for each Di. icremet Biary couter: potetial method Famous potetial fuctios Potetial fuctio. Let Φ(D) = umber of 1 bits i the biary couter D. Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. Suppose that the i th INCREMENT operatio flips t i bits from 1 to 0. The actual cost ci ti + 1. The amortized cost ĉi = ci + Φ(Di) Φ(Di 1) ci + 1 ti operatio flips at most oe bit from 0 to 1 (o bits flipped to 1 whe couter overflows) 2. Total actual cost of operatios sum of amortized costs 2. potetial method theorem potetial decreases by 1 for ti bits flipped from 1 to 0 ad icreases by 1 for bit flipped from 0 to 1 27 Fiboacci heaps. Splay trees. Move-to-frot. Preflow push. Red black trees. (T ) = (H) = 2 trees(h) + 2 marks(h) x (f) = T (T ) = w(x) = log 2 size(x) (L) = 2 iversios(l, L ) v : excess(v) > 0 x T w(x) height(v) 0 x 1 x 0 x 2 x 28

8 Multipop stack SECTION 17.4 AMORTIZED ANALYSIS biary couter multi-pop stack dyamic table Goal. Support operatios o a set of elemets: PUSH(S, x): add elemet x to stack S. POP(S): remove ad retur the most-recetly added elemet. MULTI-POP(S, k): remove the most-recetly added k elemets. MULTI-POP(S, k) FOR i = 1 TO k POP(S). Exceptios. We assume POP throws a exceptio if stack is empty. 30 Multipop stack Goal. Support operatios o a set of elemets: PUSH(S, x): add elemet x to stack S. POP(S): remove ad retur the most-recetly added elemet. MULTI-POP(S, k): remove the most-recetly added k elemets. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O( 2 ) time. Use a sigly liked list. PoP ad PUSH take O(1) time each. MULTI-POP takes O() time. overly pessimistic upper boud Multipop stack: aggregate method Goal. Support operatios o a set of elemets: PUSH(S, x): add elemet x to stack S. POP(S): remove ad retur the most-recetly added elemet. MULTI-POP(S, k): remove the most-recetly added k elemets. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. A elemet is popped at most oce for each time that it is pushed. There are PUSH operatios. Thus, there are POP operatios (icludig those made withi MULTI-POP). top

9 Multipop stack: accoutig method Multipop stack: potetial method Credits. 1 credit pays for either a PUSH or POP. Ivariat. Every elemet o the stack has 1 credit. Accoutig. PUSH(S, x): charge 2 credits. - use 1 credit to pay for pushig x ow - store 1 credit to pay for poppig x at some poit i the future POP(S): charge 0 credits. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. Ivariat umber of credits i data structure 0. Amortized cost per operatio 2. Total actual cost of operatios sum of amortized costs 2. Potetial fuctio. Let Φ(D) = umber of elemets curretly o the stack. Φ(D0) = 0. Φ(Di) 0 for each Di. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. [Case 1: push] Suppose that the i th operatio is a PUSH. The actual cost c i = 1. The amortized cost ĉ i = c i + Φ(D i) Φ(D i 1) = = 2. accoutig method theorem Multipop stack: potetial method Multipop stack: potetial method Potetial fuctio. Let Φ(D) = umber of elemets curretly o the stack. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. [Case 2: pop] Suppose that the i th operatio is a POP. The actual cost ci = 1. The amortized cost ĉi = ci + Φ(Di) Φ(Di 1) = 1 1 = 0. Potetial fuctio. Let Φ(D) = umber of elemets curretly o the stack. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. [Case 3: multi-pop] Suppose that the i th operatio is a MULTI-POP of k objects. The actual cost ci = k. The amortized cost ĉi = ci + Φ(Di) Φ(Di 1) = k k =

10 Multipop stack: potetial method Potetial fuctio. Let Φ(D) = umber of elemets curretly o the stack. Φ(D0) = 0. Φ(Di) 0 for each Di. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTI-POP operatios takes O() time. [puttig everythig together] Amortized cost ĉ i 2. 2 for push; 0 for pop ad multi-pop Sum of amortized costs ĉ i of the operatios 2. Total actual cost sum of amortized cost 2. AMORTIZED ANALYSIS biary couter multi-pop stack dyamic table potetial method theorem SECTION Dyamic table Dyamic table: isert oly Goal. Store items i a table (e.g., for hash table, biary heap). Two operatios: INSERT ad DELETE. - too may items iserted expad table. - too may items deleted cotract table. Requiremet: if table cotais m items, the space = Θ(m). Theorem. Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O( 2 ) time. Each INSERT or DELETE takes O() time. overly pessimistic upper boud 39 Whe isertig ito a empty table, allocate a table of capacity 1. Whe isertig ito a full table, allocate a ew table of twice the capacity ad copy all items. Isert item ito table. isert old capacity ew capacity isert cost copy cost Cost model. Number of items writte (due to isertio or copy). 40

11 Dyamic table: isert oly (aggregate method) Dyamic table demo: isert oly (accoutig method) Theorem. [via aggregate method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let c i deote the cost of the i th isertio. c i = i i 1 1 Startig from empty table, the cost of a sequece of INSERT operatios is: Isert. Charge 3 credits (use 1 credit to isert; save 2 with ew item). Ivariat. 2 credits with each item i right half of table; oe i left half. isert N capacity = 16 A B C D E F G H I J K L M lg c i + j=0 < +2 2 j = Dyamic table: isert oly (accoutig method) Dyamic table: isert oly (potetial method) Isert. Charge 3 credits (use 1 credit to isert; save 2 with ew item). Ivariat. 2 credits with each item i right half of table; oe i left half. [iductio] Each ewly iserted item gets 2 credits. Whe table doubles from k to 2k, k / 2 items i the table have 2 credits. - these k credits pay for the work eeded to copy the k items - ow, all k items are i left half of table (ad have 0 credits) Theorem. [via accoutig method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. slight cheat if table capacity = 1 Ivariat umber of credits i data structure 0. Amortized cost per INSERT = 3. Total actual cost of operatios sum of amortized cost 3. Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let Φ(D i) = 2 size(d i) capacity(d i). umber of elemets capacity of array immediately after doublig capacity(di) = 2 size(di) size = 6 capacity = 8 Φ = 4 accoutig method theorem 43 44

12 Dyamic table: isert oly (potetial method) Dyamic table: isert oly (potetial method) Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let Φ(D i) = 2 size(d i) capacity(d i). umber of elemets Case 0. [first isertio] Actual cost c 1 = 1. Φ(D 1) Φ(D 0) = (2 size(d 1) capacity(d 1)) (2 size(d 0) capacity(d 0)) = 1. Amortized cost ĉ i = c i + (Φ(D 1) Φ(D 0)) = = 2. capacity of array Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let Φ(D i) = 2 size(d i) capacity(d i). umber of elemets Case 1. [o array expasio] capacity(d i) = capacity(d i 1). Actual cost c i = 1. Φ(D i) Φ(D i 1) = (2 size(d i) capacity(d i)) (2 size(d i 1) capacity(d i 1)) = 2. Amortized cost ĉ i = c i + (Φ(D i) Φ(D i 1)) = = 3. capacity of array Dyamic table: isert oly (potetial method) Dyamic table: isert oly (potetial method) Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let Φ(D i) = 2 size(d i) capacity(d i). umber of elemets Case 2. [array expasio] capacity(di) = 2 capacity(di 1). Actual cost ci = 1 + capacity(di 1). Φ(Di) Φ(Di 1) = (2 size(di) capacity(di)) (2 size(di 1) capacity(di 1)) = 2 capacity(di) + capacity(di 1) = 2 capacity(di 1). Amortized cost ĉi = ci + (Φ(Di) Φ(Di 1)) = 1 + capacity(di 1) + (2 capacity(di 1)) = 3. capacity of array 47 Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Let Φ(D i) = 2 size(d i) capacity(d i). umber of elemets capacity of array [puttig everythig together] Amortized cost per operatio ĉi 3. Total actual cost of operatios sum of amortized cost 3. potetial method theorem 48

13 Dyamic table: doublig ad halvig Thrashig. INSERT: whe isertig ito a full table, double capacity. DELETE: whe deletig from a table that is ½-full, halve capacity. Efficiet solutio. Whe isertig ito a empty table, iitialize table size to 1; whe deletig from a table of size 1, free the table. INSERT: whe isertig ito a full table, double capacity. DELETE: whe deletig from a table that is ¼-full, halve capacity. Memory usage. A dyamic table uses Θ() memory to store items. Table is always betwee 25% ad 0% full. Dyamic table demo: isert ad delete (accoutig method) Isert. Charge 3 credits (1 to isert; save 2 with item if i right half). Delete. Charge 2 credits (1 to delete; save 1 i empty slot if i left half). Ivariat 1. 2 credits with each item i right half of table. Ivariat 2. 1 credit with each empty slot i left half of table. delete M capacity = 16 A B C D E F G H I J K L M Dyamic table: isert ad delete (accoutig method) Dyamic table: isert ad delete (potetial method) Isert. Charge 3 credits (1 to isert; save 2 with item if i right half). Delete. Charge 2 credits (1 to delete; save 1 i empty slot if i left half). discard ay existig or extra credits Ivariat 1. 2 credits with each item i right half of table. Ivariat 2. 1 credit with each empty slot i left half of table. Theorem. [via accoutig method] Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O() time. Ivariats umber of credits i data structure 0. Amortized cost per operatio 3. Total actual cost of operatios sum of amortized cost 3. accoutig method theorem to pay for expasio to pay for cotractio Theorem. [via potetial method] Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O() time. Pf sketch. Let α(d i) = size(d i) / capacity(d i). Defie (D i )= 2 size(d i) capacity(d i ) (D i ) 1/2 1 2 capacity(d i) size(d i ) (D i ) < 1/2 Φ(D0) = 0, Φ(Di) 0. [a potetial fuctio] Whe α(di) = 1/2, Φ(Di) = 0. [zero potetial after resizig] Whe α(di) = 1, Φ(Di) = size(di). [ca pay for expasio] Whe α(di) = 1/4, Φ(Di) = size(di). [ca pay for cotractio]