Lecture 7: Amortized analysis

Transcription

1 Lecture 7: Amortized analysis In many applications we want to minimize the time for a sequence of operations. To sum worst-case times for single operations can be overly pessimistic, since it ignores correlation effects that the operations may have on the data structure. In aggregate analysis we consider amortized time = worst sequence time/number of operations. For data structures with low amortized cost the time for operations can vary considerably. To analyze this we must be able to estimate the variations. We give two methods: Accounting method. Assign differing charges to different operations. Unused amount is stored for later operations to use. The balance is not allowed to become negative. Potential method. Stored work is seen as potential energy, which measures how prone a structure is for an expensive operation. Try to choose potential function that is precise (informative). We illustrate with two examples: 1. Stack S with operations PUSH(S, x), POP(S) of cost O(1), and MULTIPOP(S, k): while not STACK-EMPTY(S) and k > 0 do POP(S) k k 1 The cost of MULTIPOP is min(k,s), where s = S. Note: number of POP number of PUSH. 2. Incrementing a binary counter with k bits. An array A[0..k 1] stores the bits, with least significant bit in A[0]. Let x = k 1 i=0 A[i] 2i, where initially x = 0. Hence x 2 k 1. INCREMENT(A) i 0 while i < A.length and A[i] = 1 do A[i] 0 i i + 1 if i < A.length then A[i] 1 One INCREMENT takes Θ(k) time, when A consists of just 1 s. Thereby n operations from a zeroed counter should take O(nk) time. Figure 17.2 (page 455) shows that the time is < 2n. Accounting method Assign the operations coins according to an approximate cost. When the amortized cost of an operation exceeds the actual cost, assign objects in the data structure the difference as a credit. Stack. Assign PUSH two coins and none to POP and MULTIPOP. One coin pays for PUSH, the other for a later POP. For every sequence of n operations the actual time is amortized time = O(n). Binary counter. Assign two coins to set a bit to 1. One coin pays for the operation, the other coin is saved for a later operation that resets the bit to 0. Each 1 in the counter thereby has a coin stored as credit, and n INCREMENT takes O(n) time. 1

2 Potential method A potential function Φ maps data structure D i to a real number Φ(D i ), its potential energy. The amortized cost ĉ i of the ith operation is the actual cost plus the difference in potential caused by the operation: The total amortized cost of the n operations is ĉ i = c i + Φ(D i ) Φ(D i 1 ) n n n ĉ i = (c i + Φ(D i ) Φ(D i 1 )) = c i + Φ(D n ) Φ(D 0 ) i=1 i=1 i=1 Stack. Φ = number of objects on the stack, with Φ(D i ) 0 = Φ(D 0 ). If the ith operation on a stack with s objects is a PUSH, then Φ(D i ) Φ(D i 1 ) = (s + 1) s = 1. This gives ĉ i = c i + Φ(D i ) Φ(D i 1 ) = = 2. Suppose the ith operation is MULTIPOP(S,k), where k = min(k,s) objects are popped. The actual cost is k, and the potential difference is Φ(D i ) Φ(D i 1 ) = k. Thus the amortized cost of a MULTIPOP is ĉ i = c i + Φ(D i ) Φ(D i 1 ) = k k = 0. Similarly, the cost of a single POP is c i + Φ(D i ) Φ(D i 1 ) = 1+(s 1) s = 0. Hence the total cost of n operations is O(n). Binary counter. The potential after the ith INCREMENT is b i, the number of 1 s in the counter after the operation. Suppose the operation resets t i bits. The actual cost of the operation t i + 1, since at most one bit is set to 1. That is, b i b i 1 t i + 1 and The amortized cost is Φ(D i ) Φ(D i 1 ) (b i 1 t i + 1) b i 1 = 1 t i ĉ i = c i + Φ(D i ) Φ(D i 1 ) (t i + 1) + (1 t i ) = 2 Hence the total cost of n INCREMENT is O(n), if the counter starts at 0. Suppose the counter starts higher up, with initially b 0 1 s. After n INCREMENT the counter has b n 1 s, where 0 < b 0 and b n k, so n n n c i = ĉ i Φ(D n ) + Φ(D 0 ) 2 b n + b 0 = 2n b n + b 0 i=1 i=1 i=1 Note that if we execute at least n = Ω(k) INCREMENT the actual cost is O(n), no matter where the counter starts, since b 0 k. Dynamic tables Given a table, where we do not know in advance how many objects it will store. When it fills, reallocate with a larger size, copying all objects into the new, larger table. When it gets sufficiently small, we might want to reallocate with a smaller size. Goals: 2

3 1. O(1) amortized time per operation. 2. Unused space always constant fraction of allocated space. Load factor α = num/size, where num = number of items stored, size = allocated size. If size = 0, then num = 0 and α = 1. Never allow α > 1 and keep α > constant fraction (goal 2). Table expansion: Consider only insertion. When the table becomes full, double its size and reinsert all items. This guarantees that α 1/2. Each time we actually insert an item into the table, it is an elementary insertion. TABLE-INSERT(T, x) if T.size = 0 then allocate T.table with 1 slot T.size 1 if T.num = T.size then allocate newtable with 2 T.size slots insert all items in T.table in newtable free T.table T.table newtable T.size 2 T.size insert x into T.table T.num T.num + 1 (T.num elementary insertions) (one elementary insertion) Initially, T.num = T.size = 0. Running time: Charge 1 per elementary insertion. Count only elementary insertions, since all other costs are constant per call. c i is the actual cost of ith operation. If not full, c i = 1. If full, we have i 1 items in the table at the start of the ith operation. Have to copy all these i 1 items, then insert ith item c i = i. Hence, c i = O(n), and O(n 2 ) time for n operations. Of course, we do not always expand: c i = { i if i 1 is exact power of 2 1 otherwise Total cost: n i=1 log n c i n + 2 j < n + 2n = 3n j=0 Therefore, aggregate analysis gives amortized cost per operation = 3. Accounting method: Charge $3 per insertion of x, where 3

4 $1 pays for the insertion, $1 pays for x to be moved in the future, $1 pays for some other item to be moved. Suppose we have just expanded, size = m before next expansion, size = 2m after next expansion. Assume that the expansion used up all the credit, so that there is no credit stored after the expansion. Will expand again after another m insertions. Each insertion will put $1 on one of the m items that were in the table just after expansion and will put $1 on the item inserted. Having $2m of credit by next expansion, when there are 2m items to move, is just enough to pay for the expansion, with no credit left over. Potential method: Φ = 2 num size. Initially, num = size = 0 Φ = 0. Just after expansion, size = 2 num Φ = 0. Just before expansion, size = num Φ = num, so enough potential to pay for moving all items. Always have size num size/2 2 num size Φ 0. Amortized cost of ith operation: num i = num, size i = size, Φ i = Φ after ith operation. If no expansion: size i = size i 1, num i = num i 1 + 1, c i = 1. Then we have ĉ i = c i + Φ i Φ i 1 = 1 + (2 num i size i ) (2 num i 1 size i 1 ) = 1 + (2 num i size i ) (2(num i 1) size i ) = = 3 If expansion: size i = 2 size i 1, size i 1 = num i 1 = num i 1, c i = num i = num i. Then we have ĉ i = c i + Φ i Φ i 1 = num i + (2 num i size i ) (2 num i 1 size i 1 ) = num i + (2 num i 2(num i 1)) (2(num i 1) (num i 1)) = num i + 2 (num i 1) = 3 Expansion and contraction. When α drops too low, contract the table. Allocate a new, smaller table, and copy all items. Still want α bounded from below by a constant, and amortized cost per operation = O(1). Measure cost in terms of elementary insertions and deletions. Obvious strategy: Double size when inserting into a full table (when α = 1). Halve size when deletion would make table less than half full (when α = 1/2). Then always have 1/2 α 1. 4

5 However, suppose we fill table. Then insert double; 2 deletes halve; 2 inserts double; 2 deletes halve; etc. We are not performing enough operations after expansion or contraction to pay for the next one. Simple solution: When inserting with α = 1, double as before α = 1/2. When deleting with α = 1/4, halve size α = 1/2. Thus, immediately after either expansion or contraction, we have α = 1/2. And always 1/4 α 1. Intuition: Make sure that we perform enough operations between consecutive expansions/contractions to pay for the change in table size. Need to delete half the items before contraction, and to double number of items before expansion. Either way, number of operations between expansions/contractions is at least a constant fraction of number of items copied. T empty Φ = 0. Φ = { 2 num size if α 1/2 size/2 num if α < 1/2 α 1/2 num size/2 2 num size Φ 0. α < 1/2 num < size/2 Φ 0. Φ measures how far from α = 1/2 we are. α = 1/2 Φ = 2 num 2 num = 0. α = 1 Φ = 2 num num = num. α = 1/4 Φ = size/2 num = 4 num/2 num = num. Therefore, when we double or halve, have enough potential to pay for moving all num items. For α to go from 1/2 to 1, num increases from size/2 to size, for a total increase of size/2. Since Φ increases from 0 to size, Φ needs to increase by 2 for each item inserted. Hence, the coefficient 2 on the num term in the formula for Φ when α 1/2. For α to go from 1/2 to 1/4, num decreases from size/2 to size/4, for a total decrease of size/4. Since Φ increases from 0 to size/4, Φ needs to increase by 1 for each item deleted. Hence, the coefficient of 1 on the num term in the formula for Φ when α < 1/2. Amortized costs: for insert and delete, when α 1/2 and α < 1/2 (use α i, since α can vary a lot). Insert: α i 1 1/2, same analysis as before ĉ i = 3. α i 1 < 1/2 no expansion (only occurs when α i 1 = 1). α i 1 < 1/2 and α i < 1/2 ĉ i = c i + Φ i Φ i 1 = 1 + (size i /2 num i ) (num i 1 /2 num i 1 ) = 1 + (size i /2 num i ) (size i /2 (num i 1)) = 0 5

6 α i 1 < 1/2 and α i 1/2 ĉ i = 1 + (2 num i size i ) (size i 1 /2 num i 1 ) = 1 + (2(num i 1 + 1) size i 1 ) (size i 1 /2 num i 1 ) = 3 num i size i = 3 α i 1 size i size i < 3 2 (size i 1 size i 1 ) + 3 = 3 Therefore, amortized cost of insert is < 3. Delete: If α i 1 < 1/2, then α i < 1/2. No contraction ĉ i = 1 + (size i /2 num i ) (size i 1 /2 num i 1 ) = 1 + (size i /2 num i ) (size i /2 (num i + 1)) = 2 Contraction move + delete [size i /2 = size i 1 /4 = num i 1 = num i + 1] ĉ i = (num i + 1) + (size i /2 num i ) (size i 1 /2 num i 1 ) = (num i + 1) + ((num i + 1) num i ) ((2 num i + 2) (num i + 1)) = 1 If α i 1 1/2, then no contraction. α i 1/2 ĉ i = 1 + (2 num i size i ) (2 num i 1 size i 1 ) = 1 + (2 num i size i ) (2 num i + 2 size i ) = 1 α i < 1/2 with α i 1 1/2, we have num i = num i 1 1 size i 1 /2 1 = size i /2 1 Thus, ĉ i = 1 + (size i /2 num i ) (2 num i 1 size i 1 ) = 1 + (size i /2 num i ) (2 num i + 2 size i ) = size i 3 num i size i 3( 1 2 size i 1) = 2 Therefore, amortized cost of delete is 2. 6