UMass Lowell Computer Science 9.53 Graduate Analysis of Algorithms Dr. Haim Levkowitz Fall 27 Lecture 5 Tuesday, 2 Oct 27 Amortized Analysis
Overview Amortize: To pay off a debt, usually by periodic payments [Websters] 2
Amortized Analysis: creative accounting for operations Average operation cost over sequence of op s can show average cost of operation small even though single op in sequence expensive result must hold for any sequence of these op s no probability involved (unlike average-case analysis) guarantee holds in worst-case analysis method only no effect on code operation 3
Overview (continued) determine amortized cost of one operation part of sequence of operations 4
3 methods Aggregate method Accounting method Potential method 5
3 examples stack with multipop operation binary counter dynamic tables (later on) 6
Aggregate Method find upper bound T(n) on total cost of sequence of n operations amortized cost = average cost per operation = T(n)/n same for all operations in sequence 7
Accounting Method amortized cost can differ across operations overcharge some operations early in sequence store overcharge as prepaid credit on specific data structure objects 8
Potential Method amortized cost can differ across operations (as in accounting method) overcharge some operations early in sequence (as in accounting method) store overcharge as potential energy of data structure as whole (unlike accounting method) 9
Aggregate Method: Stack Operations
Traditional Stack Operations PUSH(S, x) pushes object x onto stack S POP(S) pops top of stack S, returns popped object O() time: consider cost as for our discussion Total actual cost of sequence of n PUSH/POP operations = n
Aggregate Method: Stack Operations (continued) New Stack Operation MULTIPOP(S, k) pops top k elements off stack S entire stack if stack has < k items MULTIPOP(S, k) while not STACK-EMPTY(S) and k > 2 do POP(S) 3 k k - 2 source: 9.53 textbook Cormen et al.
MULTIPOP actual cost for stack containing s items: Use cost = for each PUSH/POP Cost = min(s, k) [# iterations of while loop] Worst-case cost = O(n) 3
Sequence of n PUSH, POP, MULTIPOP operations: initially empty stack Worst-case cost of MULTIPOP O(n) Have n operations worst-case cost of sequence is O(n 2 ) 4
Observation Each object can be popped only once per time that it s pushed Have n PUSHes n POPs including those in MULTIPOP total cost = O(n) Average over n operations O() per operation on average 5
Again, notice no probability Showed worst-case O(n) cost for sequence O() per operation on average Aggregate analysis 6
Fig. 7.: Multipop demo d 7
Binary counter k-bit binary counter A[.. k ] bits A[] = LSB A[k ] = MSB Counts upward from Value of counter is Initial counter value = A[.. k ] = k i= A[ i] 2 i 8
To increment, add (mod 2 k ): INCREMENT(A, k) i while i < k and A[i ] = do A[i ] i i + if i < k then A[i ] Example k = 3 9
Example: k = 3, counter : Counter value A2 A A Cost Underlined bits flip 2 3 Cost: # flips to get here 3 4 5 6 4 7 8 7 4...... 5 2
Example: k = 3, counter : 3 4 Counter value A2 A A Cost Underlined bits flip 2 3 Cost: # flips to get here 3 4 5 6 4 7 8 7 4...... 5 2
Cost: # flips to get here Cost of INCREMENT = (# of bits flipped) Analysis: Each call could flip k bits n INCREMENTs take O(nk)time 22
Observation Not every bit flips every time See table 23
bit 2 i i k flips how often every time /2 the time /4 the time /2 i the time never times in n INCREMENTs n n/2 n/4 n/2 i 24
25 total # of flips = n INCREMENTs costs O(n) Average cost per operation = O() n n n n i i k i i 2 2 / 2 / 2 / = = < = = =
Accounting Method Diff charges to diff op s Some charged more than actual cost Some charged less 26
Amortized cost = amount we charge When amortized cost > actual cost store difference on specific objects in data structure prepaid credit Use credit later to pay for op s whose actual cost > amortized cost 27
Differs from aggregate analysis: Accounting method: diff op s can have diff costs Aggregate analysis: all op s have same cost Credit should never go negative Otherwise sequence of op s for which amortized cost is not upper bound on actual cost Amortized cost would tell us nothing 28
Let c i be actual cost of ith operation Let be amortized cost of ith operation Require ĉ i n i= cˆ i c Total amortized cost of sequence of n operations must be upper bound on total actual cost of sequence Total credit stored in data structure = i must be nonnegative for all n n i= i n = n cˆ i c i= i 29
Accounting Method: Stack Operations Operation Actual Cost Assigned Amortized Cost PUSH 2 POP MULTIPOP min(k,s) 3 source: 9.53 textbook Cormen et al.
Intuition Think trays in cafeteria : push/pop costs $ each When pushing object, pay $2 $ for PUSH $ = prepayment for it being popped by either POP MULTIPOP Each object has $ credit credit can never go negative total amortized cost = O(n) upper bound on total actual cost 3
Binary counter Charge $2 to set bit to $ pays for setting bit to $ prepayment for flipping it back to Have $ of credit for every in counter credit 32
Amortized cost of INCREMENT: Cost of resetting bits to paid by credit At most bit set to amortized cost $2 For n op s, amortized cost = O(n) 33
Potential Method Amortized cost can differ across operations (as in accounting method) Overcharge some operations early in sequence (as in accounting method) Store overcharge as potential energy of data structure as whole (unlike accounting method) Where credit stored in specific object Credit can be used for whichever most flexible of amortized analysis methods 34
Potential Method (cont.) Start w/initial data structure D Perform n ops on it (i =,..., n) Let c i be actual cost of ith op Let D i be data structure after applying ith op Let Φ(D i ) be potential associated w/ D i Amortized cost of ith operation: cˆ i = ci + Φ( Di ) Φ( Di ) Total amortized cost of n operations: n i= n cˆ i = ( ci + Φ( Di ) Φ( Di )) = ci + Φ( Dn ) Φ( D ) i= Must have: Φ( D n ) Φ( D ) to pay in advance To guarantee (n unknown), must have Φ(D ) =, Φ(D i ) for all i n i= terms telescope 35
Potential Method: Stack Operations Potential function value Φ = # items in stack (# $ bills in accounting method) D = empty stack Φ(D )= Since # objects in stack always Φ(D i ) = Φ(D ) for all i 36
Amortized operation costs (assuming stack has s items) Op Actual cost ΔΦ Φ(D i ) - Φ(D i - ) Amortized cost c i + ΔΦ PUSH (s + ) s = + = 2 POP (s ) s = = MULTIPOP k = min(k, s) (s k ) s = k k k = amortized cost of sequence of n operations = O(n) 37
Binary counter Φ = b i = # of s after ith INCREMENT Suppose ith op reset t i bits to 38
c i t i + (resets t i bits, sets bit to ) If b i =, ith op reset all k bits & didn t set one b i = t i = k b i = b i t i If b i >, ith op reset t i bits, set one b i = b i t i + Either way, b i b i t i + ΔΦ(D i ) (b i t i + ) b i = t i c^i = c i + ΔΦ(D i ) (t i + ) + ( t i ) = 2. If counter starts at, Φ(D )= amortized cost of n operations = O(n) 39
Dynamic Tables Scenario: Have table maybe hash table Don t know in advance how many objects will be stored When it fills, must reallocate with larger size copy all objects into new, larger table When it gets sufficiently small might want to reallocate with smaller size Details of table organization not important 4
Goals. O() amortized time per operation 2. Unused space always constant fraction of allocated space 4
Load factor α α(t) = num(t)/size(t), where num(t) = # items stored size(t) = allocated size If size(t) = then num(t) = Call α(t) = Never allow α > Keep α > constant fraction goal #2 42
Table expansion Consider only insertion: When table becomes full double its size reinsert all existing items Guarantees that α /2 Each time we actually insert item into table it s elementary insertion 43
Table-Insert(T, x) Linear in individual insertion time Constant overhead 5-9 then: Expansion 5-7 overhead dominated by transfer cost in 6 Elementary insertion 44
Running time: Charge per elementary insertion Count only elementary insertions since all other costs together are constant per call c i = actual cost of ith operation If not full, c i = If full have i items in table at start of ith op have to copy all i existing items then insert ith item c i = i n operations c i = O(n) O(n 2 ) time for n operations 45
We don t always expand: c i = p i i = 2 Exact power of 2 ow 46
Total cost Total cost aggregate analysis says amortized cost per operation = 3 47
Accounting Method: Charge $3 per insertion of x $ pays for x s insertion $ pays for x to be moved in future $ pays for some other item to be moved 48
Suppose just expanded size = m before next expansion size = 2m after Assume expansion used up all credit no credit stored after expansion Will expand again after another m insertions 49
Each insertion will put $ on one of m items that were in table just after expansion, and will put $ on item inserted Have $2m of credit by next expansion when there are 2m items to move Just enough to pay for expansion with no credit left over! 5
Potential method Φ(T) = 2 num[t ] size[t ] Initially num = size = Φ = Just after expansion size = 2 num Φ = Just before expansion size = num Φ = num have enough potential to pay for moving all items Need Φ, always 5
Always have size num /2 size 2 num size Φ 52
Amortized cost of ith operation: num i = num after ith op size i = size after ith op Φ i = Φ after ith op If no expansion: size i = size i num i = num i + c i = 53
Then we have c^i = c i + Φ i Φ i- = + (2 num i size i ) (2 num i size i ) = + (2 num i size i ) (2(num i ) size i ) // no expansion = + 2 = 3 54
If expansion: size i = 2 size i size i = num i = num i c i = num i + = num i 55
Then we have c^i = c i + Φ i Φ i- = num i + (2 num i size i ) (2 num i size i ) = num i + (2 num i 2(num i )) (2(num i ) (num i )) = num i + 2 (num i ) = 3 56
Expansion and contraction When α drops too low, contract table Allocate new, smaller one Copy all items Still want α bounded from below by constant amortized cost per operation = O() Measure cost in terms of elementary insertions and deletions 57
58
Obvious strategy : Double size when inserting into full table (when α =, after insertion α would become > ) Halve size when deletion would make table < half full (when α = /2, after deletion α would become < /2) Then always have /2 α 59
Example (p. 42) Perform n ops on table T n = exact power of 2 First n/2 : insertions Cost: Θ(n) At end: num[t] = size[t] = n/2 Second n/2 : I, D, D, I, I, D, D, I, I,. I = insert; D = delete 6
Example (p. 42) cont. First I expand to size n Following DD contract to n/2 II expand Etc. Cost of ea. expansion & contraction: Θ(n) There are Θ(n) of them total Θ(n 2 ) amortized cost per op = Θ(n) 6
What s the problem? Not enough Deletions after expansion to pay for contraction Insertions after contraction to pay for expansion Improve: allow load factor drop below /2 Specifically, down to /4 62
63
Simple solution: Double as before: when inserting with α = after doubling, α = /2 Halve size when deleting with α = /4 after halving, α = /2 immediately after either expansion or contraction, have α = /2 Always have /4 α 64
Intuition: Want to make sure that perform enough op s between consecutive expansions/contractions to pay for change in table size Need to delete half items before contraction double number of items before expansion Either way, # ops between expansions/contractions is at least constant fraction of # items copied 65
66
Intuition: Φ measures how far from α = /2 we are α = /2 Φ = 2 num 2 num = α = Φ = 2 num num = num α = /4 Φ = size /2 num = 4 num /2 num = num when double or halve, have enough potential to pay for moving all num items 67
Potential increases linearly between α = /2 and α = α = /2 and α = /4 68
α has diff distances to go to get to or /4, starting from /2 rate of increase of Φ differs For α to go /2 For α to go /2 /4 69
For α to go /2 num increases size/2 size for a total increase of size/2 Φ increases size Φ needs to increase by 2 for each item inserted That s why there s coeff of 2 on num[t] term in formula for Φ when α /2 7
For α to go /2 /4 num decreases size/2 size/4 for total decrease of size/4 Φ increases size/4 needs to increase by for each item deleted That s why there s a coefficient of on the num[t ] term in the formula for when α < /2 7
Amortized costs: more cases insert, delete α /2, α < /2 use αi, since α can vary a lot size does/doesn t change 72
Insert: α i /2 same analysis as before c^i = 3 α i < /2 no expansion only occurs when α i = If α i < /2 and α i < /2: 73
74
Therefore, amortized cost of insert is < 3 75
Delete 76
77
78
79
Therefore, amortized cost of delete is 2 8