4//3 RAM Mol 5-853: Algorithms in th Rl Worl Lolity I: Ch-wr lgorithms Introution Sorting List rnking B-trs Bur trs Stnr thortil mol or nlyzing lgorithms: Ininit mmory siz Uniorm ss ost Evlut n lgorithm y th numr o instrutions xut CPU RAM Rl Mhin Exmpl: Nhlm I/O Mol CPU ~ 0.4ns / instrution 8 Rgistrs L h siz: 64KB lin siz: 64B ss tim:.5ns L h siz: 56KB lin siz: 64B ss tim: 5ns CPU L L L3 L3 h siz: GB lin siz: 64B ss tim: 0ns Mmory ss tim: 00ns Dis ss tim: ~4ms = 4x0 6 ns Th ost o trnsrring t is importnt Dsign lgorithms with lolity in min Min Dis Mmory Astrts singl lvl o th mmory hirrhy Fst mmory (h) o siz M Assing st mmory is r, ut moving t rom slow mmory is xpnsiv Mmory is group into siz-b loks o ontiguous t CPU M/B Fst Mmory B B lok Slow Mmory Cost: th numr o lok trnsrs (or I/Os) rom slow mmory to st mmory.
4//3 Nottion Clriition M: th numr o ots tht it in mmory, n B: th numr o ots tht it in lok So or wor-siz (8 yt) ots, n mmory siz Myt, M = 8,000 Why -Lvl Hirrhy? It s simplr thn onsiring th multilvl hirrhy A singl lvl my omint th runtim o th pplition, so signing n lgorithm or tht lvl my suiint Consiring singl lvl xposs th lgorithmi iiultis gnrlizing to multilvl is otn strightorwr W ll s h-olivious lgorithms ltr s wy o signing or multi-lvl hirrhis Wht Improvmnt Do W Gt? Exmpls Aing ll th lmnts in siz-n rry (snning) Sorting siz-n rry Srhing siz-n t st in goo t strutur Prolm RAM Algorithm I/O Algorithm Snning Θ(N) Θ(N/B) Sorting Θ(N log N) Θ((N/B)log M/B (N/B)) Srhing Θ(log N) Θ(log B N) Prmuting Θ(N) Θ(min(N,sort(N)) For 8-yt wors on xmpl Nhlm B 8 in L-h, B 000 on is log B 3 in L-h, log B 0 on is Sorting Stnr MrgSort lgorithm: Split th rry in hl MrgSort h surry Mrg th sort surrys Numr o omputtions is O(N logn) on n N-lmnt rry How os th stnr lgorithm hv in th I/O mol?
4//3 lok B = 4 Mrging 3 8 5 0 5 6 30 45 5 63 4 6 7 0 4 33 34 36 37 3 + MrgSort Anlysis Sorting in mmory is r, so th s s is S(M) = Θ(M/B) to lo siz-m rry S(N) = S(N/) + Θ(N/B) S(N) = Θ((N/B)(log (N/M)+)) 3 4 6 7 8 A siz-n rry oupis t most N/B + loks Eh lok is lo on uring mrg, ssuming mmory siz M 3B log (N/M) M M N/ N I/O Eiint MrgSort k-wy Mrg Inst o oing -wy mrg, o Θ(M/B)- wy mrg IOMrgSort: Split th rry into Θ(M/B) surrys IOMrgSort h surry Prorm Θ(M/B)-wy mrg to omin th surrys k Assuming M/B k+, on lok rom h rry its in mmory Thror, only lo h lok on Totl ost is thus Θ(N/B) 3
4//3 IOMrgSort Anlysis Sorting in mmory is r, so th s s is S(M) = Θ(M/B) to lo siz-m rry S(N) = (M/B) S(NB/M) + Θ(N/B) S(N) = Θ((N/B)(log M/B (N/M)+)) log M/B (N/M) M /B M N/(M/B) N MrgSort Comprison Tritionl MrgSort osts Θ((N/B)log (N/M)) I/Os on siz-n rry IOMrgSort is I/O iint, osting only Θ((N/B)log M/B (N/M)) Th nw lgorithm svs Θ(log (M/B)) rtion o I/Os. How signiint is this svings? Consir L3 h to min mmory on Nhlm M = Million, B = 8, N = Billion illion / 8 x 0 vs. illion / 8 x Not hr to lult xt onstnts List Rnking Givn link list, lult th rnk o (numr o lmnts or) h lmnt List rnking in I/O mol Assum list is stor in ~N/B loks! My ump in mmory lot. Exmpl: M/B = 3, B =, lst-rntly-us vition 7 6 5 0 8 3 4 7 6 5 0 8 3 4 Trivil lgorithm is O(N) omputtion stps In gnrl, h pointr n rsult in nw lok trnsr, or O(N) I/Os 4
4//3 Why list rnking? Rovrs lolity in th list (n sort s on th rnking) List rnking outlin. Prou n inpnnt st o Θ(N) nos (i no is in th st, its sussor is not) 0 3 4 5 6 7 8 Gnrlizs to trs vi Eulr tours Usul or vrious orst/tr lgorithms lik lst ommon nstors n tr ontrtion Also us in grph lgorithms lik minimum spnning tr n onnt omponnts. Brig out inpnnt st n solv wight prolm rursivly 7 5 0 3 List rnking outlin 7 5 0 3 List rnking: ) inpnnt st Eh no lips oin {0,} A no is in th inpnnt st i it hooss n its prssor hooss 0 3. Mrg in rig-out nos 0 0 0 0 0 7 6 5 0 8 3 4 Eh no ntrs inpnnt st with pro ¼, so xpt st siz is Θ(N). 5
4//3 List rnking: ) inpnnt st 0 0 0 0 g 0h i 0 0 i Sort y sussor rss twi Intiying inpnnt-st nos iintly: Sort y sussor rss 0 List rnking: ) riging out 0 0h g h i g i h g i h g Atr sort, rquirs O(sn(N ))=O(N/B) lok trnsrs List rnking: ) riging out x+y y List rnking: ) riging out I mil no is in inpnnt st, spli it out Givs list o nw pointrs Sort k to originl orr n sn to intgrt pointr upts ol g h i Sns n sorts to omprss n rmov inpnnt st nos (homwork) upt x h 6
4//3 List rnking: 3) mrg in 7 5 0 3 Uss sorts n sns (homwork) 7 6 5 0 8 3 4 List rnking nlysis. Prou n inpnnt st o Θ(N) nos (kp rtrying until rnom st is goo nough). Brig out n solv rursivly 3. Mrg-in rig-out nos All stps us onstnt numr o sorts n sns, so xpt ost is O(sort(N)) = O((N/B) log M/B (N/B)) I/Os t this lvl o rursion Givs rurrn R(N) = R(N/) + O(sort(N)) = O(sort(N)) B-Trs B-tr/(,3)-tr A B-tr is typ o srh tr ((,)-tr) sign or goo mmory prormn Common pproh or storing srhl, orr t,.g., tss, ilsystms. Oprtions Upts: Insrt/Dlt Quris Srh: is th lmnt thr Sussor/Prssor: in th nrst ky Rng qury: rturn ll ots with kys within rng B 8 35 6 0 4 0 5 6 8 Ots stor in lvs Lvs ll hv sm pth Root hs t most B hilrn Othr intrnl nos hv twn B/ n B hilrn 5 8 5 8 35 37 43 50 43 45 50 5 53 7
4//3 B-tr srh B-tr insrt Srh or Compr srh ky ginst prtitioning kys n mov to propr hil. Cost is O(hight * no siz) Insrt 3 Srh or whr th ky shoul go. I thr s room, put it in 8 35 8 35 6 0 5 43 50 6 0 5 43 50 4 0 5 8 5 8 35 37 50 5 53 4 0 5 8 3 5 8 35 37 50 5 53 6 8 43 45 6 8 43 45 Insrt 4 6 0 8 35 B-tr insrt Srh or whr th ky shoul go. I thr s no room, split th no Splits my propgt up tr 5 3 5 43 50 B-tr insrts Splits ivi th ots / hil pointrs s vnly s possil. I th root splits, nw prnt (this is whn th hight o th tr inrss) Dlts r it mor omplit, ut similr i no rops low B/ hilrn, it is mrg or rln with som nighors. 4 0 5 8 3 5 8 35 37 50 5 53 6 8 3 4 43 45 8
4//3 B-tr nlysis Srh All nos (xpt root) hv t lst Ω(B) hilrn hight o tr is O(log B N) Eh no its in lok Totl srh ost is O(log B N) lok trnsrs. B-tr nlysis Insrt (n lt): Evry split o l rsults in n insrt into hight- no. In gnrl, hight-h split uss hight-(h+) insrt. Thr must Ω(B) insrts in no or it splits gin. An insrt thror pys or (/B) h = O(/B) splits, h osting O() lok trnsrs. Srhing n upting th kys long th rootto-l pth omints or O(log B N) lok trnsrs Sorting with srh tr? Consir th ollowing RAM sort lgorithm:. Buil ln srh tr. Rptly lt th minimum lmnt rom th tr Runtim is O(N logn) Dos this sm lgorithm work in th I/O mol? Just using B-tr is O(N log B N) whih is muh wors thn O((N/B) log M/B (N/B)) Bur tr Somwht lik B-tr: whn nos gin too mny hilrn, thy split vnly, using similr split mtho ll lvs r t th sm pth Unlik B-tr: quris r not nswr onlin (thy r rport in ths) intrnl nos hv Θ(M/B) hilrn nos hv urs o siz Θ(M)
4//3 Insrt Bur-tr insrt M ur Insrt Bur-tr insrt M ur ur ur ur ur hight O(log M/B (N/B)) M/B Strt t root. A itm to n o ur. I ur is not ull, on. Othrwis, prtition th ur n sn lmnts own to hilrn. Anlysis is: M/B Insrting into ur osts O(+k/B) or k lmnts insrt On ovrlow, prtitioning osts O(M/B) to lo ntir ur. Thn M/B sns r prorm, osting O(#Arrys + totl siz/b) = O(M/B + M/B) Flushing ur ownwrs thror osts O(M/B), moving Ω(M) lmnts, or ost o O(/B) h pr lvl An lmnt my mov own t h hight, osting totl o O((/B) hight) = O((/B)log M/B (N/B)) pr lmnt I/O Priority Quu Supporting Insrt n Extrt-Min (no Drs- Ky hr) Kp ur o Θ(M) smllst lmnts in mmory Us ur tr or rmining lmnts. Whil smllst-lmnt ur is too ull, insrt (mximum) lmnt into ur tr I smllst-lmnt ur is mpty, lush ltmost pth in ur tr n lt th ltmost ls Totl ost is O((N/B) log M/B (N/B)) or N ops. Yils optiml sort. Bur-tr vritions To support ltions, upts, n othr quris, insrt rors in th tr or h oprtion, ssoit with timstmps. As rors with th sm ky olli, mrg thm s pproprit. Exmpls o pplitions: DAG shortst pths Ciruit vlution Computtionl gomtry pplitions 0