Data Structures LECTURE 10. Huffman coding. Example. Coding: problem definition

Transcription

1 Dt Strutures, Spring 24 L. Joskowiz Dt Strutures LEURE Humn oing Motivtion Uniquel eipherle oes Prei oes Humn oe onstrution Etensions n pplitions hpter 6.3 pp in tetook Motivtion Suppose we wnt to store n trnsmit ver lrge iles messges onsisting o strings wors onstrute over n lphet o hrters letters. Representing eh hrter with ie-length oe will not result in the shortest possile ile! Emple: 8-it ASII oe or hrters some hrters re muh more requent thn others using shorter oes or requent hrters n longer ones or inrequent ones will result in shorter ile Dt Strutures, Spring 24 L. Joskowiz Frequen % Fie-length Vrile-length 45 Emple Messge: e A ile o, hrters tkes: 3, = 3, its with ie-length oe , = 224, its on verge with vrile-length oe % less Dt Strutures, Spring 24 L. Joskowiz e oing: prolem einition Represent the hrters rom n input lphet using vrile-length oe lphet, tking into ount the ourrene requen o the hrters. Desire properties: he oe must e uniquel eipherle: ever messge n e eoe in onl one w. he oe must e optiml with respet to the input proilit istriution. he oe must e eiientl eipherle prei oe: no string is prei o nother. Dt Strutures, Spring 24 L. Joskowiz Uniquel eipherle oes Deinition: he oe lphet = {, 2,, n } over the originl lphet is uniquel eipherle i ever messge onstrute rom oe-wors o n e roken own into oe-wors o in onl one w. Question: how n we test i is uniquel eipherle? Lemm: oe is uniquel eipherle i no til is oe-wor. Dt Strutures, Spring 24 L. Joskowiz erminolog Let w, p, n s e wors over the lphet. For w = ps, p is the prei n s is the sui o w. Let t e non-empt wor. t is lle til i there eist two messges 2 m n 2 n suh tht: i n j re oe-wors n i n, j m t is sui o n 2 m t = 2 n he length o wor w is lw. w is non-empt when lw >. l is the mimum length o oe-wor in. Dt Strutures, Spring 24 L. Joskowiz

2 Dt Strutures, Spring 24 L. Joskowiz Uniquel eipherle oes Proo: oe is uniquel eipherle UD i no til is oe-wor. I oe-wor is til then einition there eist two messges 2 m n 2 n whih stis 2 m = 2 n while hus there re two ws to interpret the messge. I is not UD, there eist messges whih n e interprete in more thn one w. Let µ e the shortest suh n miguous messge. hen µ = 2 k = 2 n tht is, ll i s n j s re oe-wors n. hus, without loss o generlit, k is sui o n k is til. est or unique eipherilit. For ever two oe-wors, i n j i j o: I i = j then hlt: is not UD. I or some wor s either i s = j or j s = i then put s in the set o tils 2. For ever til t in n ever oe-wor i in o: I t = j then hlt: is not UD. I or some wor s either ts = j or j s = t then put s in the set o tils. 3. Hlt: is UD. ime ompleit: On 2 l 2 Dt Strutures, Spring 24 L. Joskowiz Emple = {,,,,}. ils:. = t =. = t = 2. ils. = t = is UD Dt Strutures, Spring 24 L. Joskowiz Emple 2 = {,,,}. ils:. = t = 2. ils. = t =. = t =. = t = is oe-wor! is not UD ounter-emple: hs two menings:..... Dt Strutures, Spring 24 L. Joskowiz Dt Strutures, Spring 24 L. Joskowiz Prei oes We onsier onl prei oes: no oe-wor is prei o nother oe-wor. Prei oes re uniquel eipherle einition. A inr prei oe n e represente s inr tree: leves re oe-wors n their requen % internl noes re inr eision points: mens go to the let, mens go to the right o hrter. he inlue the sum o requenies o their hilren. he pth rom the root to the oe-wor is the inr representtion o the oe-wor. Emple: ie-length prei oe e Messge:... Dt Strutures, Spring 24 L. Joskowiz e

3 Emple: ie-length prei oe : e 9 5 Dt Strutures, Spring 24 L. Joskowiz Frequen % Emple: vrile-length prei oe Messge:... e e Dt Strutures, Spring 24 L. Joskowiz Emple: vrile-length prei oe : Frequen % 5 e 9 Dt Strutures, Spring 24 L. Joskowiz Optiml oing An optiml oe is represente s ull inr tree For oe lphet = {, 2,, n } with oewors, ll with positive requenies i >, the tree or n optiml prei oe hs etl leves n internl noes. Deinition: he ost o prei tree is eine s numer o its B require to enoe ll oe-wors B where is the epth in length o oe-wor. Dt Strutures, Spring 24 L. Joskowiz Emple Fie-length oe: Dt Strutures, Spring 24 L. Joskowiz Optiml oing = 3 Vrile-length oe: = 2.24 Optiml oe: the oe with the lowest ost: heorem: Optiml oing is hievle with prei oe. B min Humn oe: eoing Humn invente in 952 gree lgorithm or onstruting n optiml prei oe, lle Humn oe. Deoing:. Strt t the root o the oing tree, re input its. 2. Ater reing go let 3. Ater reing go right 4. I le noe hs een rehe, output the hrter store in the le, n return to the root o the tree. ompleit: On, where n is the messge length. Dt Strutures, Spring 24 L. Joskowiz

4 Humn oe: onstrution Ie: uil the tree ottom-up, strting with the oe-wors s les o the tree n reting intermeite noes merging the two lestrequent ojets, up to the root. o eiientl in the two lest-requent ojets, use minimum priorit queue. he result o the merger o two ojets is new ojet whose requen is the sum o the requenies o the merge ojets. Dt Strutures, Spring 24 L. Joskowiz Emple: Humn oe onstrution Strt: Step : Step 2: 5 e 9 2 : : e e 9 5 : 3 2 Dt Strutures, Spring 24 L. Joskowiz Emple: Humn oe onstrution 2 Emple: Humn oe onstrution 3 Step 3: : e Step 4: 45 : e 9 5 Dt Strutures, Spring 24 L. Joskowiz Dt Strutures, Spring 24 L. Joskowiz Emple: Humn oe onstrution 3 Step 5: : e 9 5 Dt Strutures, Spring 24 L. Joskowiz Humn oe onstrution lgorithm Humn n Q or i to n o llote new noe z letz] Etrt-MinQ rightz] Etrt-MinQ z + InsertQ, z return Etrt-MinQ ompleit: On lg n Dt Strutures, Spring 24 L. Joskowiz

5 Dt Strutures, Spring 24 L. Joskowiz Optimlit proo Lemm : Let e oe lphet n, two oewors in with the lowest requenies. hen there eists n optiml prei oe tree in whih n re siling leves. Proo: tke tree o n ritrr optiml prei oe where n re not silings n moi it so tht n eome silings o mimum epth n the tree remins optiml. his n e one with two trnsormtions. Dt Strutures, Spring 24 L. Joskowiz Optimlit proo 2 irst trnsormtion seon trnsormtion Dt Strutures, Spring 24 L. Joskowiz Optimlit proo 3 Let n two oe-wors tht re siling leves t mimum epth in. Assume tht n. Sine n re the two lowest requenies, n. First trnsormtion: ehnge the positions o n in to proue new tree. Seon trnsormtion: ehnge the positions o n in to proue new tree. Show tht the ost o the trees remins the sme. Dt Strutures, Spring 24 L. Joskowiz Optimlit proo 4 ] ] ] ] B B First trnsormtion: euse n Sine is optiml, B = B Dt Strutures, Spring 24 L. Joskowiz Optimlit proo 5 ] ] ] ] B B Seon trnsormtion: euse n Sine is optiml, B = B Dt Strutures, Spring 24 L. Joskowiz onlusion rom Lemm Builing up n optiml tree mergers n egin with the gree hoie o merging together the two oe-wors with the lowest requenies. his is gree hoie sine the ost o single merger is the sum o the lowest requenies, whih is the lest epensive merge.

6 Optimlit proo: lemm 2 Lemm 2: Let e n optiml prei oe tree or oe lphet. onsier n two siling oewors n in n let z e their prent in. hen, onsiering z s hrter with requen z = +, the tree = {,} represents n optiml prei oe or the oe lphet = {,} U {z}. Proo: we irst epress the ost B o tree s untion o the ost B o tree. Optimlit proo: lemm 2 2 For ll in {,}, = n thus =. Sine = = z +, we get: + = + ] z + = z z + + ] hereore, B = B + + ] B = B + ] Dt Strutures, Spring 24 L. Joskowiz Dt Strutures, Spring 24 L. Joskowiz Optimlit proo: lemm 2 3 We prove the lemm ontrition: Suppose tht oes not represent n optiml prei oe or. hen there eist tree whose ost is etter thn tht o : B < B. B Lemm, hs two silings, n. Let e the tree with the ommon prent o n reple le z with requen z = +. hen: B = B + ] < B + ] = B ieling ontrition to eing n optiml oe or. Dt Strutures, Spring 24 L. Joskowiz Optimlit proo: Humn lgorithm heorem: Humn s lgorithm proues n optiml prei oe. Proo: B inution on the size o the oe lphet, using Lemms n 2. For = 2 it is trivil, sine the tree hs two leves, ssigne to n, oth o length. Dt Strutures, Spring 24 L. Joskowiz he inution step Suppose the Humn lgorithm genertes n optiml oe or oe o size n, let us prove this or with = n +. Let e the tree generte or the Humn lgorithm, Let n e two noes with miniml requenies tht the Humn lgorithm piks irst. Suppose in ontrition tht S is tree or =n+, whih is stritl etter thn : BS< B. B Lemm, we n ssume tht S hs, s silings. Deine the noe z to e their prent, S to e the su-tree o S without n, to e the su-tree o without,. is the Humn oe generte or {,} U {z} with z = +. S esries prei oe or {,} U {z } with z = +. Dt Strutures, Spring 24 L. Joskowiz ompre now S n : = z he inution step 2 S S = BS = BS S + S ] + z S z Sine S = S = S z +, we get: BS = BS n similrl, B = B Dt Strutures, Spring 24 L. Joskowiz z

7 he inution step 3 But now i BS < B we hve tht BS < B. Sine S = = n, this ontrits the inution ssumption tht, the Humn oe or {.} U {z} is optiml! Etensions n pplitions -r oes: we merge the ojets with the lest requen t eh step, reting new ojet. whose requen is the sum o the requenies Mn more oing tehniques! Dt Strutures, Spring 24 L. Joskowiz Dt Strutures, Spring 24 L. Joskowiz