Lecture Oct. 10, 2013

Transcription

1 CS 229r: Algorths for Bg Data Fall 203 Prof. Jela Nelso Lecture Oct. 0, 203 Scrbe: To Morga Overvew I ths lecture ad the revous oe, we focused o fdg Johso-Ldestrauss atrces that allow for faster ultlcato. Last lecture, we reseted a costructo that erfored faster whe the vectors to be trasfored are sarse. Ths lecture, we wll reset a JL atrx that erfors faster ultlcato wth dese vectors. 2 Fast Johso-Ldestrauss Trasfor Here we wll reset the Fast Johso-Ldestrauss Trasfor of [AC06]. The tal dea s to use a scaled salg atrx S R. S s all zeros excet for each row havg a / a rado colu. Observe that Sx 2 2 s the scaled ercal or of rado x2 s, whch gves us our desred exectato sce, f we select r [] uforly at rado, E r x 2 r P(r ) x 2 x 2 2. The roble however, s that ths ay have hgh varace. For exale, f ost of x s ass s oe coordate, the eeds to be o the order of to exect to ht t. Our soluto to ths s to recodto x wth soe other atrx, such that the ors are reserved wll sreadg the x s ass across the coordates. [AC06] s srato for ths recodtoer coes fro the Heseberg s ucertaty rcal, whch states that a vector x ad ts Fourer trasfor ˆx caot both be sharly cocetrated. However, we caot just take the Fourer trasfor sce f the tal x s well sread, ˆx ay becoe cocetrated. [AC06] s ultate atrx was Π P HD, where D s a atrx wth rado sgs o the dagoal, H s the Fourer atrx, ad P s a rado suer-sarse atrx. Wth ths they acheved a ultlcato te betwee Π ad x of O ( log + log 2 ( ) ( ɛ 2 δ log )) δ. We wll rove a slghtly weaker verso of ths result by lettg P be a slow JL atrx tes the sale atrx S descrbed earler. 2. Ma Results There are ay ossbltes for the atrx H, other tha just the Fourer atrx. It suffces for H to satsfy the followg roertes:. Hx ca be couted O( log ) te.

2 2., j H,j / 3. H s a orthogoal atrx (HH T H T H I) Istead of the Fourer atrx, we wll use H such that H,j ( ),j, where, j s the dot roduct of the bary reresetatos of ad j. We wll requre that s a ower of 2 (f t s ot, sly ad x wth 0s utl t s). Ths H obvously satsfes roerty 2. To see that t satsfes roerty, observe that f H k s the H atrx whe k, the H k 2 [ Hk/2 H k/2 H k/2 H k/2 Ths follows fro lookg at how the ost sgfcat bt of ad j cotrbute to, j ad thus H,j. Let ad j be the ad j after reovg the ost sgfcat bt. I all but the botto rght quadrat, ether or j have a 0 ther frst bt, so, j, j, whle the botto rght quadrat, ad j both have a frst, so, j +, j. Usg ths recurrece, we ca ultly Hx te O( log ). To see that ths H satsfes roerty 3, we wll verfy that H T H I. Wrte H h h. The ad for j, (H T H) h, h (H T H) j h, h j ]. ( ( ),r ) 2 ( ) j,r 0, where s the XOR oerato. The fal equalty follows fro the fact that sce ad j dffer at least oe sot, j 0 ad so the sae uber of rs ake j, r eve as ake t odd. Now we wll ove o to rovg that Π oerates as desred. Frst, we wll show that HD successfully sreads the ass of x. Let y HDx ad z Sy. ) Lea. P D ( y c < δ 2. l(2/δ) Proof. Let α ± be the th dagoal etry of D. We wll beg by boudg the th oet of y so that we ca aly Markov s equalty to t. We wll do ths oce aga by sertg rado Gaussas g g. 2

3 y α r (h r x r ) π 2 E α r g r (h r x r ) g π α r g r (h r x r ) 2 π g 2 r (h r x r ) π (h r x r ) 2 2 g r 2 x 2 Where the frst equalty follows fro Jese s equalty ad the fal equalty follows fro the 2-stablty of Gaussas. As a asde, the roerty that for rado sgs σ, σ x x 2 s kow as Khtche s equalty. We ca ow aly Markov s equalty geeralzed to the th oet P( y > λ) < λ E y obtag ( ) l(2/δ) P y > c < ( c l(2/δ) ) ( c 2 l(2/δ) ) Whch by settg c 2 l(/δ)/e 2 ad c e s bouded by e e c2 e 2 l(2/δ) δ 2 to whch we ca sly aly a uo boud to rove our cla. Now we kow that HD sreads the ass of x as desred, ad t s easy to see that y 2 x 2. All that reas s to argue that S reserves the ors of ut y vectors. l(2/λ) Lea 2. Codtoed o the good evet that y, P ( Sy 2 ) ɛ δ/2. 3

4 Proof. We wll be usg the followg Cheroff boud. Gve deedet rado varables X,..., X, X X, µ E X, σ 2 E(X E X) 2, ad X K wth robablty, { P( X µ > λ) ax e cλ2 /σ 2, e cλ/k}. We wll wrte S r /δ r where δ r {0, } s our rado sale of the colus for row r. Thus, r, δ r. Recall that z Sy. Let q r zr 2 ( ) 2 δ r y δ r y 2 + δ r δ rj y y j j δ r y 2. We care about z 2 r q r, ad the q r s are deedet, so we ca aly Cheroff f we boud σ 2 ad K. K y 2 l(/δ). ( ) σ 2 E q 2 2 E 2 δ r y 4 y 4 y 2 y l(/δ) y 2 Pluggg ths to the Cheroff boud we get P ( Sy 2 > ɛ ) ( ) P q r > ɛ r { ax e cɛ2 / l(/δ), e cɛ/ l(/δ)}. The frst ter doates, ad so we set /ɛ 2 l(/δ) l(/δ) to get the δ/2 boud that we desre. Note that our here s losg a lttle bt fro the desred JL deso, so after alyg SHD we aly a fal atrx T whch s a oralzed rado sg atrx wth O(ɛ 2 log δ ) rows (the 4

5 sae kd we used to do slow JL) to fx t u. Thus our fal JL atrx s Π T SHD. Now let s look at the te t takes to coute Πx. Multlyg by D takes O() te, H takes O( log ), S takes O() O() te, ad T takes O ( log ( ) ɛ 2 δ log ( ) ( ɛ 2 δ log )) δ O( 3 ) te. So the total te s O( log + 3 ). 2.2 Later Work Later work focused o rovg the addtve O( 3 ) cooet of the te. [AL09] reduced t to O γ ( 2+γ ). [AL3] ad [KW] further reduced t to O(). They acheved ths by usg the sae Π SHD but usg better aalyss. I artcular, rather tha showg that HD sreads ass, they focused o SH whch they showed exhbts the soetry roerty, whch eas that t reserves the ors of k-sarse vectors. These last works however lose a lttle bt the desoalty reducto, that they have O(ɛ 2 log N(log log N) 3 ). [NPW4] ataed the sae rug te, whle rovg the deso to O(ɛ 2 log N(log log N) 2 ). 3 Other Johso-Ldestrauss Work So far we have always looked at usg lear as, but ca we do better wth other as? Defto 3. For the etrc sace (T, l 2 ), defe the doublg costat λ(t ) as the sallest uber λ such that ay l 2 ball of radus r ca be covered by at ost λ balls of radus r/2 cetered T. Defto 4. The doublg deso of T s d(t ) log 2 (λ(t )). Lea 5. Ay ebeddg of T to l 2 wth dstorto at ost C ust have d(t ). Proof. (Sketch) Take a ball the orgal sace, ad a t to l 2 usg the ebeddg. Cover ths ebedded ball wth a few (r/f(k))-balls l 2, the the take the verse ages of these balls the orgal sace. They should cover the orgal ball. Cojecture 6. ([GKL03], [LP0]) For ay T, t ca be ebedded to l 2 that C f(d(t )) f (d(t )). For soe fuctos f ad f. wth dstorto C such Ths cojecture caot be roved wth lear as. For exale, here s a exale oted out to e by Huy Nguy ê (though the exstece of slar exales was kow eve at the te of [GKL03]). Take the set { xj0 A( + j)e T + + e x j A( + j)e + + e j for soe large A. Ths set looks lke a bg le alog e + wth erodc tght clusters of ots devatg very slghtly fro the le. A lear a for ths set requres log N desos, because 5

6 for ay lear a we ust have Πx j0 Πx j Πe Πe j for whch [Alo03] gves us a boud of Ω(log N). However, f we use the olear ag f(x j0 ) (A( + j), 0, ) f(x j ) (A( + j),, 0) the we ca acheve costat deso ad dstorto. Refereces [AC06] [AL09] [AL3] Nr Alo ad Berard Chazelle. Aroxate earest eghbors ad the fast johsoldestrauss trasfor. I Proceedgs of the 38th aual ACM syosu o Theory of coutg (STOC), ages ACM, Nr Alo ad Edo Lberty. Fast Deso Reducto Usg Radeacher Seres o Dual BCH Codes. Dscrete & Coutatoal Geoetry, 42(4): , Nr Alo ad Edo Lberty. A alost otal urestrcted fast johso-ldestrauss trasfor. ACM Trasactos o Algorths (TALG), 9(3):2, 203. [Alo03] Noga Alo. Probles ad results extreal cobatorcs. Dscrete Matheatcs, 273():3 53, [GKL03] Aua Guta, Robert Krauthgaer, ad Jaes R Lee. Bouded geoetres, fractals, ad low-dstorto ebeddgs. I Proceedgs of the 44th Syosu o Foudatos of Couter Scece (FOCS), ages IEEE, [KW] Felx Kraher ad Rachel Ward. New ad roved Johso-Ldestrauss ebeddgs va the restrcted soetry roerty. SIAM Joural o Matheatcal Aalyss, 43(3):269 28, 20. [LP0] Urs Lag ad Corad Plaut. Blschtz ebeddgs of etrc saces to sace fors. Geoetrae Dedcata, 87(-3): , 200. [NPW4] Jela Nelso, Erc Prce, ad Mary Wootters. New costructos of r atrces wth fast ultlcato ad fewer rows. I Proceedgs of the 35th Aual ACM-SIAM Syosu o Dscrete Algorths (SODA),