Simple Analyses of the Sparse Johnson-Lindenstrauss Transform

Transcription

1 Smple Analyses of the Sparse Johnson-Lndenstrauss Transform Mchael B. Cohen 1, T. S. Jayram 2, and Jelan Nelson 3 1 MIT, 32 Vassar Street, Cambrdge, MA 2139, USA mcohen@mt.edu 2 IBM Almaden Research Center, 65 Harry Road, San Jose, CA 9512, USA jayram@us.bm.com 3 Harvard John A. Paulson School of Engneerng and Appled Scences, 29 Oxford Street, Cambrdge, MA 2138, USA mnlek@seas.harvard.edu Abstract For every n-pont subset X of Eucldean space and target dstorton 1+ε for < ε < 1, the Sparse Johnson Lndenstrauss Transform (SJLT) of [19] provdes a lnear dmensonalty-reducng map f : X l m 2 where f(x) = Πx for Π a matrx wth m rows where (1) m = O(ε 2 log n), and (2) each column of Π s sparse, havng only O(εm) non-zero entres. Though the constructons gven for such Π n [19] are smple, the analyses are not, employng ntrcate combnatoral arguments. We here gve two smple alternatve proofs of the man result of [19], nvolvng no delcate combnatorcs. One of these proofs has already been tested pedagogcally, requrng slghtly under forty mnutes by the thrd author at a casual pace to cover all detals n a blackboard course lecture ACM Subject Classfcaton F.2. General Keywords and phrases dmensonalty reducton, Johnson-Lndenstrauss, Sparse Johnson-Lndenstrauss Transform Dgtal Object Identfer 1.423/OASIcs.SOSA Introducton A wdely appled technque to gan speedup and reduce memory footprnt when processng hgh-dmensonal data s to frst apply a dmensonalty-reducng map whch approxmately preserves the geometry of the nput n a pre-processng step. One cornerstone result along these lnes s the followng Johnson-Lndenstrauss (JL) lemma [16]. Lemma 1 (JL lemma). For all < ε < 1, ntegers n, d > 1, and X R d wth X = n, there exsts f : X R m wth m = O(ε 2 log n) such that y, z X, (1 ε) y z 2 f(y) f(z) 2 (1 + ε) y z 2. (1) The last two authors dedcate ths work to the frst author, who n ths work and other nteractons was a constant source of energy and ntellectual enthusasm. M. B. Cohen s supported by NSF grants CCF and CCF , and by a Natonal Defense Scence and Engneerng Graduate Fellowshp. J. Nelson s supported by NSF grant IIS and CAREER award CCF-13567, ONR Young Investgator award N and DORECG award N , an Alfred P. Sloan Research Fellowshp, and a Google Faculty Research Award. Mchael B. Cohen, T. S. Jayram, and Jelan Nelson; lcensed under Creatve Commons Lcense CC-BY 1st Symposum on Smplcty n Algorthms (SOSA 218). Edtor: Ramund Sedel; Artcle No. 15; pp. 15:1 15:9 Open Access Seres n Informatcs Schloss Dagstuhl Lebnz-Zentrum für Informatk, Dagstuhl Publshng, Germany

2 15:2 Smple Analyses of the Sparse Johnson-Lndenstrauss Transform The target dmenson m gven by the JL lemma s optmal for nearly the full range of n, d, ε; n partcular, for any n, d, ε, there exsts a pont set X R d wth X = n such that any (1 + ε)-dstorton embeddng of X nto R m under the Eucldean norm must have m = Ω(mn{n, d, ε 2 log(ε 2 n)}) [21, 5]. Note that an sometrc embeddng (.e. ε = ) s always achevable nto dmenson m = mn{n 1, d}, and thus the lower bound s optmal except potentally for ε close to 1/ n. All known proofs of the JL lemma nstantate f as a lnear map. The orgnal proof n [16] pcked f(x) = Πx where Π R m d was an approprately scaled orthogonal projecton onto a unformly random m-dmensonal subspace. It was then shown that as long as m = Ω(ε 2 log(1/δ)), x R d such that x 2 = 1, P Π ( Πx > ε) < δ. (2) The JL lemma then followed by settng δ < 1/ ( n 2) and consderng x = (y z)/ y z 2 for each par y, z X, and adjustng ε by a constant factor. It s known that ths bound of m for attanng (2) s tght; that s, m must be Ω(mn{d, ε 2 log(1/δ)}) [15, 17]. One should typcally thnk of applyng dmensonalty reducton technques for applcatons as beng a two-step process: frst (1) one apples the dmenson-reducng map f to the data, then (2) one runs some algorthm on the lower dmensonal data f(x). Whle reducng m typcally speeds up the second phase, n order to speed up the frst phase t s necessary to gve an f whch can be both found and appled to data quckly. To ths end, Achloptas showed Π can be chosen wth..d. entres where Π,j = wth probablty 2/3, and otherwse Π,j s unform n ±1/ m/3 [1]. Ths was accomplshed wthout ncreasng m by even a constant factor over prevous best analyses of the JL lemma. Thus essentally a 3x speedup n step (2) s obtaned wthout any loss n the qualty of dmensonalty reducton. Later, Alon and Chazelle developed the FJLT [2] whch uses the Fast Fourer Transform to mplement a JL map Π wth m = O(ε 2 log n) supportng matrx-vector multplcaton n tme O(d log d + m 3 ). Later work of [3] gave a dfferent constructon whch, for the same m, mproved the multplcaton tme to O(d log d + m 2+γ ) for arbtrarly small γ >. More recently, a sequence of works gve embeddng tme O(d log d) but wth a suboptmal embeddng dmenson m = O(ε 2 log n poly(log log n)) [4, 2, 22, 6, 12]. Note that the lne of work begnnng wth the FJLT requres Ω(d log d) embeddng tme per pont, whch s worse than the O(m x ) tme to embed x usng a dense Π f x s suffcently sparse. Here x denotes the number of non-zero entres n x. Motvated by speedng up dmensonalty reducton further for sparse nputs, Kane and Nelson n [19], followng [1, 18, 7], ntroduced the SJLT wth m = O(ε 2 log n), and wth s = O(εm) nonzero entres per column. Ths reduced the embeddng tme to compute Πx from O(m x ) to O(s x ) = O(εm x ). The orgnal analyss of the SJLT n [19] showed Equaton (2) for m = O(ε 2 log(1/δ)), s = O(ε 1 log(1/δ)) va the moment method. Specfcally, the analyss there for x 2 = 1 defned Z = Πx (3) then used Markov s nequalty to yeld P( Z > ε) < ε q E Z q for some large even nteger q (specfcally q = Θ(log(1/δ))). The bulk of the work was n boundng E Z q, whch was accomplshed by expandng Z q as a polynomal wth exponentally many terms, groupng terms wth smlar combnatoral structure, then employng ntrcate combnatorcs to acheve a suffcently good bound.

3 M. B. Cohen, T. S. Jayram, and J. Nelson 15:3 Our Man Contrbuton. We gve two new analyses of the SJLT of [19], both of whch avod expandng Z q nto many terms and employng ntrcate combnatorcs. As mentoned n the abstract, one of these proofs has already been tested pedagogcally, requrng slghtly under forty mnutes by the thrd author at a casual pace to cover all detals n a blackboard lecture. 2 Prelmnares We say f(x) g(x) f f(x) = O(g(x)), and f(x) g(x) denotes f(x) = Θ(g(x)). For random varable X and q R, X q denotes (E X q ) 1/q. Mnkowsk s nequalty, whch we repeatedly use, states that q s a norm for q 1. If X depends on many random sources, e.g. X = X(a, b), we use X Lq(a), say, to denote the q-norm over the randomness n a (and thus the result wll be a random varable dependng only on b). A Bernoull-Rademacher random varable X = ησ wth parameter p s such that η s a Bernoull random varable (on {, 1}) wth E η = p and σ s a Rademacher random varable,.e. unform n { 1, 1}. Overloadng notaton, a random vector X whose coordnates are..d. Bernoull-Rademacher wth parameter p wll also be called by the same name. For a square real matrx A, let A be obtaned by zerong out the dagonal of A. Throughout ths paper we use F to denote Frobenus norm, and to denote l 2 l 2 operator norm. Both our SJLT analyses n ths work show Eq. (2) by analyzng tal bounds for the random varable Z defned n Eq. (3). We contnue to use the same notaton, where x R d of unt norm s as n Eq. (3). Our frst SJLT analyss uses the followng moment bounds for the bnomal dstrbuton and for quadratc forms wth Rademacher random varables. Lemma 2 ([14]). For Y dstrbuted as Bnomal(N, α) for nteger N 1 and α (, 1), let 1 p N and defne B := p/(αn). Then Y p { p log B p B f B e f B < e A more modern, general proof of the below Hanson-Wrght nequalty can be found n [23]. Theorem 3 (Hanson-Wrght nequalty [11]). For σ 1,..., σ n ndependent Rademachers and A R n n, for all q 1 σ T Aσ E σ T Aσ q q A F + q A. Our second analyss uses a standard decouplng nequalty; a proof s n [25, Remark 6.1.3] Theorem 4 (Decouplng). Let A R n n be arbtrary, and X 1,..., X n be ndependent and mean zero. Then, for every convex functon F : R R E F ( A,j X X j ) E F (4 A,j X X j) jj,j where the X are ndependent copes of the X. Before descrbng the SJLT, we descrbe the related CountSketch of [8], whch was shown to satsfy Eq. (3) n [24]. In ths constructon for Π, one pcks a hash functon h : [d] [m] from a parwse ndependent famly, and a functon σ : [d] { 1, 1} from a 4-wse ndependent famly. Then for each [d], Π h(), = σ(), and the rest of the th column s. It was shown S O S A 2 1 8

4 15:4 Smple Analyses of the Sparse Johnson-Lndenstrauss Transform m m () () Fgure 1 Both dstrbutons have s non-zeroes per column, wth each non-zero beng ndependent n ±1/ s. In (), they are n random locatons, wthout replacement. () s the CountSketch (wth s > 1), whose rows are grouped nto s blocks of sze m/s each, wth one non-zero per block per column n a unformly random locaton, ndependent of other blocks; n ths example, m = 8, s = 4. n [24] that ths dstrbuton satsfes Eq. (3) for m = Ω(1/(ε 2 δ)). Note that the column sparsty s equals 1. The analyss s smply va Chebyshev s nequalty,.e. boundng the second moment of Z. The reason for the poor dependence n m on the falure probablty δ s that we use Chebyshev s nequalty. Ths s avoded by boundng a hgher moment (as n [19], or our frst analyss n ths work), or by analyzng the moment generatng functon (MGF) (as n our second analyss n ths work). To mprove the dependence of m on 1/δ, we allow ourselves to ncrease s. Now we descrbe the SJLT. Ths s a JL dstrbuton over Π havng exactly s non-zero entres per column where each entry s a scaled Bernoull-Rademacher. Specfcally, n the SJLT, the random Π R m d satsfes Π r, = η r, σ r, / s for some nteger 1 s m. The σ r, are ndependent Rademachers and jontly ndependent of the Bernoull random varables η r, satsfyng: (a) For any [d], m r=1 η r, = s. That s, each column of Π has exactly s non-zero entres. (b) For all r [m], [d], E η r, = s/m. (c) The η r, are negatvely correlated: S [d] [n], E (r,) S η r, (r,) S E η r, = (s/m) S. See Fgure 1 for at least two natural dstrbutons satsfyng the above requrements. Thus Πx 2 2 = 1 s m r=1,j=1 d η r, η r,j σ r, σ r,j x x j. Usng (a) above we have (1/s) r η r,x 2 = x 2 2 = 1, so that Z = Πx = 1 s m η r, η r,j σ r, σ r,j x x j. (4) r=1 j Remark. In both our analyses, tem (a) above s only used to remove the dagonal = j terms from eq. (4). Thenceforth, t turns out n both analyses of SJLT that (b) and (c) mply we can assume the η r, are fully ndependent,.e., the entres of Π are fully ndependent. Ths s not the same as sayng we can replace the sketch matrx Π wth fully ndependent entres because then part (a) would be volated and t s mportant for only the cross terms n the quadratc form representng Z to be present. In the analyss we justfy ths assumpton by consderng the nteger moments of Z whch we show here cannot decrease by replacement wth fully ndependent entres. For each nteger q, each monomal n the expanson of Z q has expectaton equal to s q x d1 α 1 x dt α t (E (r,) S η r,) whenever all the d j are even,

5 M. B. Cohen, T. S. Jayram, and J. Nelson 15:5 and S contans all the dstnct (r, ) such that η r, appears n the monomal; otherwse the expectaton equals. Now, s q x d1 α 1 x dt α t s nonnegatve, and E (r,) S η r, (s/m) S. Thus monomals expectatons are term-by-term domnated by the case that all η r, are..d. Bernoull wth expectaton s/m. 3 Proof Overvew Hanson-Wrght analyss. Note Z can be wrtten as the quadratc form σ T A x,η σ, where A x,η s block dagonal wth m blocks, where the rth block s (1/s)x (r) (x (r) ) T but wth the dagonal zeroed out. Here x (r) s the vector wth (x (r) ) = η r, x. To apply Hanson-Wrght, we must then bound A x,η F p and A x,η p, over the randomness of η. Ths was done n [19], but suboptmally, leadng to a smple proof there but of a weaker result (namely, the bound on s proven there was suboptmal by a log(1/ε) factor). As already observed n [19], a smple calculaton shows A x,η 1/s wth probablty 1. In ths work we mprove the analyss of A x,η F p by a smple combnaton of the trangle and Bernsten nequaltes to yeld a tght analyss. MGF analyss. We apply the Chernoff-Rubn bound P( Z > ε) 2e tε E cosh(tz), so that we must bound E cosh(tz) (for t n some bounded range) then optmze the choce of t. We accomplsh our analyss by wrtng Z = X T A X for an approprate matrx A where X s a Bernoull-Rademacher vector, by Taylor expanson of cosh and consderatons smlar to Remark 2. We then bound E cosh(tx T A X) usng decouplng followed by arguments smlar to [13, 23]. We note one can also recover an MGF-based analyss by specalzng the analyss of [9] for analyzng sparse oblvous subspace embeddngs to the case of 1-dmensonal subspaces, though the resultng proof would be qute dfferent from the one presented here. We beleve the MGF-based analyss we gve n ths work appeals to more standard arguments, although the analyss n [9] does provde the advantage that t yelds tradeoff bounds for s, m. 4 Our SJLT analyses 4.1 A frst analyss: va the Hanson-Wrght nequalty Theorem 5. For Π comng from an SJLT dstrbuton, as long as m ε 2 log(1/δ) and s εm, x : x 2 = 1, P Π ( Πx > ε) < δ. Proof. As noted, we can wrte Z as a quadratc form Z = Πx = 1 s m def η r, η r,j σ r, σ r,j x x j = σ T A x,η σ, r=1 j Set q = Θ(log(1/δ)) = Θ(s 2 /m). By Hanson-Wrght and the trangle nequalty, Z q q A x,η F + q A x,η q q A x,η F q + q A x,η q, where A x,η s defned n Secton 3. Snce A x,η s block-dagonal, ts operator norm s the largest operator norm of any block. The egenvalue of the rth block s at most (1/s) S O S A 2 1 8

6 15:6 Smple Analyses of the Sparse Johnson-Lndenstrauss Transform max{ x (r) 2 2, x (r) 2 } 1/s, and thus A x,η 1/s wth probablty 1. Q,j = m r=1 η r,η r,j so that A x,η 2 F = 1 s 2 x 2 x 2 j Q,j. j Next, defne Suppose η r1,,..., η rs, = 1 for dstnct r t and wrte Q,j = s t=1 Y t, where Y t s an ndcator random varable for the event η rt,j = 1. By Remark 2 we may assume the Y t are ndependent, n whch case Q,j s dstrbuted as Bnomal(s, s/m). Then by Lemma 2, Q,j q q. Thus, A x,η F q = A x,η 2 F 1/2 q/2 1 s 2 x 2 x 2 j Q,j 1/2 q j 1 x 2 x 2 j Q,j q s j 1/2 (trangle nequalty) 1 m Then by Markov s nequalty and the settngs of q, s, m, P( Πx > ε) = P( σ T A x,η σ > ε) < ε q C q (m q/2 + s q ) < δ. Remark. Less general bounds than Lemma 2 would have stll suffced for our purposes. For example, Bernsten s nequalty and the trangle nequalty together mply Y p αn + p for any p 1, whch suffces for our applcaton snce we were nterested n the case p = αn. 4.2 A second analyss: boundng the MGF In ths analyss we show the followng bound on the symmetrzed MGF of the error: E cosh(tz) exp ( ) K 2 t 2 m, for t s K, where K = 4 2 (5) Usng the above, we obtan tal estmates n a standard manner. By the generc Chernoff- Rubn bound: P( Z > ε) 2e tε E cosh(tz) 2 exp ( K 2 t 2 m tε), for all t s K Optmzng over the choce of t, we obtan the tal bound: P( Z > ε) 2 max { exp( C 2 ε 2 m), exp( Cεs) }, where C = Remark. The cross-over pont for the two bounds s when s m = Θ(ε). To obtan a falure probablty of δ, ths yelds the desred s = O ( 1 ε log( )) ( 1 δ and m = O 1 ε log ( )) 1 2 δ. Our goal now s to prove eq. (5) for t satsfyng t s K. Now by Taylor expanson, we have E cosh(tz) = t q even q q! E Z q. Therefore, by secton 2, we may assume that the η r, are fully ndependent to bound E cosh(tz) from above. Now E cosh(tz) = 1 2 (E exp(tz) + E exp( tz)) max { E exp(tz), E exp( tz) }, for all t R. Let B def = 1 s xxt. Let Π = 1 s H and let Y 1, Y 2,..., Y m denote the rows of H. Then Z = m r=1 Y r T B Y r. By the ndependence

7 M. B. Cohen, T. S. Jayram, and J. Nelson 15:7 assumpton, Y are..d. Bernoull-Rademacher vectors. Lettng Y denote an dentcal copy of a sngle row of H, E exp(±tz) = r E exp(±ty T r B Y r ) = ( E exp(±ty T B Y ) ) m, for all t R (6) Let Y be an ndependent copy of Y. By decouplng (Theorem 4), E exp(ty T B Y ) E exp(4ty T BY ) = E exp(y T BY ), We show below that for all t R, where B def = 4tB (7) E exp(y T BY ) 1 + K2 t 2 m 2, provded t s K, where K = 4 2 (8) Substtutng ths bound n eq. (7) and combnng wth eq. (6), we obtan: E exp(±tz) ( ) 1 + K2 t 2 m ( m exp K ) 2 t 2 2 m, provded t s K, where K = 4 2, whch completes the proof of (5) as desred. It remans to prove eq. (8). Blnear forms of Bernoull-Rademacher random varables. The MGF of a Bernoull-Rademacher random varable X = ησ wth parameter p equals E exp(tx) = 1 p + p E exp(tσ) 1 p + p exp(t 2 /2), for all t R. Let λ(z) def = exp(z) 1. Rewrtng the above, we have E λ(tx) p λ(t 2 /2) = p E λ(tg), where g N (, 1). We show an analogous replacement nequalty for Bernoull-Rademacher vectors. Lemma 6. Let Y be a Bernoull-Rademacher vector wth parameter p. Then: E λ(b T Y ) p λ( b 2 /2) = p E λ(b T g) for all vectors b, where g N (, I n ) Proof. By stablty of Gaussans, E exp(b T g) = exp( b 2 2/2), demonstratng the last equalty above. Let g(t) def = S t S 1 S λ(b2 /2) for t. We have ( 1 + t λ(b 2 /2) ) = 1 + tg(t). Now: E exp(b T Y ) = E exp(b Y ) = ( 1 + E λ(b Y ) ) ( 1 + p λ(b 2 /2) ) = 1 + pg(p) Thus, E λ(b T Y ) pg(p) pg(1), snce g(t). To conclude, we clam that g(1) = λ( b 2 2/2). Indeed: 1 + g(1) = (1 + λ(b 2 /2)) = exp(b 2 /2) = exp ( b 2 /2 ) = 1 + λ ( b 2 2/2 ) Let p def = s m. In the left sde of eq. (8), we have E exp(y T BY ) = 1 + E λ(y T BY ). By the law of total expectaton: E Y,Y λ(y T BY ) = E Y E Y [λ((y T B)Y ) Y ] p E Y E g [λ((y T B)g ) Y ] (by lemma 6, appled to Y ) Exchange the order of expectatons of Y and g va Fubn-Tonell s theorem. Now apply lemma 6, ths tme to Y. Fnsh usng the law of total expectaton whch yelds an upper bound of p 2 E λ(g T Bg ). Thus: E exp(y T BY ) 1 + p 2 E λ(g T Bg ) (9) S O S A 2 1 8

8 15:8 Smple Analyses of the Sparse Johnson-Lndenstrauss Transform In order to be self-contaned we nclude a standard proof of the followng lemma, though note that the lemma tself s equvalent to the Hanson-Wrght nequalty for gaussan random varables snce t gves a bound on the MGF of decoupled quadratc forms n gaussan random varables. Lemma 7. E exp(g T Qg ) exp ( Q 2 F ) for ndependent g, g N (, I n ), provded Q 1 2. Proof. Let Q = UΣV T, where Σ = dag(s 1,..., s n ). So E exp(g T Qg ) = E exp(g T UΣV T g ). Snce U s orthonormal, by rotatonal nvarance, U T g N (, I n ) and s ndependent of V T g N (, I n ). Therefore, E exp(g T Qg ) = E exp(g T Σg ). Now g T Σg = s g g, therefore: E exp(g T Σg ) = E E[exp(s g g ) g ] = E exp(s 2 g 2 /2) = 1 1 s 2 Now s 2 Q for each. Use the bound e x 1 x for x 1 2 E exp(g T Qg ) exp(s 2 ) = exp( s 2 ) = exp ( Q 2 ) F so that: Note that B F = 4t B F and B = 4t B. Now B = 1 s xxt, so that B F = B = 1 s. Usng the above proposton n the rght sde of eq. (9) wth Q = B, we obtan: E exp(y T BY ) 1 + p 2 λ ( K 2 t 2 2s 2 ), provded t s K, where K = 4 2 In the rght sde above, use the bound λ(x) 2x, whch holds for x 1 2, and substtute p = s m so that E exp(y T BY ) 1 + K2 t 2 m 2, provded t s K, where K = 4 2 Ths yelds the desred bound stated n eq. (8). References 1 Dmtrs Achloptas. Database-frendly random projectons: Johnson-lndenstrauss wth bnary cons. J. Comput. Syst. Sc., 66(4): , Nr Alon and Bernard Chazelle. The fast Johnson-Lndenstrauss transform and approxmate nearest neghbors. SIAM J. Comput., 39(1):32 322, Nr Alon and Edo Lberty. Fast dmenson reducton usng Rademacher seres on dual BCH codes. Dscrete & Computatonal Geometry, 42(4):615 63, Nr Alon and Edo Lberty. An almost optmal unrestrcted fast Johnson-Lndenstrauss transform. ACM Trans. Algorthms, 9(3):21:1 21:12, Noga Alon and Bo az Klartag. Optmal compresson of approxmate nner products and dmenson reducton. In Proceedngs of the 58th Annual IEEE Symposum on Foundatons of Computer Scence (FOCS), Jean Bourgan. An mproved estmate n the restrcted sometry problem. Geometrc Aspects of Functonal Analyss, Lecture Notes n Mathematcs Volume 2116:65 7, Vladmr Braverman, Rafal Ostrovsky, and Yuval Raban. Rademacher chaos, random Euleran graphs and the sparse Johnson-Lndenstrauss transform. CoRR, abs/ , Moses Charkar, Kevn C. Chen, and Martn Farach-Colton. Fndng frequent tems n data streams. Theor. Comput. Sc., 312(1):3 15, 24.

9 M. B. Cohen, T. S. Jayram, and J. Nelson 15:9 9 Mchael B. Cohen. Nearly tght oblvous subspace embeddngs by trace nequaltes. In Proceedngs of the Twenty-Seventh Annual ACM-SIAM Symposum on Dscrete Algorthms (SODA), pages , Anrban Dasgupta, Rav Kumar, and Tamás Sarlós. A sparse Johnson-Lndenstrauss transform. In Proceedngs of the 42nd ACM Symposum on Theory of Computng (STOC), pages , Davd Lee Hanson and Farroll Tm Wrght. A bound on tal probabltes for quadratc forms n ndependent random varables. Ann. Math. Statst., 42: , Ishay Havv and Oded Regev. The restrcted sometry property of subsampled Fourer matrces. In Proceedngs of the 27th Annual ACM-SIAM Symposum on Dscrete Algorthms (SODA), pages , Potr Indyk and Assaf Naor. Nearest-neghbor-preservng embeddngs. ACM Trans. Algorthms, 3(3):31, Meena Jagadeesan. Smple analyss of sparse, sgn-consstent JL. CoRR, abs/ , T. S. Jayram and Davd P. Woodruff. Optmal bounds for Johnson-Lndenstrauss transforms and streamng problems wth subconstant error. ACM Trans. Algorthms, 9(3):26:1 26:17, Wllam B. Johnson and Joram Lndenstrauss. Extensons of Lpschtz mappngs nto a Hlbert space. Contemporary Mathematcs, 26:189 26, Danel M. Kane, Raghu Meka, and Jelan Nelson. Almost optmal explct Johnson- Lndenstrauss famles. In Proceedngs of the 15th Internatonal Workshop on Randomzaton and Computaton (RANDOM), pages , August Danel M. Kane and Jelan Nelson. A derandomzed sparse Johnson-Lndenstrauss transform. CoRR, abs/ , Danel M. Kane and Jelan Nelson. Sparser Johnson-Lndenstrauss transforms. J. ACM, 61(1):4, January 214. Prelmnary verson n SODA Felx Krahmer and Rachel Ward. New and mproved Johnson-Lndenstrauss embeddngs va the Restrcted Isometry Property. SIAM J. Math. Anal., 43(3): , Kasper Green Larsen and Jelan Nelson. Optmalty of the Johnson-Lndenstrauss lemma. In Proceedngs of the 58th Annual IEEE Symposum on Foundatons of Computer Scence (FOCS), Jelan Nelson, Erc Prce, and Mary Wootters. New constructons of RIP matrces wth fast multplcaton and fewer rows. In Proceedngs of the 25th Annual ACM-SIAM Symposum on Dscrete Algorthms (SODA), pages , Mark Rudelson and Roman Vershynn. Hanson-Wrght nequalty and sub-gaussan concentraton. Electronc Communcatons n Probablty, 18:1 9, Mkkel Thorup and Yn Zhang. Tabulaton-based 5-ndependent hashng wth applcatons to lnear probng and second moment estmaton. SIAM J. Comput., 41(2): , Roman Vershynn. Hgh-Dmensonal Probablty. May 217. Last accessed at http: //www-personal.umch.edu/~romanv/papers/hdp-book/hdp-book.pdf on August 22, 217. S O S A 2 1 8