Lecture 9: Tolerant Testing

Transcription

1 Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have bee assumg ether p = q, OR p far from q. What happes f we relax the frst assumpto? I most atural settgs of the problem the dstrbutos eed ot be exactly equal but are close to each other. Our L tester where we used a ubased estmator for p q, already had ths property. L Tester: Dstgushes p q ɛ/ vs p q ɛ O q /ɛ samples. Let us see what happes the case of dstace. Lets start wth our L detty tester ad see what kd of tolerace we acheve from t? L Idetty Tester: We splt the th b to q equal peces ad dstgush p S = q S vs p S q S > ɛ based o our L tester. Sce the L tester s already tolerat the same aalyss wll dstgush p S q S < ɛ vs p S q S > ɛ. let us see what s p S q S explctly. p S q S = = q p q q. p q q p q q def = χ p, q. Thus we are able to dstgush p q > ɛ vs χp, q < ɛ. I fact we show that χp, q upper bouds p q the followg lemma.

2 Lemma.. p q χp, q. Proof. We use Cauchy Schwarz to prove ths. [ p q = p q q ] [ p q [ q ] ] q = q p q q = χ p, q. We have a tester that dstgushes p q > ɛ vs χp, q < ɛ but ths tester does t dstgush betwee p q > ɛ vs p q < ɛ, because χp, q s a very weak upper boud o p q. For stace there could be bs for whch q s very small ad p q s small - ths has a small cotrbuto to p q but a huge cotrbuto to χp, q. So whe p q = ɛ, χp, q could be very large ad thus our tester wll fal to dstgush betwee p q > ɛ vs p q < ɛ. So we wat a tester that dstgushes p q > ɛ vs p q < ɛ. Ufortuately we wll show that ths s very hard eve the regme of costat ɛ. Lets focus o the case whe q = U. Ths kd of tester wats us to approxmate p q = p. Ituto for why L testg s hard: What ca a tester do? The order of the samples does t matter by the symmetry argumet. The oly useful formato the tester gets s the couts of the bs. Now f we look at how may bs have k samples. The expected umber s gve by e mp mp k. For small values of k these gve the momets of p upto k! the expoetals e mp. So what s really easy to do s to approxmate the momets of p or low degree polyomals of p. fp f low degree polyomal. For stace f we are dog L testg we eed to estmate p whch s exactly the above form ad thus s easy to estmate. However ths s ot the case wth orm because x s ot a polyomal. Thus we ca t expect estmatg to be easy. We could always lear the dstrbuto samples ad use that to get a ɛ approxmato to ɛ p q. However that requres a lear umber of samples. I fact we wll see that t really does take samples whe ɛ s a costat. log Research Problem: Fd out the correct depedece of the umber of samples o ad ɛ the ɛ o costat regme.

3 Theorem.. p s a dstrbuto o []. Ay tester that dstgushes betwee p U < 0 vs p U < 0 requres Ω log samples. U s the uform dstrbuto over some subset of bs. Note Ay good tolerat tester should be able to do ths because U ad U are very far apart. Ths was also used to show that estmatg etropy s hard because HU << HU. Ths shows that testg thgs that are t polyomals s gog to be dffcult. Proof. We use the adversary method to prove ths: Take X a radom bt. Oce X s fxed each p s pcked depedetly accordg to some dstrbuto. X = 0 X = p d A = some dstbuto o [0, ]. p d B. We should pck A ad B carefully. Lets fgure out what does our adversary eed to do wth A, B. Whe X = 0 we wat p U to be small. Thus we eed E[ A ] <<. Ths esures that each b we do t add too much mass to p U, so that whe we sum over all the bs we have p U <. Whe X = 0 we wat B to pck 0 half the tme ad be 0 close to half the tme. I order to quatfy ths for B, we use the Earth mover metrc. Defto.3. d EM r, s = f X r Y s E[ X Y ]. Ituto: Sce the margals of X ad Y are fxed we are takg the fmum over all possble correlatos that X ad Y could have. Say whe r = s the we ca precsely match X ad Y, that s f X takes the real t wth probablty r t the we correlate Y to take t wth s t = r t ad thus get d EM = 0. Now lets cosder the case whe s = δ 0. The the fmum would be E r X = t r t. Note that t r t says that we eed to move the mass r t through t a dstace of t to move the probablty mass r t to 0. So geeral t captures the least amout of dstace tmes the probablty mass that s beg moved so that we ca make r, s to be the same dstrbutos. So we wat A s close to δ These choces esure that X = 0 = p U < EMM, d EM A, δ / << ad d EMB, δ 0 + δ <<. ad X = = p U <. 0 0 Now we eed to fgure out what other costrats eed to be put o A ad B so that we ca actually fool the tester. We eed to put the followg two costrats for ths: If p s large the sce the umber of samples that fall b are dstrbuted P omp, there wll be more samples from b ad thus p ca be approxmated well. 3

4 To be precse lets see how far off the emprcal estmate of p would be. Let X deote the umber of samples from b. The X Pr m p > c p /m p /m c p /m The tester would wat to estmate wth a costat probablty ad thus correspods to some costat c. Thus the tester would see a error of p betwee ts emprcal m estmate of p ad actual p. Now say there are a buch of bs whose probabltes p s are all close to α. There ca be at most such bs. Now the total α L error the tester curs by choosg the emprcal estmates for p s for these bs also esurg that the tester wats to be sure wth a costat probablty s E = α α = m αm. Now we kow m log we are amg for ths lower boud, so f α >> the log E 0 ad the tester would lear these p s. We do t wat ths to happe ad thus we costra the support of A ad B to [0, log log ]. The calculato here gves, but for techcal reasos we ca allow coordates wth sze as large as log /, because you caot relably dstgush betwee these etres ad the smaller oes wth oly / log samples. We kow that the best the tester could do s compute the momets of p. So we wat the momets of A, B to be equal. Also sce A, B are supported o [0, log ] ad sce the umber of samples b s P omp whch s tghtly cocetrated aroud mp log, we wot see bs wth more tha Olog samples. Note that the expected umber of bs wth at least k samples s gve by e mp mp k.hece k! the kth momet computato correspods to lookg at the umber of bs wth k samples. Thus the tester ca compute the momets up to Olog. Hece we oly eed to esure that A ad B agree o momets up to Olog. E X A [X k ] = E X B [X k ] k Olog. I fact let D = A B be a pseudo dstrbutoeed ot be postve or ormalzed. It log s supported o [0, c ] ad satfes D ad d EM D, δ δ 0 δ s small. Its low order momets are 0, E X D [X k ] = 0 k c 3 log. Our goal s to come up wth such a D. Its postve ad egatve parts gve the requred A ad B. We thus eed D to have 0 expectato for ay low degree polyomal. There s a clever way to do ths. Its the followg lemma. Lemma.4. p s a degree d polyomal wth dstct real roots r,..., r d. Let a = the a r k = 0, for ay teger 0 k d. p r, 4

5 Note that the pseudo dstrbuto tool we wll use to costruct D. a δ r has zero low degree momets. Ths wll be the Proof. We use polyomal terpolato to fd the uque at most degree d polyomal f that satsfes fr = y [d]. The fx = y j x r j r r j. The coeffcet of x d fx s gve by y = r r j j y a. Now f we wat to prove that a r k = 0, we choose y = r k, [d]. Note that the uque at most degree d polyomal that satsfes fr = r k, [d] s just fx = x k. Sce k d ote that the x d coeffcet of f s 0. Thus we have a r k = 0, 0 k d. We get D by pckg a polyomal g ad usg the above lemma s trck wth the roots. What g do we eed to pck? We eed g to have a root at 0,, ad aother reasoable polyomal factor say T. We log expect T to have roots [0, c ]. Furthermore t eeds to have ce dervatves whch are ot huge Sce ths would esure that A, B are close. Oe atural thg to try are the Chebyshev polyomals. gx = x x x x T d c log Chebyshev Polyomals The degree-d Chebyshev polyomal s the uque polyomal that satsfes T d cos θ = cosdθ. y def = The roots of T d y are y = cos x c log m+π d log : [0, c ] [, ]. m+π, where d = c d 3 log. So the frst root s at y = π. Let c 8d 4 = π c. So the correspodg x would be at approxmately c 6c 4, 3 9 the other roots would be at c 4, c 4 5,..... Note that we choose c, c 3 such that the frst root of T d as a fucto of x above s very far rght from. Thus the three artfcally mplated 5

6 roots of g are well before the roots of T. I partcular f we take c 3 very small the T d y wll be 99/00 o [0, ]. Note that, g 0, g, g. Now we eed to fgure out g at the roots of T d. But observe T dcos θ = d sdθ. s θ At the roots of T d, cosdθ = 0, thus sdθ = ±, hece T d ±d at the roots s where θ s θ correspods to the root. We eed to compute g at these roots of T d. The roots of g are frstly the three artfcal roots 0,, ad the after a large gap followed by the roots c 4, c 4 9, c 4 5,.... Now say we evaluate g m+ c 4 the we get g m + c 4 = c 4m + 64c3 4m 6 3 d m + π c 4 m + c 4 m + log 64c3 4m 5 c 3 m 5 = c πc 5 d s m+π d Note that these have the same depedece o whe compared to g evaluated at the artfcal roots but there s a large costat c 5 before whch makes the pseudo dstrbuto d EM D, δ δ 0 δ small. We get D from ths polyomal we costructed usg the lemma above. Sce g evaluated at the roots of g all have depedece we could scale D to get rd of. Thus the pseudo dstrbuto D we get satsfes D0 D D. Ad at the other roots Now lets evaluate m + D c 4 d EM D, δ δ 0 δ = m c 5 m 5. 4c 4 m c 5 m 5 c 4 0 Note that c 5 >> c 3 4 so the above term goes to 0 wth c 4. Thus we choose the costats such that d EM D, δ δ 0 δ <. Thus, we should get a average error of /00 per 00 coordate or a total error of at most /00. Cocluso: Thus we costructed a D supported o [0, c log ] such that E X D [X k ] = 0 for k d = c 3 log. Next class we wll use ths D wth Iformato theory argumets to complete the adversaral argumet for the lower boud. dy dx 6