The VC-dimension of Unions: Learning, Geometry and Combinatorics

Transcription

1 The VC-dimension of Unions: Leaning, Geomety and Combinatoics Mónika Csikós Andey Kupavskii Nabil H. Mustafa Abstact The VC-dimension of a set system is a way to captue its complexity, and has been a key paamete studied extensively in machine leaning and geomety communities. In this pape, we make substantial pogess on bounding the VC-dimension of k-fold unions and intesections of basic geometic set systems, including settling an open question in machine leaning that was fist studied in the 1989 foundational pape of Blume, Ehenfeucht, Haussle and Wamuth [4]. Kalsuhe Institute of Technology, Kalsuhe, Gemany. Moscow Institute of Physics and Technology and EPFL, Switzeland. kupavskii@yandex.u. The wok of Andey Kupavskii has been suppoted in pat by the Swiss National Science Foundation Gants and and by the gant N of the Russian Foundation fo Basic Reseach. Univesité Pais-Est, LIGM, Equipe A3SI, ESIEE Pais, Fance. mustafan@esiee.f. The wok of Nabil H. Mustafa has been suppoted by the gant ANR SAGA JCJC-14-CE ).

2 1 Intoduction Let X, R) be a set system, whee X is a set of elements and R is a set of subsets of X. In the theoy of leaning, each set of R is also called a concept, and R a concept class on X. Fo any intege k 2, define the k-fold union of R to be the set system R k = {R 1 R k : R 1,..., R k R}. Similaly, one can define the k-fold intesection of R as the set system R k consisting of all subsets deived fom the common intesection of at most k subsets of R. Note that as the subsets R 1,..., R k need not necessaily be distinct, we have R R k and R R k. VC-dimension. One of the fundamental measues fo captuing the complexity o ichness of a set system, with applications acoss seveal fields, is the Vapnik-Chevonenkis dimension, o in shot the VC-dimension, of a set system. Given X, R), fo any set Y X, define the pojection of R onto Y as the set system: R Y = {Y R: R R}. We say that R shattes Y if R Y = 2 Y ; in othe wods, all subsets of Y can be deived by intesection with sets of R. Then the VC-dimension of R, denoted VC-dimR), is the size of the lagest subset of X that can be shatteed by R. Oiginally defined in statistics and pobability, it has tuned out to be a key paamete in seveal aeas; this pape concens thee of them leaning theoy, geomety and combinatoics. Leaning theoy. In leaning theoy, the VC-dimension of a concept class measues the difficulty of leaning a concept of the class. The foundational pape of Blume, Ehenfeucht, Haussle and Wamuth [4] states that the essential condition fo distibution-fee leanability is finiteness of the Vapnik-Chevonenkis dimension. One of the theoems they pove is the following. Theoem A Blume et al. [4]). Let X, R) be a set system, and k be any positive intege. Then Ω VC-dim R) k) = VC-dim R k ) ) = O VC-dim R) k log k. Ω VC-dim R) k) = VC-dim R k ) ) = O VC-dim R) k log k. They also consideed the question of whethe the bound of Theoem A is tight in the most basic geometic case when X R d is a set of points and R is the pojection of the family of all half-spaces of R d onto X. Hee they poved that the VC dimension of the k-fold union of half-spaces in two dimensions is exactly 2k + 1. Fo geneal dimensions d 4, they bound the VC-dimension of the k-fold union of half-spaces by Od k log k), following fom Theoem A togethe with the fact that the VC-dimension of the set system induced by half-spaces in R d is d + 1. The same bound holds fo the k-fold intesection of half-spaces in R d. Eisenstat and Angluin [8] poved, by giving a pobabilistic constuction of an abstact set system, that the uppe bound of Theoem A is asymptotically tight if VC-dim R) 5 and that fo VC-dim R) = 1, a lowe bound of k holds and that it is tight. A few yeas late, Eisenstat [7] 1

3 filled the gap by showing that VC-dim R k ) = Ω VC-dim R) k log k) even if VC-dim R) 2. Late Dobkin and Gunopulos [6] showed that the VC-dimension of the k-fold union of half-spaces in thee dimensions is uppe-bounded by 4k. Fo d 4, the cuent best uppe-bound fo the k-fold union and the k-fold intesection of halfspaces in R d is still the one given by Theoem A almost 30 yeas ago, while the lowe-bound has emained Ω VC-dimR) k). We efe the eade to the PhD thesis [11] fo a summay of the bounds on VC-dimensions of these basic combinatoial and geometic set systems. The esolution of the VC-dimension of k-fold unions and intesections of half-spaces is left as one of the main open poblems in the thesis. In fact, while the uppe-bound of Theoem A applies to geometic set systems, we did not have, till now, a single example of a geometic set system R with a non-linea lowe-bound i.e., beyond Ω VC-dim R) k) on the VC-dimension of its k-fold union o k-fold intesection. Ou fist esult poves a non-linea bound fo a geometic set system even in the plane: Theoem 1 Section 3). Let k be a given positive intege. Then thee exists a set P of points in R 2 such that the set system L induced on P by lines in the plane satisfies VC-dim L k ) ) 1 ) ) 1 ) log k 3 log k 3 = Ω VC-dimL) k = Ω k. log log k log log k Next we esolve the question fo half-spaces in R d, d 4. Theoem 2 Section 4). Let k be a given positive intege, and d 4 an intege. Then thee exists a set P of points in R d such that the set system R induced on P by half-spaces satisfies VC-dim R k ) ) ) = Ω VC-dimR) k log k = Ω d k log k. VC-dim R k ) ) ) = Ω VC-dimR) k log k = Ω d k log k. Relations to Geomety. The beakthough wok of Haussle and Wezl [10] showed that the size of a key stuctue in discete and computational geomety, ɛ-nets we efe the eade to the books [5, Chapte 4], [13, Chapte 10], [16, Chapte 15] fo detailed infomation), is diectly linked with the VC-dimension of the coesponding set system: Theoem B Epsilon-net Theoem [10]). Let X, R) be a set system and ɛ > 0 be a given paamete. Then thee exists an ɛ-net fo X, R) of size at most VC-dimR) O log 1 ). ɛ ɛ Thus the sizes of ɛ-nets can be uppe-bounded in tems of the VC-dimension of the set system. On the othe hand, Alon [1] showed a supe-linea lowe-bound fo ɛ-nets induced by lines in the plane; in a vey ecent beakthough, Balogh and Solymosi [3] impoved Alon s lowe bound to get the following: 2

4 Theoem C [3]). Given any ɛ > 0, thee exists a set P of points in the plane such that any ɛ-net fo P fo the set system induced on P by lines must have size at least ) 1 1 log 1 3 ɛ 2ɛ log log 1. ɛ We efe the eade to the chapte [14] fo futhe details on ecent pogess in the aea of ɛ-nets. 2 Ou Results Ou esults exploit links between leaning theoy, geomety and combinatoics: the open poblem on the VC-dimension of the k-fold union of half-spaces in R d in leaning theoy is esolved by noting its connection to ɛ-nets in geomety items 1. and 2. below), and the ecent beakthough of Balogh and Solymosi on ɛ-nets can be genealized by putting it in the famewok of VC-dimension of k-fold unions item 3. below). Moe pecisely: 1. The stating point of ou wok is the following obsevation that links the study of k-fold unions in leaning theoy liteatue to the one fo ɛ-nets in discete geomety. While consideable wok has been done in the pecise elation between VC-dimension of set systems and uppe-bounds on the sizes of ɛ-nets, it tuns out the connection goes both ways: Theoem 3 Section 5). Let X, R) be a set system and ɛ > 0 be a given paamete. Then any ɛ-net fo X, R) must have size at least )) VC-dimR) 1 Ω f, ɛ 2ɛ whee f : R R is any function satisfying VC-dim R k ) VC-dimR) k fk) fo k = 1 2ɛ. This can be seen as complement to Theoem B: the VC-dimension of the k-fold union gives a lowe bound fo the sizes of ɛ-nets fo R. 2. We show an optimal lowe-bound on the VC-dimension of the k-fold union and the k-fold intesection of half-spaces in R d follows via [12], thus settling affimatively one of the main open question studied by Eisenstat and Angluin [8], Johnson [11], and Eisenstat [7], and matching the Od k log k) uppe bound of Theoem A. Theoem 2 Section 4). Let k be a given positive intege, and d 4 an intege. Then thee exists a set P of points in R d such that the set system R induced on P by half-spaces satisfies VC-dim R k ) ) ) = Ω VC-dimR) k log k = Ω d k log k. VC-dim R k ) ) ) = Ω VC-dimR) k log k = Ω d k log k. 3

5 Remak 1. This statement also povides a non-pobabilistic poof of Eisenstat s lowe-bound [7]. Remak 2. Obseve that if R := {R d \ R R R}, then VC-dimR) = VC-dimR) and so R k = R k. holds by the De Mogan laws. Since fo half-spaces R = R, the fist claim of Theoem 2 implies the second one, i.e., the same lowe-bound fo R k, settling anothe question posed by Eisenstat and Angluin [8]. Remak 3. This implies that the coeset constuctions in [9] equie an additional log k facto in the coeset size coming fom the VC-dimension of the k-fold intesection of half-spaces. See [2] fo details. 3. We next tun to the set system induced on a set of points in the plane by lines. We show that the poweful technique of Balogh and Solymosi [3] can be impoved and extended to pove a lowe-bound on the VC-dimension of the k-fold union of this set system induced by lines in the plane: Theoem 1 Section 3). Let k be a given positive intege. Then thee exists a set P of points in R 2 such that the set system L induced on P by lines in the plane satisfies VC-dim L k ) ) 1 ) ) 1 ) log k 3 log k 3 = Ω VC-dimL) k = Ω k. log log k log log k As an immediate coollay using Theoem 3), we get an impovement to Theoem C: Coollay 4. Given any ɛ > 0, thee exists a set P of points in the plane such that any ɛ-net fo P fo the set system induced on P by lines must have size ) 1 Ω 1 log 1 3 ɛ ɛ log log 1. ɛ 3 Poof of Theoem 1. We fist pesent the poof outline: we stat by taking an appopiate point set P in R fo a paamete to be chosen late), with the popety that P and all its sufficiently dense subsets contain many -tuples of collinea points. We will then pick each point of P independently with pobability p a paamete to be chosen late) to get the equied point-set R. The key idea is the use of the poweful containe method to analyze the failue pobability, which in ou case will be that R contains a subset of size m a paamete to be chosen late) without any collinea points. A andom pojection onto the plane then gives the equied point set. Ou poof closely follows the one of Balogh and Solymosi, with two key diffeences: i) one can choose the constuction and paametes moe caefully to impove the bound by poly log log factos. The initial set in [3] is simply the gid [n] ; we will pick a moe subtle constuction with fine popeties which allow us to get an impoved bound, and 4

6 ii) the use of containes is flexible and poweful enough to allow one to deduce the stonge statement on lowe-bound fo the VC dimension of the k-fold union. The value of m above in [3] is set to R 2, following the initial wok of Alon [1] stated as the Alon idea in [3]. This suffices to get a lowe-bound on ɛ-nets, but not fo lowe-bounding the VC-dimension of the k-fold union of the set system on R induced by lines. Towads this end, we need to pove a stonge popety with a lowe value of m. Now we tun to the poof of the theoem. 3.1 Initial set P : pointset with many collinea tuples Denote by [a, b] the set of all integes between a and b. In what follows, all asymptotic notation will be with espect to the paamete n; fo bevity, we will denote fn) = gn) ± ogn)) by fn) gn), and fn) gn) + ogn)) by fn) gn). Let n, N and t R be paametes to be fixed late, such that 1 t = on). Ou initial stating point-set in R will be: { } P := x 1,..., x ) [ n, n] : x i n x 1 fo i = 2,...,. Now conside the family L t of all lines that pass though at least one point of P, and have the diection vecto in the following set L t : { L t := x 1,..., x ): n t x 1 2n t, x i n } t fo i = 2,...,, and x 1,..., x ae elatively pime. Note that if x 1,..., x ) ae elatively pime integes, then αx 1,..., αx ) Z if and only if α Z. 1) We ae going to wok with the set of points P and the subsets of P induced by lines in L t. The key popeties of P and L t ae summaized in the following claim. Claim 4.1. Let P and L t be as defined above. Then i) P 2n), ii) L t 2n) 2t, iii) each point of P lies on L t lines of L t, iv) each line of L t contains at most 2t points fom P, and v) 2n) 2t L t L t 22n) t L t. Poof. i) P = 2n + 1) n i=1 2 n i) + 1) 1 2n). 5

7 ii) It follows fom Nymann [15] that the numbe of k-tuples of integes fom [m] x that ae elatively pime is asymptotically mx ζx), whee ζx) = i=1 i x is the Riemann zeta function. It is easy to see that ζx) 1 as x. Theefoe, even looking at x 2,..., x [ n t, n ] t, we conclude that the numbe of such 1)-tuples that ae elatively pime is asymptotically 2n t ) 1. Theefoe, almost all -tuples in [ n t, 2n ] [ t n t, n ] 1 t ae elatively pime, and the numbe of -tuples in L is asymptotically 2n) 2t. iii) Follows fom the definition of L t. iv) Popety 1) implies that if p P lies on a line with diection vecto v L t with coefficient α, then α must be an intege. The claim follows as the fist coodinate of p lies in the inteval [ n, n] while the absolute value of the fist coodinate of the diection vecto v is at least n t. v) Note that evey line in L t passes though one of the points of the following set: U t := { x 1,..., x ): x 1 2n } t, x i [ n, n] fo i = 2,...,. Theefoe we can uppe-bound the size of L t by the numbe of incidences with U t, i.e., L t L t U t 2 2n) t L t. On the othe hand, each line fom L t passes though at most fou points of U t fo the same easons as the ones used to pove iv). So each line is counted at most fou times in the numbe of incidences with the set U t, which gives the lowe bound of L t Lt Ut 4. Lemma 5. Let γ 82 t be a given positive paamete. Then evey subset of P of size γ P contains γ n 2 t 2 collinea -tuples lying on lines fom L t. Poof. Let S be any subset of P with S = γ P. Then Claim 4.1iii) implies that the numbe of point-line incidences between S and L t is S L t = γ P L t. Thus the aveage numbe of points of S lying on a line of L t, using Claim 4.1 v), is γ P L t L t γ P L t L t 22n) t γ 2n) 22n) t = γt 2. Note that by the choice of γ, we have γt 2 4, and thus γt 2 ) γt 4. By the convexity of ) x k and Claim 4.1v), we can conclude that the numbe of collinea -tuples in S is at least γt ) γt 2 2 L t )! 2n) L t 2t γt ) 4! 2n) L t 2t whee the last inequality used the bound 4! /2). γt ) 4 2n) 2n)! 2t 2t = γ n 2 1 t 4! γ n 2 t 2, 6

8 3.2 Setting up containes In this section we pepae notations fo, and state the main technical tool the containe theoem fo hypegaphs. Roughly speaking, containe theoems state that all independent sets in a hypegaph ae contained in a elatively small numbe of subsets, each of which is elatively small. Then, afte taking a andom subset R of vetices of the hypegaph, this type of statement helps us to bound the size of the lagest independent set in R by simply applying union bound ove all subsets of each of the containes. Unfotunately, containe statements ae athe technical. Let us fist intoduce some notation. Let H be a -unifom hypegaph on vetex set V H) and let d be the aveage degee of a vetex of H. The degee ds) of a subset S V H) is simply the numbe of edges of H containing S. Fo evey j [], let us denote by j the maximum degee of a j-element subset, that is, j := max {ds): S V H), S = j}. Fo any S V H), let H[S] denote the hypegaph induced by S, and by eh[s]) its numbe of hypeedges. Fo any τ 0, 1), define H, τ) := j=2 j d τ j 1 2 j 1 2 )+1 2). The ole of H, τ), oughly speaking, is to measue the goodness of H: smalle values of H, τ) imply bette bounds. Note that, fom this pespective, we ae inteested in lowe-bounding d and uppe-bounding j. We will use the following vaiant of a containe statement: Theoem D [18]). Let H be a -unifom hypegaph on the vetex set {1,..., N}. Let 0 < ε, τ < 1/2. Suppose that τ < 1/200! 2 ) and H, τ) ε/12!). Then thee exists c = c) 1000! 3 and a collection of vetex subsets C such that 1) evey independent set in H is a subset of some A C, 2) fo evey A C, eh[a]) ε eh), and 3) log C cnτ log 1 ε) log 1 τ ). We will wok with the -unifom hypegaph H, whee V H) := P, and edges ae all -tuples of collinea points on lines fom L t. Note that L t depends on a paamete t, which we set to be t = 5 2. Next we calculate the paametes of H that ae needed to apply Theoem D. Applying Lemma 5 with γ = 1 note hee that 82 t = 16 < 1), the total numbe of collinea -tuples in P is lowebounded by n2, and so the aveage degee d of H can be lowe-bounded by 3 t 2 d n 2 t 2 P = 2n n) = n n 3+2. On the othe hand, since each line contains at most 2t = 5 points, we get that 5 ) j j 5 j) fo j = 2,...,. j 7

9 Set τ = n and be such that 1 log n 20 log log n. Then note the following inequalities: n τ j 1 = n n j 1+ j = n j n j 2 > n j n 1 2 fo all j = 1,...,. j 2 2 j j) = n o j) fo all j = 2,..., 1. log n log n) 20 log log n = n Theefoe, we may conclude that fo this choice of τ and, we have H, τ) j=2 j=2 j d τ j 1 2 j 1 2 )+1 2) n o j) o1) n j n 1 2 n 1 3. j=2 5 j)+3+2 n τ j 1 2 j2 2 5 j) j 2 j=2 n τ j 1 We apply Theoem D with ou choice of τ and, and with ɛ = n 1 4. It is easy to see that the conditions of the theoem ae satisfied. Indeed, τ n 1 n 1 6 +o1) 2 1/200! 2 ), as well as that H, τ) n 1 3 = n ε ε 12!. Note that the constant c) fom the theoem satisfies c) 3. The conclusion of the theoem gives a collection of subsets C 2 P, such that fo evey A C we have eh[a]) n 1 4 eh) n 2 1 4, 2) whee the second inequality is due to the fact that eh) n 2, and such that log C 3 n n log 2 n 3 n ) Hee the last inequality is due to the fact that log 2 2 log log n n = n log n n Using Lemma 5 with γ = 2 which is a valid choice of γ, since γt = 3 2 ), we conclude that in any subset of size at least γ P thee ae at least n 2 / 4+o) n o1) induced hypeedges. Theefoe, any subset A C has size at most γ P by inequality 2). 3.3 Constucting R In this section we use the popeties of the hypegaph H, developed in Section 3.2, to obtain a point set fo which the VC-dimension of the k-fold union of the set system induced by lines is bounded fom below. 8

10 We would like to use the point set P with lines fom L t diectly, but the poblem is that it contains many collinea + 1)-tuples. Fom Claim 4.1 iv), the numbe of collinea + 1)-tuples in the set P with lines fom L t is at most 2t + 1 ) L t 2t)+1 + 1)! 22n) t 2n) 2t = 2t)+1 2n) 2 + 1)!t +1 = )! n2 24 ) n 2. Let R be a andom subset constucted by picking each point of P with pobability p, a paamete we will set shotly. We want the numbe of points in R to be much bigge than the numbe of collinea + 1)-tuples pesent in R. In othe wods, we set p such that E [ R ] = p P p 2n) p ) n 2. 4) Using Makov s inequality, it is easy to see that this inequality is satisfied, with high pobability, fo p = 20n. Set m = p P. Now we ague that, with high pobability, R has the popety that any subset of 3/2 R of size m is not an independent set; o stated anothe way, any subset of R of size m contains collinea points. Using containes and the fact that γ = 2, we conclude via the union bound that the pobability that thee exists an independent set of size m in H[R ] is at most C ) γ P p m 2 3 n m γep P m ) m ) m = 2 3 n e 2 3 n m. 1 2 Theefoe, w.h.p., R does not contain any independent sets of size m, povided 3 n m. As P 2n), we have m 2n) 20n 1 = 2n) 1 and thus 3/2 10 3/2 m 3 n n) /2 3 n = n n 3, fo a lage-enough constant. We want the last expession to tend to infinity, which holds if n 102. This, in tun, holds fo log n = 6 log log n. 5) Note that this is the only place in the poof whee we actually need to be of ode log 0.5 o1) n, and not log 1 o1) n, and the eason fo it is the facto 3 in 3). Now delete one point in R fom each collinea + 1)-tuple of R, obtaining the set R R. Clealy, R contains no collinea + 1)-tuples, and with high pobability, it has size 1 o 1)) p P by inequality 4)), and no independent sets of size m. Poject R on the plane so that no new collineaities appea. This implies that, with set by 5), thee exists a set R whose pojection in the plane satisfies these two conditions: H[R] does not contain independent sets of size at least m, and R has size 1 o1))p P. 9

11 Given the intege k, set the value of n so that k = 2 R 2p P n 1+o1). Then log 1 2 log log n 1 2 log log k 1 log, 2 which implies that log log n 2 3 log log k and that 3 = log n 6 log log n = log k 6 log log n log k 4 log log k. We have that R = k 2 k ) 1 log k 3. 6) 2 4 log log k 3.4 Lowe-bound on VC-dimension of the k-fold union induced by lines It emains to show a lowe-bound on the VC-dimension of the k-fold union of the set system induced by lines on R. In othe wods, given any S R, we want to show that thee exist k lines in the plane such that i) S lies in the union of these k lines, and ii) no point of R \ S lies on any of these lines. This would imply that R is shatteed by the k-fold union of lines in the plane, and thus the bound in inequality 6) gives a lowe-bound on the size of a shatteed set, which then gives the equied lowe-bound on the VC-dimension of k-fold union of the set system induced by lines in the plane. Take any subset S R. We constuct the equied set A of k lines iteatively. Initially set A =. If S m, then as shown above, thee is a collinea -tuple in S. This means that thee is a line l 1 that intesects S in points and does not intesect R \ S hee we need the fact that thee ae no + 1)-tuples in R! Add l 1 to A, emove the coveed points fom S and iteate as long as the emaining set still has size at least m. This can continue fo at most R = k 2 steps, afte which it must be that the size of the emaining set S is less than m. So fa, A k 2. When S < m R k, then we add one geneal position line pe point of S to A, which gives 3/2 an additional ok) lines. We have thus added at most k lines in total to A, and by constuction, these lines cove all the points of S and no point of R \ S. This holds fo any S R, and thus R is shatteed by the set system induced by the k-fold union of lines in the plane. This completes the poof. 10

12 4 Poof of Theoem 2. Poof. The poof will need the following lemma fom [12]: Lemma 6. Let n, d 2 be integes. Then thee exists a set B of d 2 n + 3)2n 2 axis-paallel boxes in R d such that fo any subset S B, one can find a 2 n 1 -element set Q of points in R d with the popety that i) Q B fo any B B \ S, and ii) Q B = fo any B S. By a standad lifting tansfom e.g., see [17, 12]), given a set B of boxes in R d, thee exists a function π : B R 2d mapping boxes in B to points in R 2d such that fo any point p R d, thee exists a coesponding half-space in R 2d, denoted by H p, such that a box B B contains p if and only if H p contains the point fb) R 2d. Fo any B B, set πb ) = {fb): B B }. Apply Lemma 6 with n = log k + 1 in R d/2 to get a set B of boxes in R d/2. Then P = πb) will be the equied point-set in R d ; i.e., we claim that P is shatteed by the set system induced by the k-fold union of half-spaces in R d. To see that, let P be any subset of P. Set S = π 1 P \ P ). By Lemma 6, thee exists a set Q of 2 n 1 = 2 log k k points in R d/2 such that each box in S contains no point of Q, and each box in B \ S contains at least one point of Q. We show that the set HP ) = {H p : p Q} of half-spaces sepaates P fom P. By the popety of the lifting map π ), each box in S contains no point of Q = each box in B \ S contains a point of Q = each point in P \ P is contained in no half-space of HP ), each point in P is contained in some half-space in HP ). In othe wods, the union of the half-spaces in HP ) contains pecisely the set P. As this is tue fo any P P, the k-fold union of half-spaces in R d shattes P. Finally, we have as desied. P = B = d 2 log k + 3)2 log k 2 = Ωd k log k), 11

13 5 Poof of Theoem 3. Poof. Set k = 1 2ɛ, and let d = VC-dimR). As VC-dimRk ) d 2ɛ f 1 2ɛ ), we can assume that X = d 2ɛ f 1 2ɛ ) and that it is shatteed by R 1 2ɛ. We will show that if N is an ɛ-net fo R, then N X 2 = d ) 1 4ɛ f. 2ɛ Suppose that N < X 2. Since X is shatteed by R 1 2ɛ, thee exists a set S R 1 2ɛ containing pecisely the elements in X \ N. In othe wods, we can find 1 2ɛ sets S 1,..., S 1 R such that 2ɛ ) S1 S 1 X = X\N. 2ɛ Each set S i contains no point of N and by the pigeonhole pinciple, one of them must have size at least X \ N 1 2ɛ X /2 1 2ɛ = ɛ X, and which is not hit by N. This contadicts the fact that N was an ɛ-net. 12

14 Refeences [1] Noga Alon. A non-linea lowe bound fo plana epsilon-nets. Discete & Computational Geomety, 472): , [2] Olivie Bachem, Maio Lucic, and Andeas Kause. Scalable and distibuted clusteing via lightweight coesets. CoRR, abs/ , [3] J. Balogh and J. Solymosi. On the numbe of points in geneal position in the plane. AXiv e-pints, Apil [4] A. Blume, A. Ehenfeucht, D. Haussle, and M. K. Wamuth. Leanability and the Vapnik- Chevonenkis dimension. J. ACM, 364): , Octobe [5] B. Chazelle. The Discepancy Method: Randomness and Complexity. Cambidge Univesity Pess, Cambidge, New Yok, [6] D. P. Dobkin and D. Gunopulos. Concept leaning with geometic hypotheses. In Poceedings of the Eighth Annual Confeence on Computational Leaning Theoy, COLT 95, pages , New Yok, NY, USA, ACM. [7] D. Eisenstat. k-fold unions of low-dimensional concept classes. Infomation Pocessing Lettes, ): , [8] D. Eisenstat and D. Angluin. The VC dimension of k-fold union. Infomation Pocessing Lettes, 1015): , [9] Dan Feldman and Michael Langbeg. A unified famewok fo appoximating and clusteing data. In Poceedings of the 43d ACM Symposium on Theoy of Computing, STOC 2011, San Jose, CA, USA, 6-8 June 2011, pages , [10] D. Haussle and E. Welzl. ε-nets and simplex ange queies. Discete & Computational Geomety, 2: , [11] H. Johnson. Definable families of finite Vapnik Chenonenkis dimension. PhD thesis, Univesity of Mayland, [12] A. Kupavskii, N. H. Mustafa, and J. Pach. New lowe bounds fo epsilon-nets. In 32nd Intenational Symposium on Computational Geomety SoCG), pages 54:1 54:16, [13] Jií Matousek. Lectues in Discete Geomety. Spinge-Velag, New Yok, NY, [14] N. H. Mustafa and K. Vaadaajan. Epsilon-appoximations and epsilon-nets. In J. E. Goodman, J. O Rouke, and C. D. Tóth, editos, Handbook of Discete and Computational Geomety. CRC Pess LLC, 2017, to appea. [15] J.E. Nymann. On the pobability that k positive integes ae elatively pime. Jounal of Numbe Theoy, 4: , [16] J. Pach and P. K. Agawal. Combinatoial Geomety. John Wiley & Sons, New Yok, NY, [17] J. Pach and G. Tados. Tight lowe bounds fo the size of epsilon-nets. In Symposium on Computational Geomety, pages ,

15 [18] D. Saxton and A. Thomason. Hypegaph containes. Inventiones mathematicae, pages 1 68,