Tight Bounds on the Round Complexity of the Distributed Maximum Coverage Problem

Transcription

1 Tight Bounds on th Round Complxity of th Distributd Maximum Covrag Problm Sphr Assadi Sanjv Khanna Abstract W study th maximum k-st covrag problm in th following distributd stting. A collction of input sts S 1,..., S m ovr a univrs [n] is partitiond across p machins and th goal is to find k sts whos union covrs th most numbr of lmnts. Th computation procds in rounds whr in ach round machins communicat information to ach othr. Spcifically, in ach round, all machins simultanously snd a mssag to a cntral coordinator who thn communicats back to all machins a summary to guid th computation for th nxt round. At th nd of th last round, th coordinator outputs th answr. Th main masurs of fficincy in this stting ar th approximation ratio of th rturnd solution, th communication cost of ach machin, and th numbr of rounds of computation. Our main rsult is an asymptotically tight bound on th tradoff btwn ths thr masurs for th distributd maximum covrag problm. W first show that any r-round protocol for this problm ithr incurs a communication cost of k m Ω1/r) or only achivs an approximation factor of k Ω1/r). This in particular implis that any protocol that simultanously achivs good approximation ratio O1) approximation) and good communication cost Õn) communication pr machin), ssntially rquirs logarithmic in k) numbr of rounds. W complmnt our lowr bound rsult by showing that thr xist an r-round protocol that achivs an -approximation ssntially bst possibl) with a com- munication cost of k m O1/r) as wll as an r-round protocol that achivs a k O1/r) -approximation with only Õn) communication pr ach machin ssntially bst possibl). W furthr us our rsults in this distributd stting to obtain nw bounds for maximum covrag in two othr main modls of computation for massiv datasts, namly, th dynamic straming modl and th MapRduc modl. 1 Introduction A common paradigm for dsigning scalabl algorithms for problms on massiv data sts is to distribut th computation by partitioning th data across multipl machins intrconnctd via a communication ntwork. Th machins can thn jointly comput a function on th union of thir inputs by xchanging mssags. A wll-studid and important cas of this paradigm is A full vrsion of this papr is availabl on arxiv. Dpartmnt of Computr and Information Scinc, Univrsity of Pnnsylvania. Supportd in part by National Scinc Foundation grants CCF and CCF {sassadi,sanjv}@cis.upnn.du. th coordinator modl s,.g., [34, 54, 60]). In this modl, computation procds in rounds, and in ach round, all machins simultanously snd a mssag to a cntral coordinator who thn communicats back to all machins a summary to guid th computation for th nxt round. At th nd, th coordinator outputs th answr. Main masurs of fficincy in this stting ar th communication cost, i.., th total numbr of bits communicatd by ach machin, and th round complxity, i.., th numbr of rounds of computation. Th distributd coordinator modl and th closly rlatd mssag-passing modl 1 ) has bn studid xtnsivly in rcnt yars s,.g., [54, 23, 59, 60, 61], and rfrncs thrin). Traditionally, th focus in this modl has bn on optimizing th communication cost and round complxity issus hav bn ignord. Howvr, in rcnt yars, motivatd by application to big data analysis such as MapRduc computation, thr hav bn a growing intrst in obtaining round fficint protocols for various problms in this modl s,.g., [3, 4, 43, 41, 37, 52, 30, 13, 38, 10]). In this papr, w study th maximum covrag problm in th coordinator modl: A collction of input sts S := {S 1,..., S m } ovr a univrs [n] is arbitrarily partitiond across p machins, and th goal is to slct k sts whos union covrs th most numbr of lmnts from th univrs. Maximum covrag is a fundamntal optimization problm with a wid rang of applications s,.g., [47, 45, 58, 36] for som applications). As an illustrativ xampl of submodular maximization, th maximum covrag problm has bn studid in various rcnt works focusing on scalabl algorithms for massiv data sts including in th coordinator modl.g., [41, 52]), MapRduc framwork.g., [27, 48]), and th straming modl.g. [20, 51]); s Sction 1.1 for a mor comprhnsiv summary of prvious rsults. Prvious rsults for maximum covrag in th distributd modl can b dividd into two main catgoris: on on hand, w hav communication fficint protocols that only nd Õn) communication and achiv a con- 1 In absnc of any rstriction on round complxity, ths two modls ar quivalnt; s,.g., [54].

2 stant factor approximation, but rquir a larg numbr of rounds of Ωp) [15, 51] 2. On th othr hand, w hav round fficint protocols that achiv a constant factor approximation in O1) rounds of communication, but incur a larg communication cost k m Ω1) [48]. This stat-of-th-affairs, namly, communication fficint protocols that rquir a larg numbr of rounds, or round fficint protocols that rquir a larg communication cost, raiss th following natural qustion: Dos thr xist a truly fficint distributd protocol for maximum covrag, that is, a protocol that simultanously achivs Õn) communication cost, O1) round complxity, and givs a constant factor approximation? This is th prcisly th qustion addrssd in this work. 1.1 Our Contributions Our first rsult is a ngativ rsolution of th aformntiond qustion. In particular, w show that, Rsult 1. For any intgr r 1, any r-round protocol for distributd maximum covrag ithr incurs k m Ω1/r) communication pr machin or has an approximation factor of k Ω1/r). Prior to our work, th only known lowr bound for distributd maximum covrag was du to McGrgor and Vu [51] who showd an Ωm) communication lowr bound for any protocol that achivs a bttr than ) -approximation rgardlss of numbr of rounds and vn if th input is randomly distributd); s also [9]. Indyk t al. [41] also showd that no composabl corst a rstrictd family of singl round protocols) can achiv a bttr than Ω k) approximation without communicating ssntially th whol input which is known to b tight [30]). Howvr, no supr constant lowr bounds on approximation ratio wr known for this problm for arbitrary protocols vn for on round of communication. Our rsult on th othr hand implis that to achiv a constant factor approximation with an On c ) communication protocol for a fixd constant c > 0), Ω log k log log k ) rounds of communication ar rquird. In stablishing Rsult 1, w introduc a gnral framwork for proving communication complxity lowr bounds for boundd round protocols in th distributd coordinator modl. This framwork, formally introducd in Sction 4, capturs many of th xisting multi-party communication complxity lowr bounds 2 W rmark that th algorithms of [15, 51] ar originally dsignd for th straming stting and in that stting ar quit fficint as thy only rquir on or a constant numbr of passs ovr th stram. Howvr, implmnting on pass of a straming algorithm in th coordinator modl dirctly rquirs p rounds of communication. in th litratur for boundd-round protocols including [33, 46, 13, 12] for on round a.k.a simultanous protocols), and [7, 8] for multi-round protocols). W bliv our framwork will prov usful for stablishing distributd lowr bound rsults for othr problms, and is thus intrsting in its own right. W complmnt Rsult 1 by giving protocols that show that its bounds ar ssntially tight. Rsult 2. For any intgr r 1, thr xist r- round protocols that achiv: 1. an approximation factor of almost) with k m O1/r) communication pr machin, or 2. an approximation factor of Or k 1/r+1 ) with Õn) communication pr machin. Rsults 1 and 2 togthr provid a nar complt undrstanding of th tradoff btwn th approximation ratio, th communication cost, and th round complxity of protocols for th distributd maximum covrag problm for any fixd numbr of rounds. Th first protocol in Rsult 2 is quit gnral in that it works for maximizing any monoton submodular function subjct to a cardinality constraint. Prviously, it was known how to achiv a 2-approximation distributd algorithm for this problm with m O1/r) communication and r rounds of communication [48]. Howvr, th prvious bst ) -approximation distributd algorithm for this problm with sublinar in m communication du to Kumar t al. [48] rquirs at last Ωlog n) rounds of communication. As notd abov, th ) is information thortically th bst approximation ratio possibl for any protocol that uss sublinar in m communication [51]. Th scond protocol in Rsult 2 is howvr tailord havily to th maximum covrag problm. Prviously, it was known that an O k) approximation can b achivd via Õn) communication [30] pr machin, but no bttr bounds wr known for this problm in multipl rounds undr polyn) communication cost. It is worth noting that sinc an advrsary may assign all sts to a singl machin, a communication cost of Õn) is ssntially bst possibl bound. W now laborat on som applications of our rsults. Dynamic Strams. In th dynamic st) straming modl, at ach stp, ithr a nw st is insrtd or a prviously insrtd st is dltd from th stram. Th goal is to solv th maximum covrag problm on th sts that ar prsnt at th nd of th stram. A smi-straming algorithm is allowd to mak on or a small numbr of passs ovr th stram and us only

3 On poly {log m, log n}) spac to procss th stram and comput th answr. Th straming stting for th maximum covrag problm and th closly rlatd st covr problm has bn studid xtnsivly in rcnt yars [58, 28, 14, 24, 35, 32, 15, 39, 25, 11, 20, 26, 51, 9, 36]. Prvious work considrd this problm in insrtion-only strams and mor rcntly in th sliding window modl; to th bst of our knowldg, no non-trivial rsults wr known for this problm in dynamic strams. Our Rsults 1 and 2 imply th first uppr and lowr bounds for maximum covrag in dynamic strams. Rsult 1 togthr with a rcnt charactrization of multi-pass dynamic straming algorithms [5] provs that any smi-straming algorithm for maximum covrag in dynamic strams that achivs ) any constant approximation rquirs Ω log n log log n passs ovr th stram. This is in sharp contrast with insrtion-only strams in which smi-straming algorithms can achiv almost) ) 2-approximation in a singl pass [15] or almost) - approximation in a constant numbr of passs [51] constant factor approximations ar also known in th sliding window modl [26, 36]). To our knowldg, this is th first multi-pass dynamic straming lowr bound that is basd on th charactrization of [5]. Morovr, as maximum covrag is a spcial cas of submodular maximization subjct to cardinality constraint), our lowr bound xtnds to this problm and sttls an opn qustion of [36] on th spac complxity of submodular maximization in dynamic strams. W complmnt this rsult by showing that on can implmnt th first algorithm in Rsult 2 using propr linar sktchs in dynamic strams, which imply an almost) ) -approximation smi-straming algorithm for maximum covrag and monoton submodular maximization) in Olog m) passs. As a simpl application of this rsult, w can also obtain an Olog n)-approximation smi-straming algorithm for th st covr problm in dynamic stram that rquirs Olog m log n) passs ovr th stram. MapRduc Framwork. In th MapRduc modl, thr ar p machins ach with a mmory of siz s such that p s = ON), whr N is th total mmory rquird to rprsnt th input. MapRduc computation procds in synchronous rounds whr in ach round, ach machin prforms som local computation, and at th nd of th round snds mssags to othr machin to guid th computation for th nxt round. Th total siz of mssags rcivd by ach machin, howvr, is rstrictd to b Os). Following [44], w rquir both p and s to at b at most N 1 Ω1). Th main complxity masur of intrst in this modl is typically th numbr of rounds. Maximum covrag and submodular maximization hav also bn xtnsivly studid in th MapRduc modl [27, 22, 48, 53, 41, 52, 30, 31, 19]. Proving round complxity lowr bounds in th MapRduc framwork turns out to b a challnging task s,.g., [56] for implication of such lowr bounds to long standing opn problms in complxity thory). As a rsult, most prvious work on lowr bounds concrns ithr communication cost in a fixd numbr of rounds) or spcific classs of algorithms for round lowr bounds); s,.g., [1, 21, 55, 42] s [56] for mor dtails). Our rsults contribut to th lattr lin of work by charactrizing th powr of a larg family of MapRduc algorithms for maximum covrag. Many xisting tchniqus for MapRduc algorithms utiliz th following paradigm which w call th sktch-and-updat approach: ach machin snds a summary of its input, i.., a sktch, to a singl dsignatd machin which procsss ths sktchs and computs a singl combind sktch; th original machins thn rciv this combind sktch and updat thir sktch computation accordingly; this procss is thn continud on th updatd sktchs. Popular algorithmic tchniqus blonging to this framwork includ composabl corsts.g., [15, 17, 18, 41]), th filtring mthod.g., [50]), linar-sktching algorithms.g., [3, 4, 43, 2]), and th sampl-and-prun tchniqu.g., [48, 40]), among many othrs. W us Rsult 1 to prov a lowr bound on th powr of this approach for solving maximum covrag in th MapRduc modl. W show that any MapRduc algorithm for maximum covrag in th sktchand-updat framwork that uss s = m δ mmory pr machin rquirs Ω 1 δ ) rounds of computation. Morovr, both our algorithms in Rsult 2 blong to th sktch-and-updat framwork and can b implmntd in th MapRduc modl. In particular, th round complxity of our first algorithm for monoton submodular maximization subjct to cardinality constraint) in Rsult 2 matchs th bst known algorithm of [31] with th bnfit of using sublinar communication th algorithm of [31], in ach round, incurs a linar in input siz) communication cost). W rmark that th algorithm in [31] is howvr mor gnral in that it supports a largr family of constraints bsid th cardinality constraint w study in this papr. 1.2 Organization Th rst of th papr is organizd as follows. W start with prliminaris and notation in Sction 2. W thn prsnt a high lvl tchnical ovrviw of our papr in Sction 3. In Sction 4, w prsnt our framwork for proving communication complxity lowr bounds for boundd-round protocols. Sction 5.1 is ddicatd to th proof of Rsult 1. W

4 sktch th proof of Rsult 2 in Sction 6. Du to spac limitations, many of th proofs and dtails ar dfrrd to th full vrsion of th papr. In particular, w dfr th dtaild discussion on th applications of our rsults to dynamic st strams and th MapRduc modl to th full vrsion of th papr. 2 Prliminaris Notation. For a collction of sts C = {S 1,..., S t }, w dfin cc) := i [t] S i, i.., th st of lmnts covrd by C. For a tupl X = X 1,..., X t ) and indx i [t], X <i := X 1,..., X i 1 ) and X i := X 1,..., X i 1, X i+1,..., X t ). W us sans srif fonts to dnot random variabls, i.., X. For a random variabl X ovr a support Ω X, distx) dnots th distribution of X and X := log Ω X. W us HX) and IX ; Y) to dnot th Shannon ntropy of X and mutual information of X and Y, rspctivly. For any two distributions µ and ν ovr th sam probability spac, Dµ ν) and µ ν dnot th Kullback-Liblr divrgnc and th total variation distanc btwn µ and ν, rspctivly. 2.1 Tools From Information Thory Th proof of th following basic proprtis of ntropy and mutual information can b found in [29] s Chaptr 2). Fact 2.1. Lt A, B, and C b thr possibly corrlatd) random variabls HA) A, and HA) = A iff A is uniformly distributd ovr its support. 2. IA ; B C) 0. Th quality holds iff A and B ar indpndnt conditiond on C. 3. HA B, C) HA B). Th quality holds iff A C B. 4. IA, B ; C) = IA ; C) + IB ; C A) chain rul of mutual information). 5. Suppos fa) is a dtrministic function of A, thn IfA) ; B C) IA ; B C) data procssing inquality). W also us th following two standard propositions, rgarding th ffct of conditioning on mutual information. Th proofs ar standard s th full vrsion). Proposition 2.1. For variabls A, B, C, D, if A D C, thn, IA ; B C) IA ; B C, D). Proposition 2.2. For variabls A, B, C, D, if A D B, C, thn, IA ; B C) IA ; B C, D). For two distributions µ and ν ovr th sam probability spac, th Kullback-Liblr divrgnc [ btwn ] µ and ν is dfind as Dµ ν) := E a µ. Fact 2.2. For random variabls A, B, C, IA ; B C) = E b,c) log Prµa) Pr νa) [ DdistA C = c) dista B = b, C = c)) W dnot th total variation distanc btwn two distributions µ and ν ovr th sam probability spac Ω by µ ν = 1 2 x Ω Pr µx) Pr ν x). Th following Pinskrs inquality bounds th total variation distanc btwn two distributions basd on thir KL-divrgnc, Fact 2.3. Pinskr s inquality) For any two distributions µ and ν, µ ν 1 2 Dµ ν). Fact 2.4. Suppos µ and ν ar two distributions for an vnt E, thn, Pr µ E) Pr ν E) + µ ν. 2.2 Communication Complxity Modl W prov our lowr bound for distributd protocols using th framwork of communication complxity, and in particular in th numbr-in-hand) multiparty communication modl with shard blackboard: thr ar p playrs corrsponding to machins) rciving inputs x 1,..., x p ) from a prior distribution D on X 1... X p. Th communication happns in rounds and in ach round, th playrs simultanously writ a mssag to a shard blackboard visibl to all partis. Th mssag snt by any playr i in ach round can only dpnd on th input of th playr, i.., x i, th currnt contnt of th blackboard, i.., th mssags communicatd in prvious rounds, and public and privat randomnss. In addition to p playrs, thr xists a cntral party calld th rfr corrsponding to th coordinator) who only ss th contnt of th blackboard and public randomnss and is rsponsibl for outputting th answr in th final round. For a protocol π, w us Π = Π 1,..., Π p ) to dnot th transcript of th mssags communicatd by all playrs, i.., th contnt of th blackboard. Th communication cost of a protocol π, dnotd by π, is th sum of worst-cas lngth of th mssags communicatd by all playrs, i.., π := p i=1 Π i. W furthr rfr to max i [p] Π i as th pr-playr communication cost of π. W rmark that this modl is idntical to th distributd stting introducd arlir if w allow th coordinator to communicat with machins fr of charg. As a rsult, communication lowr bounds in this modl imply idntical communication lowr bounds for distributd ].

5 protocols. W rfr th radr to th xcllnt txt by Kushilvitz and Nisan [49] for mor dtails on communication complxity. 2.3 Submodular Maximization with Cardinality Constraint Lt V = {a 1,..., a m } b a ground st of m itms. For any st function f : 2 V R and any A V, w dfin th marginal contribution to f as a st function f A : 2 V R such that for all B V, f A B) = fa B) fa). Whn clar from th contxt, w abus th notation and for a V, us fa) and f A a) instad of f{a}) and f A {a}), rspctivly. A function f is submodular iff for all A B V and for all a V, f B a) f A a). A submodular function f is additionally monoton iff A B V, fa) fb). Th maximum covrag problm is a spcial cas of maximizing a monoton submodular function subjct to a cardinality constraint of k, i.., finding A arg max A: A =k fa): for any st S in maximum covrag w can hav an itm a S V and for ach A V, dfin fa) = a S A S. It is asy to vrify that f ) is monoton submodular. 3 Tchnical Ovrviw Lowr Bounds Rsult 1). Lt us start by sktching our proof for simultanous protocols. W provid ach machin with a collction of sts from a family of sts with small pairwis intrsction such that locally, i.., from th prspctiv of ach machin, all ths sts look alik. At th sam tim, w nsur that globally, on st in ach machin is spcial; think of a spcial st as covring a uniqu st of lmnts across th machins whil all othr sts ar mostly covring a st of shard lmnts. Th proof now consists of two parts: i) us th simultanity of th communication to argu that as ach machin is oblivious to idntity of its spcial st, it cannot convy nough information about this st using limitd communication, and ii) us th bound on th siz of th intrsction btwn th sts to show that this prvnts th coordinator to find a good solution. Th stratgy outlind abov is in fact at th cor of many xisting lowr bounds for simultanous protocols in th coordinator modl including [33, 46, 13, 12] a notabl xcption is th lowr bound of [12] on stimating matching siz in spars graphs). For xampl, to obtain th hard input distributions in [46, 13] for th maximum matching problm, w just nd to switch th sts in th small intrscting family abov with dg-disjoint inducd matchings in a Ruzsa-Szmrédi graph [57] s also [6] for mor dtails on ths graphs). Th first part of th proof that lowr bounds th communication cost rquird for finding th spcial inducd matchings corrsponding to spcial sts abov), rmains quit similar; howvr, w now nd an ntirly diffrnt argumnt for proving th scond part, i.., th bound obtaind on th approximation ratio. This obsrvation raiss th following qustion: can w automat th task of proving a communication lowr bound in ths argumnts so that on can focus solly on th scond part of th argumnt, i.., proving th approximation lowr bound subjct to ach machin not bing abl to find its spcial ntity,.g., spcial sts in th covrag problm and spcial inducd matchings in th maximum matching problm? W answr this qustion in th affirmativ by dsigning a framwork for proving communication lowr bounds of th aformntiond typ. W dsign an abstract hard input distribution using th idas abov and prov a gnral communication lowr bound in this abstraction. This rducs th task of proving a communication lowr bound for any spcific problm to dsigning suitabl combinatorial objcts that roughly spaking nforc th importanc of spcial ntitis discussd abov. W mphasiz that this scond part may still b a non-trivial challng; for instanc, lowr bounds for matchings in [46, 13] rly on Ruzsa-Szmrédi graphs to prov this part. Nvrthlss, automating th task of proving a communication lowr bound in our framwork allows on to focus solly on a combinatorial problm and ntirly bypass th information-thortic argumnts ndd to prov th communication lowr bound. W furthr xtnd our framwork to multi round protocols by building on th rcnt multi-party round limination tchniqu of [7] and its xtnsion in [8]. At a high lvl, in th hard instancs of r-round protocols, ach machin is providd with a collction of instancs of th sam problm but on a lowr dimnsion, i.., dfind on a smallr numbr of machins and input siz. On of ths instancs is a spcial on in that it nds to b solvd by th machins in ordr to solv th original instanc. Again, using th simultanity of th communication in on round, w show that th first round of communication cannot rval nough information about this spcial instanc and hnc th machins nd to solv th spcial instanc in only r 1 rounds of communication, which is provn to b hard inductivly. Using th abstraction in our framwork allows us to solly focus on th communication aspcts of this argumnt, indpndnt of th spcifics of th problm at hand. This allows us to provid a mor dirct and simplr proof than [7, 8], which is also applicabl to a widr rang of problms th rsults in [7, 8] ar for th stting of combinatorial auctions). Howvr, although simplr than [7, 8], this proof is still far from bing simpl - indd, it rquirs a dlicat information-thortic argumnt s Sction 4 for furthr dtails). This complxity of proving a multi-

6 round lowr bound in this modl is in fact anothr motivation for our framwork. To our knowldg, th only prvious lowr bounds spcific to boundd round protocols in th coordinator modl ar thos of [7, 8]; w hop that our framwork facilitats proving such lowr bounds in this modl undrstanding th powr of boundd round protocols is rgardd as an intrsting opn qustion in th litratur; s,.g., [60]). Finally, w prov th lowr bound for maximum covrag using this framwork by dsigning a family of sts which w call randomly narly disjoint; roughly spaking th sts in this family hav th proprty that any suitably small random subst of on st is ssntially disjoint from any othr st in th family. A radr familiar with [25] may raliz that this dfinition is similar to th dific st-systm introducd in [25]; th main diffrnc hr is that w nd vry random substs of ach st in th family to b disjoint from othr sts, as opposd to a pr-spcifid collction of sts as in difics [25]. As a rsult, th algbraic tchniqus of [25] do not sm suitabl for our purpos and w prov our rsults using diffrnt tchniqus. Th lowr bound thn follows by instantiating th hard distribution in our framwork with this family for maximum covrag and proving th approximation lowr bound. Uppr Bounds Rsult 2). W achiv th first algorithm in Rsult 2, namly an ) -approximation algorithm for maximum covrag and submodular maximization), via an implmntation of a thrsholding grdy algorithm s,.g., [16, 25]) in th distributd stting using th sampl-and-prun tchniqu of [48] a similar thrsholding grdy algorithm was usd rcntly in [51] for straming maximum covrag). Th main ida in th sampl-and-prun tchniqu is to sampl a collction of sts from th machins in ach round and snd thm to th coordinator who can build a partial grdy solution on thos sts; th coordinator thn communicats this partial solution to ach machin and in th nxt round th machins only sampl from th sts that can hav a substantial marginal contribution to th partial grdy solution maintaind by th coordinator, hnc ssntially prunning th input sts. Using a diffrnt grdy algorithm and a mor carful choic of th thrshold on th ncssary marginal contribution from ach st, w show that an ) -approximation can b obtaind in constant numbr of rounds and sublinar communication as opposd to th approach of [48] which rquirs Ωlog n) rounds). Th scond algorithm in Rsult 2, namly a k O1/r) - approximation algorithm for any numbr of rounds r, howvr is mor involvd and is basd on a nw itrativ sktching mthod spcific to th maximum covrag problm. Rcall that in our prvious algorithm th machins ar mainly obsrvrs and simply provid th coordinator with a sampl of thir input; our scond algorithm is in som sns on th othr xtrm. In this algorithm, ach machin is rsponsibl for computing a suitabl sktch of its input, which roughly spaking, is a collction of sts that tris to rprsnt ach optimal st in th input of this machin. Th coordinator is also maintaining a grdy solution that is updatd basd on th sktchs rcivd from ach machin. Th lmnts covrd by this collction ar shard by th machins to guid thm towards th sts that ar misrprsntd by th sktchs computd so far, and th machins updat thir sktchs for th nxt round accordingly. W show that ithr th grdy solution maintaind by th coordinator is alrady a good approximation or th final sktchs computd by th machins ar now a good rprsntativ of th optimal sts and hnc contain a good solution. 4 A Framwork for Distributd Lowr Bounds W introduc a gnral framwork for proving communication complxity lowr bounds for boundd round protocols in th distributd coordinator modl. Considr a dcision problm 3 P dfind by th family of functions P s : {0, 1} s {0, 1} for any intgr s 1; w rfr to s as siz of th problm and to {0, 1} s as its domain. Not that P s can b a partial function, i.., not ncssarily dfind on its whol domain. An instanc I of problm P s is simply a binary string of lngth s. W say that I is a Ys instanc if P s I) = 1 and is a No instanc if P s I) = 0. For xampl, P s can dnot th dcision vrsion of th maximum covrag problm ovr m sts and n lmnts with paramtr k in which cas s would b a fixd function of m, n, and k dpnding on th rprsntation of th input) such that thr is a rlativly larg gap as a function of, say, k) btwn th valu of optimal solution in Ys and No instancs. W can also considr th problm P s in th distributd modl, whrby w distribut ach instanc btwn th playrs. Th distributd covrag problm for instanc, can b modld hr by partitioning th sts in th instancs of P s across th playrs. To prov a communication lowr bound for som problm P, on typically nds to dsign a hard input distribution D on instancs of th problm P, and thn show that distinguishing btwn th Ys and No cass in instancs sampld from D, with som sufficintly larg probability, rquirs larg communication. Such a 3 Whil w prsnt our framwork for dcision problms, with som modifications, it also xtnds to sarch problms. W laborat mor on this in th full vrsion of th papr.

7 distribution invitably dpnds on th spcific problm P at hand. W would lik to abstract out this dpndnc to th undrlying problm and dsign a tmplat hard distribution for any problm P using this abstraction. Thn, to achiv a lowr bound for a particular problm P, on only nds to focus on th problm spcific parts of this tmplat and dsign thm according to th problm P at hand. W mphasiz that obviously w ar not going to prov a communication lowr bound for vry possibl distributd problm; rathr, our framwork rducs th problm of proving a communication lowr bound for a problm P to dsigning appropriat problm-spcific gadgts for P, which dtrmin th strngth of th lowr bound on can ultimatly prov using this framwork. With this plan in mind, w now dscrib a high lvl ovrviw of our framwork. 4.1 A High Lvl Ovrviw of th Framwork Considr any dcision problm P; w construct a rcursiv family of distributions D 0, D 1,... whr D r is a hard input distribution for r-round protocols of P sr, i.., for instancs of siz s r of th problm P, whn th input is partitiond btwn p r playrs. Each instanc in D r is a carful combination of many sub-instancs of problm P sr 1 ovr diffrnt substs of p r 1 playrs, which ar sampld ssntially) from D r 1. W nsur that a small numbr of ths sub-instancs ar spcial in that to solv th original instanc of P sr, at last on of ths instancs of P sr 1 ovr p r 1 playrs) nds to b solvd ncssarily. W hid th spcial sub-instancs in th input of playrs in a way that locally, no playr is abl to idntify thm and show that th first round of communication in any protocol with a small communication is spnt only in idntifying ths spcial sub-instancs. W thn inductivly show that as solving th spcial instanc is hard for r 1)-round protocols, th original instanc must b hard for r-round protocols as wll. W now dscrib this distribution in mor dtail. Th p r playrs in th instancs of distribution D r ar partitiond into g r groups P 1,..., P gr, ach of siz p r 1 hnc g r = p r /p r 1 ). For vry group i [g r ] and vry playr q P i, w crat w r instancs I1, i..., Iw i r of th problm P sr 1 sampld from th distribution D r 1. Th domain of ach instanc Ij i is th sam across all playrs in P i and is diffrnt i.., disjoint) btwn any two j j [w r ]; w rfr to w r as th width paramtr. Th nxt stp is to pack all ths instancs into a singl instanc I i q) for th playr q; this is on of th placs that w nd a problm spcific gadgt, namly a packing function 4 that can pack w r 4 For a radr familiar with prvious work in [12, 7, 8], w instancs of problm P sr 1 into a singl instanc of problm P s r for som s r s r. W postpon th formal dscription of th packing functions to th nxt sction, but roughly spaking, w rquir ach playr to b abl to construct th instanc I i q) from th instancs I1, i..., Iw i r and vic vrsa. As such, vn though ach playr is givn as input a singl instanc I i, w can think of ach playr as concptually playing in w r diffrnt instancs I1, i..., Iw i r of P sr 1 instad. In ach group i [g r ], on of th instancs, namly Ij i for j [w r ], is th spcial instanc of th group: if w combin th inputs of playrs in P i on thir spcial instanc Ij i, w obtain an instanc which is sampld from th distribution D r 1. On th othr hand, all othr instancs ar fooling instancs: if w combin th inputs of playrs in P i on thir instanc Ij i for j j, th rsulting instanc is not sampld from D r 1 ; rathr, it is an instanc cratd by picking th input of ach playr indpndntly from th corrsponding marginal of D r 1 D r 1 is not a product distribution, thus ths two distributions ar not idntical). Nvrthlss, by construction, ach playr is oblivious to this diffrnc and hnc is unawar of which instanc in th input is th spcial instanc sinc th marginal distribution of a playr s input is idntical undr th two distributions). Finally, w nd to combin th instancs I 1,..., I gr to crat th final instanc I. To do this, w nd anothr problm spcific gadgt, namly a rlabling function. Roughly spaking, this function taks as input th indx j, i.., th indx of th spcial instancs, and instancs I 1,..., I gr and crat th final instanc I, whil prioritizing th rol of spcial instancs in I. By prioritizing w man that in this stp, w nd to nsur that th valu of P sr on I is th sam as th valu of P sr 1 on th spcial instancs. At th sam tim, w also nd to nsur that this additional rlabling dos not rval th indx of th spcial instanc to ach individual playr, which rquirs a carful dsign dpnding on th problm at hand. Th abov family of distributions is paramtrizd by th squncs {s r } siz of instancs), {p r } numbr of playrs), and {w r } th width paramtrs), plus th packing and rlabling functions. Our main rsult in this sction is that if ths squncs and functions satisfy som natural conditions similar to what discussd not that a similar notion to a packing function is capturd via a collction of disjoint blocks of vrtics in [7] for finding larg matchings), Ruzsa-Szmrédi graphs in [12] for stimating maximum matching siz), and a family of small-intrscting sts in [8] for finding good allocations in combinatorial auctions). In this work, w us th notion of randomly narly disjoint stsystms dfind in Sction 5.1. S th full vrsion of th papr for mor dtails on th connction btwn this framwork and prvious work.

8 abov), thn any r-round protocol for th problm P sr on th distribution D r rquirs Ω r w r ) communication. W rmark that whil w stat our communication lowr bound only in trms of w r, to obtain any intrsting lowr bound using this tchniqu, on nds to nsur that th width paramtr w r is rlativly larg in th siz of th instanc s r ; this is also achivd by dsigning suitabl packing and labling functions as wll as a suitabl rprsntation of th problm). Howvr, as rlativly larg dpnds havily on th problm at hand, w do not add this rquirmnt to th framwork xplicitly. A discussion on possibl xtnsions of this framwork as wll as its connction to prvious work appars in th full vrsion of th papr. 4.2 Th Formal Dscription of th Framwork W now dscrib our framwork formally. As statd arlir, to us this framwork for proving a lowr bound for any spcific problm P, on nds to dfin appropriat problm-spcific gadgts. Ths gadgts ar functions that map multipl instancs of P s to a singl instanc P s for som s s. Th xact application of ths gadgts would bcom clar shortly in th dscription of our hard distribution. Dfinition 4.1. Packing Function) For s s 1 and w 1, w rfr to a function σ which maps any tupl of instancs I 1,..., I w ) of P s to a singl instanc I of P s as a packing function of width w. Dfinition 4.2. Labling Family) For s s 1 and g 1, w rfr to a family of functions Φ = {φ i }, whr ach φ i is a function that maps any tupl of instancs I 1,..., I g ) of P s to a singl instanc I of P s as a g-labling family, and to ach function in this family, as a labling function. W start by dsigning th following rcursiv family of hard distributions {D r } r 0, paramtrizd by squncs {p r } r 0, {s r } r 0, and {w r } r 0. W rquir {p r } r 0 and {s r } r 0 to b incrasing squncs and {w r } r 0 to b non-incrasing. In two placs markd in th distribution, w rquir on to dsign th aformntiond problm-spcific gadgts for th distribution. Distribution D r : A tmplat hard distribution for r-round protocols of P for any r 1. Paramtrs: p r : numbr of playrs, s r : siz of th instanc, w r : width paramtr, σ r : packing function, and Φ r : labling family. 1. Lt P b th st of p r playrs and dfin g r := p r p r 1 ; partition th playrs in P into g r groups P 1,..., P gr ach containing p r 1 playrs. 2. Dsign a packing function σ r of width w r which maps w r instancs of P sr 1 to an instanc of P s r for som s r 1 s r s r. 3. Pick an instanc I r D r 1 ovr th st of playrs [p r 1 ] and domain of siz s r For ach group P i for i [g r ]: a) Pick an indx j [w r ] uniformly at random and crat w r instancs I1, i..., Iw i r of problm P sr 1 as follows: i) Each instanc Ij i for j [w r] is ovr th playrs P i and domain Dj i = {0, 1} sr 1. ii) For indx j [w r ], Ij i = I r by mapping arbitrarily) [p r 1 ] to P i and domain of Ir to Dj i. iii) For any othr indx j j, Ij i Dr 1 := q P i D r 1 q), i.., th product of marginal distribution of th input to ach playr q P i in D r 1. b) Map all th instancs I1, i..., Iw i r to a singl instanc I i using th function σ r. 5. Dsign a g r -labling family Φ r which maps g r instancs of P s r to a singl instanc P sr. 6. Pick a labling function φ from Φ uniformly at random and map th g r instancs I 1,..., I gr of P s r to th output instanc I of P sr using φ. 7. Th input to ach playr q P i in th instanc I, for any i [g r ], is th input of playr q in th instanc I i, aftr applying th mapping φ to map I i to I. W rmark that in th abov distribution, th variabls in ach instanc sampld from D r ar th instancs I1, i..., Iw i r for all groups i [g r ], th indx j [w], and both th choic of labling family Φ r and th labling function φ. On th othr hand, th constants across all instancs of D r ar paramtrs p r, s r, and w r, th choic of grouping P 1,..., P gr, and th packing function σ r. To complt th dscription of this rcursiv family of distributions, w nd to xplicitly dfin th distribution D 0 btwn p 0 playrs ovr {0, 1} s0. W lt D 0 := 1 2 DYs DNo 0, whr D0 Ys is a distribution ovr Ys instancs of P s0 and D0 No is a distribution ovr No instancs. Th choic of distributions D0 Ys and D0 No

9 ar again problm-spcific. W start by dscribing th main proprtis of th packing and labling functions that ar rquird for our lowr bound. For any playr q P i, dfin I i q) := I1q), i..., Iw i r q)), whr for any j [w r ], Ij i q) dnots th input of playr q in th instanc Ij i. W rquir th packing and labling functions to b locally computabl dfind as follows. Dfinition 4.3. Locally computabl) W say that th packing function σ r and th labling family Φ r ar locally computabl iff any playr q P i for i [g r ], can comput th mapping of I i q) to th final instanc I, locally, i.., only using σ r, th sampld labling function φ Φ r, and input I i q). W us φ q to dnot th local mapping of playr q P i for mapping I i q) to I; sinc σ r is fixd in th distribution D r, across diffrnt instancs sampld from D r, φ q is only a function of φ. Th input to ach playr q P i is uniquly dtrmind by I i q) and φ q. Insid ach instanc I sampld from D r, thr xists a uniqu mbddd instanc Ir which is sampld from D r 1. Morovr, this instanc is ssntially copid g r tims, onc in ach instanc Ij i for ach group P i. W rfr to th instanc Ir as wll as its copis Ij 1,..., Igr j as spcial instancs and to all othr instancs as fooling instancs. W rquir th packing and labling functions to b prsrving, dfind as follows. Dfinition 4.4. γ-prsrving) W say that th packing function and th labling family ar γ- prsrving for a paramtr γ 0, 1), iff Pr Psr I) = P sr 1 Ir ) ) 1 γ. I D r In othr words, th valu of P sr on an instanc I should b qual to th valu of P sr 1 on th mbddd spcial instanc Ir of I w.p. 1 γ. Rcall that th packing function σ r is a dtrministic function that dpnds only on th distribution D r itslf and not any spcific instanc and hnc th undrlying spcial instancs); on th othr hand, th prsrving proprty rquirs th packing and labling functions to somhow prioritiz th spcial instancs ovr th fooling instancs in dtrmining th valu of th original instanc). To achiv this proprty, th labling family is allowd to vary basd on th spcific instanc sampld from th distribution D r. Howvr, w nd to limit th dpndnc of th labling family to th undrlying instanc, which is capturd through th dfinition of obliviousnss blow. Dfinition 4.5. W say that th labling family Φ r is oblivious iff it satisfis th following proprtis: i) Th only variabl in D r which Φ r can dpnd on is j [w r ] it can dpnd arbitrarily on th constants in D r ). ii) For any playr q P, th local mapping φ q and j ar indpndnt of ach othr in D r. Intuitivly spaking, Condition i) abov implis that a function φ Φ r can prioritiz th spcial instancs basd on th indx j, but it cannot us any furthr knowldg about th spcial or fooling instancs. For xampl, on may b abl to us φ to distinguish spcial instancs from othr instancs, i.., dtrmin j, but would not b abl to infr whthr th spcial instanc is a Ys instanc or a No on only basd on φ. Condition ii) on th othr hand implis that for ach playr q, no information about th spcial instanc is rvald by th local mapping φ q. This mans that givn th function φ q and not φ as a whol), on is not abl to dtrmin j. Finally, w say that th family of distributions {D r } is a γ-hard rcursiv family, iff i) it is paramtrizd by incrasing squncs {p r } and {s r }, and non-incrasing squnc {w r }, and ii), th packing and labling functions in th family ar locally computabl, γ-prsrving, and oblivious. W ar now rady to prsnt our main thorm of this sction. Thorm 4.1. Lt R 1 b an intgr and suppos {D r } R r=0 is a γ-hard rcursiv family for som γ 0, 1); for any r R, any r-round protocol for P sr on D r which rrs w.p. at most 1/3 r γ rquirs Ωw r /r 4 ) total communication. 4.3 Corrctnss of th Framwork: Proof of Thorm 4.1 W first st up som notation. For any r-round protocol π and any l [r], w us Π l := Π l,1,..., Π l,pr ) to dnot th random variabl for th transcript of th mssag communicatd by ach playr in round l of π. W furthr us Φ rsp. Φ q ) to dnot th random variabl for φ rsp. local mapping φ q ) and J to dnot th random variabl for th indx j. Finally, for any i [g r ] and j [w r ], I i j dnots th random variabl for th instanc Ij i. W start by stating a simpl proprty of oblivious mapping functions. Proposition 4.1. For any i [g r ] and any playr q P i, conditiond on input I i q), φ q ) to playr q, th indx j [w r ] is chosn uniformly at random. Proof. By Condition ii) of obliviousnss in Dfinition 4.5, Φ q J, and hnc J Φ q = φ q. Morovr, by Condition i) of Dfinition 4.5, Φ q cannot dpnd on I i q) and hnc I i q) Φ q = φ q also. Now notic

10 that whil th distribution of I i j and Ii j for j j, i.., Dr 1 and D r 1 ar diffrnt, th distribution of I i j q) and I i j q) ar idntical by dfinition of D r 1. As such, I i q) and j ar also indpndnt of ach othr conditiond on Φ q = φ q, finalizing th proof. W show that any protocol with a small communication cost cannot larn ssntially any usful information about th spcial instanc I r in its first round. Lmma 4.6. For any dtrministic protocol π for D r, II r ; Π 1 Φ, J) Π 1 /w r. Proof. Th first stp is to show that th information rvald about I r via Π 1 can b partitiond ovr th mssags snt by ach individual playr about thir own input in thir spcial instanc. Claim 4.7. II r ; Π 1 Φ, J) q P II rq) ; Π 1,q Φ, J). Proof. Intuitivly, th claim is tru bcaus aftr conditioning on Φ and J, th input of playrs bcom indpndnt of ach othr on all fooling instancs, i.., vry instanc xcpt for thir copy of Ir. As a rsult, th mssags communicatd by on playr do not add xtra information to mssags of anothr on about Ir. Morovr, sinc ach playr q is obsrving Ir q), th information rvald by this playr can only b about Ir q) and not Ir. W now provid th formal proof. Rcall that Π 1 = Π 1,1,..., Π 1,pr ). By chain rul of mutual information, II r ; Π 1 Φ, J) = Fact 2.1-4) q P W first show that for ach q P, 4.1) II r ; Π 1,q Π <q 1, Φ, J). II r ; Π 1,q Π <q 1, Φ, J) II r ; Π 1,q Φ, J). Rcall that, for any playr q, Iq) dnots th input to playr q in all instancs in which q is participating, and dfin I q) as th collction of th inputs to all othr playrs across all instancs. W argu that Iq) I q) I r, Φ, J. Th rason is simply bcaus aftr conditioning on I r, th only variabls in Iq) and I q) ar fooling instancs that ar sampld from Dr 1 which is a product distribution across playrs. This implis that IIq) ; I q) I r, Φ, J) = 0 by Fact 2.1-2)). Now, notic that th input to ach playr q is uniquly idntifid by Iq), Φ) by locally computabl proprty in Dfinition 4.3) and hnc conditiond on I r, Φ, J, th mssag Π 1,q is a dtrministic function of Iq). As such, by th data procssing inquality Fact 2.1-5)), w hav that IΠ 1,q ; Π <q 1 I r, Φ, J) = 0; by Proposition 2.2, this implis Eq 4.1) hr, conditioning on Π <q 1 in RHS of Eq 4.1) can only dcras th mutual information). Dfin I r q) as th input to all playrs in I r xcpt for playr q; hnc I r = I rq), I r q)). By chain rul of mutual information Fact 2.1-4)), II r ; Π 1,q Φ, J) = II rq) ; Π 1,q Φ, J) + II r q) ; Π 1,q I rq), Φ, J) = II rq) ; Π 1,q Φ, J), sinc II r q) ; Π 1,q I rq), Φ, J) = 0 as Π 1,q is indpndnt of I r q) aftr conditioning on I rq) and Fact 2.1-2)). Th claim now follows from Eq 4.1) and abov quation. Nxt, w us a dirct-sum styl argumnt to show that as ach playr is oblivious to th idntity of th spcial instanc in th input, th mssag snt by this playr cannot rval much information about th spcial instanc, unlss it is too larg. Claim 4.8. For any group P i and playr q P i, II rq) ; Π 1,q Φ, J) Π 1,q /w r. Proof. W first argu that, 4.2) II rq) ; Π 1,q Φ, J) II rq) ; Π 1,q Φ q, J). Lt Φ = Φ q, Φ q ) whr Φ q dnots th rst of th mapping function Φ byond Φ q. W hav, Π 1,q Φ q Φ q, J, I rq) sinc aftr conditioning on J, Φ dos not dpnd on any othr variabl in D r by obliviousnss proprty in Dfinition 4.5), and hnc th input to playr q and as a rsult Π 1,q ar indpndnt of Φ q aftr conditioning on both Φ q and J. Eq 4.2) now follows from th indpndnc of Π 1,q and Φ q and Proposition 2.2 as conditioning on Φ <q in RHS of Eq 4.2) can only dcras th mutual information). W can bound th RHS of Eq 4.2) as follows, II rq) ; Π 1,q Φ q, J) = E j [w r] [ II rq) ; Π 1,q Φ q, J = j) = 1 w r II i w jq) ; Π 1,q Φ q, J = j). r j=1 Rcall that j is chosn uniformly at random from [w r ] and I r = I i j conditiond on J = j. Our goal now is to drop th conditioning on th vnt J = j. By Dfinition 4.5, Φ q is indpndnt of J = j. Morovr, I i j q) is sampld from D r 1q) both in D r 1 and in Dr 1 ) and hnc is indpndnt of J = j, vn conditiond on Φ q. Finally, by Proposition 4.1, th input to playr q is indpndnt of J = j and as Π 1,q is a dtrministic function of th input to playr q, Π 1,q ]

11 is also indpndnt of J = j, vn conditiond on Φ q and I i j q). This mans that th joint distribution of I i j q), Π 1,q, and Φ q is indpndnt of th vnt J = j and hnc w can drop this conditioning in th abov trm, and obtain that, 1 w r j=1 w r II i jq) ; Π 1,q Φ i, J = j) 1 w r = 1 w r II i w jq) ; Π 1,q Φ i ) r w r j=1 j=1 II i jq) ; Π 1,q I i,<j q), Φ i ) = 1 w r II i q) ; Π 1,q Φ i ), whr th inquality holds sinc I i j q) Ii,<j q) Φ i and hnc conditioning on I i,<j q) can only incras th mutual information by Proposition 2.1. Finally, 1 w r II i q) ; Π 1,q Φ i ) Fact 2.1-1) Fact 2.1-3) Fact 2.1-1) 1 w 1 HΠ 1,q Φ i ) 1 w r HΠ 1,q ) 1 w r Π 1,q. Lmma 4.6 now follows from th prvious two claims: II r ; Π 1 Φ, J) II rq) ; Π 1,q Φ, J) Claim 4.7 Claim 4.8 q P 1 Π 1,q = 1 Π 1. w r w r q P For any tupl Π 1, φ, j), w dfin th distribution ψπ 1, φ, j) as th distribution of I r in D r conditiond on Π 1 = Π 1, Φ = φ, and J = j. Rcall that th original distribution of I r is D r 1. In th following, w show that if th first mssag snt by th playrs is not too larg, and hnc dos not rval much information by about I r by Lmma 4.6, vn aftr th aformntiond conditioning, distribution of I r dos not chang by much in avrag. Formally, Lmma 4.9. If Π 1 = ow r /r 4 ), thn [ ] E Π 1,φ,j) ψπ 1, φ, j) D r 1 = o1/r 2 ). Proof. [Proof Sktch] Sinc Ir is indpndnt of φ and j in D r, w hav D r 1 = disti r) = disti r Φ, J). As such, it suffics to show that disti r Φ, J) is clos to th distribution of disti r Π 1, Φ, J). By Lmma 4.6 and th assumption Π 1 = ow r /r 4 ), w know that th information rvald about Ir by Π 1, conditiond on Φ, J is quit small, i.., o1/r 4 ). This intuitivly mans that having an xtra knowldg of Π 1 would not b abl to chang th distribution of Ir by much. To mak this formal, w us th connction btwn mutual information and KL-divrgnc Fact 2.2) btwn th two distributions abov and us Pinskr s inquality Fact 2.3) to rlat th KL-divrgnc to total variation distanc and obtain th final bound. W dfr this simpl calculation to th full vrsion of th papr. Dfin th rcursiv function δr) := δr 1) o1/r 2 ) γ with bas δ0) = 1/2. W hav, Lmma For any dtrministic δr)-rror r-round protocol π for D r, w hav π = Ωw r /r 4 ). Proof. Proof is by induction on th numbr of rounds. Bas cas: Th bas cas rfrs to 0-round protocols for D 0, i.., protocols that ar not allowd any communication. As in th distribution D 0, Ys and No instancs happn w.p. 1/2 ach and th coordinator has no input, any 0-round protocol can only output th corrct answr w.p. 1/2, proving th induction bas. Induction stp: Suppos th lmma holds for all intgrs up to r and w prov it for r-round protocols. Th proof is by contradiction. Givn an r-round protocol π r violating th induction hypothsis, w crat an r 1)-round protocol π r 1 which also violats th induction hypothsis, a contradiction. Givn an instanc I r 1 of P sr 1 ovr playrs P r 1 and domain D r 1 = {0, 1} sr 1, th protocol π r 1 works as follows: 1. Lt P r = [p r ] and partition P r into g r qual-siz groups P 1,..., P gr as is don in D r. Crat an instanc I r of D r as follows: 2. Using public randomnss, th playrs in P r 1 sampl R := Π 1, φ, j ) distπ r ), D r ), i.., from th joint) distribution of protocol π r ovr distribution D r. 3. Th q-th playr in P r 1 in instanc I r 1 ) mimics th rol of th q-th playr in ach group P i for i [g r ] in I r, dnotd by playr i, q), as follows: a) St th input for i, q) in th spcial instanc I i j q) of I r as th original input of q in I r 1, i.., I r 1 q) mappd via σ r and φ to I as

12 is don in I r to th domain Dj i ). This is possibl by th locally computabl proprty of σ r and φ in Dfinition 4.3. b) Sampl th input for i, q) in all th fooling instancs Ij iq) of I r for any j j using privat randomnss from th corrlatd distribution D r I r = I r 1, Π 1, Φ, J) = R). This sampling is possibl by Proposition 4.2 blow. 4. Run th protocol π r from th scond round onwards on I r assuming that in th first round th communicatd mssag was Π 1 and output th sam answr as π r. In Lin 3b), th distribution th playrs ar sampling from dpnds on Π 1, φ, j which ar public knowldg through sampling via public randomnss), as wll as Ir which is not a public information as ach playr q only knows Ir q) and not all of Ir. Morovr, whil random variabls I i j q) for j j ) ar originally indpndnt across diffrnt playrs q as thy ar sampld from th product distribution Dr 1 ), conditioning on th first mssag of th protocol, i.., Π 1 corrlats thm, and hnc a-priori it is not clar whthr th sampling in Lin 3b) can b don without any furthr communication. Nvrthlss, w prov that this is th cas and to sampl from th distribution in Lin 3b), ach playr only nds to know Ir q) and not Ir. Proposition 4.2. Suppos I is th collction of all instancs in th distribution D r and Iq) is th input to playr q in instancs in which q participats; thn, disti I r = I r 1, Π 1, Φ, J) = R) = X q P distiq) I rq) = I r 1 q), Π 1, Φ, J) = R). Proof. Fix any playr q P, and rcall that I q) is th collction of th inputs to all playrs othr than q across all instancs spcial and fooling). W prov that Iq) I q) I rq), Π 1, Φ, J) in D r, which immdiatly implis th rsult. To prov this claim, by Fact 2.1-2), it suffics to show that IIq) ; I q) I rq), Π 1, Φ, J) = 0. Dfin Π q 1 as th st of all mssags in Π 1 xcpt for th mssag of playr q, i.., Π 1,q. W hav, IIq) ; I q) I rq), Π 1, Φ, J) IIq) ; I q) I rq), Π 1,q, Φ, J), sinc Iq) Π q 1 I q), I rq), Π 1,q, Φ, J as th input to playrs P \ {q} is uniquly dtrmind by I q), Φ by th locally computabl proprty in Dfinition 4.3) and hnc Π q 1 is dtrministic aftr th conditioning; this indpndnc mans that conditioning on Π q 1 in th RHS abov can only dcras th mutual information by Proposition 2.2. W can furthr bound th RHS abov by, IIq) ; I q) I rq), Π 1,q, Φ, J) IIq) ; I q) I rq), Φ, J), sinc I q) Π 1,q Iq), I rq), Φ, J as th input to playr q is uniquly dtrmind by Iq), Φ again by Dfinition 4.3) and hnc aftr th conditioning, Π 1,q is dtrministic; this implis that conditioning on Π 1,q in RHS abov can only dcras th mutual information by Proposition 2.2. Finally, obsrv that IIq) ; I q) I rq), Φ, J) = 0 by Fact 2.1-2), sinc aftr conditioning on Ir q), th only rmaining instancs in Iq) ar fooling instancs which ar sampld from th distribution Dr 1 which is indpndnt across th playrs. This implis that IIq) ; I q) I rq), Π 1, Φ, J) = 0 also which finalizs th proof. Having provd Proposition 4.2, it is now asy to s that π r 1 is indd a valid r 1 round protocol for distribution D r 1 : ach playr q can prform th sampling in Lin 3b) without any communication as I q), Π 1, Φ, J) ar all known to q; this allows th playrs to simulat th first round of protocol π r without any communication and hnc only nd r 1 rounds of communication to comput th answr of π r. W can now prov that, Claim Assuming π r is a δ-rror protocol on D r, π r 1 would b a δ + γ + o 1 r 2 ) ) -rror protocol on D r 1. Proof. [Proof Sktch] Our goal is to calculat th probability that π r 1 rrs on an instanc I r 1 D r 1. For th sak of analysis, suppos that I r 1 is instad sampld from th distribution ψ for a randomly chosn tupl Π 1, φ, j ) dfind bfor Lmma 4.9). Notic that by Lmma 4.9, ths two distributions ar quit clos to ach othr in total variation distanc, and hnc if π r 1 has a small rror on distribution ψ it would ncssarily has a small rror on D r 1 as wll by Fact 2.4). Using Proposition 4.2, it is asy to vrify that if I r 1 is sampld from ψ, thn th instanc I r constructd by π r 1 is sampld from D r and morovr Ir = I r 1. As such, sinc i) π r is a δ-rror protocol for D r, ii) th answr to I r and Ir = I r 1 ar th sam w.p. 1 γ by γ-prsrving proprty in Dfinition 4.4), and iii) π r 1 outputs th sam answr as π r, protocol π r 1 is a δ + γ)-rror protocol for ψ. W dfr th formal proof and dtaild calculation of this probability of rror to th full vrsion of th papr.