Fundamental Limits on Data Acquisition: Trade-offs between Sample Complexity and Query Difficulty

Transcription

1 Funamntal Limit on Data Acquiition: Tra-off btwn Sampl Complxity an Qury Difficulty Hy Won Chung, Ji Oon L, an Alfr O. Hro arxiv: v2 [c.it] 2 Jan 208 Abtract W conir qury-ba ata acquiition an th corrponing information rcovry problm, whr th goal i to rcovr binary variabl (information bit from parity maurmnt of tho variabl. Th quri an th corrponing parity maurmnt ar ign uing th ncoing rul of Fountain co. By uing Fountain co, w can ign potntially limitl numbr of quri, an corrponing parity maurmnt, an guarant that th original information bit can b rcovr with high probability from any ufficintly larg t of maurmnt of iz n. In th qury ign, th avrag numbr of information bit that i aociat with on parity maurmnt i call qury ifficulty ( an th minimum numbr of maurmnt rquir to rcovr th information bit for a fix i call ampl complxity (n. W analyz th funamntal tra-off btwn th qury ifficulty an th ampl complxity, an how that th ampl complxity of n = c max{, ( log / } for om contant c > 0 i ncary an ufficint to rcovr information bit with high probability a. Inx Trm Sampl complxity, qury ifficulty, Fountain co, Soliton itribution, crowourcing. I. INTRODUCTION Qury-ba ata acquiition ari in ivr application incluing: crowourcing [], [2]; activ larning [3], [4]; xprimntal ign [5], [6]; an community rcovry or clutring in graph [7], [8]. In th application, qury-ba ata acquiition can b mol a a 20 qution problm [9] btwn an oracl (or oracl an a playr whr th oracl now th valu of information bit that th playr aim to rcovr, whil th playr ign quri to th oracl an rciv anwr from th oracl. In thi papr, w conir qury-ba ata acquiition with th goal of rcovring th valu of variabl (x,..., x (information bit. Whn w aum that th variabl an th anwr (maurmnt ar binary, w can conir a parity chc um a a typ of maurmnt, which corrpon to xcluiv or (XOR or moulo 2 um of om ubt of th information bit. Qurying parity ymbol of th information bit gnraliz th 20 qution mol of [9], [0]. Thi gnralization i th focu of thi papr an ha a wi rang of application, in particular to crowourcing ytm []. Conir a crowourcing ytm coniting of a numbr of worr an a particular ta that thy ar xpct to wor on. Aum that th ta i to claify a collction of imag into two xcluiv group,.g., whthr or not an imag i uitabl for chilrn. A worr (oracl in th ytm i givn a qury about om ubt of th imag an a to provi a binary anwr rgaring tho imag. Aum that th worr can ip th qury if th worr i unur of th anwr. Th probability that on worr ip a qury i unnown at th tag of qury ign an it can b iffrnt for ach of th worr in th ytm, pning on thir abiliti or ffort. Thrfor, from th qury ignr point of viw, it i natural to aum that only a ranom ubt of th ign quri will b anwr by th worr in th crowourcing ytm. Th qury ignr objctiv i to ign quri uch that whn th rciv numbr of anwr xc om thrhol, rgarl of which ubt of th anwr wa collct, th original binary bit can b rcovr with high probability. In thi papr w how that Fountain co [] ar naturally uit to thi crowourcing qury ign problm. Fountain co ar a typ of forwar rror corrcting co uitabl for binary raur channl (BEC with unnown raur probabiliti. Thi typ of co ha bn th ubjct of much rarch for rliabl intrnt pact tranmiion whn th pact tranmitt from th ourc ar ranomly lot bfor thy arriv at th tination. Hy Won Chung (hwchung@ait.ac.r i with th School of Elctrical Enginring at KAIST in South Kora. Ji Oon L (jioon.l@ait.u i with th Dpartmnt of Mathmatical Scinc at KAIST in South Kora. Alfr O. Hro (hro@c.umich.u i with th Dpartmnt of EECS at th Univrity of Michigan. Thi wor wa partially upport by National Rarch Founation of Kora unr grant numbr 207REAA , an by Unit Stat Army Rarch Offic unr grant W9NF

2 2 For input ymbol (x, x 2,..., x, Fountain co prouc a potntially limitl numbr of parity maurmnt, which ar alo call output ymbol. By uing wll-ign Fountain co, on can guarant that, givn any t of output ymbol of iz ( + δ with mall ovrha δ > 0, th input ymbol can b rcovr with high probability. Exampl of Fountain co ar LT-co [2] an Raptor co [3]. By uing th Fountain co, w can ign potntially limitl numbr of quri (an corrponing parity maurmnt with th ir proprti uitabl for th crowourcing xampl. Howvr, th Fountain co framwor mut b xtn in orr to account for th worr limit capacity to anwr ifficult quri. Th qury ifficulty i fin a th avrag numbr of input ymbol rquir to comput a ingl parity maurmnt. Th qury ifficulty i rlat to th ncoing complxity for on-tag ncoing, but it i iffrnt from th ncoing complxity whn th ncoing i on in multipl tag. Th qury ifficulty rprnt th numbr of input ymbol on avrag th worr mut now to calculat on parity maurmnt. Dpning on th qury ifficulty, th numbr of anwr (parity maurmnt rquir to rcovr input ymbol may vary gratly. W call th minimum numbr of maurmnt rquir to rcovr input ymbol th ampl complxity. Th ampl complxity i a function of th qury ifficulty a wll a th numbr of input ymbol to b rcovr. Lt u conir two xtrm ca. Firt, conir th ca whn th qury ifficulty i qual to. Mor pcifically, w aum that ach qury a th valu of only on variabl in (x, x 2,..., x at a tim. Sinc it i not nown which of th quri woul b anwr by th worr at th tag of qury ign, a t of quri i ign by uniformly an ranomly picing on variabl in (x, x 2,..., x at a tim. For uch qurying cnario, with ranomly lct n maurmnt, in orr to rcovr th information bit with rror probability l than / u for om contant u > 0, th rquir numbr n of quri cal a log. On th othr han, whn ach qury i ign to gnrat a parity maurmnt of ranomly lct /2 bit at a tim, th rquir numbr n of maurmnt i only + c log for om contant c > 0 [3]. Thrfor, in th two xtrm ca, w can obrv that whn th qury ifficulty i qual to, th ampl complxity cal a log, whra for th qury ifficulty of orr th ampl complxity cal a. Thn th qution i how th ampl complxity cal a th qury ifficulty incra from to Θ(. In thi papr, w aim to analyz th funamntal tra-off btwn th ampl complxity an th qury ifficulty in rcovring information bit. Thr hav bn papr that hav analyz uch tra-off whn it i aum that th parity maurmnt involv only a fix numbr of input ymbol. In [4], th ca of pairwi maurmnt ( = 2 wa conir an in [5], a gnral intgr wa conir. Not that thr ar a total of ( poibl parity maurmnt for a fix. In both papr, it wa aum that ach maurmnt i inpnntly obrv with probability p ob. It wa hown that th numbr n of maurmnt to rcovr input ymbol with high probability cal a n = p ob ( = c max{, ( log /} for om contant c > 0. In thi papr, w gnraliz th wor in [5] by not fixing th numbr but inta allowing that th numbr follow a itribution (Ω 0, Ω,..., Ω whr Ω not th probability that th valu i chon, whr =0 Ω =. W conir th avrag qury ifficulty = =0 Ω an analyz th ampl complxity n a a function of th qury ifficulty an th numbr of input ymbol. By auming that follow th prcrib itribution, w can gnrat potntially limitl parity maurmnt by uing th ncoing rul mploy by Fountain co; guaranting that for any t of fix numbr of maurmnt it i poibl to rcovr th input ymbol with high probability. Thi framwor i thu mor uitabl for th ituation whr th parity maurmnt ar ra with arbitrary (unnown probabiliti an thu it i rquir to hav th ability to gnrat potntially limitl numbr of quri an corrponing parity maurmnt. For th prviou framwor in [4], [5], th maximum numbr of parity maurmnt i rtrict to ( for a fix. Our main contribution in thi papr i to pcify th funamntal tra-off btwn th ampl complxity (n an th qury ifficulty ( in thi gnraliz maurmnt mol. W how that th ampl complxity n ncary an ufficint to rcovr input ymbol with high probability cal a n = c max {, ( log / } ( for om contant c > 0. Not that for = O(log, th ampl complxity n i invrly proportional to th qury ifficulty. In particular, whn qury ifficulty = Θ(, th ampl complxity cal a log, whra whn = Θ(log, th ampl complxity cal a.

3 3 Th rt of thi papr i organiz a follow. In Sction II, w xplain th ncoing rul of Fountain co (how to gnrat potntially limitl numbr of parity maurmnt an tat th main problm of thi papr. In Sction III, w provi th main rult, howing th funamntal tra-off btwn th ampl complxity an th qury ifficulty. In Sction IV, w prov th main thorm. Mor tchnical tail for th proof ar prnt in Appnic. In Sction V, om imulation rult ar provi, which furthr upport our thortical rult. In Sction VI, w provi concluion an icu poibl futur rarch irction. A. Notation W u th notation for XOR of binary variabl, i.., for a, b {0, }, a b = 0 iff a = b an a b = iff a b. W not by j th -imnional unit vctor with it j-th lmnt qual to. For a vctor x, x not th numbr of in th vctor x. For vctor x an y, th innr prouct btwn x an y i not by x y. For two intgr α an β, w u th notation α β to inicat that mo(α, 2 = mo(β, 2. For two vctor x = (x, x 2,..., x an y = (y, y 2,..., y, whn w writ x y, it man that mo(x i, 2 = mo(y i, 2 for all i {, 2,..., }. W u th O( an Θ( notation to crib th aymptotic of ral qunc {a } an {b }: a = O(b impli that a Mb for om poitiv ral numbr M for all 0 ; a = Θ(b impli that a Mb an a M b for om poitiv ral numbr M an M for all 0. Th logarithmic function log i with ba. II. MODEL AND PROBLEM STATEMENT Conir a -imnional binary ranom vctor x = (X, X 2,..., X T, which i uniformly an ranomly itribut ovr {0, }. W call X, X 2,..., X th input ymbol. W aim to larn th valu of (X, X 2,..., X by obrving a total of n parity maurmnt of iffrnt ubt of tho bit. Conir -imnional binary vctor v i = (v i, v i2,..., v i, i =,..., n. Th parity maurmnt aociat with th vctor v i i fin by Y i = mo v ij X j, 2 = v i X v i X, (2 j= for i =,..., n. W call uch parity maurmnt (Y, Y 2,..., Y n th output ymbol. Each v i {0, } trmin which ubt of (X, X 2,..., X i to b pic in calculating th i-th parity maurmnt. Th proc of igning {v i } i call qury ign or ncoing. W u Fountain co, alo nown a raur ratl co, for th ncoing. Lt (Ω 0, Ω,..., Ω b a itribution on {0,,..., } whr Ω not th probability that th valu i chon an =0 Ω =. In th ncoing of Fountain co, ach vctor v i i gnrat inpnntly an ranomly by firt ampling a wight {0,,..., } from th itribution (Ω 0,..., Ω an thn lcting a -imnion vctor of wight uniformly at ranom from all th vctor of {0, } with wight. Conir an arbitrary t of n output ymbol (Y,..., Y n gnrat by th abov ncoing rul. Th rlationhip btwn th input ymbol an th n output ymbol can b pict by a bipartit graph with input no on on i an n output no on th othr i a hown in Fig. Dnot by th avrag gr of th output no, = Ω. (3 Thi numbr inicat th avrag numbr of input ymbol involv in on parity maurmnt (output ymbol an i rlat to th ifficulty in calculating on parity maurmnt. W call thi numbr qury ifficulty. Th proc of rcovring th input ymbol from th n output ymbol i call information rcovry or coing. Dnot by ˆx(Y th timat of x givn Y n P ( an fin th probability of rror a = min ˆx( Pr(ˆx(Y x. (4 With th propr choic of th itribution (Ω 0,..., Ω, th Fountain co guarant that P ( 0 a with n largr than om thrhol. Th minimum numbr of n rquir to guarant P ( 0 a, minimiz ovr all (Ω 0,..., Ω for a fix an, i call ampl complxity. W aim to fin th funamntal limit on n to guarant rliabl information rcovry of input ymbol for a fix qury ifficulty.

4 4 Input no x x 2 x 3 x 4 x 5 x 6 x. Output no. y y 2 y 3 y 4 y 5 y 6 y 7 y n Fig.. Bipartit graph btwn input no an output no. III. MAIN RESULTS: FUNDAMENTAL TRADE-OFFS BETWEEN SAMPLE COMPLEXITY AND QUERY DIFFICULTY In thi ction, w tat our main rult that th ampl complxity n that i ncary an ufficint to ma P ( 0 a cal in trm of an a n = c max{, ( log / } for om contant c > 0 inpnnt of an, whn th parity maurmnt ar gnrat by th ncoing rul of Fountain co a xplain in Sction II. W firt tat th wll-nown lowr boun on th ampl complxity n of Fountain co prnt in [3]. Propoition : To rliably rcovr input ymbol with P ( / u for om contant u > 0 from parity maurmnt gnrat by Fountain co, it i ncary that n c l max {, log } for om contant c l > 0. ( Proof: Showing th firt conition n c l i traightforwar. Each output ymbol Y i = mo j= v ijx j, 2 rprnt a linar quation of unnown input ymbol (X, X 2,..., X. Sinc thr ar unnown, it i ncary to hav at lat n = linar quation to olv thi linar ytm rliably. Th con conition n (c l log / i from a proprty of ranom graph. In th bipartit graph btwn input no an output no, w ay that an input no i iolat if it i not connct to any of th output no. W analyz th probability that an input no i iolat whn th g ar ign by th ncoing rul of Fountain co. Th rror probability P ( i boun blow by th probability that an input no i iolat, inc whn an input no i iolat th coing rror happn. Conir an output no with gr. Th probability that an input no i not connct to thi output no of gr qual /. Sinc an output no ha gr with probability Ω, th probability that an input no i not connct to an output no qual Ω ( / = /. (6 =0 Sinc thr ar n output no an th output no ar ampl inpnntly, th probability that an input no i iolat (not connct to any of tho output no qual ( n. (7 By th man valu thorm, w can how that ( / n α/( α/n whr α = n /. Sinc th coing rror probability P ( i lowr boun by th probability that an input no i iolat, to atify P ( / u (5

5 5 for om contant u > 0, it i ncary that α/( α/n / u, which i quivalnt to u α log + (u log /n u log + (u log / u log + (u log 3/3 c l log for om contant c l > 0. By plugging in α = n /, w gt n c l log. (9 (8 Th main contribution of thi papr i howing that th boun in (5 i in achivabl (up to contant caling by proprly ign Fountain co for any from Θ( to Θ(log. W provi a particular output gr itribution (Ω 0, Ω,..., Ω for which w can control th qury ifficulty from Θ( to Θ(log an how that it i poibl to rliably rcovr information bit with ampl complxity obying { n = c u max, log } (0 for om contant c u > 0. Thrfor, by combining (0 with (5, w conclu that n = c max{, ( log / } i ncary an ufficint for rliabl rcovry of information bit whn i th avrag qury ifficulty. Suppo that th law of Ω i givn by an ial Soliton itribution D if = Ω = ( if 2 D ( 0 if > D or = 0, for om D {2, 3,..., }. Hr, for implicity, w aum that 3. Not that th qury ifficulty cal a log D inc log(d + < = D D + D = < log D +. =2 Thrfor, a D incra from 2 to, th qury ifficulty cal from log 3 to log. Thorm : For th Soliton itribution ( with D {2, 3,..., }, th input ymbol can b rliably rcovr, i.., P ( 0 a, with ampl complxity { n = c u max, log } (2 for om contant c u > 0. Th proof of Thorm will b prnt in Sction IV. Thorm tat that for qury ifficulty = O(log, th ampl complxity n to rliably rcovr input ymbol i invrly proportional to th qury ifficulty. Whn th qury ifficulty o not incra in, i.., = Θ(, it i ncary an ufficint to hav n = Θ( log to rliably rcovr th information bit. In thi rgim, th ratio btwn an n convrg to 0 a. On th othr han, whn w incra th qury ifficulty to = Θ(log, it i nough to hav n = Θ( ampl, which rult in a poitiv limit of /n a. Whn log, incraing th qury ifficulty no longr hlp in rucing th ampl complxity. By uing th Soliton itribution ( an th ncoing rul of Fountain co, w can ign potntially limitl numbr of quri about (x, x 2,..., x an th corrponing parity maurmnt. Thorm how that with any t of maurmnt of iz n no largr than (2, w can rliably rcovr th information bit a. Morovr, thi ampl iz i optimal up to contant a hown by Propoition. Thu, our rult provi th optimal qury ign tratgy for rliabl information rcovry from an arbitrary t of parity maurmnt, optimal in trm of th ampl complxity (up to contant for a fix qury ifficulty.

6 6 IV. PROOF OF THEOREM In thi ction, w prov Thorm by proviing an uppr boun on P ( an howing that th ampl complxity n ufficint to ma thi uppr boun convrg to 0 a i qual to c u max{, ( log / } for om contant c u > 0. Conir P ( fin in (4. Th optimal coing rul ˆx( that minimiz th probability of rror i th maximum lilihoo (ML coing for th uniformly itribut ( input ymbol. Aum that w collct n parity n maurmnt (Y,..., Y n ach of which qual Y i = mo j= v ijx j, 2. Conir a matrix A who i-th row i v i = (v i, v i2,..., v i, i.., A := [v ; v 2 ;... ; v n ]. (3 W call A a ampling matrix. Givn (Y,..., Y n an th ampling matrix A, th ML coing rul fin x = (X, X 2,..., X T {0, } uch that Ax (Y, Y 2,..., Y n T. (4 If thr i a uniqu olution x {0, } for thi linar ytm, thn it i claim that ˆx(Y = x. If thr i mor than on x atifying thi linar ytm, thn an rror i clar. Th probability of rror i thu qual to P ( = x {0,} 2 Pr( x x uch that Ax Ax. (5 Du to ymmtry, th probabiliti Pr( x x uch that Ax Ax ar qual for vry x {0, }. Thu, w focu on th ca whr x i th vctor of all zro an conir By uing th union boun, it can b hown that P ( = Pr( x 0 uch that Ax 0. (6 P ( x 0 Pr(Ax 0 = = = ( ( Pr A = x = Pr(Ax 0 ( (7 i 0 whr i i th i-th tanar unit vctor. Th lat quality follow from th ymmtry of th ampling matrix A. Sinc all th output ampl ar gnrat inpnntly by th intically itribut v i, ach of which ha wight with probability Ω, ( ( ( ( n Pr W nxt analyz P ( = i= v i 0 = i= (( ( ( ( n Ω Pr v i 0 v =. = i= (8 ( ( Pr v i 0 v =. (9 i= Not that v ( i= i 0 if an only if thr ar vn numbr of in th firt ntri of v. Thi probability qual ( ( ( i i( i Pr v i 0 vt i i vn = = (. (20 i=

7 7 W nxt provi an uppr boun on (20. Dfin I = i i i vn ( i ( i. (2 In th following lmma, w provi an uppr boun on I a a multipl of (. Th proof of thi lmma i ba on that of th imilar lmma provi in [5], whr th uppr boun on I i tat pning on th rgim of for a fix. W provi an altrnativ vrion whr th uppr boun on I pn on th rgim of for a fix. Lmma : Conir th ca that 2 (i..,. Dfin κ( = (22 For 2 + (or,, whn w fin α =, {( ( 2 I 5α, whn < κ(, (23, whn κ(. 4 5( 2 For > 2 (or, <, whn w fin α = +, {( ( 2 I 5α, whn > κ(, (24, whn κ(. 4 5( In th ca > 2, w can obtain th boun for I imply by changing to. Proof: Appnix A. By uing Lmma an (20, th uppr boun on P ( in (8 can b furthr boun by whr w lt P ( 2 2 = 2 2 Σ = ( κ( = κ( κ( κ( + ( 2 κ( Ω + 5α = κ( + ( ( Σ n 2 2 Ω + Ω + + ( 2 5α = κ( ( nσ, = κ( + Ω ( Ω + Suppo that th law of Ω i givn by a Soliton itribution provi in (. Hr, for implicity, w aum that D {2, 3,..., } an 3. For thi Soliton itribution, w provi an uppr boun on ( nσ in (25 for 2 pning on th rgim of κ( with conition on th ampl complxity n. Lmma 2: With th ampl complxity { n c u max, log } (27 for om contant c u > 0, th trm ( nσ i boun abov a follow. 4 5 Ω n. (25 (26

8 8 Fig. 2. Mont Carlo imulation (5000 run of th probability of rror P ( with = 300 for thr iffrnt (th qury ifficulti. Th ampl complxity i normaliz by ( log /. W can obrv th pha tranition for P ( aroun th normaliz ampl complxity qual to for all th thr qury ifficulti conir. If κ( > D, ( nσ <. (28 2 If 4 κ( D ( { if, nσ 2 2 if < /2. 3 If κ( 3, ( nσ 2. (30 Proof: Appnix B. W rmar that th ca 2 o not happn whn D {2, 3}. From Lmma 2, whn th ampl complxity n atifi (27 w can furthr boun P ( P ( 2 2 ( in (25 by (29 c ( (3 for om contant c > 0. Not that thi uppr boun convrg to 0 a. V. SIMULATIONS In thi ction, w provi mpirical prformanc analyi for th probability of rror in th rcovry of information bit, a a function of th ampl complxity an qury ifficulty. In Fig. 2, w provi Mont Carlo imulation rult for th probability of rror P (, fin in (4, whr th numbr of information bit to rcovr i fix a = 300. W plot P ( in trm of th normaliz ampl complxity, normaliz by ( log / whr i th qury ifficulty. W run th imulation for thr iffrnt qury ifficulti, =4, 4.7, 5.6. Th parity maurmnt (output ymbol ar ign by firt ampling (th numbr of input ymbol rquir to comput a ingl parity maurmnt from th Soliton itribution ( an thn gnrating th maurmnt by th ncoing rul of Fountain co.

9 9 Fig. 3. Sam imulation conition a in Fig 2 xcpt that th horizontal axi i th un-normaliz ampl complxity. A th qury ifficulty incra, th ampl complxity to ma P ( clo to 0 cra. Thi illutrat th tra-off btwn th qury ifficulty an th ampl complxity. Obrv th pha tranition of P ( aroun th normaliz ampl complxity qual to. In Thorm, w tat that with ampl complxity of c u max{, ( log / } for om contant c u > 0, w can guarant P ( 0 a. Th imulation rult how that c u i ufficint to prouc a ramatic cra of P (. Sinc th pha tranition occur in th vicinity of normaliz ampl complxity qual to, th figur montrat th tra-off btwn th qury ifficulty an th ampl complxity. Spcifically, th rquir numbr of parity maurmnt to rliably rcovr information bit i invrly proportional to th qury ifficulty whn = O(log. Not that for th Soliton itribution (, th qury ifficulty i O(log, an thu max{, ( log / } = Θ(( log /. In Fig 3, w how th am imulation with un-normaliz ampl complxity inxing th horizontal axi. From thi plot, w can obrv that a th qury ifficulty incra, th rquir numbr of ampl to ma P ( clo to 0 cra. VI. CONCLUSIONS In thi papr, w analyz th funamntal tra-off btwn qury ifficulty an ampl complxity n in a qury-ba ata acquiition ytm aociat with a crowourcing ta with worr who may b non-rponiv to crtain quri (channl raur. W conir th information rcovry of binary variabl (x, x 2,..., x from parity maurmnt of ubt of th variabl. W u a qury ign ba on th ncoing rul of Fountain co, with which w can ign potntially limitl numbr of quri. W how that th propo qury ign policy guarant that th original information bit can b rcovr with high probability from any t of maurmnt of iz n largr than om thrhol. W obtain ncary an ufficint conition on ampl complxity n c max{, ( log / }. Thr ar vral intrting futur rarch irction rlat to thi wor. On of uch irction inclu analyzing tra-off btwn qury ifficulty an ampl complxity for partial information rcovry problm. In thi papr, w conir xact information rcovry, maning w aim to rcovr all th information bit with high probability. But pning on cnario, it coul b nough to rcovr only α of information bit for α (0,. Thn, th qution i how much thi rlax rcovry conition woul hlp in rucing th ampl complxity for a givn qury ifficulty. Epcially, on intrting qution might b whthr it i poibl to rcovr α information bit with only n = Θ( maurmnt vn with th vry low qury ifficulty = Θ(, which o not incra in. In th xact rcovry problm, it wa impoibl to rliably rcovr input ymbol with th ampl complxity n = Θ( whn th qury ifficulty i = Θ(. With = Θ(, it wa ncary to hav at lat n = Θ( log ampl complxity for th xact rcovry, which ma th ratio /n go to 0 a. Thrfor, it woul b intrting to whthr th ampl complxity of n = Θ( i ufficint for th partial rcovry problm vn with th qury ifficulty of = Θ(.

10 0 Anothr intrting irction i to apply th propo qury ign to ral crowourcing ytm an to analyz th xprimntal tra-off btwn th qury ifficulty an th ampl complxity. Epcially, whn th collct maurmnt contain inaccurat anwr an th probability that th maurmnt inclu inaccurat anwr chang pning on th qury ifficulty, th corrponing ampl complxity might b a iffrnt function of an. Thrfor, it woul b intrting to fin th qury ifficulty that minimiz th ampl complxity in crowourcing ytm with ranom raur an inaccurat anwr, an thi irction of rarch woul hlp guiing th ign of ampl-fficint crowourcing ytm. APPENDIX A PROOF OF LEMMA To prov thi lmma, w { rfr to th imilar } boun provi { in [5]. } Lmma 3: Lt β = max + 2+, + 2( + an α = max +, +. Thn w hav 2 ( ( 5α(, whn < β, i i 5(, whn β β, (32, whn β <. Not that i i i o ( = i i i o ( i 2( 5α ( i ( + i i i vn Thrfor, by uing Lmma 3, w can fin an uppr boun on i W fin ( i i i vn ( i I = ( ( i i i i i vn ( ( 2 5α, whn < β, 4 ( 5(, whn β β, 2( 5α, whn β <. (. (33 ( i( i a a caling of ( uch that (34 κ( = (35 W firt conir th ca 2 (i..,. Sinc β attain it maximum 3 at = or =, w fin that β < 2. Hnc, β > an th lat ca in (34 cannot happn. For 2 (or,, + β =, α = Not that κ(+ 2κ(+ + =. Sinc 2+ i an incraing function of, if < κ( thn β >. Thu, {( ( 2 I 5α, whn < κ(, (36, whn κ(. 4 5( 2 For > 2 (or, <, + β =, α = + 2( +.

11 Procing a abov, w gt {( ( 2 I 5α, whn > κ(, (37, whn κ(. 4 5( In th ca > 2, w can obtain th boun for I imply by changing to. APPENDIX B PROOF OF LEMMA 2 In thi lmma, w prov an uppr boun on ( nσ whr For th Soliton itribution w hav th qury ifficulty Σ = κ( = κ( κ( Ω + Ω + + = κ( + D if = Ω = ( if 2 D 0 if > D or = 0, log(d + < = D + D For implicity, hr w aum that D 2. Rcall that =2 which i a craing function of, an κ( > 0 for 2. If κ( > D, Thu, if n 5 log, Σ If 4 κ( D, w firt notic that κ( ( Ω +. D = < log D +. (38 κ( = + 2 +, (39 Ω + > 2 5 D Ω = 2 5. ( ( nσ < 2n xp 2 =. 5 < 2 κ( > 3 κ( 4. (40 7 Thu 2 7 an κ( = ( In thi ca, Σ 2 5 κ( > 2 κ( 5 =2 > log Ω + log( κ( ( 4. 2

12 2 Morovr, inc 7, if n C for om ufficintly larg C, (C 68 uffic nσ 2C ( 4 5 log 2 ( ( 2C 4 log log 7 + 2C ( 4 5 log 2 ( 4 log. From Stirling formula, w alo hav that 2πn n+ 2 n n! n n+ 2 n, hnc Thu, if n 68, Not that ( + 2 2π( + 2 ( ( ( ( ( ( ( 2 = xp ( log xp ( 2 log (. ( ( nσ xp 2 log ( = + 2 ( 2. ( { 2 if, 2 2 if < /2. 3 If κ( = 3, w fin from (40 that 2 7. Thn, by coniring th ca = 2, Σ 2 5 κ( 2 5( 2( 2 35( 35 Ω + for 3. Thu, if n 35, 4 If κ( =, 2, Thu, if n 0, Σ 5 ( nσ 2. κ( = κ( Ω Ω 2 5 = 0. ( nσ 2.

13 3 REFERENCES [] D. R. Kargr, S. Oh, an D. Shah, Bugt-optimal ta allocation for rliabl crowourcing ytm, Opration Rarch, vol. 62, no., pp. 24, 204. [2] M. S. Brntin, J. Brant, R. C. Millr, an D. R. Kargr, Crow in two con: Enabling raltim crow-powr intrfac, in Procing of th 24th annual ACM ympoium on Ur intrfac oftwar an tchnology. ACM, 20, pp [3] D. J. MacKay, Information-ba objctiv function for activ ata lction, Nural computation, vol. 4, no. 4, pp , 992. [4] B. Sttl, Activ larning litratur urvy, Univrity of Wiconin, Maion, vol. 52, no , p., 200. [5] D. V. Linly, On a maur of th information provi by an xprimnt, Th Annal of Mathmatical Statitic, pp , 956. [6] V. V. Forov, Thory of optimal xprimnt. Elvir, 972. [7] E. Abb an C. Sanon, Community tction in gnral tochatic bloc mol: Funamntal limit an fficint algorithm for rcovry, in Founation of Computr Scinc (FOCS, 205 IEEE 56th Annual Sympoium on. IEEE, 205, pp [8] B. Haj, Y. Wu, an J. Xu, Information limit for rcovring a hin community, IEEE Tranaction on Information Thory, 207. [9] H. W. Chung, B. M. Salr, L. Zhng, an A. O. Hro, Unqual rror protction qurying polici for th noiy 20 qution problm, IEEE Tranaction on Information Thory, DOI: 0.09/TIT , 207. [0] T. Tiligarii, B. M. Salr, an A. O. Hro, Collaborativ 20 qution for targt localization, IEEE Tranaction on Information Thory, vol. 60, no. 4, pp , 204. [] D. J. MacKay, Fountain co, IEE Procing-Communication, vol. 52, no. 6, pp , [2] M. Luby, LT co, in Procing. Th 43r Annual IEEE Sympoium on Founation of Computr Scinc, IEEE, 2002, pp [3] A. Shorollahi, Raptor co, IEEE Tranaction on Information Thory, vol. 52, no. 6, pp , [4] Y. Chn, C. Suh, an A. J. Golmith, Information rcovry from pairwi maurmnt, IEEE Tranaction on Information Thory, vol. 62, no. 0, pp , 206. [5] K. Ahn, K. L, an C. Suh, Community rcovry in hyprgraph, in th Annual Allrton Confrnc on Communication, Control, an Computing (Allrton. IEEE, 206, pp