Linear-Time Encodable and Decodable Error-Correcting Codes MIT. ipped by the channel. codewords of the code are the individual images under

Transcription

1 Linear-Time Encoable an Decoable Error-Correctin Coes Daniel A. Spielman MIT Abstract We present a class of asymptotically oo errorcorrectin coes that can be both encoe an ecoe in linear sequential time an loarithmic parallel time with a linear number of processors. We present both ranomize an explicit constructions of these coes. An important step in our construction is the introuction of error-reucin coes: coes that have a ecoer that can very quickly remove a constant fraction of the errors from a corrupte coewor.. Introuction Error-correctin coes were evelope to enable communication over noisy channels. The situation is escribe by Fiure, which appears in the beinnin of almost every book on error-correctin coes. Errors/Noise Sener Encoer Channel Decoer Receiver Fiure : Communication over a noisy channel. The sener passes a messae to the encoer, which then as reunant information to the messae to form a coewor. Distinct coewors shoul ier in many places so that, if the channel corrupts a coewor in a Dept. of Mathematics, Massachusetts Institute of Technoloy, Cambrie, MA 39. spielman@math.mit.eu. Partially supporte by the Fannie an John Hertz Founation, Air Force Contract F96-9-J-, DARPA N-9-J-799, an NSF rant 98CCR. only few places, the ecoer will still be able to etermine which coewor was sent. It is stanar to assume that the channel creates errors in a transmitte messae by ippin some of the bits in the messae. A wiely stuie channel is the binary symmetric channel, in which the channel ips each bit inepenently with a xe probability p < =. We will say that a bit has been \corrupte" if it has been ippe by the channel. Formally, we ene an error correctin coe of blocklenth n, rate r <, an relative minimum istance to be the imae of a function f: f; rn! f; n such that for all istinct x; y f; rn we have that (f(x); f(y)) n, where (u; v) is the Hammin istance (the number of bits in which u an v ier). The coewors of the coe are the iniviual imaes uner f of f; rn. Encoin is the process of mappin x f; rn to f(x), an ecoin is the process of mappin some u f; n to the x which minimizes (f(x); u). If a coe has relative minimum istance, then there is a ecoin alorithm that will prouce x from a corrupte version of f(x) in which fewer than n= bits have been ippe. Of course, some ecoin alorithms will be able to correct more errors than others. We will say that a ecoin alorithm can correct n errors if, for all u f; n an an x f; rn such that (f(x); u) < n, the ecoin alorithm outputs x on input u. Coin schemes that use this type of coe are calle block coes: when a lon messae is to be sent, it is rst ivie into pieces of lenth rn. Each piece is encoe to form a block of lenth n. These blocks are then sent over the channel. A central problem of coin theory is to n families of coes for which an r remain constant as the block-lenth rows. Such families are calle asymptotically oo. Asymptotically oo families are esirable because they enable one to encoe ata in fewer blocks of larer size, thereby ecreasin the probability that

2 an uncorrectable number of errors will occur in any one block. No aitional communication costs are incurre because the rate of the coes remains constant even as the block-lenth rows. However, the amount of computation neee to encoe an ecoe a block coe usually rows isproportionately with the lenth of the coe. In this paper, we present the rst asymptotically oo family of error-correctin coes in which the work neee to encoe an ecoe the coes is strictly proportional to the lenth of the coes... Previous work Early formalizations of the problem of nin computationally ecient error-correctin coes can be foun in [Ziv67, BZP77, Sav7]. Certain Ree-Solomon an Goppa coes can be encoe in rouhly O(n lo n) time an ecoe in rouhly O(n lo n) time [Jus76, Sar77]. While these coes are not necessarily asymptotically oo, they can be concatenate with Justesen coes to obtain asymptotically oo coes with similar encoin an ecoin times. Other oo coin schemes that have been more ecient in one operation have suere in the other. Gelfan, Dobrushin an Pinsker [GDP73] presente a ranomize construction of asymptotically oo errorcorrectin coes that coul be encoe in linear time. They i not suest an alorithm for ecoin their coes. Zyablov an Pinsker [ZP76] an Kuznetsov [Kuz73] showe that it is possible to ecoe Gallaer's ranomly chosen low-ensity parity-check coes [Gal63] in loarithmic time with a linear number of processors. Since these coes are linear, they can be encoe by performin a binary matrix-vector multiplication. We are not aware of a more ecient means of encoin these coes. Sipser an Spielman [SS9] use expaner raphs to construct error-correctin coes that can be ecoe in linear sequential time an loarithmic parallel time with a linear number of processors. These \expaner coes" are linear coes so they can be encoe usin quaratic work. They presente both ranomize an explicit constructions of their coes. We buil on the techniques evelope in their paper... Superconcentrator coes The coes that we present in this paper have encoin circuits that stronly resemble superconcentrators. Accorinly, we will refer to them as \superconcentrator coes". The intuition behin why we can ecoe these coes eciently is that each of the expaner raphs use No knowlee of superconcentrators is assume in this paper. If the reaer woul like to learn more about superconcentrators, we recommen reain [Pip77]. in the construction of the \superconcentrators" can be ecoe usin the expaner coe ecoin alorithms of [SS9]. All the coes that we iscuss in this paper will be systematic linear coes. This means that the coewors can be ivie into rn \messae bits" an (? r)n \parity bits". The messae bits contain the messae that the sener wants to encoe, so there are no restrictions on their values. The parity bits contain the reunant information that the encoer as to the messae bits, an are uniquely etermine by the messae bits. They are calle parity bits because each is etermine by some linear combination over GF () of the messae bits. These coes are linear because the set of coewors form a vector space over GF ()..3. Outline of paper We bein in Section by explainin why linear-size encoin circuits must look like superconcentrators. In Section 3, we introuce the concept of an errorreuction coe an present ranomize constructions of these coes that can be encoe an ecoe in lineartime. These coes are use in Section to construct superconcentrator coes. We conclue in Section by presentin explicit constructions of linear-time encoable an ecoable error-reuction coes. We then show how to use these to construct linear-time encoable an ecoable error-correctin coes.. Motivatin the construction It is not an accient that our linear-time encoin circuits look like superconcentrators. They have to. In this section, we will explain why. While this fact was the motivation behin our construction, one shoul be able to unerstan our construction without reain this section. Consier a circuit C that takes as input some bits x ; : : : ; x m, an prouces as output bits p ; : : :; p n such that the wors x ; : : :; x m ; p ; : : : ; p n form a oo errorcorrectin coe. This means that there is a constant such that even if we erase any m of the input bits an any n of the output bits, we can still recover the erase m input bits. We will show that this means that there Usually, we iscuss errors in which a bit's value is ippe. It is also possible to consier the case in which no value is receive for some bit. In this case, the ecoer knows that the value of the bit has been lost. These errors are calle erasure errors. Any error-correctin coe that can tolerate the rst type of error can also tolerate erasure errors. If one is uncomfortable with erasure errors, then just ip a coin an assin its value to each erase bit. One will probably be left with half as many errors of the rst type.

3 must be ate-isjoint paths from the erase inputs to some subset of the un-erase outputs. Assume that we cannot n m vertex-isjoint paths from the erase inputs to the un-erase outputs. Then, Mener's Theorem implies that there is some set of m? ates in the circuit such that all paths in the circuit from the erase inputs to the un-erase outputs must o throuh these ates. This contraicts our assumption that it is possible to recover the values of the erase inputs because there are m bits of information in the erase input ates, but only m? bits of information can et throuh to the un-erase output ates. Thus, we see that vertex isjoint paths can be rawn in the unerlyin raph from any m input ates into any (? )n output ates. While this property is not quite as stron as the property require of superconcentrators, it is suciently close that we ecie that the easiest way to create linear-size encoin circuits woul be to base them on Valiant's [Val76] construction of linear-size superconcentrators. 3. Error-reucin coes In this section, we introuce the concept of an errorreucin coe. While we are not sure whether this iea will have practical applications, it will be useful for unerstanin our main construction. An error-reucin coe has the property that if a ecoer receives a partially corrupte coewor, then it can correctly compute a lare fraction of its messae bits. The fraction of the messae bits that the ecoer cannot compute shoul be smaller than the fraction of the bits of the coewor that were corrupte in transmission. One coul imaine usin such a coe if one is encoin ata that is alreay encoe by an errorcorrectin coe. In such a situation, it miht only be necessary to remove most of the errors from a messae. We will ene an error-reucin coe of rate r <, error-reuction <, an reucible istance < to be the imae of a function f : f; rn! f; n such that there exists a function : f; n! f; rn such that for all x f; rn an z f; n such that (f(x); z), we have (x; (z)) < (f(x); z). f is the encoin function, an is the ecoin (errorreuction) function. We will now moify an iea from [SS9] to construct an error-reucin coe. Let B be an unbalance bipartite expaner raph with n noes of eree on one sie an n= noes of eree on the other (we will hereafter call such a raph a (; )-reular raph). We will turn this raph into a circuit by irectin all the ees from the lare sie of the raph to the small sie, lettin the noes on the lare sie be the input ates, an lettin the noes on the small sie be parity ates (see Fiure ). These parity ates are the outputs of the circuit. Fiure : A circuit that encoes an error-reucin coe. We now use this circuit to ene a coe R n; of rate =3 by placin the messae bits at the inputs to the circuit an usin the outputs of the parity ates as the parity bits. The coe we thereby obtain is a horrible error-correctin coe: it has a wor of weiht + (see Fiure 3). However, if the raph is a oo expaner Fiure 3: A low-weiht coewor: only one input bit is, an only those parity ates that rea this bit are ; all others are. raph, then this coe is a oo error-reucin coe. It is obvious that we can encoe this coe in linear time (we nee merely compute the values of the parity ates, each of which has a constant number of inputs). We now will show that it is possible to perform error-reuction on this coe in linear time. The ecoer naturally associates each bit of the wor that it receives with one of the ates in the encoin

4 circuit. We will say that a parity ate in the circuit is satise by a wor if the bit associate with the ate is the parity of the bits associate with its inputs. Otherwise, we will say that it is unsatise (see Fiure ). The (a) Fiure : (a) is a satise parity ate. (b) is unsatise. ecoer will successively ip the bits associate with the input ates in an eort to ecrease the number of unsatise parity ates. Sequential error-reuction alorithm: If there is an input that is an input to more unsatise than satise parity ates, then ip the value of that input. Repeat until no such inputs remain. We note that it is easy to implement this alorithm so that it takes constant time for each iteration. At each iteration, it ecreases the total number of unsatise parity ates, so it can run for at most a linear number of iterations. Lemma. Let R n; be erive from a eree (; ) bipartite raph between a set of n inputs an n= parity ates such that all sets of at most n inputs expan by a factor of at least? 3 +. Assume that the sequential error-reuction alorithm is iven a wor that resembles a coewor of R n; except that at most n= of the inputs have been corrupte an at most n= of the ates have been corrupte. Then, after the termination of the sequential error-reuction alorithm, at most n= of the inputs will be corrupte. Proof: We will let V enote the corrupte inputs, v the size of V, u the number of unsatise parity ates with inputs in V, an s the number of satise parity ates with inputs in V. We will view the pair (u; v) as the state of the alorithm. We will rst show that if n= v n, then there is some input in more unsatise than satise parity ates. The expansion of the raph implies that 3 u + s + v: () Each ate with an input in V accounts for at least one wire leavin V. It is possible that as many as n= of (b) the satise parity ates with inputs in V are satise because they have only one wire from V, but they have been corrupte. The rest must have two wires from V. By countin the v wires leavin V, we obtain v u + s + (s? n=) ) v + n= u + s () Combinin equations () an (), we n s? v + n=; an u + v? n=: (3) When n v n=, we have u > v=, so there must be some input in more unsatise than satise parity ates. To show that the alorithm must terminate with v n=, we show that v must always be less than n. We assume that when the alorithm beins v n= an therefore u n= + n=. As the alorithm procees, u must steaily ecrease. However, if the alorithm is ever in a state (u; v) in which v = n, then equation (3) woul imply that u n= + 3n=, which woul be a contraiction. Thus, the alorithm must always maintain the conition that v < n. This implies that the alorithm cannot terminate unless it is in a state in which v < n=. We have prove that R n; is an error-reucin coe of rate =3 an error-reuction =. It is also possible to perform error-reuction on this coe in parallel. Parallel error-reuction alorithm For each input, count the number of unsatise parity ates to which it is an input. For each input that is an input to more unsatise than satise parity ates, ip the value of that input. It is easy to implement this alorithm as a constantepth circuit of linear size. Lemma. Let R n; be erive from a eree (; ) bipartite raph between a set of n inputs an n= parity ates such that all sets of at most n inputs expan by a factor of at least? 3 +, for any > =. Assume that the parallel error-reuction alorithm is iven a wor that resembles a coewor of R n; except that at most v n= of the inputs have been corrupte, an at most b n= of the ates have been corrupte. Then, after the execution of the parallel error-reuction alorithm, at most v + b (6 + 8) : () +

5 of the inputs will be corrupte. Proof: We will let V enote the set of corrupte inputs, F the set of corrupte inputs that fail to ip, an C the set of inputs that were oriinally clean, but which become corrupte by the parallel error-reuction alorithm. We will let N(V ), the neihbors of V, enote the set of parity ates that contain inputs in V. We will let v = jv j, v = jf j, v = jcj, an v = jn(v )j. We bein by obtainin a boun on in terms of. Every input in F is an input to at least as many satise as unsatise parity ates. At most b of these satise parity ates are satise because they have been corrupte. Thus, at least v? b of the wires leavin F en in a parity ate that contains a wire from another element of V. Thus, the set V can have at most v? v + b neihbors. Because we have set the number of neihbors of V to be v, we obtain v v? v + b v (? )v + b (? ) + b v Next, we will boun in terms of by showin that () <?? b v + : (6) Assume by way of contraiction that this is false, an let C be a subset of C of size exactly?( 3 +)+ b v. Each + input in C is an input to at least = unsatise parity ates. At most b of these ates can be unsatise because their parity ate has been corrupte. The others must be unsatise because they have an input in V. Thus, the number of parity ates containin inputs in the set V [ C is at most v + jc j + b: We will set > =, which will imply that jc [V j < n, so this set must expan by a factor of at least? 3 +. Thus, we obtain 3 + jc [ V j < v + jc j + b; which contraicts our assumption about the size of C. By combinin equations () an (6), we n that the number of corrupte inputs after an execution of the (a) messae bits parity bits R k C k R k+ Goo Coe (b) Fiure : The recursive construction of C k. (a) is the error-reucin coe on the new messae bits. (b) is the error-reucin coe place on top of the error-correctin coe. parallel error-reuction alorithm is at most ( + ) v v?? b v + + (? ) + b = v? + 3? + b +? + b + < v + b (6 + 8) : + We can iterate this alorithm a constant number of times in orer to reuce the number of corrupte input bits.. Error-correctin coes The main iea in our construction of linear-time encoable an ecoable error-correctin coes is to use the linear-time encoable an ecoable error-reucin coes recursively. Imaine what happens when we take a oo error-correctin coe of lenth n an use the wors of this coe as the messae bits of an errorreucin coe (See Fiure, part (b)). We now have an error-correctin coe of reater lenth with a hiher error-tolerance; however, it has the same number of messae bits as the oriinal coe. To increase the number of messae bits in the coe, we will create a new set of more messae bits, an encoe these messae bits in an error-reuction coe. We then use the parity bits of this error-reuction coe as the messae bits of the error-correctin coe we just constructe (See Fiure ).

6 When we apply this construction recursively, we n that not only have we ene the error-correctin coe, but we have create a linear-size circuit that encoes the error-correctin coe. Moreover, the output of each ate of this circuit is a parity bit of the error-correctin coe. This will be true with one small exception: we shoul choose a better base case for our recursion. The performance of our coe will be ictate by the quality of the coe that we choose as our base. Thus, we may want to choose a particularly oo coe of small block lenth. Another constraint is that we on't know that we will be able to n oo expaner raphs of small size. However, there enitely is some constant size after which we will be able to n oo expaner raphs, an which we will choose to be the block lenth of our base coe. We will now present a formal escription of one family of superconcentrator coes. We provie this escription by escribin the encoin circuits for these coes. Description of superconcentrator coes: Choose absolute constants b an. Choose a coe C b of lenth b, rate =, an as lare minimum istance as possible. Let C b be a circuit that takes b bits as input an prouces 3 b bits as output so that these bits taken toether form a coewor of C b. (The b inputs bits are the messae bits of the coe, an the others are the parity bits.) For k > b, let R k be a circuit that encoes an errorreuction coe of rate =3 with k messae bits an k = parity bits, as escribe in Section 3. To form circuit C k from C k?, take a copy of R k an use the inputs of R k as the inputs of C k. A C k? to the circuit by ientifyin the output ates of R k with the inputs of C k?. Finally, attach a copy of R k+ that we will call R k + by ientifyin all the input an output ates of the copy of C k? with the inputs of R. The output ates of k+ C k will be all the input an output ates of C k? alon with the output ates of R k+ (See Fiure ). Let C k be the rate = coe obtaine by takin k messae bits, feein them into C k, an usin the 3 k output bits as the parity bits of the coe. We can ecoe these coes in linear sequential time. Sequential superconcentrator coe ecoin alorithm: If k = b, then ecoe C b usin an arbitrary ecoin alorithm. If k > b, then apply the sequential error-reuction alorithm to the noes in the R k+. Now, recursively ecoe the noes in the copy of C k? usin the sequential ecoin alorithm for C k?. Finish by applyin the sequential error-reuction alorithm to the copy of R k. Theorem 3. If the superconcentrator coe C k is constructe from eree (; ) raphs such that in each raph, every at most fraction of inputs expans by a factor of at least ( 3 + ), an if C b is chosen to be a coe in which any = fraction of errors can be correcte, then the sequential superconcentrator coe ecoin alorithm will correct up to an =8 fraction of errors an will run in linear time. Proof: We assume that there are at most k = errors in the noes of C k. By Lemma, after we apply the sequential error-reuction alorithm R k+, there will be at most k = errors in the noes of the copy of C k?. We can now assume by inuction that the ecoin alorithm for C k? will correct all the errors in its input an output noes. As the input noes of C k? are now the parity bits of the error-reuction coe R k corresponin to the messae oriinally containe at the inputs of the copy of R k, an there are at most k = errors in these messae bits, we can use the sequential error-reuction alorithm to correct all the errors in the input noes of R k (This can be easily observe from the proof of Lemma, or from the analysis of the sequential expaner coe ecoin alorithm of [SS9]). Since the inputs of R k are the inputs of C k, we have remove all the errors from the messae bits of the coe. To see that this alorithm runs in linear time, observe that each error-reuction step runs in time that is linear in the number of bits that it is actin on, an each step acts on half as many bits as the previous step i. It is easy to see that there is a constant that satises the requirements of Theorem 3, but we will not attempt to optimize the constant in this paper. We will note that the main constraint on the constant is in the analysis of the quality of expansion obtaine by a ranomly chosen raph. The best analysis of this expansion of which we are aware appears in the appenix of [SS9]. Remark. We have only constructe coes of lenths k where k is an inteer. It is easy to use similar techniques to construct coes of other lenths. Remark. We remin the reaer that we have only shown that this alorithm corrects some constant fraction of errors. It oes not prouce the coewor closest to an arbitrary wor, an we suspect that an ecient alorithm that oes this woul be icult to construct.

7 We nee to be slihtly trickier to ecoe the superconcentrator coes in parallel loarithmic time. The problem that we must overcome is that if we iterate the parallel error-reuction alorithm enouh times to remove all the errors from the input bits of C i, we will nee to o throuh O(i) iterations. If we i this for each C i, then we woul have an O(lo n) time alorithm. To overcome this problem, we will perform the error-reuctions from the input bits of C i? to the input bits of C i simultaneously for all i. Thus, while the input ates of C i? are bein use to reuce the errors in the input ates of C i, the input ates in C i are bein use to reuce the errors in the input ates of C i+. In orer to show that the reuction of errors of the bits of C i usin the output bits of R i+ works, we will assume = +, for > an > 6. We now wish to observe that if b n=, an n= v n=, then after one roun of the parallel error-reuction alorithm, v will ecrease by a constant multiplicative factor. From equation (), we can see that this constant factor will be boune by the ecrease that occurs when v = n= an b = n=. By pluin these values into equation (), we can see that this constant is less than. We now know that if v n= an b n=, then after a constant number of rouns, we will have v n=. Let c be this constant. To prove the correctness of the error-reuctions on the input noes of the C i 's, we will assume that there is a w n= such that v w an b w=. By substitutin into equation (), we n that after one ecoin roun the number of corrupte inputs is boune by A < w Let =? ? We can now state the parallel superconcentrator coe ecoin alorithm. Parallel superconcentrator coe ecoin alorithm: For i = k? to b: Apply c rouns of the parallel errorreuction alorithm usin the inputs an outputs of C i as the messae bits, an the outputs of R k+ to which they are attache as the parity bits. Decoe the errors in C b usin any ecoin alorithm. For lo = k rouns: Apply the parallel errorreuction alorithm to the copy of R i between the inputs of C i an the inputs noes of C i+, simultaneously for all b i k?. : Theorem 6. If the superconcentrator coe C k is constructe from eree (; ) raphs such that in each raph, every at most fraction of inputs expans by a factor of at least ( ), for some > an > 6, an if C b is chosen to be a coe in which any = fraction of errors can be correcte, then the parallel superconcentrator coe ecoin alorithm will correct up to an =8 fraction of errors in loarithmic time with a linear number of processors. Proof: We bein by assumin that there are at most k = errors in the bits of C k. After we apply c rouns of the parallel error-reuction alorithm to R k+, there will be at most k = errors in the bits of the copy of C k?. Similarly, after we have nishe the i-th stae of the alorithm, there will be at most k = i+ errors in the bits of C k?i. Thus, C b will have fewer than b = errors, so the ecoin alorithm for C b will correct all the errors in C b. We can now move on to the ecoin of the input bits of the C i 's. We have alreay observe that the input bits of C i have at most i = errors an that the input bits of C b are free of error. Thus, after we apply one roun of the error-reuction alorithm simultaneously to all of the R i 's, the input bits of C i will have at most i = errors. Similarly, after we apply the errorreuction alorithm for lo = k rouns, there will be no more errors in any of the input noes of C k.. Explicit Constructions The obstacle to obtainin explicit constructions of the superconcentrator coes escribe in Section is that we are unaware of any explicit constructions of eree expaner raphs that have expansion reater than =, let alone the 3= that is require for these constructions. 3 To et aroun this problem, we will moify our construction so that a lower level of expansion will suce. This means of obtainin explicit constructions is analoous to that which appeare in [SS9]. We will bein by sayin a little about what types of expaner raphs we can construct, an then showin how to prouce explicit constructions of error-reuction coes from them. Denition 7. We will say that a family of raphs G is a family of oo expaner raphs if G contains raphs G n; of n noes an eree so that For an innite number of values of, there exists an innite number of raphs G n; G, an 3 For a proof that Ramanujan raphs have expansion approachin =, see [Kah93].

8 for each of these values, the secon-larest eienvalues of G n; are boune from above by constants such that lim! = =. The expaner raphs constructe by Lubotzky, Phillips an Sarnak [LPS88] an by Marulis [Mar88] are such a family. Pippener [Pip93] points out that one can also obtain such a family by exponentiatin the expaner raphs of Gabber an Galil [GG8]. From a -reular raph G on n vertices, we will erive a (; )-reular raph with n= vertices on one sie, an n vertices on the other. Denition 8. Let G be a raph with ee set E an vertex set V. The ee-vertex incience raph of G is the bipartite raph with vertex set E [ V an ee set f(e; v) E V : v is an enpoint of e : We can now ene the circuits, R(G; C), that will encoe our explicit error-reuction coes. Explicit error-reuction coes: Let G be a -reular raph on n vertices, an let C be a linear error-correctin coe of block-lenth l with messae bits. Let B be the ee-vertex incience raph of G. The circuit R(G; C) has n= input ates an (l? )n parity ates. Each noe on the lare sie of B will correspon to an input of R(G; C). The parity ates are arrane into clusters of size l?, an each cluster is ientie with one of the noes on the small sie of B. The input ates that are neihbors of a cluster will be calle the inputs of the cluster. The parity ates are connecte to the input ates so that for each cluster, if the inputs of that cluster are the messae bits of a coewor of C, then the parity ates in the cluster compute the parity bits of that coewor. Let R(G; C) enote the coe obtaine by usin the inputs of R(G; C) as messae bits an the outputs of the circuit as parity bits. To prove that R(G; C) is a oo error-reuction coe if G is a oo expaner raph, we will make use of the followin result ue to Alon an Chun: Lemma 9 ([AC88]). Let G n; be a -reular raph on n noes with secon-larest eienvalue boune by. Let S be a subset of the vertices of G n; of size n. Then, the number of ees containe in the subraph inuce by S in G n; is at most n + (? ) : To perform error-reuction on R(G; C), we will associate each bit of a receive wor with an input or ate of R(G; C), as we i in Section 3. Parallel explicit error-reuction alorithm: In parallel, for each cluster, if the bits associate with the inputs an ates of a cluster are within of a 6 coewor of C, then sen a \ip" messae to every input that nees to be ippe to obtain that coewor. In parallel, every input that receives the messae \ip", ips its value. We can now prove a lemma for our explicit construction that is analoous to Lemma. Lemma. Let fg n; be a family of oo expaner raphs. There exist constants an such that if the parallel explicit error-reuction alorithm is iven as input a wor that resembles a coewor of R(G n; ; C) except that for some w n at most w v w inputs are corrupte an at most b w parity ates are corrupte, then after the execution of the alorithm, the number of corrupte inputs will ecrease by a constant multiplicative factor, an if at most v w of the inputs are corrupte an at most b w of the parity ates are corrupte, then after the execution of the alorithm, at most w will be corrupte. Proof: Let V be the set of v corrupte inputs. Set an so that v = n an b = n. We will say that a cluster is confuse if it sens a \ip" messae to an input that is not corrupt. In orer for a cluster to be confuse, it must have at least 6 corrupt inputs an ates. Thus, there can be at most n ( + ) 6 confuse clusters. Each of these can sen at most 6 \ip" messaes, so at most n ( + ) 6 = n ( + ) 6 uncorrupte inputs can receive \ip" sinals. We will call a cluster unhelpful if it has a noe of V amon its inputs, but it fails to sen a \ip" sinal to that noe. There can be at most n ( + ) 6

9 unhelpful clusters. By Lemma 9, there can be at most n! inputs both of whose neihbors are unhelpful clusters. The total number of corrupte inputs after the alorithm is run will be at most n Set w = n. We want to show that ! : < ; for [; ] an [ ; ]. Because the function in consieration is ecreasin in, an quaratic in with positive coecients, n it suces o to consier the function at the points ; an =. For =, we nee 6 > + 6 ; an for =, we nee > + : We now see that, iven, we can choose an then so that both of these inequalities hol. For lower values of w, the claim follows from a similar analysis. To prove the secon claim, it suces to observe that the function in question is ecreasin in for >. We will n it convenient to let C be a coe of lenth = an rate =. By the Gilbert-Varshmov boun [MS77], there exists a coe with lenth, rate, an minimum istance where H() = (where H(x) =?x lo x?(?x) lo (?x) enotes the binary entropy function). To construct superconcentrator coes as we i in Section, we will nee error-reuction coes of very specic sizes. To prouce these, we rst observe that explicit constructions of oo expaner raphs meet the followin enition. Denition. Let G = fg i i be a sequence of raphs such that G i has n i vertices, an n i+ > n i. We say that G is ense if n i+? n i = o(n i ). One can observe that by exponentiatin the expaner raphs of Gabber an Galil, one obtains a ense family of expaner raphs. The results of [Hei33] can be use to prove that the expaner raphs constructe by Lubotzky, Phillips an Sarnak [LPS88] an by Marulis [Mar88] are ense. We can use a ense family of -reular oo expaner raphs to construct a family of oo expaner raphs of every suciently lare number of vertices. Let G be a -reular raph on n vertices. Let S be an inepenent subset of the vertices of G such that no two vertices in S have a common neihbor. Sets with this property can be foun in any size up to n=( + ). Consier the raph obtaine by removin the vertices of S from G. Every noe has eree or?, an the inuce subraphs in this raph are all inuce subraphs of G. The fact that the vertices in these raphs have eree either or? has only a neliible impact on the analysis of the error-reuction coes that we erive from them. Thus, we will assume hereafter that we can obtain oo error-reuction coes of any suciently lare size, an that they perform as escribe in Lemma. Explicit superconcentrator coes: Choose a family of oo expaner raphs, G = fg n; such that the subfamily for each contains a ense subsequence. Choose a constant an a coe C of rate =, lenth = an minimum relative istance where H(=). (As escribe in the previous pararaph, we will assume that G contains a raph of every suciently lare number of vertices in which every vertex has eree or?.) Choose a constant b an a coe C b of lenth b, rate =, an as lare minimum istance as possible. Let C b be a circuit that takes b bits as input an prouces 3 b bits as output so that these bits taken toether form a coewor of C b. (The b inputs bits are the messae bits of the coe, an the others are the parity bits.) For k > b, let R k be the circuit R(G k ;; C ) ene earlier in this section. We have chosen the coe C so that this circuit has k input ates an k = parity ates (If the raph isn't -reular, then we miht have to moify the coes at the clusters slihtly to make sure that we have k = parity ates, but this eect is insinicant). To form circuit C k from C k?, take a copy of R k an use the inputs of R k as the inputs of C k. A C k? to the circuit by ientifyin the output ates of R k with the inputs of C k?. Finally, attach a copy of R k+ that we will call R k + by ientifyin all the input an output ates of the copy of C k? with the inputs of R. The output ates of k+ C k will be all the input an output ates of C k? alon with the output ates of R. k+

10 Let C k be the coe obtaine by takin k messae bits, feein them into C k, an usin the 3 k output bits as the parity bits of the coe. Lemma implies that there is a constant c such that if v w an b w, where w k =, then after c executions of the parallel explicit error-reuction alorithm, we will have at most w= corrupte inputs. Similarly, there exits a constant < such that if v w an b w, then after the execution of the errorreuction alorithm there will be at most v corrupte inputs. Parallel explicit error-correction alorithm: Ientical to the alorithm presente in Section, but usin the enitions of C k an R k presente in this section. Theorem. There exist settins of the parameters of the constructions of the rate = superconcentrator coes C k presente in this section, as well as settins of an, such that the parallel explicit superconcentrator coe ecoin alorithm will correct up to an = fraction of errors in loarithmic time with a linear number of processors. Moreover, this alorithm can be simulate in linear sequential time. Proof: [Sketch] The rst claim follows from the iscussion in this section an the proof of Theorem 6. By keepin track of which clusters of parity ates are \unsatise", it is fairly simple to simulate this alorithm in linear time on a sequential machine (assumin that pointer references have unit cost). The iea is similar to that use in Theorem 9 of [SS9]. 6. Open question There are many ways to vary the parameters of our constructions. However, we are not aware of any way of moifyin our construction to prouce coes that approach the Gilbert-Varshmov boun. We woul like to know if it possible to prouce linear-time encoable an ecoable error-correctin coes that can be ecoe up to the Gilbert-Varshmov boun. 7. Acknowlements I woul like to thank Michael Sipser for many helpful conversations. I woul like to thank Brenan Hassett for pointin me to [Hei33]. References [AC88] Noa Alon an F.R.K. Chun. Explicit construction of linear size tolerant networks. Discrete Mathematics, 7:{9, 988. [BZP77] L. A. Bassalyo, V. V. Zyablov, an M. S. Pinsker. Problems of complexity in the theory of correctin coes. Problems of Information Transmission, 3(3):66{7, 977. [Gal63] R. G. Gallaer. Low Density Parity-Check Coes. MIT Press, Cambrie, MA, 963. [GDP73] S. I. Gelfan, R. L. Dobrushin, an M. S. Pinsker. On the complexity of coin. In Secon International Symposium on Information Theory, paes 77{8, Akaemiai Kiao, Buapest, 973. [GG8] O. Gabber an Z. Galil. Explicit constructions of linear-size superconcentrators. Journal of Computer an System Sciences, :7{, 98. [Hei33] Hans Heilbronn. Uber en primzahlsatz von Herrn Hoheisel. Mathematische Zeitschrift, 36:39{3, 933. [Jus76] Jorn Justesen. On the complexity of ecoin Ree- Solomon coes. IEEE-TIT, (?):37{38, March 976. [Kah93] Nabil Kahale. On the secon eienvalue an linear expansion of reular raphs. In Proc. of the 33r FOCS, paes 96{33, 993. [Kuz73] A. V. Kuznetsov. Information storae in a memory assemble from unreliable components. Problems of Information Transmission, 9(3):{6, 973. [LPS88] A. Lubotzky, R. Phillips, an P. Sarnak. Ramanujan raphs. Combinatorica, 8(3):6{77, 988. [Mar88] G. A. Marulis. Explicit roup theoretical constructions of combinatorial schemes an their application to the esin of expaners an concentrators. Problems of Information Transmission, ():39{6, July 988. [MS77] F. J. MacWilliams an N. J. A. Sloane. The Theory of Error-Correctin Coes. North Hollan, Amsteram, 977. [Pip77] Nicholas Pippener. Superconcentrators. SIAM Journal on Computin, 6:98{3, 977. [Pip93] Nicholas Pippener. Self-routin superconcentrators. In Proc. of the th STOC, paes 3{36, 993. [Sar77] Dilip V. Sarwate. On the complexity of ecoin Goppa coes. IEEE-TIT, 3(?):{6, July 977. [Sav7] John E. Savae. The complexity of ecoers{part II: Computational work an ecoin time. IEEE-TIT, 7():77{8, January 97. [SS9] Michael Sipser an Daniel A. Spielman. Expaner coes. In Proc. of the 3th FOCS, paes 66{76, 99. [Val76] L. G. Valiant. Graph-theoretic properties in computational complexity. Journal of Computer an System Sciences, 3:3{3, 976. [Ziv67] Jacob Ziv. Asymptotic performance an complexity of a coin scheme for memoryless channels. IEEE-TIT, 3(3):36{39, July 967. [ZP76] V. V. Zyablov an M. S. Pinsker. Estimation of the error-correction complexity of Gallaer low-ensity coes. Problems of Information Transmission, ():8{8, May 976.