Fountain Codes Amin Shokrollahi EPFL
Content Fountain Codes LT-Codes Raptor Codes Extensions
Transmission Problem Transmit information from one sender to multiple receivers where the channel between sender and each receiver is an erasure channel with unknown erasure probability. Want the transmission to be such that the transmission rate is very close to the capacity on all the transmssion channels, simultaneously. Current protocols do not meet our demands (TCP/IP, UDP,...) Traditional codes are not able to meet our demands. Fountain codes are the right objects!
Fountain Codes D k probability distribution on F k 2. A Fountain code with parameters (k, D k ) is a k-dimensional subpsace of F N 2 in which every coordinate function is sampled independently from D k. The coordinates of the input vector in F k 2 are called the input symbols. The coordinates of the code are called the output symbols. Example: (X 4, X 1 + X 2, X 1 + X 6 + X 93, X 7 + X 512,...) Unlike traditional codes, fountain codes are usually constructed on the fly.
Fountain codes Input Symbols Output Symbols
Encoding Encoding (x 1,..., x k ) F k 2: to compute value of a coordinate, sample from D k to obtain linear from X i1 + + X il, and add x i1 + + x il. Encoding algorithm uses algorithm for sampling from D k. Count number of arithmetic operations over F 2. Cost of encoding: Expected number of arithmetic operations to compute one coordinate. Cost(Encoding) [Cost(D k ) +] E Dk [Hamming weight].
Example: Random Fountain Codes D k is the uniform distribution on F k 2. Encoding: For every coordinate position take a random subset of {1,..., k} and add the input elements according to that subset. Cost of encoding is k/2 ( = expected weight of an element in F k 2 under uniform distribution).
C fountain code of dimension k. Decoding A decoding algorithm of overhead ε is an algorithm that can recover the k input symbols of the code from any set of k(1 + ε) output symbols and errs with probability at most 1/k c for some c > 0. The cost of the decoding algorithm is the expected number of operations for recovering the k input symbols.
Transmission Problem: Solution For sending k symbols to all the receivers use a fountain code of dimension k which has a decoding algorithm of overhead ε, for small ε. On any of the erasure channels the code used is ε away from the capacity of the channel. But do such fountain codes exist?
Example: Random Fountain Codes Theorem. Maximum Likelihood is a decoding algorithm of overhead O(log(k)/k) for random fountain codes. Proof. ML decoding is equivalnet to solving system of equations. M has k columns and k(1 + ε) rows, entries binary i.i.d. random with prob 1/2, then probability that rank of M is not full is at most 2 kε. For ε = O(log(k)/k) probability that rank is not full is at most 1/k c for some constant c.
Efficiency Want to construct fountain codes that have low cost encoders and decoders. Example: Gaussian elimination for uniform D k : Encoding cost is O(k), decoding cost is O(k 3 ). But want: Constant encoding cost. Linear decoding cost.
LT-Codes First class of efficient fountain codes invented by Michael Luby in 1998 and published in 2002. Ω probability distribution on {1,..., k} determined by Ω 1, Ω 2,..., Ω k. Distribution D k : x F k 2: Pr[x] = Ω w, where w = wgt(x). ( w) k Encoding cost: Ω (1). Want this to be small. How small can we get it?
Lower Bound on the Cost of Encoding/Decoding Theorem. C LT-code of dimension k and distribution Ω(x) with decoding algorithm of overhead ε. for some c > 0. Ω (1) c ln(k)/(1 + ε). Proof. Collect output symbols and consider covered input symbols. Decoding successful only if all input symbols are covered. E = k(1 + ε)ω (1). ( 1 1 ) E 1 k k c e E/k 1 k c Ω (1) c ln(k)/(1 + ε)
LT-Codes: Heuristic Analysis Design degree distribution Ω(x). Call output node released at time d if d input symbols recovered and output symbol becomes of degree one when (d + 1)-st input symbol recovered. Pr[Node released at time d weight = w] = ( ) 1 d 1 k 1 k w (w 1) ( ) d w 2 k. E[Nodes released at time d] = ( 1 d 1 ) ( ) d Ω. k k (1 x)ω (x) = 1 Ω(x) = c 1 + c 2 x + x2 2 + x3 6 + + xl l(l 1) +
Put Ω 1 = 1/k, Ω i = 1 i(i 1) Soliton Distribution for 1 i k and normalize. Average degree is H(k) ln(k). (Essentially optimal!) In expectation exactly one output node is released at each round of decoding. But: variance!
LT-Codes: BP Decoding and Robust Soliton Distribution Luby has found a robust Soliton distribution such that: Corresponding codes have average degree O(ln(k)), They have decoding algorithms of overhead O(log 2 (k)/ k) and cost O(k log(k)) (BP decoding). Absolutely remarkable, since this matches the information theoretic lower bound!
Raptor Codes Want linear time encoding and decoding, but how to circumvent lower bound? Idea: Pre-code the input symbols with a traditional code prior to encoding them with an appropriate LT-code. Precode can recover from any (1 δ)- fraction w.h.p. Precode LT-code can recover a (1 δ)-fraction w.h.p. LT-code Raptor codes are fountain codes!
Different types of Raptor Codes LT-codes: Raptor codes with trivial pre-code. Pre-code-only (PCO) codes: Trivial decoding algorithm has overhead ln(1 R)/R which approaches 0 only if R approaches 0.
Asymptotically Optimal Raptor Codes Precode: Appropriate LDPC-code. LT-code: Use degree distribution 1 µ + 1 (µx + x2 1 2 + + x D (D 1) D + xd+1 D where µ ε/2 and D 2/ε to obtain BP decoding algorithm of overhead ε and linear time decoding. )
Finite Length Analysis of Raptor Codes Finite Length analysis of LT-codes gives recursion for state generating function (joint work with Dick Karp and Michael Luby) P u 1 (x, y) = P ( u x(1 pu ) + yp u, 1 u + y ( ( 1 u)) 1 Pu x(1 pu ), 1 u y Here p u is the probability that an output node of residual degree larger than one is released when u input nodes remain unrecovered. The expression for this probability can be found on the next page. ).
p u = 1 1 K u 1 K K u D i 2 Ω i u(u 1) i=1 K u D i 1 Ω i u D K i Ω i K u i, a b := ( ) a b!. b i=1 K i i=1 K i
Finite Length Analysis of Raptor Codes For right-poisson LDPC code with n left and r right nodes and left node degree fractions Λ i the block error probability can be estimated (joint work with Rüdiger Urbanke): A 0 (r, 0) := 1, A 0 (z, o) := 0 for (z, o) (r, 0), A n+1 (z, o) := A n (l, k) l,k d for n 0. ( l )( k l z Λ d )( r l k k+l z o d k 2l+2z+o ( r d) ) Block error probability is upper bounded by ( ) r ( n A s (z, 0) 1 ( r z )) n s Λ d d ( s r. z=0 d d)
Combination: A Practical Raptor Code Precode 1: Hamming code. Precode 2: Left-regular, right-poisson LDPC code with left degree 4. LT-code: Use degree distribution Ω(x) = 0.008x + 0.494x 2 + 0.166x 3 + 0.073x 4 + with average degree 5.86. 0.083x 5 + 0.056x 8 + 0.037x 9 + 0.056x 19 +0.025x 65 + 0.003x 66, Resulting code has block error probability at most 10 14 for k 65536.
Systematic raptor codes. Further Work Raptor codes for error correction on the q-ary symmetric channel, q large (with Dick Karp, Michael Luby, and Avi Wigderson). Raptor codes for general memoryless symmetric channels (with Omid Etesami and Mehdi Molkaraie). Inactivation decoding for raptor codes and its analysis. Raptor codes for data compression (with Guiseppe Caire, Shlomo Shamai, and Sergio Verdu).
What Can We Do With Raptor Codes? Scalable Point-to-Multipoint Data Distribution See accompanying ppt slide.
What Can We Do With Raptor Codes? Point-to-point transmission: Send encoded data via UDP and not TCP. Transmission speed increases 10-100 fold. But: need multirate flow control algorithm. Easy with Fountain Codes! Content Different parts of encoding offered at different rates Clients pull content at their own rate
What Can We Do With Raptor Codes? Multipoint-to-multipoint transmission: Receive encoded data from several Fountain encoders encoding the same object. See accompanying ppt slide
Conclusion and Further Remarks Fountain Codes are a new class of codes designed for robust, unsynchronized, and scalable transmission of information from multiple senders to multiple receivers over computer networks. The theory of Fountain Codes is very interesting, and also provides new insights into the theory of LDPC codes. They can also be used for channels other than the erasure channel. New asynchronous multicast applications can be based on Fountain Codes, such as asynchronous video on demand, or multiple rate congestion control.