Algebra for error control codes EE 387, Notes 5, Handout #7 EE 387 concentrates on block codes that are linear: Codewords components are linear combinations of message symbols. g 11 g 12 g 1n g 21 g 22 g 2n [m 1,m 2,...,m k ] }{{}..... = [c 1,c 2,...,c n ]. }{{} message m codeword c g k1 g k2 g kn }{{} generator matrix G Error detection begins with the syndrome, also a linear combination of codeword symbols. h 11 h 12 h 1n h 21 h 22 h 2n [r 1,r 2,...,r n ] }{{}..... = [s 1,s 2,...,s n k ]. }{{} senseword r syndrome s h k1 h k2 h kn }{{} parity-check matrix G EE 387, September 30, 2015 Notes 5, Page 1
Algebra for error control codes (cont.) Nonlinear algebra is also needed. Error correction requires finding the zeroes of polynomials whose coefficients are rational functions of the syndrome components. [s 1,s 2,...,s n k ] }{{} syndrome s PGZ or Berlekamp-Massey or Euclidean } {{ } decoding algorithm [Λ 1,Λ 2,...,Λ ν ] }{{} error locator polynomial s All of these steps require that we can add and multiply channel symbols. Decoding also requires division every nonzero symbol needs a reciprocal. Fields are algebraic structures with invertible addition and multiplication. Unlike floating point arithmetic, finite field computations are exact. Fields inherit properties from groups and rings, and field elements are the scalars for vector spaces. So we also define groups, rings, and vector spaces. EE 387, September 30, 2015 Notes 5, Page 2
Number theory and modular arithmetic: motivation Error-control codes use check equations. These equations require that arithmetic operations be defined for codeword symbols. Finite-precision arithmetic is easier to implement than unlimited precision. Finite fields (+,,, ) are defined using modular arithmetic: Integer arithmetic modulo a prime number, (2 31 1): 12345678 87654321 mod 2147483647 = 652912986 Polynomial arithmetic modulo a prime polynomial (lsb first): (x 3 +x+1) (x 2 +1) mod (x 4 +x+1) = 1 1101 1010 = 1000 Note that the product of two 4-bit vectors is also a 4-tuple. Other applications of modular arithmetic: Pseudo-random number generation Public-key cryptography EE 387, September 30, 2015 Notes 5, Page 3
Multiples and divisors Let a, b, m be integers with a b = m. m is product or multiple of a and b a,b are factors or divisors of m Terminology: a divides m. Notation: a m or a\m. Obvious: every nonzero integer m has divisors ±1 and ±m. A proper divisor of m is a divisor a such that 1 < a < m. m proper divisors of m 6 2,3 28 2,4,7,14 256 2,4,8,16,32,64,128 2 16 1 = 66535 3,5,17,257 2 32 +1 = 4294967297 641,6700417 A positive integer p is prime if it has no proper divisors. Note: 6 = 1+2+3 and 28 = 1+2+4+7+14 are perfect numbers. All even perfect numbers are (2 p 1)(2 p 1 ) where p is prime (2 p 1 is a Mersenne prime). Open questions: is there an odd perfect number or are there infinitely many perfect number? EE 387, September 30, 2015 Notes 5, Page 4
Distribution of prime numbers The first few prime numbers are 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31. The only even prime 2 is excluded in many theorems about finite fields. But p = 2 is vital to applications of fields to error-correcting codes. Theorem: (Euclid) There are infinitely many prime numbers. Proof: Suppose there are only finitely many primes, {p 1,p 2,...,p t }. Then m = (p 1 p 2 p t )+1 is not divisible by any p i. So either m is prime or has a prime divisor different from all p i. Prime Number Theorem: Let π(x) be the number of primes less than x. lim x π(x) x/lnx = 1 = π(x) x lnx = p n nlnn. Fact: (Bertrand) For every integer n 2 there is a prime between n and 2n. In particular, there is at least one m-bit prime for every m 1. Similarly, there is at least one prime binary polynomial of degree m 1. (In fact, there are 2 m /m prime polynomials). EE 387, September 30, 2015 Notes 5, Page 5
Division algorithm The division algorithm expresses the dividend n as the sum of a multiple, qd, of the divisor d and a remainder r: n = (ndivd)d+(n mod d) = qd+r, where 0 r < d. Fact: quotient and remainder produced by the division algorithm are unique. The method of this proof will be used repeatedly in this course. Suppose q 1 d+r 1 = q 2 d+r 2, where 0 r 1 r 2 < d. Combine the above equality and inequalities: 0 r 2 r 1 = (q 1 q 2 )d r 2 < d. Thus r 2 r 1 is a nonnegative multiple of d that is less than d. Therefore r 2 r 1 = 0, hence r 2 = r 1, hence q 2 = q 1. Division algorithm 0: repeatedly subtract d from n while incrementing q. More efficient procedure (nonrestoring division): First find the largest m such that 2 m n. Then fo = m,m 1,...,1,0 if n 2 i, subtract 2 i d from n and add 2 i to q. EE 387, September 30, 2015 Notes 5, Page 6
Greatest common divisor The greatest common divisor gcd(m,n) of two integers m and n is the largest integer that divides both m and n. Example: divisors of 12: 1,2,3,4,6,12 divisors of 30: 1,3,5,6,10,15,30 common divisors: 1,3,6 greatest common divisor: gcd(12,30) = 6 Theorem: gcd(m, n) is the smallest positive integer linear combination d = am+bn where a,b are integers. Proof: Obviously every common divisor of m and n is a divisor of d. So we must show that d divides m and n. First m. Use the division algorithm: m = qd+r = q(am+bn)+r, where 0 r < d = am+bn. The remainder s also an integer combination of m and n: r = m q(am+bn) = (1 qa)m+(qb)n < d. Since d is the least positive combination, r must be 0; i.e., That is, d m. In the same way we show that d n. EE 387, September 30, 2015 Notes 5, Page 7
GCD examples We can find gcd by inspection (and factoring) for small cases: gcd(4,12) = 4 = 1 4+0 12 gcd(12,28) = 4 = 2 12+28 gcd(17,37) = 1 = 13 17+6 37 = 24 17 = 408 = 1 mod 37 The Euclidean algorithm is an efficient method for computing both the greatest common divisor and the coefficients a and b. Example: To find a and b such that gcd(17,37) = 17a+37b. a i b i 37 0 1 17 1 0 3 2 2 1 2 5 11 5 1 1 13 6 0 1 2 = 1 + = 2 1 a i = a i 2 a i 1 b i = b i 2 b i 1 To check this result: 17 ( 13)+37 6 = 221+222 = 1. EE 387, September 30, 2015 Notes 5, Page 8
Relatively prime numbers Two different integers m and n are relatively prime or coprime if they have no common proper divisors, i.e., their greatest common divisos 1. If m and n are relatively prime, then there are integers a and b such that 1 = gcd(m,n) = am+bn (Obviously, if a > 0 then b 0, and vice versa.) Therefore bn = 1 am 1 mod m. In other words, b is the multiplicative inverse (reciprocal) of n modulo m. When m is prime, every n such that 0 < n < m is relatively prime to m. Corollary: Integers mod p form a finite field if (and only if) p is prime. Addition, subtraction, and multiplication mod p have associative and commutative properties. And division works for every nonzero divisor. EE 387, September 30, 2015 Notes 5, Page 9
Useful property of coprime numbers Lemma: If d = gcd(r,s) and m 0, then gcd(mr,ms) = md. Proof: Obviously, md is a common divisor of mr and ms. Conversely, d = gcd(r,s) = ar +bs = md = m(ar +bs) = a(mr)+b(ms). This shows that every common divisor of mr and ms is a divisor of md. Theorem: If m rs and gcd(m,r) = 1, then m s. Proof: Trivially true if s = 0. If s > 0 then by the previous lemma gcd(ms,rs) = s gcd(m,r) = s 1 = s = s = a(ms)+b(rs) is the sum of two multiples of m. Thus s is a multiple of m, that is, m s. Important special case of the previous result: Lemma: If p is prime and p ab, then p a or p b (or both). Proof: Since p is prime, p a or gcd(a,p) = 1. If gcd(a,p) = 1 then p b. EE 387, September 30, 2015 Notes 5, Page 10
Fundamental Theorem of Arithmetic Fundamental Theorem of Arithmetic: Every integer 2 has a unique factorization into primes, apart from the order of the factors. Proof: First show that every integer m 2 can be factored into primes. We use complete mathematical induction. If m is prime, its factorization is simply m = m. Otherwise let m = ab with a < m and b < m. By induction, a and b have prime factorizations a = p 1 p r and b = q 1 q s. So m = ab = p 1 p r q 1 q s is a prime factorization of m. Uniqueness: suppose there is an integer with two different factorizations. Divide out primes common to the representations to obtain p 1 p 2 p r = q 1 q 2 q s where p j and q j are primes and no p i equals any q j. But p 1 q 1 q 2 q s implies that p 1 must be a divisor of some q j. This is a contradiction. EE 387, September 30, 2015 Notes 5, Page 11
Euclidean algorithm We can find gcd(r, s) by reducing to a smaller problem: gcd(s qr, r). Every common divisor of r,s is a common divisor of r,s qr and vice versa. The Euclidean algorithm generates sequence of remainders r 1 > r 2 > > r n > 0 where the final remaindes the greatest common denominator, r n = gcd(r,s). r 1 = s = Q 1 r 0 +r 1 0 < r 1 < r r 0 = r = Q 2 r 1 +r 2 0 < r 2 < r 1 r 1 = Q 3 r 2 +r 3 0 < r 3 < r 2. 2 = 1 +. r n 2 = Q n r n 1 +r n r n 1 = Q n+1 r n 0 < < 1 0 < r n < r n 1 r n r n 1 This procedure halts after a finite number of steps because each remainder is a positive number smaller than the preceding remainder. EE 387, September 30, 2015 Notes 5, Page 12
Euclidean algorithm: integer examples The third tableau shows that division can be sloppy; quotients are powers of 2 More steps are needed, but the steps are simpler. 37 17 3 2 2 5 1 1 0 2 109 32 13 3 6 2 1 2 0 6 109 32 45 2 32 0 13 1 6 2 1 2 0 6 34 21 13 1 8 1 5 1 3 1 2 1 1 1 0 2 The fourth example shows that the worst case running time. This occurs when inputs are consecutive Fibonacci numbers, 1,2,3,5,8,13,21,34,... F n = F n 1 +F n 2 ; initial conditions F 0 = 0,F 1 = 1. EE 387, September 30, 2015 Notes 5, Page 13
Euclidean algorithm: worst case Fibonacci numbers are the worst case for the Euclidean algorithm. 34 21 13 1 8 1 5 1 3 1 2 1 1 1 35 21 14 1 7 1 0 2 34 22 12 1 10 1 2 1 0 5 55 21 13 2 8 1 5 1 3 1 2 1 1 1 89 21 5 4 1 4 0 5 0 2 0 2 Nonconsecutive Fibonacci numbers are easier. See the last two tableaux. Fact: gcd(f i,f j ) = F gcd(i,j). EE 387, September 30, 2015 Notes 5, Page 14
Extended Euclidean algorithm Every remainder is an integer combination of r and s: = a i r +b i s This is obvious fo = 1 and r 1 i = 0 r 0. + 1 s r 0 = 1 r + 0 s i = 1,0,1,...,n The other coefficients a i and b i can be computed iteratively: = 1 + 2 = (a i 1 r +b i 1 s) + (a i 2 r +b i 2 s) = ( a i 1 +a i 2 )r + ( b i 1 +b i 2 )s = a i r + b i s The sequences {a i } and {b i } satisfy same linear recurrence that defines { }: a i = a i 1 + a i 2 and b i = b i 1 + b i 2 ri 2 = is integer part of quotient of two previous remainders. 1 EE 387, September 30, 2015 Notes 5, Page 15
Extended Euclidean algorithm: reciprocals Find reciprocal of 17 in GF(37). a i b i 37 0 1 17 1 0 3 2 2 1 2 5 11 5 1 1 13 6 Answer: 17 1 = 13 = 24. Check: 17 24 = 408 = 11 37+1. Find reciprocal of x 3 +x 2 mod x 4 +x+1 over GF(2). Answer: x 3 +x. (x) (x) a i (x) x 4 +x+1 0 x 3 +x 2 1 x 2 +x+1 x+1 x+1 x x x 2 +x+1 1 x+1 x 3 +x (x) (x) a i (x) 10011 0 1100 1 111 11 11 10 10 111 1 11 1010 0 10 EE 387, September 30, 2015 Notes 5, Page 16