ALGEBRA+NUMBER THEORY +COMBINATORICS COMP 321 McGill University These slides are mainly compiled from the following resources. - Professor Jaehyun Park slides CS 97SI - Top-coder tutorials. - Programming Challenges books.
Outline Algebra. Number Theory Combinatorics
Numerical Bases Binary Base-2 numbers are made up of the digits 0 and 1. They provide the integer representation used within computers, because these digits map naturally to on/off or high/low states. Octal Base-8 numbers are useful as a shorthand to make it easier to read binary numbers, since the bits can be read off from the right in groups of three. Thus 11111001 2 = 371 8 = 249 10. Why do programmers think Christmas is Halloween? Because 31 Oct = 25 Dec! Decimal We use base-10 numbers because we learned to count on our ten fingers.
Numerical Bases Hexadecimal Base-16 numbers are an even easier shorthand to represent binary numbers Alphanumeric Base-36 numbers are the highest you can represent using the 10 numerical digits with the 26 letters of the alphabet.
Real Numbers The most important thing to remember about real numbers is that they are not real real numbers. Floating point arithmetic has limited precision. The fact that there always exists a number c between a and b if a < b. This is not true in real numbers as they are represented in a computer. The associativity of addition guarantees that (a + b ) + c = a + (b + c ). Unfortunately, this is not necessarily true in computer arithmetic because of round-off errors.
Types of Numbers Integers These are the counting numbers,,..., 2, 1, 0, 1, 2,...,. Rational Numbers These are the numbers which can be expressed as the ratio of two integers, i.e. c is rational if c = a/b for integers a and b Irrational Numbers There does not exist any pair of integers x and y such that x/y equals any of these numbers. Examples include π = 3. 1415926..., 2 = 1. 41421..., and e = 2. 71828.... They can be computed using Taylor series expansions, but for all practical purposes it suffices to approximate them using the ten digits or so.
Dealing with Numbers Rounding Rounding is used to get a more accurate value for the least significant digit. Truncating Truncation is exemplified by the floor function, which converts a real number of an integer by chopping off the fractional part.
Fractions Exact rational numbers x/y are best represented by pairs of integers x, y, where x is the numerator and y is the denominator of the fraction. Addition We must find a common denominator before adding fractions Substraction Same as addition, since c d = c + 1 X d
Fractions Multiplication Since multiplication is repeated addition Division To divide fractions you multiply by the reciprocal of the denominator Note: It is important to reduce fractions to their simplest representation, i.e., replace 2/ 4 by 1/ 2 (use GCD).
Manipulating Polynomials Evaluation. Brute Force: computing each term c i x n independently and adding them together. It costs O(n 2 ) multiplications. Improvement 1: Note that x i = x i-1 x, so if we compute the terms from smallest degree to highest degree we can keep track of the current power of x, and get away with two multiplications per term (x i-1 x, and then c i x i ). Improvement 2: Employ Horner s rule.
Manipulating Polynomials Addition/Substraction. Easier than the same operations on long integers, since there is no borrowing or carrying. Simply add or subtract the coefficients of the ith terms for all i from zero to the maximum degree. Multiplication. The product of polynomials P(x) and Q(x) is the sum of the product of every pair of terms, where each term comes from a different polynomial: Such an all-against-all operation is called a convolution
Root Finding Given a polynomial P(x) and a target number t, the problem of root finding is identifying any or all x such that P(x) = t. If P(x) is a first-degree polynomial If P(x) is a second-degree polynomial
Root Finding There are more complicated formulae for solving thirdand fourth-degree polynomials. Beyond quadratic equations, numerical methods are typically used. Newton-Raphson (the basic idea is that of binary search)
Logarithms A logarithm is simply an inverse exponential function. Logarithms are still useful for multiplication A direct consequence of this is that So how can we compute a b for any a and b using the exp(x) and ln(x) functions? So the problem is reduced to one multiplication plus one call of each of these functions
Sum of Powers Pretty useful in many random situations
Fast Exponentiation Recursive computation of a n
Implementation Running time O(log n).
Outline Algebra Number Theory Combinatorics
Number Theory: Prime numbers A natural number starting from 2: {2, 3, 4,...} is considered as a prime if it is only divisible by 1 or itself. The first (and the only even) prime is 2. The next prime numbers are: 3, 5, 7, 11, 13, 17, 19, 23, 29,..., and infinitely many more primes. There are 25 primes in range [0... 100], 168 primes in [0... 1000], 1000 primes in [0... 7919], 1229 primes in [0... 10000], etc...
Number Theory: Prime Testing Function Test by definition. Test if N is divisible by divisor [2...N-1] runs in O(N) Improvement 1 Test if N is divisible by a divisor [2... N] if N is divisible by p, then N = p X q. If q were smaller than p, then q or a prime factor of q would have divided N earlier. This is O( N) Improvement 2 test if N is divisible by divisor [3, 5, 7... N] (only test odd numbers). there is only one even prime number, i.e. number 2, which can be tested separately. This is O( N/2)
Number Theory: Prime Testing Function Improvement 3. Test if N is divisible by prime divisors N This is O( #primes N ) This improvement is already good enough for contest problems. There are 500 odd numbers in [1... (10 6 )], but there are only 168 primes in the same range. This is O( N/ln( N)).
Generating List of Prime Numbers Use the Sieve of Eratosthenes algorithm. First, it sets all numbers in the range to be probably prime but set numbers 0 and 1 to be not prime. Then, it takes 2 as prime and crosses out all multiples of 2 Then it takes the next non-crossed number 3 as a prime and crosses out all multiples of 3. Then it takes 5 and crosses out all multiples of 5. After that, whatever left uncrossed within the range [0...N] are primes. This is roughly O(N log logn) opt sieve for smaller primes and reserve optimized prime testing function for larger primes
Finding Prime Factors A composite numbers N, i.e. the non-primes, can be written uniquely it as a multiplication of its prime factors. N = 240 = 2 X 2 X 2 X 2 X 3 X 5 = 2 4 X 3 X 5 (the latter form is called prime-power factorization). Divide and Conquer Spirit: An integer N can be expressed as: N = PF X N, where PF is a prime factor and N is another number which is N/PF i.e. we can reduce the size of N by taking out its factor PF We can keep doing this until eventually N = 1. Special case if N is actually a prime number. This is O(π( N)) = O( N/ln N).
Greatest Common Divisor (GCD) The largest positive integer d such that d a and d b where x y implies that x divides y. One practical usage of GCD is to simplify fraction, e.g. Used very frequently in number theoretical problems. Some facts: gcd(a, b) = gcd(a, b a) gcd(a, 0) = a
GCD: Euclidian Algorithm Repeated use of gcd(a, b) = gcd(a, b a) Example:
GCD: Euclidian Algorithm-Implementation Running time: O(log(a + b))
LCM: Least Common Multiple. The smallest positive integer l such that a l and b l lcm(a, b) = a X b/gcd(a, b). Implementation: The GCD of more than 2 numbers, e.g. gcd(a, b, c) is equal to gcd(a, gcd(b, c)), etc, and similarly for LCM. Both GCD and LCM algorithms run in O(log 10 n), where n = max(a, b).
Number Theory: Relatively Prime Two integers a and b are said to be relatively prime if gcd(a, b) = 1 e.g. 25 and 42. Problem of finding positive integers below N that are relatively prime to N. Use Euler s Totient (Phi) function. For example:
Number Theory: Solving Diophantine Equation David buys apples and oranges at a total cost of 8.39CAD. If an apple is 25 cents and an orange is 18 cents, how many of each type of fruit does David buys? This problem can be modeled as a linear equation with two variables: 25x + 18y = 839. Since we know that both x and y must be integers, this linear equation is called the Linear Diophantine Equation. We can solve Linear Diophantine Equation with two variables even if we only have one equation!
Number Theory: Solving Diophantine Equation Let a and b be integers with d = gcd(a, b). The equation ax + by = c has no integral solutions if d c is not true. But if d c, then there are infinitely many integral solutions. The first solution (x 0, y 0 ) can be found using the Extended Euclid algorithm, and the rest can be derived from x = x 0 + (b/d)n, y = y 0 (a/d)n, where n is an integer.
Number Theory: Extended Euclid
Number Theory: Solving Diophantine Equation For our problem above: 25x + 18y = 839, we have: a = 25, b = 18, extendedeuclid(25, 18) = (( 5, 7), 1), or 25 X ( 5) + 18 X 7 = 1. Multiplying the left and right hand side of the equation above by 839/gcd(25, 18) = 839, we have: 25 X ( 4195) + 18 X 5873 = 839. Thus, x = 4195 + (18/1)n, y = 5873 (25/1)n. Since we need to have non-negative x and y, we have: 4195 + 18n 0 and 5873 25n 0, or 4195/18 n 5873/25, or 233.05 n 234.92. The only possible integer for n is 234.
Number Theory: Solving Diophantine Equation Thus x = 4195 + 18 X 234 = 17 and y = 5873 25 X 234 = 23, i.e. 17 apples (of 25 cents each) and 23 oranges (of 18 cents each) of a total of 8.39 CAD.
Number Theory: Fibonacci Numbers Leonardo Fibonacci s numbers are defined as fib(0) = 0, fib(1) = 1, and fib(n) = fib(n 1) + fib(n 2) for n 2. This generates the following familiar patterns: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 which can be derived with an O(n) DP technique. (One of their properties) Zeckendorf s theorem: every positive integer can be written in a unique way as a sum of one or more distinct Fibonacci numbers such that the sum does not include any two consecutive Fibonacci numbers.
Number Theory: Fibonacci Numbers Implementation 1 (recursive solution) Exponential time complexity. Implementation 2 (Dynamic Programming) O(n) complexity.
Number Theory: Fibonacci Numbers Implementation 3 (matrix exponentiation) Use fast exponentiation to compute the matrix power