3 The Fundamentals:Algorithms, the Integers, and Matrices

Transcription

1 3 The Fundamentals:Algorithms, the Integers, and Matrices 3.1 Algrithms Definition 1 An algorithm is a finite set of instructions for performing a computation or solving a problem. Dijkstra s mini language S ::= x 1,, x n := E 1,, E n S; S skip abort if B 1 SL 1 B n SL n fi do B 1 SL 1 B n SL n od if x > y m := x if x y m := x if x y m := x x y m := y x < y m := y x y m := y fi fi fi 3/28/07 Kwang-Moo Choe 1

2 (m = x m = y) m x m y. Algorithm 1 Finding athe maximal element in a finite sequence procedure max(a 1, a 2,, a n : integers) max := a 1 ; do i n if max a n max = a n ; i := i+1 max a n i := i+1 fi od 3/28/07 Kwang-Moo Choe 2

3 Algorithm 2 The linear search algorithm procedure linear search(x: integer; a 1, a 2,, a n : distinct integers) i := 1; do (i n x a i ) i := i+1 od if i n location := i i=n+1 location = 0 fi i := 1; location := 0; do ((i n) (location = 0)) if x = a i location := i; x a i i := i+1 fi od 3/28/07 Kwang-Moo Choe 3

4 Algorithm 3 The binary search algorithm procedure binary search(x: integer; a 1, a 2,, a n : increasing integers) i, j := 1, n; location := 0; do ((i j) (location = 0)) m := (i+j)/2 ; if x > a m i := m+1 x = a m location := m; x < a m j := m 1 fi od do (i j) m := (i+j)/2 ; if x > a m i := m+1 x = a m location := m; i, j := m, m-1; x < a m j := m 1 3/28/07 Kwang-Moo Choe 4

5 od fi procedure bubble sort(a 1, a 2,, a n : distinct integers with n 2) i := 1; do (i n-1) j := 1; do (j n-i) if a j a j+1 a j, a j+1 := a j+1, a j ; j := j+1 a j a j+1 j := j+1 fi; od; i := i + 1 od 3/28/07 Kwang-Moo Choe 5

6 procedure insertion sort(a 1, a 2,, a n : distinct integers with n 2) j := 2; do (j n) i := 1; do (a j > a i ) i := i+1; m := a j ; k := 0; do (k j - i -1) a j-k := a j-k-1 ; od; a i := m od; i := i + 1 od 3/28/07 Kwang-Moo Choe 6

7 Greedy algorithm Optimization problem to find a solution to the given problem that either minimizes or maximizes the value of somr parameters (global) optimal to select the best choise considering all sequences of steps local optimal(greddy algorithm) to select the best choise at each step Algorithm 6 Greedy Change-Making Algorithm procedure change(n 1, n 2,, n r : number of coins where c 1 > c 2 > > c r is the value for each coin i ; n: positive integer) n 1, n 2,, n r := 0, 0,, 0; i := 1; do (i r) do (n c i ) n i := n i + 1; n := n - c i od; i:= i+1 od 3/28/07 Kwang-Moo Choe 7

8 The Halting Problem Unsolvable problems Halting problem where program will be halt or not(infinite loop) Alan Turing, 1936 No algorithm that computes it! Three cases i) (totally) solvable(computable; recursive) always terminate algorithms ii) partially solvable(partially computable;recursively enumerable) algorithm(program) exits but may halts or loops forever iii) unsolvable(uncomputable; Not recursively enumerable) No algorithm(program) exists 3/28/07 Kwang-Moo Choe 8

9 proof Assume a procedure H(P: program, I: input data) exists where If P runs with I and halts print halt as output. P with I loops forever print loops forever as output. Program also can be a input data(compiler) Condider procedure K(P) as follows procedure K(P: program) if H(P, P) prints halts loops forevar H(P, P) prints loops forever halts fi Now consider K(K) if H(K, K) print loops forever K(K) halts H(K, K) print halts K(K) loops forever Contradiction No algorithm for halting program! 3/28/07 Kwang-Moo Choe 9

10 3.2 The Groth of Functions Big-O Notation Definition 1 Let f: N R or R R. We say f(x) is O(g(x)), if c, k R.. f(x) c g(x) x > k. We also write f(x) = O(g(x)) or even f(x) O(g(x)). The constant c and k are called the witnesses of... f(x) is O(g(x)). Assume c and k are witnesses, c, k.. c < c, x < x, c and k are also witnesses g(x) is faster or equl(not slower) glowing than f(x) when x becomes large enough and ignoring multiplying some constant c. Example 1, 2, 3, 4 3/28/07 Kwang-Moo Choe 10

11 Theorem 1 Let f(x) = a n x n + a n-1 x n a 1 x + a 0. Then f(x) is O(x n ). proof easy with witness k=1 and c = Σ a i. Example 6 n! n! n n. n! is O(n n ) with witness k=1 and c=1. log n! log n n = n log n 1, log n, n, nlog n, n 2, 2 n, n! Definition Let g: N or R R. O(g) = {f: N or R R c, k > 0 x > k: f(x) c g(x) } we may write f O(g). O(1) O(log n) O(n) O(nlog n) O(n 2 ) O(n 3 ) O(2 n ) O(n!) 3/28/07 Kwang-Moo Choe 11

12 Theorem 2, 3 Let f 1 O(g 1 ), f 2 (x) O(g 2 ). Then f 1 +f 2 O(g 1 +g 2 ) = O(max( g 1, g 2 ) and f 1 f 2 O(g 1 g 2 ). Colollary 1 Let f 1, f 2 O(g). Then f 1 +f 2 O(g). Big-Omega and Big-Theta Notation Definition 2 Ω(g) = {f: N or R R c, k > 0 x > k: f(x) c g(x) } Definition 3 Θ(g) = {f: N R c 1, c 2, k > 0, x > k: c 1 g(x) f(x) c 2 g(x) } f Θ(g), iff f O(g) f Ω(g). We say f is order of g, if f Θ(g). Theorem 4 Let f(x) = a n x n + a n-1 x n a 1 x + a 0 with a n 0. Then f(x) Θ(x n ). 3/28/07 Kwang-Moo Choe 12

13 3.3 Complexity of Algorithms Computational complexity time complexity space complexity Worst-case analysis Average-case analysis Θ(1) Θ(log n) Θ(n) Θ(n k ) Θ(k n ) Θ(n!) constant complexity logarithmic complexity linear complexity polinomial complexity exponential complexity factorial complexity 3/28/07 Kwang-Moo Choe 13

14 (totally) solvable tractable polynomial intractable exponential class P polynomial, tractable class NP(P NP) Nondeterministc polynomial exponetial(upper bound) lower bound? tractable or intractable NP-complete problems subset of class NP If any of these problems can be solved in polynomial time, all the problems in class in NP also in class P(P = NP). 3/28/07 Kwang-Moo Choe 14

15 3.4 Integer and Division Definition 1 Let a, b Z a 0. We say a divies b, if c Z.. b = ac. We say a is a factor of b and b is a multiple of a. a b denotes a divides b, and a / b to denote a does not divide b. Theorem 1 a, b, c Z. i) a b a c a (b+c). ii) a b a bc. iii) a b b c a c. Corollary 1 a, b, c Z. a b a c a (mb+nc) m, n Z. 3/28/07 Kwang-Moo Choe 15

16 Theorem 2 The Division Algorithm a Z, d Z + 1 q, 1 r Z: 0 r < d.. a = dq + r. Definition 2 d is called the divisor, a is called dividened, q is called the quotient, and q is called the remainder. q = a div d, r = a mod d. Modular Arithmetic Definition 3 Let a, b Z, m Z +. Then a is congruent to b modulo m, written a b (mod m) or b [a] mod m, if m (a - b). Theorem 3 a b (mod m), iff a mod m = b mod m. or (a - b) mod m = 0. Theorem 4 a b (mod m), iff k Z: a = b + km. Theorem 5 If a b (mod m) c d (mod m), then a + c b + d (mod m) ac bd (mod m). 3/28/07 Kwang-Moo Choe 16

17 3.5 Primes and Greatest Common Divisors Definition 1 p Z + is prime, p > 1 a Z + : (1 < a < p a p). Otherwise composite. Theorem 1 The Fundamental Theorem of Arithmetic Every positive integer has a unique representation as the product of nondecreasing series of zero or more primes. Theorem 2 If n is composite integer, then n has a prime divisor less than or equal to SQRT(n). Theorem 3 There are infinitely many primes. proof proof by contradiction Assume there are only finitelt many primes, p 1, p 2, p n. Let Q = p 1 p 2 p n /28/07 Kwang-Moo Choe 17

18 Q is prime or product of two or more primes. If p j Q, p j Q - p 1 p 2 p n = p j 1. j: 1 < j < n, p j Q. There exist other prime not in the list p 1, p 2, p n or Q is a prime. There are infinitely many primes. Conjectures and Open Problems about Primes Goldbach s conjecture Every even integer n, n > 2, is the sum of two prime numbers The Twin Prime conjecture primes that differs two 16,8699,8733, ,1960 ± 1 are primes with 5,1779 digits. 3/28/07 Kwang-Moo Choe 18

19 Graetest common divisors and Least common multiples Definition 2 Let a, b Z and not both zero. d = gcd(a, b) = max(d: d a d b), d a d b e Z, (e a e b) d e. Definition 3 The integers a and b are relatively prime(coprime) if gcd(a, b) = 1. Definition 4 The integers a 1, a 2, a n are pairwise relatively prime, if gcd(a i, a j ) = 1 whenever 1 i < j n. Definition 5 Let a, b Z and not both zero. d = lcm(a, b) = min(m: a m b m), a m b m n Z, (n a n b) m n. 3/28/07 Kwang-Moo Choe 19

20 Let a = p 1 a1 p 2 a2 p n an and b = p 1 b1 p 2 b2 p n bn. gcd(a, b) = p 1 min(a1, b1) p 2 min(a2, b2) p n min(an, bn). lcm(a, b) = p 1 max(a1, b1) p 2 max(a2, b2) p n max(an, bn). Theorem 5 Let a and b be positive integer. ab = gcd(a, b) lcm(a, b) 3/28/07 Kwang-Moo Choe 20

21 3.6 Integer and Algorithms Theorem 1 Let b Z +, b > 1. Then n Z +, n = a k b k + a k-1 b k a 1 b + a 0. k 0, 0 a 0, a 1,, a k < b, a k 0. unique repesentation base expansion of n written n = (a k a k-1 a 1 a 0 ) b. Lemma 1 Let a = bq + r, a, b, q, r Z. Then gcd(a, b) = gcd(b, r) proof d, d a d b, then d a - bq = r.(corollary 1) d, d b d r, then d bq + r = a. 3/28/07 Kwang-Moo Choe 21

22 Algorithm 6 Euclid algorithm procedure gcd(a, b: positive integer) do b 0 r := a mod b; a := b; b := r od; return a procedure gcd(a, b: positive integer) Dijkstra s algorithm do a > b a := a b a < b b := b a od; return a 3.7 Application of Number Theory 3.8 Matrices 3/28/07 Kwang-Moo Choe 22