Many algorithms do not fall into this class. Example: The travelling salesperson problem (TSP).

Transcription

1 Exponential Complexity Algorithms that have a complexity of O(n p ) (where n is a measure of the problem size) are said to have polynomial complexity. They are considered reasonably efficient, specially if p is small. Many algorithms do not fall into this class. Example: The travelling salesperson problem (TSP). A salesperson has to visit n towns. Each pair of towns is joined by a route of a given length. Find the shortest possible route that visits all the towns and returns to the starting point. 1

2 A naïve approach would be as follows. 1 Find all possible routes 2 Compute their lengths 3 Choose the shortest route However, even step 1 has a complexity bigger than O(n p ) for any p. By complexity here we mean the number of routes between towns. 2

3 Theorem 1: when there are n + 1 towns, the complexity of step 1 is n! Proof: since we are looking for a circuit that includes all the towns, it is immaterial which town we start with, so pick one at random. Then our first journey goes to any one of the n other towns, the second to any one of the n 1 remaining towns, and so on, until finally we visit the last remaining town and then return to our starting point. There are n! possibilities. 3

4 A complexity growth of n! is very high. Theorem 2: as n increases, (n ), n! grows faster than n p for any positive integer p. Idea of Proof: It s important to notice that we fix p and then let n, so we can assume that n p. Then n! = n(n 1)... (n p + 1)(n p) n p (n p)!, so n!/n p (n p)! as n, as required. 4

5 Rigorous Proof: fix a value for p, a positive integer. Then, for n > p, we can write n! = (n p)![(n p + 1)... n] } {{ } p terms (n p)!(n p + 1) p ( n p (n p)! + 1 p = (n p)! = ( 1 p ( ) p n p ) p (n p)! n p. Since (1/p) p is just a constant and (n p)! as n, the result follows. ) p 5

6 In fact, it can be shown that n! increases exponentially for large n, and hence this algorithm for the TSP has exponential complexity. There are clever ways of doing the TSP that are quicker. (See MATLAB demo; click on More Examples.) Many of these algorithms do have a lower complexity than n!, but it is not known whether there is an algorithm for TSP that has polynomial complexity. In fact it is thought to be extremely unlikely that there is; if there is, then there are algorithms of polynomial complexity for a whole host of other difficult problems. 6

7 If you can find a polynomial algorithm for TSP (or prove that one does not exist) then you have solved one of the most important problems in Mathematics (called the P vs NP problem). There is a million-dollar prize available for this; see the unit web-site and for more information. 7

8 Sorting Algorithms Consider the general problem of sorting n numbers into increasing order. Here is a simple algorithm to do this. 8

9 Algorithm: Bubble Sort 1 Input the numbers x(1),..., x(n) (as a vector). 2 Input (or compute) n. 3 for k = 1,..., n 1 for j = 1,... n k if(x(j) > x(j + 1)) swop x(j) and x(j + 1) end if end for j end for k 4 Output the sorted list 9

10 Example: n = 5, x = [27, 13, 9, 5, 3]. On first passage through k-loop k = 1, j = 1, 2, 3, 4. The largest number 27 rises to the end of the list: x becomes [13,9,5,3,27] On second passage through k-loop k = 2, j = 1, 2, 3. The second largest number 13 rises to second last: x becomes [9,5,3,13,27] etc... Program in bubble.m. 10

11 function y = bubble(x) n = length(x); for k = 1:n-1 for j = 1:n-k if(x(j)>x(j+1)) % then we want to swop them oldxj = x(j); % remember old value of x_j x(j) = x(j+1); % new x_j equals old x_j+1 x(j+1) = oldxj;% new x_j+1 equals old x_j end % if end % for j end % for k y = x; 11

12 Analysis of Bubble Sort Script complexity2.m illustrates the complexity of Bubble Sort, but we can also analyse this complexity mathematically. There is no arithmetic at all, but there are (n 1)+(n 2)+(n 3) = n(n 1)/2 comparisons and (at most) the same number of swops. This is an O(n 2 ) algorithm. 12

13 % script complexity2.m % illustrates the complexity of bubble sort times_bubble = []; % we shall store times for various n here; % initialise to an empty vector range = 100:100:600; for n = range % range of n to consider n % output current n to screen x = n:-1:1; % apply bubble to a vector x % given in decreasing order, % the most time-consuming case 13

14 % The next three statements are crucial clock_begin = clock; % start the clock y = bubble(x); % sort with bubble, % calling the function we defined time = etime(clock,clock_begin); % find the time for this n times_bubble = [times_bubble,time]; % and add this newly calculated time % to the list we had before % (using MATLAB dynamic storage capability) % NB case of first passage through loop end % for n plot(range,times_bubble, r- ) title( computation time for bubble sort ) 14