CSE 101 Algorithm Design and Analysis Miles Jones mej016@eng.ucsd.edu Office 4208 CSE Building Lecture 1: Introduction
LOGISTICS Book: Algorithms by Dasgupta, Papadimitriou and Vazirani Homework: Due Wednesdays on gradescope with groups of size 1-4. Your lowest Homework grade will be dropped. Website: cse101wi17ucsd.wordpress.com There will be three exams. (week 4, week 6, week 9) 40% Final, 10% each exam, 30% homework 50% Final, 10% for your two best exams, 30% homework
LOGISTICS In discussion, you will discuss. The TA will reinforce what was taught in class that week. The class website is cse101w17ucsd.wordpress.com Here you can find homework assignments, lecture slides, office hours, class schedule, and syllabus. Office hours: I will hold office hours in room 4208 CSE building on Wednesday 2:30-3:30 and Monday/Wednesday 10:30-11:30. I understand that not everybody can make those times so I will take appointments also, just give me at least a day in advance.
What do we assume you know? Material covered in CSE 20, 21 You can describe algorithms and their correctness using precise mathematical terminology and techniques. For example: Basic math notation (logic, sets, functions, ) Proofs (contradiction, induction.) Asymptotic notation (big-o ) pseudocode basic counting strategies graph terminology
WHAT ARE SOME EXAMPLES OF ALGORITHMS?
WHAT ARE SOME EXAMPLES OF ALGORITHMS? Graph algorithms Divide and Conquer Greedy Algorithms Dynamic Programming Other topics Linear Programming Iterative Improvement P vs NP
QUESTIONS ABOUT ALGORITHMS Does the algorithm work? Does the algorithm terminate? How many operations does the algorithm take?
MUHAMMAD IBN MUSA AL-KHWARIZMI 9 th Century, Bagdad Al wrote The Compendious Book on Calculation, By completion and Balancing,
THE COMPENDIOUS BOOK ON CALCULATION, BY COMPLETION AND BALANCING Compilation of known rules for: Adding Subtracting Dividing Multiplying Solving quadratic and linear equations Algebra is named after Al-Jabr, one of his operations
SPREAD OF IDEAS Al Khwarizmi Algoritmi
ROMAN NUMERALS Exercise: MCDXLVIII + DCCCXII
LEONARDO BONACCI In 1202 Bonacci wrote Liber Abaci (book of calculation) Latin translation of Al s methods Introduction of the decimal system into the Western World
FIBONACCI SEQUENCE Today, Leonardo Bonacci AKA Fibonacci is mostly known for a sequence of integers 1,1,2,3,5,8,13,21,34,55 This sequence came about from modeling the population growth of bunnies with ideal conditions
POPULATION OF BUNNIES
FIBONACCI SEQUENCE DEFINITION F(0) = 0 F(1) = 1 F(n) = F(n-1) + F(n-2)
FIBONACCI SEQUENCE ALGORITHM 1 function fib1(n) if n = 0 then return 0 if n = 1 then return 1 return fib1(n-1)+fib1(n-2) Does it work? Does it terminate? How long does it take? Can we do better?
FIBONACCI SEQUENCE ALGORITHM 1 function fib1(n) if n = 0 then return 0 if n = 1 then return 1 return fib1(n-1)+fib1(n-2) Let T(n) be the number of computer steps it takes to calculate fib1(n) If n < 2 then T(n) < 3 If n > 2 then T(n) > T(n - 1) + T(n - 2) So we have that T(n) > F(n). Fibonacci numbers grow FAST!!!!!!! F(n)~1.6^n
COMPUTER SPEED The fastest computers can do about 2^45 calculations in one second. That s 3,518,372,088,832. And 2^45~1.6^66 So, it would take the fastest computer to calculate F(66) in one second.
COMPUTER SPEED
WHY DOES IT TAKE SO LONG? F(5) F(4) F(3) F(3) F(2) F(2) F(1) F(2) F(1) F(1) F(1) F(0) F(0) F(1) F(0)
A BETTER APPROACH? function fib2(n) if n = 0 then return 0 create array f[0 n] f[0] := 0 f[1] := 1 for i = 2 n: f[i]:=f[i 1] + f[i 2] return f[n] The for loop consists of a single computer step so in order to compute f[n], you need n 1 + 2 computer steps!!!!! This is a huge improvement
EXPONENTIAL VS POLYNOMIAL Polynomial Exponential Θ(1) Θ(log(n)) Θ(n) Θ(nlog(n)) Θ(n^2) Θ(n^k) Θ(2^n) Θ(n!)
BASIC COMPUTER STEPS Primitives: Branching Storing Comparing (small numbers) Simple addition (small numbers) Array look up
ADDITION AS A COMPUTER STEP Adding small numbers together should be one computer step. Adding n bit numbers in general takes Θ(n) steps if n is large (like 1000 digits long).
RUNTIME OF FIB1 AND FIB2 The nth Fibonacci number is F(n)<2^n so it has less than n bits and the procedure fib1 takes O(2^n) additions and each addition takes O(n) time so the runtime of fib1 is O(n2^n). the procedure fib2 takes O(n) additions and each addition takes O(n) time so the runtime of fib2 is O(n*n).
AMBIGUITY IN COMPUTER STEPS. We say that the time it takes to execute fib2 is proportional to n^2. What does that mean?? 2n^2? 3n^2? 100n^2? May depend on the computer.
BIG-O BIG-Θ BIG-Ω f(n) = O(g(n)) means that There exists a constant c such that f(n) cg(n) for all n large enough. f(n) = Ω(g(n)) means that g(n) = O(f(n)) f(n) = Θ(g(n)) means that f(n) = O(g(n)) and g(n) = O(f(n))
THOUGHT EXPERIMENT Suppose there was an alien species that was way more advanced than us. They had computers that could run 1 billion times faster than our computers. We were racing to see who could figure out how the planets in our solar system interacted. Our algorithm takes O(n 2 ) operations and their algorithm Takes O(2 n ). More specifically, our algorithm takes n 2 computer steps and theirs takes 2 n computer steps.
THOUGHT EXPERIMENT Let s say our computers can do 1 computer step in 1 nanosecond. That means the aliens can do 1 million computer steps in 1 nanosecond or 1 in 10 9 nanoseconds. That means that to figure out the interaction within our solar system, Our computer would take 9 2 1 = 81 nanoseconds and theirs would take 2 9 10 9 = 5.12 10 7 nanoseconds. There is a point where our computers would run faster than the aliens. How many planets must there be for this to happen?
QUICK RULES FOR BIG-O Any polynomial is Big-O of its highest power Exponentials dominate polynomials. If f/g goes to 0 then f=o(g) If f/g goes to infinity then f=ω(g) If f/g goes to c>0 then f=θ(g), f=o(g) and f=ω(g)