Input Decidable Language -- Program Halts on all Input Encoding of Input -- Natural Numbers Encoded in Binary or Decimal, Not Unary
|
|
- Gwendoline Nelson
- 5 years ago
- Views:
Transcription
1 Complexity Analysis
2 Complexity Theory Input Decidable Language -- Program Halts on all Input Encoding of Input -- Natural Numbers Encoded in Binary or Decimal, Not Unary Output TRUE or FALSE Time and Space Problems How much Time does the Algorithm Take to Execute? How much Memory is Required to Execute the Algorithm? As Functions of the Size of the Input
3 Examples Hamiltonian Circuit -- Decision Problem Input = Matrix Describing a Graph Output = YES if Graph has a Hamiltonian Circuit = NO if Graph does not have a Hamiltonian Circuit Traveling Salesman -- Optimization Problem Input = Matrix Describing the Cost of Each Edge Max Cost = Maximal Allowable Cost of Circuit Output = YES if Graph has Circuit with Cost Max Cost = NO if Graph does not have a Circuit with Cost Max Cost
4 Complexity and Turing Machines One Tape Deterministic Turing Machines Time Complexity = Number of Commands Executed Space Complexity = Number of Tape Squares Visited Multiple Tape Turing Machines and Actual Digital Computers Complexity Differs from Single Tape Turing Machines by Only a Polynomial Factor Non-Deterministic Turing Machines Require Different Complexity Analysis from Deterministic Turing Machines
5 Complexity of Turing Machines Setup M = Turing Machine that Halts on All Input n = Size of Input Time and Space Complexity Worst Case Analysis t M : N N -- t M (n) = Maximal Time M Takes to Execute on a Problem of Size n s M : N N -- s M (n) = Maximal Memory Required by M to Execute a Problem of Size n For Non-Deterministic Machines, Consider All Possible Paths
6 Asymptotic Complexity Classes Big O -- Upper Bound Big Ω -- Lower Bound Small o -- Much Bigger Bound Small ω -- Much Smaller Bound Big Theta Θ -- Tight Bound
7 Big O and Big Ω Big O -- Upper Bound f = O(g) f O(g) f is bounded above by g O(g) = {h h(n) C h g(n) for all n k h for some constant C h } Big Ω -- Lower Bound f = Ω(g) f Ω(g) f is bounded below by g Ω(g) = {h h(n) C h g(n) for all n k h for some constant C h } Observations k and C are not unique g is not unique
8 Big O and Big Ω and Limits Limits lim n f (n) g(n) < f O(g) lim n f (n) g(n) > 0 f Ω(g)
9 Small o and Small ω Definitions f is o(g) lim n f (n) g(n) = 0 (much smaller) f is ω (g) lim n f (n) g(n) = (much bigger) Observations f is o(g) f is O(g) (much smaller smaller) f is ω (g) f is Ω(g) (much bigger bigger)
10 Big-Θ -- Tight Bound Definition f = Θ(g) f Θ(g) f is bounded above and below by a constant times g Θ(g) = {h Dg(h) h(n) C h g(n) for all n k h for some constants C h, D h } Meaning f Θ(g) f Ω(g) and f O(g) Observations k, c, C are not unique g is not unique Limits 0 < lim n f (n) g(n) < f Θ(g)
11 Most Important Big O is the most important complexity class, since we are most interested in an upper bound on the time of an algorithm.
12 Big-O Definition f is O(g) means there are numbers k and C such that -- f (n) Cg(n) for all n > k. Meaning For all sufficiently large integers, f (n) is less than a constant multiple of g(n). g is only an Upper Bound on the size of f. Observations k and C are not unique. g is not unique. g is usually chosen to be well-known function: log(n), n p, 2 n.
13 Examples 1. c p n p + + c 1 n + c 0 = O(n p ) 2. c p n p + + c 1 n + c 0 d q n q + + d 1 n + d 0 = O(n p q ) n 3. k p = O(n p+1 ) k =1 n 4. 2 p = O(2 n ) p=1 5. n! = O(n n )
14 Most Common Orders 1, log(n), n, nlog(n), n 2, n p, 2 p, n!, n n Constant, Logarithmic, Linear, Polynomial, Exponential
15 Properties of Big O 0. f = O( f ) 1. f = O(g) and g = O(h) f = O(h) 2. f = O(g) f + c = O(g) 3. f = O(g) c f = O(g) 4. f 1 = O(g 1 ) and f 2 = O(g 2 ) f 1 + f 2 = O ( max(g 1, g 2 )) 5. f 1 = O(g 1 ) and f 2 = O(g 2 ) f 1 f 2 = O(g 1 g 2 )
16 Properties of Big O (continued) FALSE Property 2. f = O(g) f + c = O(g) Example 1 n 2 = O 1 n 1 n 2 +1 O 1 n True Property 2*. f = O(g) and 1 = O( f ) f + c = O(g)
17 Properties of Big O for Special Functions 6. p q n p = O(n q ) 7. c p n p + + c 1 n + c 0 = O(n p ) 8, a < b a n = O(b n ) 9. a,b > 1 O ( Log a (n)) = O ( Log b (n)) -- Log a (n) = Log a (b)log b (n) 10. O(n p ) O ( n p Log b (n)) O(n p +1 ) n = O(n!) 12. n! = O(n n )
18 Examples Find Simple O(g) estimates for the following functions: 1. (n 2 + 8)(n +1) 2. (nlog(n) + n 2 )(n 3 + 3) 3. ( n!+ 2 n )( n 3 + log(n 2 +1) )
19 True or False Prove or give a counterexample For all positive functions f and g, either f = O(g) or g = O( f ).
20 True or False Prove or give a counterexample For all positive functions f and g, either f = O(g) or g = O( f ). Counterexample f (n) = 1+ ncos 2 nπ 2 g(n) = 1 + nsin 2 n π 2
21 Examples of o( f ) lim n ln(n) n = 0 ln(n) is o(n) ( ln(n) ) p lim n n = 0 ( ln(n) ) p is o(n) lim n n p ln(n) n p +1 = 0 n p ln(n) is o(n p+1 ) lim n n p 2 n = 0 n p is o(2 n ) lim n P(n) 2 n = 0 P(n) is o(2 n ), P(n) Polynomial
22 Examples of Θ( f ) 1. c nn p + + c 1 n + c 0 = Θ(n p ) 2. c p n p + + c 1 n + c 0 d q n q + + d 1 n + d 0 = Θ(n p q ) n 3. k p = Θ(n p +1 ) (Hint: Integrate) k =1 n 4. 2 p = Θ(2 n ) p=1
23 Recursion vs. Dynamic Programming Polynomial Interpolation Neville s Algorithm (Time and Space) O(n 2 ) vs. O(2 n ) Fibonacci Numbers Memoization O(n) vs. O(2 n )
24 Neville s Algorithm (Polynomial Interpolation) -- Recursive Implementation I 1,,n (x) I 1,,n 1 (x) I 2,,n (x) I 1,2 (x) I 1,,n 2 (x) I 2,,n 1 (x) I 2,,n 1(x) I 3,,n 1(x) I n 1, n (x) y 1 y 2 y n 1 y n n 1 Levels 2 n 2 = O(2 n ) multiplications
25 Neville s Algorithm (Polynomial Interpolation) -- Iterative Implementation I 1, 2, 3, 4 I 1, 2, 3 I 2, 3, 4 I 1,2 I 2, 3 I 3, 4 y 1 y 2 y 3 y 4 n 1 n 1 Levels 2 k = (n 1)n = O(n 2 ) multiplications k =1
26 Fibonacci Recurrence f n f n 1 f n 2 f 3 f n 2 f n 3 f n 3 f n 4 f 3 f 1 f 2 f 1 f 2 f n = f n 1 + f n 2 T ( f n ) = T ( f n 1 ) +T ( f n 2 )
27 Fibonacci Numbers -- Memoization f 1 f 3 f 5 f 2n 3 f 2n 1 f 2 f 4 f 6 f 2n 2 f 2n T ( f 2n ) = 2n 2 = O(n) Additions
28 Polynomial Time Algorithms Polynomial Evaluation Horner s Method (Time) O(n) vs. O(n 2 ) Polynomial Multiplication Fast Fourier Transform (FFT) O ( n log(n) ) vs. O(n 2 ) Matrix Multiplication Strassen s Algorithm O n 2.8 ( ) vs. O(n 3 )
29 Horner s Method Example 7x 3 3x 2 +11x multiplies ((7x 3)x +11)x multiplies General Case a nx n + a n 1 x n 1 n + + a 1 x + a 0 -- k = k=0 n(n +1) 2 = O(n 2 ) multiplies ((a n x + a n 1 )x + + a 1 )x + a 0 -- n = O(n) multiplies
30 Searching Algorithms Linear Search Worst Case Analysis -- O(n) Average Case Analysis -- O(n) Binary Search -- O ( Log(n) ) Root Finding (Mathematica Code) Ray Tracing
31
32
33
34
35 Example Integer Division Input: m, n = integers Output: q, r = integers (quotient and remainder) -- n = mq + r with 0 r < m Algorithm q = 0 r = n While r m q = q +1 Loop Invariant n = mq + r r = r m
36 Proofs Proof of Loop Invariance By induction on the number of iteration of the loop. Base Case: q = 0 and r = n n = mq + r. Induction: Suppose that n = mq + r after k iterations of the loop. Proof of Program Correctness Must show that n = mq + r after k +1 iterations of the loop. But after k +1 iterations of the loop: (mq + r) k +1 = mq k +1 + r k+1 = m (q k +1) + r k m = m q k + r k (inductive hypothesis) = n. 1. The loop will terminate with r < m. 2. When the loop terminates n = m q + r. (Loop Invariance)
37 Complexity of Integer Division Worst Case: m = 2, n odd O(n / 2)
38 Example GCD -- Greatest Common Divisor Input: m, n = integers Output: GCD(m,n) Euclidean Algorithm x = m y = n While y 0 Loop Invariant r = remainder of x divided by y GCD(x, y) = GCD(m, n) (use an efficient division algorithm) x = y y = r Output: x is GCD(m, n)
39 Proof of Loop Invariance By induction on the number of iteration of the loop. Base Case: x = m and y = n GCD(x, y) = GCD(m, n). Induction: Suppose that GCD(x, y) = GCD(m, n) after k iterations of the loop. Must show that GCD(x, y) = GCD(m, n) after k +1 iterations. After the first line of the (k +1) st iteration of the loop: x = yq + r r = x yq GCD(y, r) = GCD(x, y) and by the inductive hypothesis GCD(x, y) = GCD(m, n) so GCD(y, r) = GCD(m, n). But after the (k +1) st iteration of the loop: x = y and y = r so GCD(x, y) = GCD(y, r) = GCD(m, n).
40 Complexity GCD Algorithm Standard GCD Algorithm Factor m and n into Prime Factors Find Common Prime Factors No Efficient Algorithm for Step 1 Euclid s GCD Algorithm m n / 2: (n, m) (m,r) r < n / 2 m < n / 2: (n, m) (m,r) m < n / 2 -- Second Parameter Decreases by at Least 1/ 2 in at Most 2 Step -- O ( Log 2 (n))
41 Searching Algorithms Depth First Less Space -- 1 Path at a Time More Time -- May Explore Bad Path Breadth First More Space -- Stores all Paths Simultaneously Less Time -- May Find Good Path and Halt Iterative Deepening All Paths to Depth 1, All Paths to Depth 2, Space Complexity = Space Complexity of Depth First Search Time Complexity Breadth First Search Compromise Between Depth First and Breadth First Search
Recursive Definitions. Recursive Definition A definition is called recursive if the object is defined in terms of itself.
Recursion Recursive Definitions Recursive Definition A definition is called recursive if the object is defined in terms of itself. Base Case Recursive definitions require a base case at which to either
More informationRecursive Definitions. Recursive Definition A definition is called recursive if the object is defined in terms of itself.
Recursion Recursive Definitions Recursive Definition A definition is called recursive if the object is defined in terms of itself. Base Case Recursive definitions require a base case at which to either
More informationTime Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1)
Time Complexity (1) CSCI 2670 Original Slides were written by Dr. Frederick W Maier Spring 2014 Time Complexity So far we ve dealt with determining whether or not a problem is decidable. But even if it
More informationModule 1: Analyzing the Efficiency of Algorithms
Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Associate Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu Based
More informationGrowth of Functions (CLRS 2.3,3)
Growth of Functions (CLRS 2.3,3) 1 Review Last time we discussed running time of algorithms and introduced the RAM model of computation. Best-case running time: the shortest running time for any input
More informationHarvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time
Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time Harry Lewis November 12, 20123 Review of Asymptotic Notation For f, g : N R + f = O(g): c > 0 s.t. f(n) c g(n) for all sufficiently large n. f
More informationBig , and Definition Definition
Big O, Ω, and Θ Big-O gives us only a one-way comparison; if f is O(g) then g eventually is bigger than f from that point on, but in fact f could be very small in comparison. Example; 3n is O(2 2n ). We
More informationLecture 2. Fundamentals of the Analysis of Algorithm Efficiency
Lecture 2 Fundamentals of the Analysis of Algorithm Efficiency 1 Lecture Contents 1. Analysis Framework 2. Asymptotic Notations and Basic Efficiency Classes 3. Mathematical Analysis of Nonrecursive Algorithms
More informationAnalysis of Algorithm Efficiency. Dr. Yingwu Zhu
Analysis of Algorithm Efficiency Dr. Yingwu Zhu Measure Algorithm Efficiency Time efficiency How fast the algorithm runs; amount of time required to accomplish the task Our focus! Space efficiency Amount
More informationMid-term Exam Answers and Final Exam Study Guide CIS 675 Summer 2010
Mid-term Exam Answers and Final Exam Study Guide CIS 675 Summer 2010 Midterm Problem 1: Recall that for two functions g : N N + and h : N N +, h = Θ(g) iff for some positive integer N and positive real
More informationAnalysis of Algorithms
October 1, 2015 Analysis of Algorithms CS 141, Fall 2015 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization
More informationModule 1: Analyzing the Efficiency of Algorithms
Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?
More informationCSC Design and Analysis of Algorithms. Lecture 1
CSC 8301- Design and Analysis of Algorithms Lecture 1 Introduction Analysis framework and asymptotic notations What is an algorithm? An algorithm is a finite sequence of unambiguous instructions for solving
More informationName CMSC203 Fall2008 Exam 2 Solution Key Show All Work!!! Page (16 points) Circle T if the corresponding statement is True or F if it is False.
Name CMSC203 Fall2008 Exam 2 Solution Key Show All Work!!! Page ( points) Circle T if the corresponding statement is True or F if it is False T F GCD(,0) = 0 T F For every recursive algorithm, there is
More informationBig O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013
/4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop
More informationTaking Stock. IE170: Algorithms in Systems Engineering: Lecture 3. Θ Notation. Comparing Algorithms
Taking Stock IE170: Algorithms in Systems Engineering: Lecture 3 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University January 19, 2007 Last Time Lots of funky math Playing
More informationAnalysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College
Analysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College Why analysis? We want to predict how the algorithm will behave (e.g. running time) on arbitrary inputs, and how it will
More information2301 Assignment 1 Due Friday 19th March, 2 pm
Show all your work. Justify your solutions. Answers without justification will not receive full marks. Only hand in the problems on page 2. Practice Problems Question 1. Prove that if a b and a 3c then
More informationCS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationComputational Complexity - Pseudocode and Recursions
Computational Complexity - Pseudocode and Recursions Nicholas Mainardi 1 Dipartimento di Elettronica e Informazione Politecnico di Milano nicholas.mainardi@polimi.it June 6, 2018 1 Partly Based on Alessandro
More informationRemainders. We learned how to multiply and divide in elementary
Remainders We learned how to multiply and divide in elementary school. As adults we perform division mostly by pressing the key on a calculator. This key supplies the quotient. In numerical analysis and
More informationCpt S 223. School of EECS, WSU
Algorithm Analysis 1 Purpose Why bother analyzing code; isn t getting it to work enough? Estimate time and memory in the average case and worst case Identify bottlenecks, i.e., where to reduce time Compare
More informationChapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens
Chapter 2: The Basics slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens Problem instances vs. decision problems vs. search problems Decision
More information3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need?
3. Algorithms We will study algorithms to solve many different types of problems such as finding the largest of a sequence of numbers sorting a sequence of numbers finding the shortest path between two
More informationCOMP 382: Reasoning about algorithms
Fall 2014 Unit 4: Basics of complexity analysis Correctness and efficiency So far, we have talked about correctness and termination of algorithms What about efficiency? Running time of an algorithm For
More informationOrder Notation and the Mathematics for Analysis of Algorithms
Elementary Data Structures and Algorithms Order Notation and the Mathematics for Analysis of Algorithms Name: Email: Code: 19704 Always choose the best or most general answer, unless otherwise instructed.
More informationDivisibility in the Fibonacci Numbers. Stefan Erickson Colorado College January 27, 2006
Divisibility in the Fibonacci Numbers Stefan Erickson Colorado College January 27, 2006 Fibonacci Numbers F n+2 = F n+1 + F n n 1 2 3 4 6 7 8 9 10 11 12 F n 1 1 2 3 8 13 21 34 89 144 n 13 14 1 16 17 18
More informationDefine Efficiency. 2: Analysis. Efficiency. Measuring efficiency. CSE 417: Algorithms and Computational Complexity. Winter 2007 Larry Ruzzo
CSE 417: Algorithms and Computational 2: Analysis Winter 2007 Larry Ruzzo Define Efficiency Runs fast on typical real problem instances Pro: sensible, bottom-line-oriented Con: moving target (diff computers,
More information3 The Fundamentals:Algorithms, the Integers, and Matrices
3 The Fundamentals:Algorithms, the Integers, and Matrices 3.1 Algrithms Definition 1 An algorithm is a finite set of instructions for performing a computation or solving a problem. Dijkstra s mini language
More informationLimitations of Algorithm Power
Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying
More informationComputer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms
Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds
More informationLecture 2: Asymptotic Notation CSCI Algorithms I
Lecture 2: Asymptotic Notation CSCI 700 - Algorithms I Andrew Rosenberg September 2, 2010 Last Time Review Insertion Sort Analysis of Runtime Proof of Correctness Today Asymptotic Notation Its use in analyzing
More informationMAT 243 Test 2 SOLUTIONS, FORM A
MAT Test SOLUTIONS, FORM A 1. [10 points] Give a recursive definition for the set of all ordered pairs of integers (x, y) such that x < y. Solution: Let S be the set described above. Note that if (x, y)
More information1. (16 points) Circle T if the corresponding statement is True or F if it is False.
Name Solution Key Show All Work!!! Page 1 1. (16 points) Circle T if the corresponding statement is True or F if it is False. T F The sequence {1, 1, 1, 1, 1, 1...} is an example of an Alternating sequence.
More informationAsymptotic Notation. such that t(n) cf(n) for all n n 0. for some positive real constant c and integer threshold n 0
Asymptotic Notation Asymptotic notation deals with the behaviour of a function in the limit, that is, for sufficiently large values of its parameter. Often, when analysing the run time of an algorithm,
More informationThe Time Complexity of an Algorithm
Analysis of Algorithms The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input. Purpose To estimate how long a program will run. To estimate the largest input
More informationCSE 421: Intro Algorithms. 2: Analysis. Winter 2012 Larry Ruzzo
CSE 421: Intro Algorithms 2: Analysis Winter 2012 Larry Ruzzo 1 Efficiency Our correct TSP algorithm was incredibly slow Basically slow no matter what computer you have We want a general theory of efficiency
More informationDefining Efficiency. 2: Analysis. Efficiency. Measuring efficiency. CSE 421: Intro Algorithms. Summer 2007 Larry Ruzzo
CSE 421: Intro Algorithms 2: Analysis Summer 2007 Larry Ruzzo Defining Efficiency Runs fast on typical real problem instances Pro: sensible, bottom-line-oriented Con: moving target (diff computers, compilers,
More informationCSC 8301 Design & Analysis of Algorithms: Lower Bounds
CSC 8301 Design & Analysis of Algorithms: Lower Bounds Professor Henry Carter Fall 2016 Recap Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized Simplex
More informationCMSC Discrete Mathematics SOLUTIONS TO SECOND MIDTERM EXAM November, 2005
CMSC-37110 Discrete Mathematics SOLUTIONS TO SECOND MIDTERM EXAM November, 2005 Instructor: László Babai Ryerson 164 e-mail: laci@cs This exam contributes 20% to your course grade. 1. (6 points) Let a
More informationThe P-vs-NP problem. Andrés E. Caicedo. September 10, 2011
The P-vs-NP problem Andrés E. Caicedo September 10, 2011 This note is based on lecture notes for the Caltech course Math 6c, prepared with A. Kechris and M. Shulman. 1 Decision problems Consider a finite
More informationCS1800 Discrete Structures Fall 2016 Profs. Gold & Schnyder April 25, CS1800 Discrete Structures Final
CS1800 Discrete Structures Fall 2016 Profs. Gold & Schnyder April 25, 2017 CS1800 Discrete Structures Final Instructions: 1. The exam is closed book and closed notes. You may not use a calculator or any
More informationReading 10 : Asymptotic Analysis
CS/Math 240: Introduction to Discrete Mathematics Fall 201 Instructor: Beck Hasti and Gautam Prakriya Reading 10 : Asymptotic Analysis In the last reading, we analyzed the running times of various algorithms.
More informationCS 344 Design and Analysis of Algorithms. Tarek El-Gaaly Course website:
CS 344 Design and Analysis of Algorithms Tarek El-Gaaly tgaaly@cs.rutgers.edu Course website: www.cs.rutgers.edu/~tgaaly/cs344.html Course Outline Textbook: Algorithms by S. Dasgupta, C.H. Papadimitriou,
More informationCS Non-recursive and Recursive Algorithm Analysis
CS483-04 Non-recursive and Recursive Algorithm Analysis Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/
More informationAutomata Theory CS Complexity Theory I: Polynomial Time
Automata Theory CS411-2015-17 Complexity Theory I: Polynomial Time David Galles Department of Computer Science University of San Francisco 17-0: Tractable vs. Intractable If a problem is recursive, then
More informationAlgorithms and Their Complexity
CSCE 222 Discrete Structures for Computing David Kebo Houngninou Algorithms and Their Complexity Chapter 3 Algorithm An algorithm is a finite sequence of steps that solves a problem. Computational complexity
More informationTime Complexity. CS60001: Foundations of Computing Science
Time Complexity CS60001: Foundations of Computing Science Professor, Dept. of Computer Sc. & Engg., Measuring Complexity Definition Let M be a deterministic Turing machine that halts on all inputs. The
More informationcse 311: foundations of computing Fall 2015 Lecture 12: Primes, GCD, applications
cse 311: foundations of computing Fall 2015 Lecture 12: Primes, GCD, applications n-bit unsigned integer representation Represent integer x as sum of powers of 2: If x = n 1 i=0 b i 2 i where each b i
More informationLimits of Feasibility. Example. Complexity Relationships among Models. 1. Complexity Relationships among Models
Limits of Feasibility Wolfgang Schreiner Wolfgang.Schreiner@risc.jku.at Research Institute for Symbolic Computation (RISC) Johannes Kepler University, Linz, Austria http://www.risc.jku.at 1. Complexity
More informationFall 2017 Test II review problems
Fall 2017 Test II review problems Dr. Holmes October 18, 2017 This is a quite miscellaneous grab bag of relevant problems from old tests. Some are certainly repeated. 1. Give the complete addition and
More informationWhat is Performance Analysis?
1.2 Basic Concepts What is Performance Analysis? Performance Analysis Space Complexity: - the amount of memory space used by the algorithm Time Complexity - the amount of computing time used by the algorithm
More informationAnalysis of Algorithms. Unit 5 - Intractable Problems
Analysis of Algorithms Unit 5 - Intractable Problems 1 Intractable Problems Tractable Problems vs. Intractable Problems Polynomial Problems NP Problems NP Complete and NP Hard Problems 2 In this unit we
More informationAdvanced Algorithmics (6EAP)
Advanced Algorithmics (6EAP) MTAT.03.238 Order of growth maths Jaak Vilo 2017 fall Jaak Vilo 1 Program execution on input of size n How many steps/cycles a processor would need to do How to relate algorithm
More informationData Structures in Java
Data Structures in Java Lecture 21: Introduction to NP-Completeness 12/9/2015 Daniel Bauer Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways
More informationAlgebra for error control codes
Algebra for error control codes EE 387, Notes 5, Handout #7 EE 387 concentrates on block codes that are linear: Codewords components are linear combinations of message symbols. g 11 g 12 g 1n g 21 g 22
More informationModels of Computation
Models of Computation Analysis of Algorithms Week 1, Lecture 2 Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Models of Computation (RAM) a) Random Access Machines
More informationThe Time Complexity of an Algorithm
CSE 3101Z Design and Analysis of Algorithms The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input. Purpose To estimate how long a program will run. To estimate
More informationTheory of Computation Time Complexity
Theory of Computation Time Complexity Bow-Yaw Wang Academia Sinica Spring 2012 Bow-Yaw Wang (Academia Sinica) Time Complexity Spring 2012 1 / 59 Time for Deciding a Language Let us consider A = {0 n 1
More informationComputer Sciences Department
Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y
More informationAnalysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort
Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map
More informationPrinciples of Algorithm Analysis
C H A P T E R 3 Principles of Algorithm Analysis 3.1 Computer Programs The design of computer programs requires:- 1. An algorithm that is easy to understand, code and debug. This is the concern of software
More informationCS Data Structures and Algorithm Analysis
CS 483 - Data Structures and Algorithm Analysis Lecture II: Chapter 2 R. Paul Wiegand George Mason University, Department of Computer Science February 1, 2006 Outline 1 Analysis Framework 2 Asymptotic
More information5 + 9(10) + 3(100) + 0(1000) + 2(10000) =
Chapter 5 Analyzing Algorithms So far we have been proving statements about databases, mathematics and arithmetic, or sequences of numbers. Though these types of statements are common in computer science,
More informationComputational Models Lecture 11, Spring 2009
Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 11, Spring 2009 Deterministic Time Classes NonDeterministic Time Classes
More informationComputability and Complexity
Computability and Complexity Lecture 10 More examples of problems in P Closure properties of the class P The class NP given by Jiri Srba Lecture 10 Computability and Complexity 1/12 Example: Relatively
More information2. THE EUCLIDEAN ALGORITHM More ring essentials
2. THE EUCLIDEAN ALGORITHM More ring essentials In this chapter: rings R commutative with 1. An element b R divides a R, or b is a divisor of a, or a is divisible by b, or a is a multiple of b, if there
More informationChinese Remainder Theorem
Chinese Remainder Theorem Theorem Let R be a Euclidean domain with m 1, m 2,..., m k R. If gcd(m i, m j ) = 1 for 1 i < j k then m = m 1 m 2 m k = lcm(m 1, m 2,..., m k ) and R/m = R/m 1 R/m 2 R/m k ;
More informationComplexity. Complexity Theory Lecture 3. Decidability and Complexity. Complexity Classes
Complexity Theory 1 Complexity Theory 2 Complexity Theory Lecture 3 Complexity For any function f : IN IN, we say that a language L is in TIME(f(n)) if there is a machine M = (Q, Σ, s, δ), such that: L
More informationLecture 6: Introducing Complexity
COMP26120: Algorithms and Imperative Programming Lecture 6: Introducing Complexity Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2015 16 You need this book: Make sure you use the up-to-date
More informationTopic 17. Analysis of Algorithms
Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm
More informationRecursion. Computational complexity
List. Babes-Bolyai University arthur@cs.ubbcluj.ro Overview List 1 2 List Second Laboratory Test List Will take place week 12, during the laboratory You will receive one problem statement, from what was
More informationCP405 Theory of Computation
CP405 Theory of Computation BB(3) q 0 q 1 q 2 0 q 1 1R q 2 0R q 2 1L 1 H1R q 1 1R q 0 1L Growing Fast BB(3) = 6 BB(4) = 13 BB(5) = 4098 BB(6) = 3.515 x 10 18267 (known) (known) (possible) (possible) Language:
More informationBig O Notation. P. Danziger
1 Comparing Algorithms We have seen that in many cases we would like to compare two algorithms. Generally, the efficiency of an algorithm can be guaged by how long it takes to run as a function of the
More informationBig O Notation. P. Danziger
1 Comparing Algorithms We have seen that in many cases we would like to compare two algorithms. Generally, the efficiency of an algorithm can be guaged by how long it takes to run as a function of the
More informationCS Asymptotic Notations for Algorithm Analysis
CS483-02 Asymptotic Notations for Algorithm Analysis Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/
More informationCISC 4090 Theory of Computation
CISC 4090 Theory of Computation Complexity Professor Daniel Leeds dleeds@fordham.edu JMH 332 Computability Are we guaranteed to get an answer? Complexity How long do we have to wait for an answer? (Ch7)
More informationCISC 235: Topic 1. Complexity of Iterative Algorithms
CISC 235: Topic 1 Complexity of Iterative Algorithms Outline Complexity Basics Big-Oh Notation Big-Ω and Big-θ Notation Summations Limitations of Big-Oh Analysis 2 Complexity Complexity is the study of
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 5
CS 70 Discrete Mathematics and Probability Theory Summer 2014 James Cook Note 5 Modular Arithmetic In several settings, such as error-correcting codes and cryptography, we sometimes wish to work over a
More informationIn complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.
10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called
More informationcse 311: foundations of computing Spring 2015 Lecture 12: Primes, GCD, applications
cse 311: foundations of computing Spring 2015 Lecture 12: Primes, GCD, applications casting out 3s Theorem: A positive integer n is divisible by 3 if and only if the sum of its decimal digits is divisible
More informationPart I: Definitions and Properties
Turing Machines Part I: Definitions and Properties Finite State Automata Deterministic Automata (DFSA) M = {Q, Σ, δ, q 0, F} -- Σ = Symbols -- Q = States -- q 0 = Initial State -- F = Accepting States
More informationAlgorithms Design & Analysis. Analysis of Algorithm
Algorithms Design & Analysis Analysis of Algorithm Review Internship Stable Matching Algorithm 2 Outline Time complexity Computation model Asymptotic notions Recurrence Master theorem 3 The problem of
More informationAlgorithms (II) Yu Yu. Shanghai Jiaotong University
Algorithms (II) Yu Yu Shanghai Jiaotong University Chapter 1. Algorithms with Numbers Two seemingly similar problems Factoring: Given a number N, express it as a product of its prime factors. Primality:
More informationGrowth of Functions. As an example for an estimate of computation time, let us consider the sequential search algorithm.
Function Growth of Functions Subjects to be Learned Contents big oh max function big omega big theta little oh little omega Introduction One of the important criteria in evaluating algorithms is the time
More informationFundamental Algorithms
Fundamental Algorithms Chapter 1: Introduction Michael Bader Winter 2011/12 Chapter 1: Introduction, Winter 2011/12 1 Part I Overview Chapter 1: Introduction, Winter 2011/12 2 Organizational Stuff 2 SWS
More informationThe running time of Euclid s algorithm
The running time of Euclid s algorithm We analyze the worst-case running time of EUCLID as a function of the size of and Assume w.l.g. that 0 The overall running time of EUCLID is proportional to the number
More informationIntro to Theory of Computation
Intro to Theory of Computation LECTURE 22 Last time Review Today: Finish recursion theorem Complexity theory Exam 2 solutions out Homework 9 out Sofya Raskhodnikova L22.1 I-clicker question (frequency:
More informationCh 01. Analysis of Algorithms
Ch 01. Analysis of Algorithms Input Algorithm Output Acknowledgement: Parts of slides in this presentation come from the materials accompanying the textbook Algorithm Design and Applications, by M. T.
More informationCSE 417: Algorithms and Computational Complexity
CSE 417: Algorithms and Computational Complexity Lecture 2: Analysis Larry Ruzzo 1 Why big-o: measuring algorithm efficiency outline What s big-o: definition and related concepts Reasoning with big-o:
More informationChapter 5.1: Induction
Chapter.1: Induction Monday, July 1 Fermat s Little Theorem Evaluate the following: 1. 1 (mod ) 1 ( ) 1 1 (mod ). (mod 7) ( ) 8 ) 1 8 1 (mod ). 77 (mod 19). 18 (mod 1) 77 ( 18 ) 1 1 (mod 19) 18 1 (mod
More informationDiscrete Math in Computer Science Solutions to Practice Problems for Midterm 2
Discrete Math in Computer Science Solutions to Practice Problems for Midterm 2 CS 30, Fall 2018 by Professor Prasad Jayanti Problems 1. Let g(0) = 2, g(1) = 1, and g(n) = 2g(n 1) + g(n 2) whenever n 2.
More informationAlgorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ. Instructor: X. Zhang
Algorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Last class Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive
More informationAlgorithm efficiency analysis
Algorithm efficiency analysis Mădălina Răschip, Cristian Gaţu Faculty of Computer Science Alexandru Ioan Cuza University of Iaşi, Romania DS 2017/2018 Content Algorithm efficiency analysis Recursive function
More informationUnit 1A: Computational Complexity
Unit 1A: Computational Complexity Course contents: Computational complexity NP-completeness Algorithmic Paradigms Readings Chapters 3, 4, and 5 Unit 1A 1 O: Upper Bounding Function Def: f(n)= O(g(n)) if
More informationMat Week 8. Week 8. gcd() Mat Bases. Integers & Computers. Linear Combos. Week 8. Induction Proofs. Fall 2013
Fall 2013 Student Responsibilities Reading: Textbook, Section 3.7, 4.1, & 5.2 Assignments: Sections 3.6, 3.7, 4.1 Proof Worksheets Attendance: Strongly Encouraged Overview 3.6 Integers and Algorithms 3.7
More informationAnalysis of Algorithms
September 29, 2017 Analysis of Algorithms CS 141, Fall 2017 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization
More informationStudent Responsibilities Week 8. Mat Section 3.6 Integers and Algorithms. Algorithm to Find gcd()
Student Responsibilities Week 8 Mat 2345 Week 8 Reading: Textbook, Section 3.7, 4.1, & 5.2 Assignments: Sections 3.6, 3.7, 4.1 Induction Proof Worksheets Attendance: Strongly Encouraged Fall 2013 Week
More informationSpace Complexity of Algorithms
Space Complexity of Algorithms So far we have considered only the time necessary for a computation Sometimes the size of the memory necessary for the computation is more critical. The amount of memory
More information