Chapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens

Size: px
Start display at page:

Download "Chapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens"

Transcription

1 Chapter 2: The Basics slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens

2 Problem instances vs. decision problems vs. search problems Decision problem instance: is this graph Eulerian? (i.e., does it have an Eulerian path?) Decision problem: Given: a graph G Question: is G Eulerian? Search problem: Given: a graph G Find: an Eulerian path in G if one exists Search problem: A function f: {0,1}* {0,1}* Decision problem: A function φ: {0,1}* {0,1} a.k.a., predicate Decision problem: The set of all Eulerian graphs,,,, { }, Decision problem: The subset L EG {0,1}* of binary strings encoding Eulerian graphs (a.k.a. language) L EG = {0, 10,,, } : Problems and Solutions

3 The definition of algorithm A problem is solved by an algorithm that computes the function/predicate associated to the problem Formal definition of algorithm (for this class): program written in your favorite programming language (e.g., Python, C++, Java, Haskell, ) Equivalent formal definition: Turing machine The textbook defers discussion of Turing machines until we have developed most of the basics of computational complexity theory Turing machines are historically important, and still relevant, but not required to understand computational complexity or computability 2.1: Problems and Solutions 3

4 Euclid s algorithm for greatest common divisor (300 BC) Given: integers a,b Find: gcd(a,b) d is a common divisor of a and b if and only if it is a common divisor of b and a mod b Thus gcd(a,b) = gcd(b, a mod b) Python code def gcd(a,b): if b==0: return a else: return gcd(b, a%b) a b a%b 66 mod 24 = mod 18 = 6 18 mod 6 = 0 gcd(66,24) = 6 66 mod 24 = 18 How long does this algorithm take in the worst case? 2.1: Problems and Solutions x6 square 4 66

5 Time and scaling We measure running time as a function of input size; e.g., n 2 steps on inputs of size n. What is the size n of an instance G=(V,E) of the Eulerian path problem? n = V? n = V + E? n = s G = V 2, where s G {0,1}* is the adjacency matrix of G? What is the size n of an instance (a,b) of the GCD problem? n = a+b? n = max(a,b)? n = log 2 (a)+log 2 (b) is the number of bits we need to represent a and b 2.2: Time, Space and Scaling 5

6 Definition of running time What is a step? reasonable definition depends on programming language atomic or basic line of code in (imperative) program, e.g. a = 4 a = b+c (is this reasonable?) string = abcde while a < b: not a line that takes several steps or calls a subroutine, e.g. multiples_of_3 = [3*n for n in range(n)] if a < largest_prime_in_range(2,n): string = other_string.upper_case() single transition of a Turing machine single application of update rule in cellular automaton (many cell updates in parallel) step in a functional or logical programming language? (definition depends on how interpreter works) We will not worry too much about the precise definition since much computational complexity theory ignores polynomial differences. 2.2: Time, Space and Scaling 6

7 Time complexity of Euclidean algorithm input: (a,b); size = n = # bits of a and b (assume both are n bits) we will count how many divisions we need; if it is t(n) and each division takes time d(n), then the total time complexity is t(n)*d(n) Exercise: If a b, then a mod b < a/2 Exercise: Euclid s algorithm performs at most 2 log a = 2n divisions. Takes O(n 2 ) time if we use grade-school algorithm for division (Problem 2.11) 2.2: Time, Space and Scaling 7

8 Time complexity of factoring input: m; size = n = log m, so m 2 n find: a factor of m in the range [2, m-1] if it exists why is this enough to help us find the whole prime factorization? Obvious algorithm (trial division): time complexity (# of divisions): m 1/2 2 n/2 find_factor(m): for d in [2,3,, m 1/2 +1]: if m%d == 0: return d Best known algorithm: number field sieve, time complexity 2 nn1/3 It is believed by many (not everyone) that there is no polynomial-time algorithm for factoring (there is a n 3 quantum algorithm) 2.2: Time, Space and Scaling 8

9 Asymptotic notation t(n) = O(f(n)) what does it mean? intuitively, t f more properly written, t = O(f) even more properly written, t O(f) t = o(f) intuitively, t < f formally: lim nn tt nn ff nn = 0 t = O(f) if and only if f o(t) if and only if there is a constant C so that, for all n, t(n) C*f(n) t = Θ(f) if and only if t = O(f) and f = O(t) intuitively t = f t = Ω(f) if and only if f = O(t) intuitively t f t = ω(f) if and only if f = o(t) intuitively t > f 3n 2 = O(n 2 ) n 2 = O(3n 2 ) 100n 2 = o(n 2.1 ) n 2.1 = o(n 3 ) n 100 = o(2 n ) 2 n = o(2 3n ) if we say f(n) = O(log n), why don t we bother writing the base of the logarithm? 2.2: Time, Space and Scaling 9

10 Intrinsic complexity: algorithms vs. problems Grade school algorithm to multiply n-digit numbers x and y: O(n 2 ) time better algorithm: break x and y into two pieces, e.g., x = y = a = b = c = d = x = 10 n/2 a + b y = 10 n/2 c + d xy = (10 n/2 a + b)(10 n/2 c + d) = 10 n ac + 10 n/2 (ad+bc) + bd note (a+b)(c+d) ac bd = ad + bc we can do three multiplications (ac, bd, and (a+b)(c+d)) on n/2 digit integers, and 6 additions/subtractions and 2 shifts (which take O(n) time each) time complexity: t(n) = 3t(n/2) + O(n) = O(n log 3 ) = O(n ) Fast Fourier Transform algorithm takes time O(n log n) Open question: is there an O(n) time algorithm? Lesson: although grade school algorithm seems most natural; the problem itself has an intrinsic time complexity; grade school algorithm doesn t meet it 2.3: Intrinsic Complexity belief that maybe there s a better algorithm than the first one you thought of n x uncanny valley of understanding that complexity is intrinsic to problems, not algorithms biologist n 2 mathematician engineer physicist skill at programming computer scientist 10

11 Why focus on polynomial running times? Running time of this algorithm on G=(V,E)? loop executes n= V times how long does each loop take? depends on time to evaluate deg(node) adjacency matrix: Ω(n) time, so n 2 for whole algorithm could be more if matrix lookups are not O(1) time adjacency list: O(1) time, so n for whole algorithm euler(g): y = 0 for each node in g.nodes: if deg(node) is odd: y = y + 1 if y > 2: return false return true These choices will affect running time measurements by no more than a polynomial factor ( most choices affect only by a linear factor) Conclusion: to understand fundamental properties of computation, and not details of individual modeling choices, consider equivalences up to a polynomial factor Computer science is no more about [technical details of] computers than astronomy is about telescopes folklore, quote often misattributed to Edsgar Dijkstra Rather than studying the art of grinding lenses and mirrors, let us turn our attention to the stars real quote from textbook (Cris Moore and Stephan Mertens) 2.4: The Importance of Being Polynomial 11

12 Our first complexity classes: P, TIME(t(n)), EXP P is the class of problems solvable in polynomial time Formal version for decision problems: Given A {0,1}*, A P if there is a program Q and a constant c such that, for all x {0,1}*, Q on input x halts in at most x c steps, and Q(x) = 1 if and only if x A For t: N N, TIME(t(n)) is the class of problems solvable in time O(t(n)). P = TIME(n c ) cc N poly(n) is a shorthand for O(n c ) for some constant c P = TIME(poly(n)) EXP = TIME(2 poly(n) ) = TIME(2 nncc ) cc N since for every c, n c = O(2 n ), this means P EXP 2.4: The Importance of Being Polynomial 1

13 Tractability versus mathematical insight Definition: P = class of decision problems solvable in polynomial time P = set of predicates φ: {0,1}* {0,1} such that there is a program Q and a constant c such that, for all strings x {0,1}* where n= x, Q(x) = φ(x) and Q halts on input x in at most n c steps This is often paraphrased as tractable decision problems Not really: n 100 is intractable However, not being in P generally implies intractable Real goal of studying P: mathematical insight problems in P have some special structure that allows algorithms to avoid an exponential time brute force search once this insight is made, others tend to follow, and so do efficient algorithms We think some problems in NP don t have this structure (i.e., P NP) The difference between polynomial and exponential time is one of kind, not of degree. textbook 2.5: Tractability and Mathematical Insight 13

14 Chapter 3: Insights and Algorithms We won t go through this in lecture. It reviews the big ideas from ECS 122a. I will assume everyone is familiar with the techniques and terminology from chapter 3, especially the following recursion (3.1) divide-and-conquer (3.2) dynamic programming (3.3) graph reachability; breadth-first and depth-first search (3.4) graph algorithms: shortest path (3.4), minimum spanning tree (3.5), min-cut (3.7) reductions (3.8) Chapter 3 14

Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1)

Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1) Time Complexity (1) CSCI 2670 Original Slides were written by Dr. Frederick W Maier Spring 2014 Time Complexity So far we ve dealt with determining whether or not a problem is decidable. But even if it

More information

Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time

Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time Harry Lewis November 12, 20123 Review of Asymptotic Notation For f, g : N R + f = O(g): c > 0 s.t. f(n) c g(n) for all sufficiently large n. f

More information

Computational Models Lecture 11, Spring 2009

Computational Models Lecture 11, Spring 2009 Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 11, Spring 2009 Deterministic Time Classes NonDeterministic Time Classes

More information

CISC 4090 Theory of Computation

CISC 4090 Theory of Computation CISC 4090 Theory of Computation Complexity Professor Daniel Leeds dleeds@fordham.edu JMH 332 Computability Are we guaranteed to get an answer? Complexity How long do we have to wait for an answer? (Ch7)

More information

Lecture 18: P & NP. Revised, May 1, CLRS, pp

Lecture 18: P & NP. Revised, May 1, CLRS, pp Lecture 18: P & NP Revised, May 1, 2003 CLRS, pp.966-982 The course so far: techniques for designing efficient algorithms, e.g., divide-and-conquer, dynamic-programming, greedy-algorithms. What happens

More information

Computational complexity

Computational complexity COMS11700 Computational complexity Department of Computer Science, University of Bristol Bristol, UK 2 May 2014 COMS11700: Computational complexity Slide 1/23 Introduction If we can prove that a language

More information

Input Decidable Language -- Program Halts on all Input Encoding of Input -- Natural Numbers Encoded in Binary or Decimal, Not Unary

Input Decidable Language -- Program Halts on all Input Encoding of Input -- Natural Numbers Encoded in Binary or Decimal, Not Unary Complexity Analysis Complexity Theory Input Decidable Language -- Program Halts on all Input Encoding of Input -- Natural Numbers Encoded in Binary or Decimal, Not Unary Output TRUE or FALSE Time and Space

More information

Lecture 6: Introducing Complexity

Lecture 6: Introducing Complexity COMP26120: Algorithms and Imperative Programming Lecture 6: Introducing Complexity Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2015 16 You need this book: Make sure you use the up-to-date

More information

Computability and Complexity

Computability and Complexity Computability and Complexity Lecture 10 More examples of problems in P Closure properties of the class P The class NP given by Jiri Srba Lecture 10 Computability and Complexity 1/12 Example: Relatively

More information

Computer Sciences Department

Computer Sciences Department Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

More information

Time Complexity. CS60001: Foundations of Computing Science

Time Complexity. CS60001: Foundations of Computing Science Time Complexity CS60001: Foundations of Computing Science Professor, Dept. of Computer Sc. & Engg., Measuring Complexity Definition Let M be a deterministic Turing machine that halts on all inputs. The

More information

CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION CSE 105 THEORY OF COMPUTATION Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/ Logistics HW7 due tonight Thursday's class: REVIEW Final exam on Thursday Dec 8, 8am-11am, LEDDN AUD Note card allowed

More information

Remainders. We learned how to multiply and divide in elementary

Remainders. We learned how to multiply and divide in elementary Remainders We learned how to multiply and divide in elementary school. As adults we perform division mostly by pressing the key on a calculator. This key supplies the quotient. In numerical analysis and

More information

CS 350 Algorithms and Complexity

CS 350 Algorithms and Complexity CS 350 Algorithms and Complexity Winter 2019 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower

More information

CS 350 Algorithms and Complexity

CS 350 Algorithms and Complexity 1 CS 350 Algorithms and Complexity Fall 2015 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower

More information

Intractable Problems [HMU06,Chp.10a]

Intractable Problems [HMU06,Chp.10a] Intractable Problems [HMU06,Chp.10a] Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions A 10 Minute Motivation https://www.youtube.com/watch?v=yx40hbahx3s 1 Time-Bounded TM s A Turing

More information

3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need?

3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need? 3. Algorithms We will study algorithms to solve many different types of problems such as finding the largest of a sequence of numbers sorting a sequence of numbers finding the shortest path between two

More information

P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

More information

Great Theoretical Ideas in Computer Science. Lecture 9: Introduction to Computational Complexity

Great Theoretical Ideas in Computer Science. Lecture 9: Introduction to Computational Complexity 15-251 Great Theoretical Ideas in Computer Science Lecture 9: Introduction to Computational Complexity February 14th, 2017 Poll What is the running time of this algorithm? Choose the tightest bound. def

More information

1 Computational problems

1 Computational problems 80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

More information

Theory of Computation

Theory of Computation Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 38 Lecture 21: Overview Big-Oh notation. Little-o notation. Time Complexity Classes Non-deterministic TMs The Class

More information

CP405 Theory of Computation

CP405 Theory of Computation CP405 Theory of Computation BB(3) q 0 q 1 q 2 0 q 1 1R q 2 0R q 2 1L 1 H1R q 1 1R q 0 1L Growing Fast BB(3) = 6 BB(4) = 13 BB(5) = 4098 BB(6) = 3.515 x 10 18267 (known) (known) (possible) (possible) Language:

More information

34.1 Polynomial time. Abstract problems

34.1 Polynomial time. Abstract problems < Day Day Up > 34.1 Polynomial time We begin our study of NP-completeness by formalizing our notion of polynomial-time solvable problems. These problems are generally regarded as tractable, but for philosophical,

More information

Notes for Lecture Notes 2

Notes for Lecture Notes 2 Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

Limitations of Algorithm Power

Limitations of Algorithm Power Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying

More information

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta CSE 2001: Introduction to Theory of Computation Fall 2013 Suprakash Datta datta@cse.yorku.ca Office: CSEB 3043 Phone: 416-736-2100 ext 77875 Course page: http://www.eecs.yorku.ca/course/2001 9/10/2013

More information

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta

CSE 2001: Introduction to Theory of Computation Fall Suprakash Datta CSE 2001: Introduction to Theory of Computation Fall 2012 Suprakash Datta datta@cse.yorku.ca Office: CSEB 3043 Phone: 416-736-2100 ext 77875 Course page: http://www.cs.yorku.ca/course/2001 9/6/2012 CSE

More information

Computational Complexity

Computational Complexity CS311 Computational Structures Computational Complexity Lecture 16 Andrew P. Black Andrew Tolmach 1 So, itʼs computable! But at what cost? Some things that are computable in principle are in practice intractable

More information

Advanced topic: Space complexity

Advanced topic: Space complexity Advanced topic: Space complexity CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 1/28 Review: time complexity We have looked at how long it takes to

More information

1 Computational Problems

1 Computational Problems Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

5 + 9(10) + 3(100) + 0(1000) + 2(10000) =

5 + 9(10) + 3(100) + 0(1000) + 2(10000) = Chapter 5 Analyzing Algorithms So far we have been proving statements about databases, mathematics and arithmetic, or sequences of numbers. Though these types of statements are common in computer science,

More information

Notes on Complexity Theory Last updated: December, Lecture 2

Notes on Complexity Theory Last updated: December, Lecture 2 Notes on Complexity Theory Last updated: December, 2011 Jonathan Katz Lecture 2 1 Review The running time of a Turing machine M on input x is the number of steps M takes before it halts. Machine M is said

More information

Automata Theory CS Complexity Theory I: Polynomial Time

Automata Theory CS Complexity Theory I: Polynomial Time Automata Theory CS411-2015-17 Complexity Theory I: Polynomial Time David Galles Department of Computer Science University of San Francisco 17-0: Tractable vs. Intractable If a problem is recursive, then

More information

COMPUTER ALGORITHMS. Athasit Surarerks.

COMPUTER ALGORITHMS. Athasit Surarerks. COMPUTER ALGORITHMS Athasit Surarerks. Introduction EUCLID s GAME Two players move in turn. On each move, a player has to write on the board a positive integer equal to the different from two numbers already

More information

Acknowledgments 2. Part 0: Overview 17

Acknowledgments 2. Part 0: Overview 17 Contents Acknowledgments 2 Preface for instructors 11 Which theory course are we talking about?.... 12 The features that might make this book appealing. 13 What s in and what s out............... 14 Possible

More information

INTRODUCTION TO THE THEORY OF NP-COMPLETENESS

INTRODUCTION TO THE THEORY OF NP-COMPLETENESS INTRODUCTION TO THE THEORY OF NP-COMPLETENESS CSI4105 (3 TO 5 LECTURES) Lucia Moura University of Ottawa Winter 2006 These notes/slides are intended as an introduction to the theory of NP-completeness,

More information

CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION CSE 105 THEORY OF COMPUTATION "Winter" 2018 http://cseweb.ucsd.edu/classes/wi18/cse105-ab/ Today's learning goals Sipser Ch 7 Distinguish between computability and complexity Articulate motivation questions

More information

Great Theoretical Ideas in Computer Science. Lecture 7: Introduction to Computational Complexity

Great Theoretical Ideas in Computer Science. Lecture 7: Introduction to Computational Complexity 15-251 Great Theoretical Ideas in Computer Science Lecture 7: Introduction to Computational Complexity September 20th, 2016 What have we done so far? What will we do next? What have we done so far? > Introduction

More information

Complexity and NP-completeness

Complexity and NP-completeness Lecture 17 Complexity and NP-completeness Supplemental reading in CLRS: Chapter 34 As an engineer or computer scientist, it is important not only to be able to solve problems, but also to know which problems

More information

1a. Introduction COMP6741: Parameterized and Exact Computation

1a. Introduction COMP6741: Parameterized and Exact Computation 1a. Introduction COMP6741: Parameterized and Exact Computation Serge Gaspers 12 1 School of Computer Science and Engineering, UNSW Sydney, Asutralia 2 Decision Sciences Group, Data61, CSIRO, Australia

More information

Theory of Computation Time Complexity

Theory of Computation Time Complexity Theory of Computation Time Complexity Bow-Yaw Wang Academia Sinica Spring 2012 Bow-Yaw Wang (Academia Sinica) Time Complexity Spring 2012 1 / 59 Time for Deciding a Language Let us consider A = {0 n 1

More information

Big , and Definition Definition

Big , and Definition Definition Big O, Ω, and Θ Big-O gives us only a one-way comparison; if f is O(g) then g eventually is bigger than f from that point on, but in fact f could be very small in comparison. Example; 3n is O(2 2n ). We

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning 236756 Prof. Nir Ailon Lecture 4: Computational Complexity of Learning & Surrogate Losses Efficient PAC Learning Until now we were mostly worried about sample complexity

More information

Topic 17. Analysis of Algorithms

Topic 17. Analysis of Algorithms Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm

More information

Number Theory. Zachary Friggstad. Programming Club Meeting

Number Theory. Zachary Friggstad. Programming Club Meeting Number Theory Zachary Friggstad Programming Club Meeting Outline Factoring Sieve Multiplicative Functions Greatest Common Divisors Applications Chinese Remainder Theorem Throughout, problems to try are

More information

Intractable Problems. Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions

Intractable Problems. Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions Intractable Problems Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions 1 Time-Bounded TM s A Turing machine that, given an input of length n, always halts within T(n) moves is said

More information

COT 3100 Applications of Discrete Structures Dr. Michael P. Frank

COT 3100 Applications of Discrete Structures Dr. Michael P. Frank University of Florida Dept. of Computer & Information Science & Engineering COT 3100 Applications of Discrete Structures Dr. Michael P. Frank Slides for a Course Based on the Text Discrete Mathematics

More information

How many hours would you estimate that you spent on this assignment?

How many hours would you estimate that you spent on this assignment? The first page of your homework submission must be a cover sheet answering the following questions. Do not leave it until the last minute; it s fine to fill out the cover sheet before you have completely

More information

In complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.

In complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size. 10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called

More information

Complexity. Complexity Theory Lecture 3. Decidability and Complexity. Complexity Classes

Complexity. Complexity Theory Lecture 3. Decidability and Complexity. Complexity Classes Complexity Theory 1 Complexity Theory 2 Complexity Theory Lecture 3 Complexity For any function f : IN IN, we say that a language L is in TIME(f(n)) if there is a machine M = (Q, Σ, s, δ), such that: L

More information

The Complexity of Optimization Problems

The Complexity of Optimization Problems The Complexity of Optimization Problems Summary Lecture 1 - Complexity of algorithms and problems - Complexity classes: P and NP - Reducibility - Karp reducibility - Turing reducibility Uniform and logarithmic

More information

CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1

CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1 CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY E. Amaldi Foundations of Operations Research Politecnico di Milano 1 Goal: Evaluate the computational requirements (this course s focus: time) to solve

More information

Lecture 1: Asymptotics, Recurrences, Elementary Sorting

Lecture 1: Asymptotics, Recurrences, Elementary Sorting Lecture 1: Asymptotics, Recurrences, Elementary Sorting Instructor: Outline 1 Introduction to Asymptotic Analysis Rate of growth of functions Comparing and bounding functions: O, Θ, Ω Specifying running

More information

CS6901: review of Theory of Computation and Algorithms

CS6901: review of Theory of Computation and Algorithms CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational

More information

Computational Complexity - Pseudocode and Recursions

Computational Complexity - Pseudocode and Recursions Computational Complexity - Pseudocode and Recursions Nicholas Mainardi 1 Dipartimento di Elettronica e Informazione Politecnico di Milano nicholas.mainardi@polimi.it June 6, 2018 1 Partly Based on Alessandro

More information

CS154, Lecture 13: P vs NP

CS154, Lecture 13: P vs NP CS154, Lecture 13: P vs NP The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms Polynomial-Time Turing Machines More generally: TM can simulate every reasonable model of

More information

Lecture 29: Tractable and Intractable Problems

Lecture 29: Tractable and Intractable Problems Lecture 29: Tractable and Intractable Problems Aims: To look at the ideas of polynomial and exponential functions and algorithms; and tractable and intractable problems. Page 1 of 10 To look at ways of

More information

Theory of Computation

Theory of Computation Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi

More information

CSC Design and Analysis of Algorithms. Lecture 1

CSC Design and Analysis of Algorithms. Lecture 1 CSC 8301- Design and Analysis of Algorithms Lecture 1 Introduction Analysis framework and asymptotic notations What is an algorithm? An algorithm is a finite sequence of unambiguous instructions for solving

More information

Complete problems for classes in PH, The Polynomial-Time Hierarchy (PH) oracle is like a subroutine, or function in

Complete problems for classes in PH, The Polynomial-Time Hierarchy (PH) oracle is like a subroutine, or function in Oracle Turing Machines Nondeterministic OTM defined in the same way (transition relation, rather than function) oracle is like a subroutine, or function in your favorite PL but each call counts as single

More information

Complexity (Pre Lecture)

Complexity (Pre Lecture) Complexity (Pre Lecture) Dr. Neil T. Dantam CSCI-561, Colorado School of Mines Fall 2018 Dantam (Mines CSCI-561) Complexity (Pre Lecture) Fall 2018 1 / 70 Why? What can we always compute efficiently? What

More information

Analysis of Algorithms

Analysis of Algorithms Presentation for use with the textbook Data Structures and Algorithms in Java, 6th edition, by M. T. Goodrich, R. Tamassia, and M. H. Goldwasser, Wiley, 2014 Analysis of Algorithms Input Algorithm Analysis

More information

Ch 4.2 Divisibility Properties

Ch 4.2 Divisibility Properties Ch 4.2 Divisibility Properties - Prime numbers and composite numbers - Procedure for determining whether or not a positive integer is a prime - GCF: procedure for finding gcf (Euclidean Algorithm) - Definition:

More information

Asymptotic Notation. such that t(n) cf(n) for all n n 0. for some positive real constant c and integer threshold n 0

Asymptotic Notation. such that t(n) cf(n) for all n n 0. for some positive real constant c and integer threshold n 0 Asymptotic Notation Asymptotic notation deals with the behaviour of a function in the limit, that is, for sufficiently large values of its parameter. Often, when analysing the run time of an algorithm,

More information

Optimisation and Operations Research

Optimisation and Operations Research Optimisation and Operations Research Lecture 12: Algorithm Analysis and Complexity Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/

More information

Discrete Mathematics GCD, LCM, RSA Algorithm

Discrete Mathematics GCD, LCM, RSA Algorithm Discrete Mathematics GCD, LCM, RSA Algorithm Abdul Hameed http://informationtechnology.pk/pucit abdul.hameed@pucit.edu.pk Lecture 16 Greatest Common Divisor 2 Greatest common divisor The greatest common

More information

Algorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ. Instructor: X. Zhang

Algorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Algorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Last class Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive

More information

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1

U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1 U.C. Berkeley CS278: Computational Complexity Handout N1 Professor Luca Trevisan August 30, 2004 Notes for Lecture 1 This course assumes CS170, or equivalent, as a prerequisite. We will assume that the

More information

Induction and recursion. Chapter 5

Induction and recursion. Chapter 5 Induction and recursion Chapter 5 Chapter Summary Mathematical Induction Strong Induction Well-Ordering Recursive Definitions Structural Induction Recursive Algorithms Mathematical Induction Section 5.1

More information

Algorithm Efficiency. Algorithmic Thinking Luay Nakhleh Department of Computer Science Rice University

Algorithm Efficiency. Algorithmic Thinking Luay Nakhleh Department of Computer Science Rice University Algorithm Efficiency Algorithmic Thinking Luay Nakhleh Department of Computer Science Rice University 1 All Correct Algorithms Are Not Created Equal When presented with a set of correct algorithms for

More information

Analysis of Algorithms

Analysis of Algorithms October 1, 2015 Analysis of Algorithms CS 141, Fall 2015 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization

More information

NP-Completeness. Until now we have been designing algorithms for specific problems

NP-Completeness. Until now we have been designing algorithms for specific problems NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower

More information

CS154, Lecture 13: P vs NP

CS154, Lecture 13: P vs NP CS154, Lecture 13: P vs NP The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms Polynomial-Time Turing Machines More generally: TM can simulate every reasonable model of

More information

CS294: Pseudorandomness and Combinatorial Constructions September 13, Notes for Lecture 5

CS294: Pseudorandomness and Combinatorial Constructions September 13, Notes for Lecture 5 UC Berkeley Handout N5 CS94: Pseudorandomness and Combinatorial Constructions September 3, 005 Professor Luca Trevisan Scribe: Gatis Midrijanis Notes for Lecture 5 In the few lectures we are going to look

More information

Asymptotic Analysis and Recurrences

Asymptotic Analysis and Recurrences Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several

More information

CSCI3390-Lecture 14: The class NP

CSCI3390-Lecture 14: The class NP CSCI3390-Lecture 14: The class NP 1 Problems and Witnesses All of the decision problems described below have the form: Is there a solution to X? where X is the given problem instance. If the instance is

More information

15-251: Great Theoretical Ideas in Computer Science Lecture 7. Turing s Legacy Continues

15-251: Great Theoretical Ideas in Computer Science Lecture 7. Turing s Legacy Continues 15-251: Great Theoretical Ideas in Computer Science Lecture 7 Turing s Legacy Continues Solvable with Python = Solvable with C = Solvable with Java = Solvable with SML = Decidable Languages (decidable

More information

3 The Fundamentals:Algorithms, the Integers, and Matrices

3 The Fundamentals:Algorithms, the Integers, and Matrices 3 The Fundamentals:Algorithms, the Integers, and Matrices 3.1 Algrithms Definition 1 An algorithm is a finite set of instructions for performing a computation or solving a problem. Dijkstra s mini language

More information

8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

More information

NP-Completeness. Algorithmique Fall semester 2011/12

NP-Completeness. Algorithmique Fall semester 2011/12 NP-Completeness Algorithmique Fall semester 2011/12 1 What is an Algorithm? We haven t really answered this question in this course. Informally, an algorithm is a step-by-step procedure for computing a

More information

CPSC 467: Cryptography and Computer Security

CPSC 467: Cryptography and Computer Security CPSC 467: Cryptography and Computer Security Michael J. Fischer Lecture 9 September 30, 2015 CPSC 467, Lecture 9 1/47 Fast Exponentiation Algorithms Number Theory Needed for RSA Elementary Number Theory

More information

1 Non-deterministic Turing Machine

1 Non-deterministic Turing Machine 1 Non-deterministic Turing Machine A nondeterministic Turing machine is a generalization of the standard TM for which every configuration may yield none, or one or more than one next configurations. In

More information

CS3719 Theory of Computation and Algorithms

CS3719 Theory of Computation and Algorithms CS3719 Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational tool - analysis

More information

P and NP. Or, how to make $1,000,000.

P and NP. Or, how to make $1,000,000. P and NP Or, how to make $1,000,000. http://www.claymath.org/millennium-problems/p-vs-np-problem Review: Polynomial time difference between single-tape and multi-tape TMs Exponential time difference between

More information

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

More information

Asymptotic Analysis. Slides by Carl Kingsford. Jan. 27, AD Chapter 2

Asymptotic Analysis. Slides by Carl Kingsford. Jan. 27, AD Chapter 2 Asymptotic Analysis Slides by Carl Kingsford Jan. 27, 2014 AD Chapter 2 Independent Set Definition (Independent Set). Given a graph G = (V, E) an independent set is a set S V if no two nodes in S are joined

More information

Announcements. CSE332: Data Abstractions Lecture 2: Math Review; Algorithm Analysis. Today. Mathematical induction. Dan Grossman Spring 2010

Announcements. CSE332: Data Abstractions Lecture 2: Math Review; Algorithm Analysis. Today. Mathematical induction. Dan Grossman Spring 2010 Announcements CSE332: Data Abstractions Lecture 2: Math Review; Algorithm Analysis Dan Grossman Spring 2010 Project 1 posted Section materials on using Eclipse will be very useful if you have never used

More information

Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity. The space complexity of a program is how much memory it uses. Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

More information

You separate binary numbers into columns in a similar fashion. 2 5 = 32

You separate binary numbers into columns in a similar fashion. 2 5 = 32 RSA Encryption 2 At the end of Part I of this article, we stated that RSA encryption works because it s impractical to factor n, which determines P 1 and P 2, which determines our private key, d, which

More information

CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION CSE 105 THEORY OF COMPUTATION Spring 2016 http://cseweb.ucsd.edu/classes/sp16/cse105-ab/ Today's learning goals Sipser Ch 7.2, 7.3 Distinguish between polynomial and exponential DTIME Define nondeterministic

More information

Reductions. Reduction. Linear Time Reduction: Examples. Linear Time Reductions

Reductions. Reduction. Linear Time Reduction: Examples. Linear Time Reductions Reduction Reductions Problem X reduces to problem Y if given a subroutine for Y, can solve X. Cost of solving X = cost of solving Y + cost of reduction. May call subroutine for Y more than once. Ex: X

More information

CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010

CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010 CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010 Computational complexity studies the amount of resources necessary to perform given computations.

More information

CS 241 Analysis of Algorithms

CS 241 Analysis of Algorithms CS 241 Analysis of Algorithms Professor Eric Aaron Lecture T Th 9:00am Lecture Meeting Location: OLB 205 Business Grading updates: HW5 back today HW7 due Dec. 10 Reading: Ch. 22.1-22.3, Ch. 25.1-2, Ch.

More information

Friday Four Square! Today at 4:15PM, Outside Gates

Friday Four Square! Today at 4:15PM, Outside Gates P and NP Friday Four Square! Today at 4:15PM, Outside Gates Recap from Last Time Regular Languages DCFLs CFLs Efficiently Decidable Languages R Undecidable Languages Time Complexity A step of a Turing

More information

2. ALGORITHM ANALYSIS

2. ALGORITHM ANALYSIS 2. ALGORITHM ANALYSIS computational tractability asymptotic order of growth survey of common running times Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos

More information

1 Circuit Complexity. CS 6743 Lecture 15 1 Fall Definitions

1 Circuit Complexity. CS 6743 Lecture 15 1 Fall Definitions CS 6743 Lecture 15 1 Fall 2007 1 Circuit Complexity 1.1 Definitions A Boolean circuit C on n inputs x 1,..., x n is a directed acyclic graph (DAG) with n nodes of in-degree 0 (the inputs x 1,..., x n ),

More information

Could we potentially place A in a smaller complexity class if we consider other computational models?

Could we potentially place A in a smaller complexity class if we consider other computational models? Introduction to Complexity Theory Big O Notation Review Linear function: r(n) =O(n). Polynomial function: r(n) =2 O(1) Exponential function: r(n) =2 no(1) Logarithmic function: r(n) = O(log n) Poly-log

More information

NP and Computational Intractability

NP and Computational Intractability NP and Computational Intractability 1 Review Basic reduction strategies. Simple equivalence: INDEPENDENT-SET P VERTEX-COVER. Special case to general case: VERTEX-COVER P SET-COVER. Encoding with gadgets:

More information

This is a recursive algorithm. The procedure is guaranteed to terminate, since the second argument decreases each time.

This is a recursive algorithm. The procedure is guaranteed to terminate, since the second argument decreases each time. 8 Modular Arithmetic We introduce an operator mod. Let d be a positive integer. For c a nonnegative integer, the value c mod d is the remainder when c is divided by d. For example, c mod d = 0 if and only

More information