Problem. Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26

Size: px
Start display at page:

Download "Problem. Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26"

Transcription

1 Binary Search

2 Introduction

3 Problem Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26

4 Strategy 1: Random Search Randomly select a page until the page containing the word is found. 4 / 26

5 Strategy 1: Random Search Randomly select a page until the page containing the word is found. Very inefficient, no guarantee that the correct page is chosen. Does not detect if word is not in the dictionary. 4 / 26

6 Strategy 2: Linear Search Check all pages sequentially starting at page 1. 5 / 26

7 Strategy 2: Linear Search Check all pages sequentially starting at page 1. Acceptable efficiency, but... strategy is not using all known properties. 5 / 26

8 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? 6 / 26

9 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. < < < < < < < < < 6 / 26

10 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. 6 / 26

11 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. 6 / 26

12 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. Question: Which page should be selected? 6 / 26

13 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. Question: Which page should be selected? The middle!? Why? 6 / 26

14 Algorithm Input: A sorted array A Some value k / 26

15 Algorithm 1. Set l := 0 and r := A.Length While l r l r / 26

16 Algorithm 2.1. Set m := l+r 2. l m r / 26

17 Algorithm 2.2. If A[m] < k Then Set l := m + 1. m l r / 26

18 Algorithm 2.3. If A[m] > k Then Set r := m 1. l r m / 26

19 Algorithm 2.3. If A[m] = k Then Return m. m r / 26

20 Algorithm Binary Search Input: A sorted array A Some value k 1. Set l := 0 and r := A.Length While l r 2.1 Set m := l+r If A[m] < k Then Set l := m If A[m] > k Then Set r := m If A[m] = k Then Return m. 3. Return 1. 8 / 26

21 Correctness and Invariant

22 Correctness What means Binary Search is correct? 10 / 26

23 Correctness What means Binary Search is correct? Theorem The algorithm Binary Search returns the index of k in A if k A, otherwise it returns 1. We have to show two cases k A k / A We will show the second case first. 10 / 26

24 Proof of Correctness k / A Some notation: l i and r i denote the value of l and r at the beginning of the i-th loop iteration, i. e., l 0 = 0 and r 0 = A.Length 1. m i denotes the value of m assigned in the i-th loop iteration. 11 / 26

25 Proof of Correctness k / A Some notation: l i and r i denote the value of l and r at the beginning of the i-th loop iteration, i. e., l 0 = 0 and r 0 = A.Length 1. m i denotes the value of m assigned in the i-th loop iteration. We have to show that we reach line 3. Because k / A, we never run line 2.4. Because l i m i r i, r i+1 l i+1 < r i l i (line 2.2 and line 2.3). Thus, the algorithm reaches line 3 after a finite amount of iterations. 11 / 26

26 Proof of Correctness k A How do we show that we find the right index? 12 / 26

27 Proof of Correctness k A How do we show that we find the right index? We show an invariant. 12 / 26

28 Proof of Correctness k A How do we show that we find the right index? We show an invariant. Invariant The invariant of a loop is a property that is true every time before and after every iteration of the loop. 12 / 26

29 Proof of Correctness k A How do we show that we find the right index? We show an invariant. Invariant The invariant of a loop is a property that is true every time before and after every iteration of the loop. Invariants often lead to the correctness of an algorithm. The most common way to prove them is induction. 12 / 26

30 Proof of Correctness k A Invariant for Binary Search 13 / 26

31 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i 13 / 26

32 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length / 26

33 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length 1. Inductive Step By induction, l i Ind(k) r i. If k = A[m i ], the algorithm returns m i. Done. If k < A[m i ], them Ind(k) < m i. Thus, l i+1 = l i Ind(k) r i+1 = m i 1 < m i r i. If k > A[m i ], them Ind(k) > m i. Thus, l i m i < m i + 1 = l i+1 Ind(k) r i+1 = r i. 13 / 26

34 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length 1. Inductive Step By induction, l i Ind(k) r i. If k = A[m i ], the algorithm returns m i. Done. If k < A[m i ], them Ind(k) < m i. Thus, l i+1 = l i Ind(k) r i+1 = m i 1 < m i r i. If k > A[m i ], them Ind(k) > m i. Thus, l i m i < m i + 1 = l i+1 Ind(k) r i+1 = r i. Because r i+1 l i+1 < r i l i, there is an iteration j with l j = Ind(k) = r j. 13 / 26

35 Runtime

36 Runtime How fast is Binary Search? 15 / 26

37 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 15 / 26

38 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 For which j is r j l j 1? 15 / 26

39 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 For which j is r j l j 1? r j l j 1 2 (r j 1 l j 1 ) 1 4 (r j 2 l j 2 ). 2 j (r 0 l 0 ) = 2 j (n 1) 1 log 2 n j 15 / 26

40 Runtime Theorem The algorithm Binary Search requires at most log 2 (n+1) iterations, where n = A. 16 / 26

41 Logarithmic Runtime If A = 2 n we need approx. n iterations. Also, Input size (n) Runtime (log 2 n) 1, ,000, ,000,000, atoms in the universe / 26

42 Lower Time Bound Is there a faster way than Binary Search? We allow a free preprocessing of A. We want to know if k is in A and where. 18 / 26

43 Lower Time Bound Is there a faster way than Binary Search? We allow a free preprocessing of A. We want to know if k is in A and where. Theorem In general, finding an element in an array cannot be done in less than logarithmic time. 18 / 26

44 Decision Tree Model of computation: comparison model only operations allowed are comparisons (<,, >,, =, ) time cost is number of comparisons 19 / 26

45 Decision Tree Model of computation: comparison model only operations allowed are comparisons (<,, >,, =, ) time cost is number of comparisons Decision Tree algorithm is tree of comparisons internal nodes are comparisons leafs are outcomes of the algorithm 19 / 26

46 Decision Tree Example Binary Search for A = 3 (For simplicity we ignore the case A[i] = k.) No A[1] < k? Yes A[0] < k? A[2] < k? No Yes No Yes k A[0] A[0] < k A[1] A[1] < k A[2] A[2] < k 20 / 26

47 Decision Tree Example Linear Search for A = 3 (For simplicity we ignore the case A[i] = k.) No A[0] < k? Yes k A[0] No A[1] < k? Yes A[0] < k A[1] No A[2] < k? Yes A[1] < k A[2] A[2] < k 21 / 26

48 Decision Tree vs. Algorithm + Proof Decision Tree internal node leaf root-to-leaf path path length height of tree Algorithm binary decision found answer algorithm execution running time worst case running time 22 / 26

49 Decision Tree vs. Algorithm + Proof Decision Tree internal node leaf root-to-leaf path path length height of tree Algorithm binary decision found answer algorithm execution running time worst case running time Proof of theorem: Decision tree is binary Tree contains each possible answer as leaf, i. e., n + 1 leafs Height log 2 (number of leafs). 22 / 26

50 Other Variants

51 Finding 01 in a Bit Array Given a bit array A where A[0] = 0 and A[n 1] = 1. Find an i such that A[i] = 0 and A[i + 1] = / 26

52 Finding 01 in a Bit Array Given a bit array A where A[0] = 0 and A[n 1] = 1. Find an i such that A[i] = 0 and A[i + 1] = 1. Using binary search: If A[m] = 1, search left. If A[m] = 0, search right. Invariant: A[l] = 0 and A[r] = / 26

53 Analog Digital Converter Convert an analog input (voltage) into a digital representation. Logic creates digital value (starting with most significant bit). Digital-to-analog converter (DAC) creates the analog equivalent U d. Comparator takes both analog siganls an outputs if U d > U in. If U d > U in, logic resets bit. Logic DAC Com. U in 25 / 26

54 Nodes in a Complete Binary Tree Complete binary tree All layers (except lowest) are full. Lowest layer is filled from left to right. Question: How many nodes are in the tree? 26 / 26

55 Nodes in a Complete Binary Tree Naive solution Count all nodes. Runtime: n 26 / 26

56 Nodes in a Complete Binary Tree Using binary search Determine height h l of left subtree (left side). Determine height h r of right subtree (left side). If h l = h r, lowest layer of left subtree is full (continue with right subtree). Otherwise, lowest layer of right subtree is empty (continue with left subtree). Runtime: log 2 n 26 / 26

57 Nodes in a Complete Binary Tree Using binary search Determine height h l of left subtree (left side). Determine height h r of right subtree (left side). If h l = h r, lowest layer of left subtree is full (continue with right subtree). Otherwise, lowest layer of right subtree is empty (continue with left subtree). Runtime: log 2 n 26 / 26

Sorting algorithms. Sorting algorithms

Sorting algorithms. Sorting algorithms Properties of sorting algorithms A sorting algorithm is Comparison based If it works by pairwise key comparisons. In place If only a constant number of elements of the input array are ever stored outside

More information

Week 5: Quicksort, Lower bound, Greedy

Week 5: Quicksort, Lower bound, Greedy Week 5: Quicksort, Lower bound, Greedy Agenda: Quicksort: Average case Lower bound for sorting Greedy method 1 Week 5: Quicksort Recall Quicksort: The ideas: Pick one key Compare to others: partition into

More information

Sorting Algorithms. We have already seen: Selection-sort Insertion-sort Heap-sort. We will see: Bubble-sort Merge-sort Quick-sort

Sorting Algorithms. We have already seen: Selection-sort Insertion-sort Heap-sort. We will see: Bubble-sort Merge-sort Quick-sort Sorting Algorithms We have already seen: Selection-sort Insertion-sort Heap-sort We will see: Bubble-sort Merge-sort Quick-sort We will show that: O(n log n) is optimal for comparison based sorting. Bubble-Sort

More information

Amortized analysis. Amortized analysis

Amortized analysis. Amortized analysis In amortized analysis the goal is to bound the worst case time of a sequence of operations on a data-structure. If n operations take T (n) time (worst case), the amortized cost of an operation is T (n)/n.

More information

Outline. 1 Introduction. 3 Quicksort. 4 Analysis. 5 References. Idea. 1 Choose an element x and reorder the array as follows:

Outline. 1 Introduction. 3 Quicksort. 4 Analysis. 5 References. Idea. 1 Choose an element x and reorder the array as follows: Outline Computer Science 331 Quicksort Mike Jacobson Department of Computer Science University of Calgary Lecture #28 1 Introduction 2 Randomized 3 Quicksort Deterministic Quicksort Randomized Quicksort

More information

Randomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th

Randomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th CSE 3500 Algorithms and Complexity Fall 2016 Lecture 10: September 29, 2016 Quick sort: Average Run Time In the last lecture we started analyzing the expected run time of quick sort. Let X = k 1, k 2,...,

More information

Lecture 5: Loop Invariants and Insertion-sort

Lecture 5: Loop Invariants and Insertion-sort Lecture 5: Loop Invariants and Insertion-sort COMS10007 - Algorithms Dr. Christian Konrad 11.01.2019 Dr. Christian Konrad Lecture 5: Loop Invariants and Insertion-sort 1 / 12 Proofs by Induction Structure

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the

More information

Central Algorithmic Techniques. Iterative Algorithms

Central Algorithmic Techniques. Iterative Algorithms Central Algorithmic Techniques Iterative Algorithms Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length;

More information

Lecture 1: Asymptotics, Recurrences, Elementary Sorting

Lecture 1: Asymptotics, Recurrences, Elementary Sorting Lecture 1: Asymptotics, Recurrences, Elementary Sorting Instructor: Outline 1 Introduction to Asymptotic Analysis Rate of growth of functions Comparing and bounding functions: O, Θ, Ω Specifying running

More information

Sorting. Chapter 11. CSE 2011 Prof. J. Elder Last Updated: :11 AM

Sorting. Chapter 11. CSE 2011 Prof. J. Elder Last Updated: :11 AM Sorting Chapter 11-1 - Sorting Ø We have seen the advantage of sorted data representations for a number of applications q Sparse vectors q Maps q Dictionaries Ø Here we consider the problem of how to efficiently

More information

Data selection. Lower complexity bound for sorting

Data selection. Lower complexity bound for sorting Data selection. Lower complexity bound for sorting Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 12 1 Data selection: Quickselect 2 Lower complexity bound for sorting 3 The

More information

Lecture 6 September 21, 2016

Lecture 6 September 21, 2016 ICS 643: Advanced Parallel Algorithms Fall 2016 Lecture 6 September 21, 2016 Prof. Nodari Sitchinava Scribe: Tiffany Eulalio 1 Overview In the last lecture, we wrote a non-recursive summation program and

More information

Outline. Computer Science 331. Cost of Binary Search Tree Operations. Bounds on Height: Worst- and Average-Case

Outline. Computer Science 331. Cost of Binary Search Tree Operations. Bounds on Height: Worst- and Average-Case Outline Computer Science Average Case Analysis: Binary Search Trees Mike Jacobson Department of Computer Science University of Calgary Lecture #7 Motivation and Objective Definition 4 Mike Jacobson (University

More information

Heaps Induction. Heaps. Heaps. Tirgul 6

Heaps Induction. Heaps. Heaps. Tirgul 6 Tirgul 6 Induction A binary heap is a nearly complete binary tree stored in an array object In a max heap, the value of each node that of its children (In a min heap, the value of each node that of its

More information

ASYMPTOTIC COMPLEXITY SEARCHING/SORTING

ASYMPTOTIC COMPLEXITY SEARCHING/SORTING Quotes about loops O! Thou hast damnable iteration and art, indeed, able to corrupt a saint. Shakespeare, Henry IV, Pt I, 1 ii Use not vain repetition, as the heathen do. Matthew V, 48 Your if is the only

More information

Lecture 7: Amortized analysis

Lecture 7: Amortized analysis Lecture 7: Amortized analysis In many applications we want to minimize the time for a sequence of operations. To sum worst-case times for single operations can be overly pessimistic, since it ignores correlation

More information

CS 161 Summer 2009 Homework #2 Sample Solutions

CS 161 Summer 2009 Homework #2 Sample Solutions CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than

More information

Analysis of Algorithm Efficiency. Dr. Yingwu Zhu

Analysis of Algorithm Efficiency. Dr. Yingwu Zhu Analysis of Algorithm Efficiency Dr. Yingwu Zhu Measure Algorithm Efficiency Time efficiency How fast the algorithm runs; amount of time required to accomplish the task Our focus! Space efficiency Amount

More information

Data Structures. Outline. Introduction. Andres Mendez-Vazquez. December 3, Data Manipulation Examples

Data Structures. Outline. Introduction. Andres Mendez-Vazquez. December 3, Data Manipulation Examples Data Structures Introduction Andres Mendez-Vazquez December 3, 2015 1 / 53 Outline 1 What the Course is About? Data Manipulation Examples 2 What is a Good Algorithm? Sorting Example A Naive Algorithm Counting

More information

Reminder of Asymptotic Notation. Inf 2B: Asymptotic notation and Algorithms. Asymptotic notation for Running-time

Reminder of Asymptotic Notation. Inf 2B: Asymptotic notation and Algorithms. Asymptotic notation for Running-time 1 / 18 Reminder of Asymptotic Notation / 18 Inf B: Asymptotic notation and Algorithms Lecture B of ADS thread Let f, g : N! R be functions. We say that: I f is O(g) if there is some n 0 N and some c >

More information

Sorting. CMPS 2200 Fall Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk

Sorting. CMPS 2200 Fall Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk CMPS 2200 Fall 2017 Sorting Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 11/17/17 CMPS 2200 Intro. to Algorithms 1 How fast can we sort? All the sorting algorithms

More information

CS 310 Advanced Data Structures and Algorithms

CS 310 Advanced Data Structures and Algorithms CS 310 Advanced Data Structures and Algorithms Runtime Analysis May 31, 2017 Tong Wang UMass Boston CS 310 May 31, 2017 1 / 37 Topics Weiss chapter 5 What is algorithm analysis Big O, big, big notations

More information

Notes on induction proofs and recursive definitions

Notes on induction proofs and recursive definitions Notes on induction proofs and recursive definitions James Aspnes December 13, 2010 1 Simple induction Most of the proof techniques we ve talked about so far are only really useful for proving a property

More information

Outline. 1 Merging. 2 Merge Sort. 3 Complexity of Sorting. 4 Merge Sort and Other Sorts 2 / 10

Outline. 1 Merging. 2 Merge Sort. 3 Complexity of Sorting. 4 Merge Sort and Other Sorts 2 / 10 Merge Sort 1 / 10 Outline 1 Merging 2 Merge Sort 3 Complexity of Sorting 4 Merge Sort and Other Sorts 2 / 10 Merging Merge sort is based on a simple operation known as merging: combining two ordered arrays

More information

Data Structures and Algorithms

Data Structures and Algorithms Data Structures and Algorithms Spring 2017-2018 Outline 1 Sorting Algorithms (contd.) Outline Sorting Algorithms (contd.) 1 Sorting Algorithms (contd.) Analysis of Quicksort Time to sort array of length

More information

Quiz 1 Solutions. (a) f 1 (n) = 8 n, f 2 (n) = , f 3 (n) = ( 3) lg n. f 2 (n), f 1 (n), f 3 (n) Solution: (b)

Quiz 1 Solutions. (a) f 1 (n) = 8 n, f 2 (n) = , f 3 (n) = ( 3) lg n. f 2 (n), f 1 (n), f 3 (n) Solution: (b) Introduction to Algorithms October 14, 2009 Massachusetts Institute of Technology 6.006 Spring 2009 Professors Srini Devadas and Constantinos (Costis) Daskalakis Quiz 1 Solutions Quiz 1 Solutions Problem

More information

INF2220: algorithms and data structures Series 1

INF2220: algorithms and data structures Series 1 Universitetet i Oslo Institutt for Informatikk I. Yu, D. Karabeg INF2220: algorithms and data structures Series 1 Topic Function growth & estimation of running time, trees (Exercises with hints for solution)

More information

Weighted Activity Selection

Weighted Activity Selection Weighted Activity Selection Problem This problem is a generalization of the activity selection problem that we solvd with a greedy algorithm. Given a set of activities A = {[l, r ], [l, r ],..., [l n,

More information

Review Of Topics. Review: Induction

Review Of Topics. Review: Induction Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case

More information

Problem 5. Use mathematical induction to show that when n is an exact power of two, the solution of the recurrence

Problem 5. Use mathematical induction to show that when n is an exact power of two, the solution of the recurrence A. V. Gerbessiotis CS 610-102 Spring 2014 PS 1 Jan 27, 2014 No points For the remainder of the course Give an algorithm means: describe an algorithm, show that it works as claimed, analyze its worst-case

More information

Sorting. CS 3343 Fall Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk

Sorting. CS 3343 Fall Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk CS 3343 Fall 2010 Sorting Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk 10/7/10 CS 3343 Analysis of Algorithms 1 How fast can we sort? All the sorting algorithms we

More information

Data Structures and Algorithms Chapter 2

Data Structures and Algorithms Chapter 2 1 Data Structures and Algorithms Chapter 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples

More information

Algorithms and Data Structures 2016 Week 5 solutions (Tues 9th - Fri 12th February)

Algorithms and Data Structures 2016 Week 5 solutions (Tues 9th - Fri 12th February) Algorithms and Data Structures 016 Week 5 solutions (Tues 9th - Fri 1th February) 1. Draw the decision tree (under the assumption of all-distinct inputs) Quicksort for n = 3. answer: (of course you should

More information

CS6375: Machine Learning Gautam Kunapuli. Decision Trees

CS6375: Machine Learning Gautam Kunapuli. Decision Trees Gautam Kunapuli Example: Restaurant Recommendation Example: Develop a model to recommend restaurants to users depending on their past dining experiences. Here, the features are cost (x ) and the user s

More information

Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018

Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of

More information

Solution suggestions for examination of Logic, Algorithms and Data Structures,

Solution suggestions for examination of Logic, Algorithms and Data Structures, Department of VT12 Software Engineering and Managment DIT725 (TIG023) Göteborg University, Chalmers 24/5-12 Solution suggestions for examination of Logic, Algorithms and Data Structures, Date : April 26,

More information

CS Analysis of Recursive Algorithms and Brute Force

CS Analysis of Recursive Algorithms and Brute Force CS483-05 Analysis of Recursive Algorithms and Brute Force Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/

More information

Quicksort. Where Average and Worst Case Differ. S.V. N. (vishy) Vishwanathan. University of California, Santa Cruz

Quicksort. Where Average and Worst Case Differ. S.V. N. (vishy) Vishwanathan. University of California, Santa Cruz Quicksort Where Average and Worst Case Differ S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu February 1, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 28 Basic Idea Outline

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)

More information

Lecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan

Lecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Lecture 2 More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Announcements Assignment #1 is posted online It is directly related to MCSS which we will be talking about today or Monday. There are

More information

Asymptotic Running Time of Algorithms

Asymptotic Running Time of Algorithms Asymptotic Complexity: leading term analysis Asymptotic Running Time of Algorithms Comparing searching and sorting algorithms so far: Count worst-case of comparisons as function of array size. Drop lower-order

More information

Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013

Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013 /4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop

More information

Slides for CIS 675. Huffman Encoding, 1. Huffman Encoding, 2. Huffman Encoding, 3. Encoding 1. DPV Chapter 5, Part 2. Encoding 2

Slides for CIS 675. Huffman Encoding, 1. Huffman Encoding, 2. Huffman Encoding, 3. Encoding 1. DPV Chapter 5, Part 2. Encoding 2 Huffman Encoding, 1 EECS Slides for CIS 675 DPV Chapter 5, Part 2 Jim Royer October 13, 2009 A toy example: Suppose our alphabet is { A, B, C, D }. Suppose T is a text of 130 million characters. What is

More information

Algorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE

Algorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE Algorithms ROBERT SEDGEWICK KEVIN WAYNE 2.2 MERGESORT Algorithms F O U R T H E D I T I O N mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE http://algs4.cs.princeton.edu

More information

Module 1: Analyzing the Efficiency of Algorithms

Module 1: Analyzing the Efficiency of Algorithms Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?

More information

Searching Algorithms. CSE21 Winter 2017, Day 3 (B00), Day 2 (A00) January 13, 2017

Searching Algorithms. CSE21 Winter 2017, Day 3 (B00), Day 2 (A00) January 13, 2017 Searching Algorithms CSE21 Winter 2017, Day 3 (B00), Day 2 (A00) January 13, 2017 Selection Sort (MinSort) Pseudocode Rosen page 203, exercises 41-42 procedure selection sort(a 1, a 2,..., a n : real numbers

More information

Complexity Theory Part I

Complexity Theory Part I Complexity Theory Part I Problem Problem Set Set 77 due due right right now now using using a late late period period The Limits of Computability EQ TM EQ TM co-re R RE L D ADD L D HALT A TM HALT A TM

More information

Linear Selection and Linear Sorting

Linear Selection and Linear Sorting Analysis of Algorithms Linear Selection and Linear Sorting If more than one question appears correct, choose the more specific answer, unless otherwise instructed. Concept: linear selection 1. Suppose

More information

Lecture 4. Quicksort

Lecture 4. Quicksort Lecture 4. Quicksort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018 Networking

More information

1 Quick Sort LECTURE 7. OHSU/OGI (Winter 2009) ANALYSIS AND DESIGN OF ALGORITHMS

1 Quick Sort LECTURE 7. OHSU/OGI (Winter 2009) ANALYSIS AND DESIGN OF ALGORITHMS OHSU/OGI (Winter 2009) CS532 ANALYSIS AND DESIGN OF ALGORITHMS LECTURE 7 1 Quick Sort QuickSort 1 is a classic example of divide and conquer. The hard work is to rearrange the elements of the array A[1..n]

More information

searching algorithms

searching algorithms searching algorithms learning objectives algorithms your software system software hardware learn what the searching problem is about learn two algorithms for solving this problem learn the importance of

More information

Data Structures and Algorithms " Search Trees!!

Data Structures and Algorithms  Search Trees!! Data Structures and Algorithms " Search Trees!! Outline" Binary Search Trees! AVL Trees! (2,4) Trees! 2 Binary Search Trees! "! < 6 2 > 1 4 = 8 9 Ordered Dictionaries" Keys are assumed to come from a total

More information

CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms

CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions

More information

Limitations of Algorithm Power

Limitations of Algorithm Power Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying

More information

CprE 281: Digital Logic

CprE 281: Digital Logic CprE 281: Digital Logic Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs/classes/ Synchronous Sequential Circuits Basic Design Steps CprE 281: Digital Logic Iowa State University, Ames,

More information

Data Structures and Algorithms

Data Structures and Algorithms Data Structures and Algorithms Autumn 2018-2019 Outline 1 Algorithm Analysis (contd.) Outline Algorithm Analysis (contd.) 1 Algorithm Analysis (contd.) Growth Rates of Some Commonly Occurring Functions

More information

Problem-Solving via Search Lecture 3

Problem-Solving via Search Lecture 3 Lecture 3 What is a search problem? How do search algorithms work and how do we evaluate their performance? 1 Agenda An example task Problem formulation Infrastructure for search algorithms Complexity

More information

Lecture 1 - Preliminaries

Lecture 1 - Preliminaries Advanced Algorithms Floriano Zini Free University of Bozen-Bolzano Faculty of Computer Science Academic Year 2013-2014 Lecture 1 - Preliminaries 1 Typography vs algorithms Johann Gutenberg (c. 1398 February

More information

Fundamental Algorithms

Fundamental Algorithms Fundamental Algorithms Chapter 5: Searching Michael Bader Winter 2014/15 Chapter 5: Searching, Winter 2014/15 1 Searching Definition (Search Problem) Input: a sequence or set A of n elements (objects)

More information

Lower and Upper Bound Theory

Lower and Upper Bound Theory Lower and Upper Bound Theory Adnan YAZICI Dept. of Computer Engineering Middle East Technical Univ. Ankara - TURKEY 1 Lower and Upper Bound Theory How fast can we sort? Most of the sorting algorithms are

More information

16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ]

16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ] 418 16. Binary Search Trees [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1-12.3] 419 Dictionary implementation Hashing: implementation of dictionaries with expected very fast access times. Disadvantages

More information

V. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005

V. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005 V. Adamchi Recurrences Victor Adamchi Fall of 00 Plan Multiple roots. More on multiple roots. Inhomogeneous equations 3. Divide-and-conquer recurrences In the previous lecture we have showed that if the

More information

26 Chapter 4 Classification

26 Chapter 4 Classification 26 Chapter 4 Classification The preceding tree cannot be simplified. 2. Consider the training examples shown in Table 4.1 for a binary classification problem. Table 4.1. Data set for Exercise 2. Customer

More information

16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ]

16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ] 423 16. Binary Search Trees [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1-12.3] Dictionary implementation 424 Hashing: implementation of dictionaries with expected very fast access times. Disadvantages

More information

Fundamental Algorithms

Fundamental Algorithms Chapter 2: Sorting, Winter 2018/19 1 Fundamental Algorithms Chapter 2: Sorting Jan Křetínský Winter 2018/19 Chapter 2: Sorting, Winter 2018/19 2 Part I Simple Sorts Chapter 2: Sorting, Winter 2018/19 3

More information

Fundamental Algorithms

Fundamental Algorithms Fundamental Algorithms Chapter 2: Sorting Harald Räcke Winter 2015/16 Chapter 2: Sorting, Winter 2015/16 1 Part I Simple Sorts Chapter 2: Sorting, Winter 2015/16 2 The Sorting Problem Definition Sorting

More information

CSCE 222 Discrete Structures for Computing

CSCE 222 Discrete Structures for Computing CSCE 222 Discrete Structures for Computing Algorithms Dr. Philip C. Ritchey Introduction An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem.

More information

Computer Algorithms CISC4080 CIS, Fordham Univ. Outline. Last class. Instructor: X. Zhang Lecture 2

Computer Algorithms CISC4080 CIS, Fordham Univ. Outline. Last class. Instructor: X. Zhang Lecture 2 Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2 Outline Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive formula

More information

Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2

Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2 Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2 Outline Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive formula

More information

Asymptotic redundancy and prolixity

Asymptotic redundancy and prolixity Asymptotic redundancy and prolixity Yuval Dagan, Yuval Filmus, and Shay Moran April 6, 2017 Abstract Gallager (1978) considered the worst-case redundancy of Huffman codes as the maximum probability tends

More information

Solutions. Problem 1: Suppose a polynomial in n of degree d has the form

Solutions. Problem 1: Suppose a polynomial in n of degree d has the form Assignment 1 1. Problem 3-1 on p. 57 2. Problem 3-2 on p. 58 3. Problem 4-5 on p. 86 4. Problem 6-1 on p. 142 5. Problem 7-4 on p. 162 6. Prove correctness (including halting) of SelectionSort (use loop

More information

Search Trees. Chapter 10. CSE 2011 Prof. J. Elder Last Updated: :52 AM

Search Trees. Chapter 10. CSE 2011 Prof. J. Elder Last Updated: :52 AM Search Trees Chapter 1 < 6 2 > 1 4 = 8 9-1 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 2 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 3 - Binary Search Trees Ø A binary search

More information

CSE 202 Homework 4 Matthias Springer, A

CSE 202 Homework 4 Matthias Springer, A CSE 202 Homework 4 Matthias Springer, A99500782 1 Problem 2 Basic Idea PERFECT ASSEMBLY N P: a permutation P of s i S is a certificate that can be checked in polynomial time by ensuring that P = S, and

More information

Algorithms and Their Complexity

Algorithms and Their Complexity CSCE 222 Discrete Structures for Computing David Kebo Houngninou Algorithms and Their Complexity Chapter 3 Algorithm An algorithm is a finite sequence of steps that solves a problem. Computational complexity

More information

Time Analysis of Sorting and Searching Algorithms

Time Analysis of Sorting and Searching Algorithms Time Analysis of Sorting and Searching Algorithms CSE21 Winter 2017, Day 5 (B00), Day 3 (A00) January 20, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Big-O : How to determine? Is f(n) big-o of g(n)?

More information

CS6901: review of Theory of Computation and Algorithms

CS6901: review of Theory of Computation and Algorithms CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational

More information

Introduction to Divide and Conquer

Introduction to Divide and Conquer Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward

More information

Optimal Tree-decomposition Balancing and Reachability on Low Treewidth Graphs

Optimal Tree-decomposition Balancing and Reachability on Low Treewidth Graphs Optimal Tree-decomposition Balancing and Reachability on Low Treewidth Graphs Krishnendu Chatterjee Rasmus Ibsen-Jensen Andreas Pavlogiannis IST Austria Abstract. We consider graphs with n nodes together

More information

CSCI 3110 Assignment 6 Solutions

CSCI 3110 Assignment 6 Solutions CSCI 3110 Assignment 6 Solutions December 5, 2012 2.4 (4 pts) Suppose you are choosing between the following three algorithms: 1. Algorithm A solves problems by dividing them into five subproblems of half

More information

Problem Set 2 Solutions

Problem Set 2 Solutions Introduction to Algorithms October 1, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Piotr Indyk and Charles E. Leiserson Handout 9 Problem Set 2 Solutions Reading: Chapters 5-9,

More information

Recap: Prefix Sums. Given A: set of n integers Find B: prefix sums 1 / 86

Recap: Prefix Sums. Given A: set of n integers Find B: prefix sums 1 / 86 Recap: Prefix Sums Given : set of n integers Find B: prefix sums : 3 1 1 7 2 5 9 2 4 3 3 B: 3 4 5 12 14 19 28 30 34 37 40 1 / 86 Recap: Parallel Prefix Sums Recursive algorithm Recursively computes sums

More information

Search Trees. EECS 2011 Prof. J. Elder Last Updated: 24 March 2015

Search Trees. EECS 2011 Prof. J. Elder Last Updated: 24 March 2015 Search Trees < 6 2 > 1 4 = 8 9-1 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 2 - Learning Outcomes Ø From this lecture, you should be able to: q Define the properties of a binary search

More information

Notes on Machine Learning for and

Notes on Machine Learning for and Notes on Machine Learning for 16.410 and 16.413 (Notes adapted from Tom Mitchell and Andrew Moore.) Learning = improving with experience Improve over task T (e.g, Classification, control tasks) with respect

More information

3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need?

3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need? 3. Algorithms We will study algorithms to solve many different types of problems such as finding the largest of a sequence of numbers sorting a sequence of numbers finding the shortest path between two

More information

PARALLEL DIGITAL-ANALOG CONVERTERS

PARALLEL DIGITAL-ANALOG CONVERTERS CMOS Analog IC Design Page 10.2-1 10.2 - PARALLEL DIGITAL-ANALOG CONVERTERS CLASSIFICATION OF DIGITAL-ANALOG CONVERTERS CMOS Analog IC Design Page 10.2-2 CURRENT SCALING DIGITAL-ANALOG CONVERTERS GENERAL

More information

Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees!

Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees! Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees! Summary! Input Knowledge representation! Preparing data for learning! Input: Concept, Instances, Attributes"

More information

Math 391: Midterm 1.0 Spring 2016

Math 391: Midterm 1.0 Spring 2016 Math 391: Midterm 1.0 Spring 2016 James Skon skonjp@kenyon.edu Test date: October 5 Submission Instructions: Give all your assumptions. Show all your work. Be as complete as possible. Grading Problem 1

More information

Space Complexity of Algorithms

Space Complexity of Algorithms Space Complexity of Algorithms So far we have considered only the time necessary for a computation Sometimes the size of the memory necessary for the computation is more critical. The amount of memory

More information

Binary Decision Diagrams. Graphs. Boolean Functions

Binary Decision Diagrams. Graphs. Boolean Functions Binary Decision Diagrams Graphs Binary Decision Diagrams (BDDs) are a class of graphs that can be used as data structure for compactly representing boolean functions. BDDs were introduced by R. Bryant

More information

SIGNAL COMPRESSION Lecture 7. Variable to Fix Encoding

SIGNAL COMPRESSION Lecture 7. Variable to Fix Encoding SIGNAL COMPRESSION Lecture 7 Variable to Fix Encoding 1. Tunstall codes 2. Petry codes 3. Generalized Tunstall codes for Markov sources (a presentation of the paper by I. Tabus, G. Korodi, J. Rissanen.

More information

Machine Learning 2nd Edi7on

Machine Learning 2nd Edi7on Lecture Slides for INTRODUCTION TO Machine Learning 2nd Edi7on CHAPTER 9: Decision Trees ETHEM ALPAYDIN The MIT Press, 2010 Edited and expanded for CS 4641 by Chris Simpkins alpaydin@boun.edu.tr h1p://www.cmpe.boun.edu.tr/~ethem/i2ml2e

More information

Learning Decision Trees

Learning Decision Trees Learning Decision Trees Machine Learning Fall 2018 Some slides from Tom Mitchell, Dan Roth and others 1 Key issues in machine learning Modeling How to formulate your problem as a machine learning problem?

More information

arxiv: v2 [cs.ds] 3 Oct 2017

arxiv: v2 [cs.ds] 3 Oct 2017 Orthogonal Vectors Indexing Isaac Goldstein 1, Moshe Lewenstein 1, and Ely Porat 1 1 Bar-Ilan University, Ramat Gan, Israel {goldshi,moshe,porately}@cs.biu.ac.il arxiv:1710.00586v2 [cs.ds] 3 Oct 2017 Abstract

More information

b + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d

b + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we

More information

Decision Trees. Data Science: Jordan Boyd-Graber University of Maryland MARCH 11, Data Science: Jordan Boyd-Graber UMD Decision Trees 1 / 1

Decision Trees. Data Science: Jordan Boyd-Graber University of Maryland MARCH 11, Data Science: Jordan Boyd-Graber UMD Decision Trees 1 / 1 Decision Trees Data Science: Jordan Boyd-Graber University of Maryland MARCH 11, 2018 Data Science: Jordan Boyd-Graber UMD Decision Trees 1 / 1 Roadmap Classification: machines labeling data for us Last

More information

Lecture 2. Fundamentals of the Analysis of Algorithm Efficiency

Lecture 2. Fundamentals of the Analysis of Algorithm Efficiency Lecture 2 Fundamentals of the Analysis of Algorithm Efficiency 1 Lecture Contents 1. Analysis Framework 2. Asymptotic Notations and Basic Efficiency Classes 3. Mathematical Analysis of Nonrecursive Algorithms

More information

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabás Póczos Contents Decision Trees: Definition + Motivation Algorithm for Learning Decision Trees Entropy, Mutual Information, Information

More information

Mathematical Background. Unsigned binary numbers. Powers of 2. Logs and exponents. Mathematical Background. Today, we will review:

Mathematical Background. Unsigned binary numbers. Powers of 2. Logs and exponents. Mathematical Background. Today, we will review: Mathematical Background Mathematical Background CSE 373 Data Structures Today, we will review: Logs and eponents Series Recursion Motivation for Algorithm Analysis 5 January 007 CSE 373 - Math Background

More information