Problem. Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26
|
|
- Cameron Woods
- 6 years ago
- Views:
Transcription
1 Binary Search
2 Introduction
3 Problem Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26
4 Strategy 1: Random Search Randomly select a page until the page containing the word is found. 4 / 26
5 Strategy 1: Random Search Randomly select a page until the page containing the word is found. Very inefficient, no guarantee that the correct page is chosen. Does not detect if word is not in the dictionary. 4 / 26
6 Strategy 2: Linear Search Check all pages sequentially starting at page 1. 5 / 26
7 Strategy 2: Linear Search Check all pages sequentially starting at page 1. Acceptable efficiency, but... strategy is not using all known properties. 5 / 26
8 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? 6 / 26
9 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. < < < < < < < < < 6 / 26
10 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. 6 / 26
11 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. 6 / 26
12 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. Question: Which page should be selected? 6 / 26
13 Strategy 3: Binary Search Question: What is the main property of words in a dictionary? Words are sorted. After looking on a page, we can say if word is on an earlier or later page. Select a page. If the word is smaller, look on an earlier pages. If the word is larger, look on a later pages. Question: Which page should be selected? The middle!? Why? 6 / 26
14 Algorithm Input: A sorted array A Some value k / 26
15 Algorithm 1. Set l := 0 and r := A.Length While l r l r / 26
16 Algorithm 2.1. Set m := l+r 2. l m r / 26
17 Algorithm 2.2. If A[m] < k Then Set l := m + 1. m l r / 26
18 Algorithm 2.3. If A[m] > k Then Set r := m 1. l r m / 26
19 Algorithm 2.3. If A[m] = k Then Return m. m r / 26
20 Algorithm Binary Search Input: A sorted array A Some value k 1. Set l := 0 and r := A.Length While l r 2.1 Set m := l+r If A[m] < k Then Set l := m If A[m] > k Then Set r := m If A[m] = k Then Return m. 3. Return 1. 8 / 26
21 Correctness and Invariant
22 Correctness What means Binary Search is correct? 10 / 26
23 Correctness What means Binary Search is correct? Theorem The algorithm Binary Search returns the index of k in A if k A, otherwise it returns 1. We have to show two cases k A k / A We will show the second case first. 10 / 26
24 Proof of Correctness k / A Some notation: l i and r i denote the value of l and r at the beginning of the i-th loop iteration, i. e., l 0 = 0 and r 0 = A.Length 1. m i denotes the value of m assigned in the i-th loop iteration. 11 / 26
25 Proof of Correctness k / A Some notation: l i and r i denote the value of l and r at the beginning of the i-th loop iteration, i. e., l 0 = 0 and r 0 = A.Length 1. m i denotes the value of m assigned in the i-th loop iteration. We have to show that we reach line 3. Because k / A, we never run line 2.4. Because l i m i r i, r i+1 l i+1 < r i l i (line 2.2 and line 2.3). Thus, the algorithm reaches line 3 after a finite amount of iterations. 11 / 26
26 Proof of Correctness k A How do we show that we find the right index? 12 / 26
27 Proof of Correctness k A How do we show that we find the right index? We show an invariant. 12 / 26
28 Proof of Correctness k A How do we show that we find the right index? We show an invariant. Invariant The invariant of a loop is a property that is true every time before and after every iteration of the loop. 12 / 26
29 Proof of Correctness k A How do we show that we find the right index? We show an invariant. Invariant The invariant of a loop is a property that is true every time before and after every iteration of the loop. Invariants often lead to the correctness of an algorithm. The most common way to prove them is induction. 12 / 26
30 Proof of Correctness k A Invariant for Binary Search 13 / 26
31 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i 13 / 26
32 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length / 26
33 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length 1. Inductive Step By induction, l i Ind(k) r i. If k = A[m i ], the algorithm returns m i. Done. If k < A[m i ], them Ind(k) < m i. Thus, l i+1 = l i Ind(k) r i+1 = m i 1 < m i r i. If k > A[m i ], them Ind(k) > m i. Thus, l i m i < m i + 1 = l i+1 Ind(k) r i+1 = r i. 13 / 26
34 Proof of Correctness k A Invariant for Binary Search If k A, then l i Ind(k) r i. (Ind(k) is the index of k in A) r i+1 l i+1 < r i l i Base Case Is true, because l 0 = 0 and r 0 = A.Length 1. Inductive Step By induction, l i Ind(k) r i. If k = A[m i ], the algorithm returns m i. Done. If k < A[m i ], them Ind(k) < m i. Thus, l i+1 = l i Ind(k) r i+1 = m i 1 < m i r i. If k > A[m i ], them Ind(k) > m i. Thus, l i m i < m i + 1 = l i+1 Ind(k) r i+1 = r i. Because r i+1 l i+1 < r i l i, there is an iteration j with l j = Ind(k) = r j. 13 / 26
35 Runtime
36 Runtime How fast is Binary Search? 15 / 26
37 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 15 / 26
38 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 For which j is r j l j 1? 15 / 26
39 Runtime How fast is Binary Search? We count number of iterations. r 0 l 0 = n 1 r i+1 l i+1 (r i l i )/2 For which j is r j l j 1? r j l j 1 2 (r j 1 l j 1 ) 1 4 (r j 2 l j 2 ). 2 j (r 0 l 0 ) = 2 j (n 1) 1 log 2 n j 15 / 26
40 Runtime Theorem The algorithm Binary Search requires at most log 2 (n+1) iterations, where n = A. 16 / 26
41 Logarithmic Runtime If A = 2 n we need approx. n iterations. Also, Input size (n) Runtime (log 2 n) 1, ,000, ,000,000, atoms in the universe / 26
42 Lower Time Bound Is there a faster way than Binary Search? We allow a free preprocessing of A. We want to know if k is in A and where. 18 / 26
43 Lower Time Bound Is there a faster way than Binary Search? We allow a free preprocessing of A. We want to know if k is in A and where. Theorem In general, finding an element in an array cannot be done in less than logarithmic time. 18 / 26
44 Decision Tree Model of computation: comparison model only operations allowed are comparisons (<,, >,, =, ) time cost is number of comparisons 19 / 26
45 Decision Tree Model of computation: comparison model only operations allowed are comparisons (<,, >,, =, ) time cost is number of comparisons Decision Tree algorithm is tree of comparisons internal nodes are comparisons leafs are outcomes of the algorithm 19 / 26
46 Decision Tree Example Binary Search for A = 3 (For simplicity we ignore the case A[i] = k.) No A[1] < k? Yes A[0] < k? A[2] < k? No Yes No Yes k A[0] A[0] < k A[1] A[1] < k A[2] A[2] < k 20 / 26
47 Decision Tree Example Linear Search for A = 3 (For simplicity we ignore the case A[i] = k.) No A[0] < k? Yes k A[0] No A[1] < k? Yes A[0] < k A[1] No A[2] < k? Yes A[1] < k A[2] A[2] < k 21 / 26
48 Decision Tree vs. Algorithm + Proof Decision Tree internal node leaf root-to-leaf path path length height of tree Algorithm binary decision found answer algorithm execution running time worst case running time 22 / 26
49 Decision Tree vs. Algorithm + Proof Decision Tree internal node leaf root-to-leaf path path length height of tree Algorithm binary decision found answer algorithm execution running time worst case running time Proof of theorem: Decision tree is binary Tree contains each possible answer as leaf, i. e., n + 1 leafs Height log 2 (number of leafs). 22 / 26
50 Other Variants
51 Finding 01 in a Bit Array Given a bit array A where A[0] = 0 and A[n 1] = 1. Find an i such that A[i] = 0 and A[i + 1] = / 26
52 Finding 01 in a Bit Array Given a bit array A where A[0] = 0 and A[n 1] = 1. Find an i such that A[i] = 0 and A[i + 1] = 1. Using binary search: If A[m] = 1, search left. If A[m] = 0, search right. Invariant: A[l] = 0 and A[r] = / 26
53 Analog Digital Converter Convert an analog input (voltage) into a digital representation. Logic creates digital value (starting with most significant bit). Digital-to-analog converter (DAC) creates the analog equivalent U d. Comparator takes both analog siganls an outputs if U d > U in. If U d > U in, logic resets bit. Logic DAC Com. U in 25 / 26
54 Nodes in a Complete Binary Tree Complete binary tree All layers (except lowest) are full. Lowest layer is filled from left to right. Question: How many nodes are in the tree? 26 / 26
55 Nodes in a Complete Binary Tree Naive solution Count all nodes. Runtime: n 26 / 26
56 Nodes in a Complete Binary Tree Using binary search Determine height h l of left subtree (left side). Determine height h r of right subtree (left side). If h l = h r, lowest layer of left subtree is full (continue with right subtree). Otherwise, lowest layer of right subtree is empty (continue with left subtree). Runtime: log 2 n 26 / 26
57 Nodes in a Complete Binary Tree Using binary search Determine height h l of left subtree (left side). Determine height h r of right subtree (left side). If h l = h r, lowest layer of left subtree is full (continue with right subtree). Otherwise, lowest layer of right subtree is empty (continue with left subtree). Runtime: log 2 n 26 / 26
Sorting algorithms. Sorting algorithms
Properties of sorting algorithms A sorting algorithm is Comparison based If it works by pairwise key comparisons. In place If only a constant number of elements of the input array are ever stored outside
More informationWeek 5: Quicksort, Lower bound, Greedy
Week 5: Quicksort, Lower bound, Greedy Agenda: Quicksort: Average case Lower bound for sorting Greedy method 1 Week 5: Quicksort Recall Quicksort: The ideas: Pick one key Compare to others: partition into
More informationSorting Algorithms. We have already seen: Selection-sort Insertion-sort Heap-sort. We will see: Bubble-sort Merge-sort Quick-sort
Sorting Algorithms We have already seen: Selection-sort Insertion-sort Heap-sort We will see: Bubble-sort Merge-sort Quick-sort We will show that: O(n log n) is optimal for comparison based sorting. Bubble-Sort
More informationAmortized analysis. Amortized analysis
In amortized analysis the goal is to bound the worst case time of a sequence of operations on a data-structure. If n operations take T (n) time (worst case), the amortized cost of an operation is T (n)/n.
More informationOutline. 1 Introduction. 3 Quicksort. 4 Analysis. 5 References. Idea. 1 Choose an element x and reorder the array as follows:
Outline Computer Science 331 Quicksort Mike Jacobson Department of Computer Science University of Calgary Lecture #28 1 Introduction 2 Randomized 3 Quicksort Deterministic Quicksort Randomized Quicksort
More informationRandomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th
CSE 3500 Algorithms and Complexity Fall 2016 Lecture 10: September 29, 2016 Quick sort: Average Run Time In the last lecture we started analyzing the expected run time of quick sort. Let X = k 1, k 2,...,
More informationLecture 5: Loop Invariants and Insertion-sort
Lecture 5: Loop Invariants and Insertion-sort COMS10007 - Algorithms Dr. Christian Konrad 11.01.2019 Dr. Christian Konrad Lecture 5: Loop Invariants and Insertion-sort 1 / 12 Proofs by Induction Structure
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the
More informationCentral Algorithmic Techniques. Iterative Algorithms
Central Algorithmic Techniques Iterative Algorithms Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length;
More informationLecture 1: Asymptotics, Recurrences, Elementary Sorting
Lecture 1: Asymptotics, Recurrences, Elementary Sorting Instructor: Outline 1 Introduction to Asymptotic Analysis Rate of growth of functions Comparing and bounding functions: O, Θ, Ω Specifying running
More informationSorting. Chapter 11. CSE 2011 Prof. J. Elder Last Updated: :11 AM
Sorting Chapter 11-1 - Sorting Ø We have seen the advantage of sorted data representations for a number of applications q Sparse vectors q Maps q Dictionaries Ø Here we consider the problem of how to efficiently
More informationData selection. Lower complexity bound for sorting
Data selection. Lower complexity bound for sorting Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 12 1 Data selection: Quickselect 2 Lower complexity bound for sorting 3 The
More informationLecture 6 September 21, 2016
ICS 643: Advanced Parallel Algorithms Fall 2016 Lecture 6 September 21, 2016 Prof. Nodari Sitchinava Scribe: Tiffany Eulalio 1 Overview In the last lecture, we wrote a non-recursive summation program and
More informationOutline. Computer Science 331. Cost of Binary Search Tree Operations. Bounds on Height: Worst- and Average-Case
Outline Computer Science Average Case Analysis: Binary Search Trees Mike Jacobson Department of Computer Science University of Calgary Lecture #7 Motivation and Objective Definition 4 Mike Jacobson (University
More informationHeaps Induction. Heaps. Heaps. Tirgul 6
Tirgul 6 Induction A binary heap is a nearly complete binary tree stored in an array object In a max heap, the value of each node that of its children (In a min heap, the value of each node that of its
More informationASYMPTOTIC COMPLEXITY SEARCHING/SORTING
Quotes about loops O! Thou hast damnable iteration and art, indeed, able to corrupt a saint. Shakespeare, Henry IV, Pt I, 1 ii Use not vain repetition, as the heathen do. Matthew V, 48 Your if is the only
More informationLecture 7: Amortized analysis
Lecture 7: Amortized analysis In many applications we want to minimize the time for a sequence of operations. To sum worst-case times for single operations can be overly pessimistic, since it ignores correlation
More informationCS 161 Summer 2009 Homework #2 Sample Solutions
CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than
More informationAnalysis of Algorithm Efficiency. Dr. Yingwu Zhu
Analysis of Algorithm Efficiency Dr. Yingwu Zhu Measure Algorithm Efficiency Time efficiency How fast the algorithm runs; amount of time required to accomplish the task Our focus! Space efficiency Amount
More informationData Structures. Outline. Introduction. Andres Mendez-Vazquez. December 3, Data Manipulation Examples
Data Structures Introduction Andres Mendez-Vazquez December 3, 2015 1 / 53 Outline 1 What the Course is About? Data Manipulation Examples 2 What is a Good Algorithm? Sorting Example A Naive Algorithm Counting
More informationReminder of Asymptotic Notation. Inf 2B: Asymptotic notation and Algorithms. Asymptotic notation for Running-time
1 / 18 Reminder of Asymptotic Notation / 18 Inf B: Asymptotic notation and Algorithms Lecture B of ADS thread Let f, g : N! R be functions. We say that: I f is O(g) if there is some n 0 N and some c >
More informationSorting. CMPS 2200 Fall Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 2200 Fall 2017 Sorting Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 11/17/17 CMPS 2200 Intro. to Algorithms 1 How fast can we sort? All the sorting algorithms
More informationCS 310 Advanced Data Structures and Algorithms
CS 310 Advanced Data Structures and Algorithms Runtime Analysis May 31, 2017 Tong Wang UMass Boston CS 310 May 31, 2017 1 / 37 Topics Weiss chapter 5 What is algorithm analysis Big O, big, big notations
More informationNotes on induction proofs and recursive definitions
Notes on induction proofs and recursive definitions James Aspnes December 13, 2010 1 Simple induction Most of the proof techniques we ve talked about so far are only really useful for proving a property
More informationOutline. 1 Merging. 2 Merge Sort. 3 Complexity of Sorting. 4 Merge Sort and Other Sorts 2 / 10
Merge Sort 1 / 10 Outline 1 Merging 2 Merge Sort 3 Complexity of Sorting 4 Merge Sort and Other Sorts 2 / 10 Merging Merge sort is based on a simple operation known as merging: combining two ordered arrays
More informationData Structures and Algorithms
Data Structures and Algorithms Spring 2017-2018 Outline 1 Sorting Algorithms (contd.) Outline Sorting Algorithms (contd.) 1 Sorting Algorithms (contd.) Analysis of Quicksort Time to sort array of length
More informationQuiz 1 Solutions. (a) f 1 (n) = 8 n, f 2 (n) = , f 3 (n) = ( 3) lg n. f 2 (n), f 1 (n), f 3 (n) Solution: (b)
Introduction to Algorithms October 14, 2009 Massachusetts Institute of Technology 6.006 Spring 2009 Professors Srini Devadas and Constantinos (Costis) Daskalakis Quiz 1 Solutions Quiz 1 Solutions Problem
More informationINF2220: algorithms and data structures Series 1
Universitetet i Oslo Institutt for Informatikk I. Yu, D. Karabeg INF2220: algorithms and data structures Series 1 Topic Function growth & estimation of running time, trees (Exercises with hints for solution)
More informationWeighted Activity Selection
Weighted Activity Selection Problem This problem is a generalization of the activity selection problem that we solvd with a greedy algorithm. Given a set of activities A = {[l, r ], [l, r ],..., [l n,
More informationReview Of Topics. Review: Induction
Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case
More informationProblem 5. Use mathematical induction to show that when n is an exact power of two, the solution of the recurrence
A. V. Gerbessiotis CS 610-102 Spring 2014 PS 1 Jan 27, 2014 No points For the remainder of the course Give an algorithm means: describe an algorithm, show that it works as claimed, analyze its worst-case
More informationSorting. CS 3343 Fall Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk
CS 3343 Fall 2010 Sorting Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk 10/7/10 CS 3343 Analysis of Algorithms 1 How fast can we sort? All the sorting algorithms we
More informationData Structures and Algorithms Chapter 2
1 Data Structures and Algorithms Chapter 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
More informationAlgorithms and Data Structures 2016 Week 5 solutions (Tues 9th - Fri 12th February)
Algorithms and Data Structures 016 Week 5 solutions (Tues 9th - Fri 1th February) 1. Draw the decision tree (under the assumption of all-distinct inputs) Quicksort for n = 3. answer: (of course you should
More informationCS6375: Machine Learning Gautam Kunapuli. Decision Trees
Gautam Kunapuli Example: Restaurant Recommendation Example: Develop a model to recommend restaurants to users depending on their past dining experiences. Here, the features are cost (x ) and the user s
More informationLecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018
CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of
More informationSolution suggestions for examination of Logic, Algorithms and Data Structures,
Department of VT12 Software Engineering and Managment DIT725 (TIG023) Göteborg University, Chalmers 24/5-12 Solution suggestions for examination of Logic, Algorithms and Data Structures, Date : April 26,
More informationCS Analysis of Recursive Algorithms and Brute Force
CS483-05 Analysis of Recursive Algorithms and Brute Force Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/
More informationQuicksort. Where Average and Worst Case Differ. S.V. N. (vishy) Vishwanathan. University of California, Santa Cruz
Quicksort Where Average and Worst Case Differ S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu February 1, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 28 Basic Idea Outline
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)
More informationLecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan
Lecture 2 More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Announcements Assignment #1 is posted online It is directly related to MCSS which we will be talking about today or Monday. There are
More informationAsymptotic Running Time of Algorithms
Asymptotic Complexity: leading term analysis Asymptotic Running Time of Algorithms Comparing searching and sorting algorithms so far: Count worst-case of comparisons as function of array size. Drop lower-order
More informationBig O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013
/4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop
More informationSlides for CIS 675. Huffman Encoding, 1. Huffman Encoding, 2. Huffman Encoding, 3. Encoding 1. DPV Chapter 5, Part 2. Encoding 2
Huffman Encoding, 1 EECS Slides for CIS 675 DPV Chapter 5, Part 2 Jim Royer October 13, 2009 A toy example: Suppose our alphabet is { A, B, C, D }. Suppose T is a text of 130 million characters. What is
More informationAlgorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE
Algorithms ROBERT SEDGEWICK KEVIN WAYNE 2.2 MERGESORT Algorithms F O U R T H E D I T I O N mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE http://algs4.cs.princeton.edu
More informationModule 1: Analyzing the Efficiency of Algorithms
Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?
More informationSearching Algorithms. CSE21 Winter 2017, Day 3 (B00), Day 2 (A00) January 13, 2017
Searching Algorithms CSE21 Winter 2017, Day 3 (B00), Day 2 (A00) January 13, 2017 Selection Sort (MinSort) Pseudocode Rosen page 203, exercises 41-42 procedure selection sort(a 1, a 2,..., a n : real numbers
More informationComplexity Theory Part I
Complexity Theory Part I Problem Problem Set Set 77 due due right right now now using using a late late period period The Limits of Computability EQ TM EQ TM co-re R RE L D ADD L D HALT A TM HALT A TM
More informationLinear Selection and Linear Sorting
Analysis of Algorithms Linear Selection and Linear Sorting If more than one question appears correct, choose the more specific answer, unless otherwise instructed. Concept: linear selection 1. Suppose
More informationLecture 4. Quicksort
Lecture 4. Quicksort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018 Networking
More information1 Quick Sort LECTURE 7. OHSU/OGI (Winter 2009) ANALYSIS AND DESIGN OF ALGORITHMS
OHSU/OGI (Winter 2009) CS532 ANALYSIS AND DESIGN OF ALGORITHMS LECTURE 7 1 Quick Sort QuickSort 1 is a classic example of divide and conquer. The hard work is to rearrange the elements of the array A[1..n]
More informationsearching algorithms
searching algorithms learning objectives algorithms your software system software hardware learn what the searching problem is about learn two algorithms for solving this problem learn the importance of
More informationData Structures and Algorithms " Search Trees!!
Data Structures and Algorithms " Search Trees!! Outline" Binary Search Trees! AVL Trees! (2,4) Trees! 2 Binary Search Trees! "! < 6 2 > 1 4 = 8 9 Ordered Dictionaries" Keys are assumed to come from a total
More informationCS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationLimitations of Algorithm Power
Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying
More informationCprE 281: Digital Logic
CprE 281: Digital Logic Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs/classes/ Synchronous Sequential Circuits Basic Design Steps CprE 281: Digital Logic Iowa State University, Ames,
More informationData Structures and Algorithms
Data Structures and Algorithms Autumn 2018-2019 Outline 1 Algorithm Analysis (contd.) Outline Algorithm Analysis (contd.) 1 Algorithm Analysis (contd.) Growth Rates of Some Commonly Occurring Functions
More informationProblem-Solving via Search Lecture 3
Lecture 3 What is a search problem? How do search algorithms work and how do we evaluate their performance? 1 Agenda An example task Problem formulation Infrastructure for search algorithms Complexity
More informationLecture 1 - Preliminaries
Advanced Algorithms Floriano Zini Free University of Bozen-Bolzano Faculty of Computer Science Academic Year 2013-2014 Lecture 1 - Preliminaries 1 Typography vs algorithms Johann Gutenberg (c. 1398 February
More informationFundamental Algorithms
Fundamental Algorithms Chapter 5: Searching Michael Bader Winter 2014/15 Chapter 5: Searching, Winter 2014/15 1 Searching Definition (Search Problem) Input: a sequence or set A of n elements (objects)
More informationLower and Upper Bound Theory
Lower and Upper Bound Theory Adnan YAZICI Dept. of Computer Engineering Middle East Technical Univ. Ankara - TURKEY 1 Lower and Upper Bound Theory How fast can we sort? Most of the sorting algorithms are
More information16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ]
418 16. Binary Search Trees [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1-12.3] 419 Dictionary implementation Hashing: implementation of dictionaries with expected very fast access times. Disadvantages
More informationV. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005
V. Adamchi Recurrences Victor Adamchi Fall of 00 Plan Multiple roots. More on multiple roots. Inhomogeneous equations 3. Divide-and-conquer recurrences In the previous lecture we have showed that if the
More information26 Chapter 4 Classification
26 Chapter 4 Classification The preceding tree cannot be simplified. 2. Consider the training examples shown in Table 4.1 for a binary classification problem. Table 4.1. Data set for Exercise 2. Customer
More information16. Binary Search Trees. [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap ]
423 16. Binary Search Trees [Ottman/Widmayer, Kap. 5.1, Cormen et al, Kap. 12.1-12.3] Dictionary implementation 424 Hashing: implementation of dictionaries with expected very fast access times. Disadvantages
More informationFundamental Algorithms
Chapter 2: Sorting, Winter 2018/19 1 Fundamental Algorithms Chapter 2: Sorting Jan Křetínský Winter 2018/19 Chapter 2: Sorting, Winter 2018/19 2 Part I Simple Sorts Chapter 2: Sorting, Winter 2018/19 3
More informationFundamental Algorithms
Fundamental Algorithms Chapter 2: Sorting Harald Räcke Winter 2015/16 Chapter 2: Sorting, Winter 2015/16 1 Part I Simple Sorts Chapter 2: Sorting, Winter 2015/16 2 The Sorting Problem Definition Sorting
More informationCSCE 222 Discrete Structures for Computing
CSCE 222 Discrete Structures for Computing Algorithms Dr. Philip C. Ritchey Introduction An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem.
More informationComputer Algorithms CISC4080 CIS, Fordham Univ. Outline. Last class. Instructor: X. Zhang Lecture 2
Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2 Outline Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive formula
More informationComputer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2
Computer Algorithms CISC4080 CIS, Fordham Univ. Instructor: X. Zhang Lecture 2 Outline Introduction to algorithm analysis: fibonacci seq calculation counting number of computer steps recursive formula
More informationAsymptotic redundancy and prolixity
Asymptotic redundancy and prolixity Yuval Dagan, Yuval Filmus, and Shay Moran April 6, 2017 Abstract Gallager (1978) considered the worst-case redundancy of Huffman codes as the maximum probability tends
More informationSolutions. Problem 1: Suppose a polynomial in n of degree d has the form
Assignment 1 1. Problem 3-1 on p. 57 2. Problem 3-2 on p. 58 3. Problem 4-5 on p. 86 4. Problem 6-1 on p. 142 5. Problem 7-4 on p. 162 6. Prove correctness (including halting) of SelectionSort (use loop
More informationSearch Trees. Chapter 10. CSE 2011 Prof. J. Elder Last Updated: :52 AM
Search Trees Chapter 1 < 6 2 > 1 4 = 8 9-1 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 2 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 3 - Binary Search Trees Ø A binary search
More informationCSE 202 Homework 4 Matthias Springer, A
CSE 202 Homework 4 Matthias Springer, A99500782 1 Problem 2 Basic Idea PERFECT ASSEMBLY N P: a permutation P of s i S is a certificate that can be checked in polynomial time by ensuring that P = S, and
More informationAlgorithms and Their Complexity
CSCE 222 Discrete Structures for Computing David Kebo Houngninou Algorithms and Their Complexity Chapter 3 Algorithm An algorithm is a finite sequence of steps that solves a problem. Computational complexity
More informationTime Analysis of Sorting and Searching Algorithms
Time Analysis of Sorting and Searching Algorithms CSE21 Winter 2017, Day 5 (B00), Day 3 (A00) January 20, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Big-O : How to determine? Is f(n) big-o of g(n)?
More informationCS6901: review of Theory of Computation and Algorithms
CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational
More informationIntroduction to Divide and Conquer
Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward
More informationOptimal Tree-decomposition Balancing and Reachability on Low Treewidth Graphs
Optimal Tree-decomposition Balancing and Reachability on Low Treewidth Graphs Krishnendu Chatterjee Rasmus Ibsen-Jensen Andreas Pavlogiannis IST Austria Abstract. We consider graphs with n nodes together
More informationCSCI 3110 Assignment 6 Solutions
CSCI 3110 Assignment 6 Solutions December 5, 2012 2.4 (4 pts) Suppose you are choosing between the following three algorithms: 1. Algorithm A solves problems by dividing them into five subproblems of half
More informationProblem Set 2 Solutions
Introduction to Algorithms October 1, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Piotr Indyk and Charles E. Leiserson Handout 9 Problem Set 2 Solutions Reading: Chapters 5-9,
More informationRecap: Prefix Sums. Given A: set of n integers Find B: prefix sums 1 / 86
Recap: Prefix Sums Given : set of n integers Find B: prefix sums : 3 1 1 7 2 5 9 2 4 3 3 B: 3 4 5 12 14 19 28 30 34 37 40 1 / 86 Recap: Parallel Prefix Sums Recursive algorithm Recursively computes sums
More informationSearch Trees. EECS 2011 Prof. J. Elder Last Updated: 24 March 2015
Search Trees < 6 2 > 1 4 = 8 9-1 - Outline Ø Binary Search Trees Ø AVL Trees Ø Splay Trees - 2 - Learning Outcomes Ø From this lecture, you should be able to: q Define the properties of a binary search
More informationNotes on Machine Learning for and
Notes on Machine Learning for 16.410 and 16.413 (Notes adapted from Tom Mitchell and Andrew Moore.) Learning = improving with experience Improve over task T (e.g, Classification, control tasks) with respect
More information3. Algorithms. What matters? How fast do we solve the problem? How much computer resource do we need?
3. Algorithms We will study algorithms to solve many different types of problems such as finding the largest of a sequence of numbers sorting a sequence of numbers finding the shortest path between two
More informationPARALLEL DIGITAL-ANALOG CONVERTERS
CMOS Analog IC Design Page 10.2-1 10.2 - PARALLEL DIGITAL-ANALOG CONVERTERS CLASSIFICATION OF DIGITAL-ANALOG CONVERTERS CMOS Analog IC Design Page 10.2-2 CURRENT SCALING DIGITAL-ANALOG CONVERTERS GENERAL
More informationSupervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees!
Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees! Summary! Input Knowledge representation! Preparing data for learning! Input: Concept, Instances, Attributes"
More informationMath 391: Midterm 1.0 Spring 2016
Math 391: Midterm 1.0 Spring 2016 James Skon skonjp@kenyon.edu Test date: October 5 Submission Instructions: Give all your assumptions. Show all your work. Be as complete as possible. Grading Problem 1
More informationSpace Complexity of Algorithms
Space Complexity of Algorithms So far we have considered only the time necessary for a computation Sometimes the size of the memory necessary for the computation is more critical. The amount of memory
More informationBinary Decision Diagrams. Graphs. Boolean Functions
Binary Decision Diagrams Graphs Binary Decision Diagrams (BDDs) are a class of graphs that can be used as data structure for compactly representing boolean functions. BDDs were introduced by R. Bryant
More informationSIGNAL COMPRESSION Lecture 7. Variable to Fix Encoding
SIGNAL COMPRESSION Lecture 7 Variable to Fix Encoding 1. Tunstall codes 2. Petry codes 3. Generalized Tunstall codes for Markov sources (a presentation of the paper by I. Tabus, G. Korodi, J. Rissanen.
More informationMachine Learning 2nd Edi7on
Lecture Slides for INTRODUCTION TO Machine Learning 2nd Edi7on CHAPTER 9: Decision Trees ETHEM ALPAYDIN The MIT Press, 2010 Edited and expanded for CS 4641 by Chris Simpkins alpaydin@boun.edu.tr h1p://www.cmpe.boun.edu.tr/~ethem/i2ml2e
More informationLearning Decision Trees
Learning Decision Trees Machine Learning Fall 2018 Some slides from Tom Mitchell, Dan Roth and others 1 Key issues in machine learning Modeling How to formulate your problem as a machine learning problem?
More informationarxiv: v2 [cs.ds] 3 Oct 2017
Orthogonal Vectors Indexing Isaac Goldstein 1, Moshe Lewenstein 1, and Ely Porat 1 1 Bar-Ilan University, Ramat Gan, Israel {goldshi,moshe,porately}@cs.biu.ac.il arxiv:1710.00586v2 [cs.ds] 3 Oct 2017 Abstract
More informationb + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d
CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we
More informationDecision Trees. Data Science: Jordan Boyd-Graber University of Maryland MARCH 11, Data Science: Jordan Boyd-Graber UMD Decision Trees 1 / 1
Decision Trees Data Science: Jordan Boyd-Graber University of Maryland MARCH 11, 2018 Data Science: Jordan Boyd-Graber UMD Decision Trees 1 / 1 Roadmap Classification: machines labeling data for us Last
More informationLecture 2. Fundamentals of the Analysis of Algorithm Efficiency
Lecture 2 Fundamentals of the Analysis of Algorithm Efficiency 1 Lecture Contents 1. Analysis Framework 2. Asymptotic Notations and Basic Efficiency Classes 3. Mathematical Analysis of Nonrecursive Algorithms
More informationComputer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms
Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabás Póczos Contents Decision Trees: Definition + Motivation Algorithm for Learning Decision Trees Entropy, Mutual Information, Information
More informationMathematical Background. Unsigned binary numbers. Powers of 2. Logs and exponents. Mathematical Background. Today, we will review:
Mathematical Background Mathematical Background CSE 373 Data Structures Today, we will review: Logs and eponents Series Recursion Motivation for Algorithm Analysis 5 January 007 CSE 373 - Math Background
More information