Tutorial Session 5 Tuesday, 19 th of March 2019 Discussion of Exercise 4, Preview on Exercise 5, Preparation for Midterm 1 Running Time Analysis, Asymptotic Complexity 16.15 17.45 / 18.15 19.45 BIN 0.B.06
Agenda Outlook on Midterm 1 Discussion of Exercise 4 Midterm Preparation Preview on Exercise 5 AINF1169 Informatics II 2
Outlook on Midterm 1 General Information What to Expect Exam Strategy Remarks on the Cheat Sheet
Date and Location Date, time, location: Hints: Friday, March 22 nd, 12.00 13.45 h 90 minutes exam time in HAH-E-03 and HAH-F-01 (both at Häldeliweg 2, at Zentrum) seating plan will be published on OLAT Don t forget your student ID (Legi). Either this midterm or the next midterm will count 25% towards your final grade. Things will probably be significantly more relaxed if you score a good grade now Failure to show up leads to grade 1 in this midterm. AINF1169 Informatics II 4
General Information: Topics and Allowed Auxiliary Materials Topics covered: SL01 to SL03 complete Allowed auxiliary materials: one A4 sheet (both-sided), notes can be handwritten or printed or photocopied pocket calculator with no communication capabilities and no storage / memory, e.g. TI-30 XII B/S AINF1169 Informatics II 5
What to Expect There might (or might not) be some changes to the style of the midterm when compared to earlier years since the midterms now count towards the final grade and are no longer graded on a pass/fail basis. I expect the passing grade threshold (grade 4.0) to remain at about 50% of reachable points and I also expect the amount of tasks to stay about the same, i.e. a total 60 points, distributed on 3 to 4 exercises worth 10 to 20 points each. Standard tasks of midterm 1 include: Asymptotic complexity and recurrences Running time analysis Recursion / Divide and conquer algorithm Have a look at previous year s midterms which are available on the tutorial web page. Also, old assignments with solutions are available on the tutorial web page and can be used for additional practice. AINF1169 Informatics II 6
What to Expect Some data from midterm 1 of previous semesters (numbers without no-shows, maximum 60 points each): Semester Pass Fail Median points FS 2018 141 76.2% FS 2017 110 79.7% FS 2016 106 80.9% 44 23.8% 28 20.3% 25 19.1% Average points Almost full point range was used in previous years. Points to pass 27 27.2 21 39 37.7 30 39 36.7 30 Points distribution in midterm 1 of FS 2018: number of participants 50 45 40 35 30 25 20 15 10 5 0 5 10 15 20 25 30 35 40 45 50 55 points reached AINF1169 Informatics II 7
Exam Strategy Make sure, you always at least write something. Note down observations and thoughts for example. If you struggle with formal notation, use natural language additionally (or alternatively). Start with tasks which you feel comfortable to solve. Never give up. AINF1169 Informatics II 8
Remarks on the Cheat Sheet The cheat sheet can be a nice help. It saves you from learning stupid things by heart. It may protect from «blackout situations» where you suddenly can t remember things that you actually know. Although: The cheat sheet is useless and can t save you, if you don t understand the content and did not practice its application. Preparing the cheat sheet may be a good preparation and repetition. I suggest, though, to not spend too much time into preparing the perfect «God cheat sheet» solving example tasks (e.g. old midterms and assignments) is far more important and rewarding in my opinion. AINF1169 Informatics II 9
Last Minute Q&A Session For people who want to ask questions, I will be available this Friday, 22 rd of March from 09.00 h here in BIN 0.B.06 (until about 11.00 at the latest) AINF1169 Informatics II 10
Review of Exercise 4 Task 1: Ternary Search Visualization Task 2: Ternary Search Implementation Task 3: Ternary Search Analysis Task 4: Towers of Hanoi: Recurrence Relation Task 5: Towers of Hanoi: Implementation Task 6: Recursion Tree & Substitution Method Task 7: Master Method
Exercise 4 Task 1: Ternary Search Task Description: «Based on the three way max finder description, draw a tree to illustrate the process of finding the array s maximum value, given A = [24, 7, -18, 9, -7, -6, 15, 2, 1]. Also write the recurrence relation for this three way max finder algorithm.» left boundary right boundary A = 24 7-18 9-7 -6 15 2 1 0 1 2 3 4 5 6 7 8 AINF1169 Informatics II 12
Exercise 4 Task 1: Ternary Search Visualizaion AINF1169 Informatics II 13
Exercise 4 Task 2: Ternary Search: Implementation left boundary right boundary current maximum value base case as long as boundaries are valid, i.e. r might not be smaller than l if the left and right boundary have met (single remaining element) compare value of single remaining element to current maximum value calculate boundary indices for subarrays recursive calls on subarrays AINF1169 Informatics II 14
Exercise 4 Task 2: Ternary Search: Implementation in C int ternarymaxsearch(int A[], int l, int r, int k) { int mid1, mid2; int val, max1, max2, max3; if (r >= l) { if (l == r) { if (A[l] > k) { return A[l]; } else { return k; } } } } mid1 = l + (r - l) * 1 / 3; mid2 = l + (r - l) * 2 / 3; max1 = ternarymaxsearch(a, l, mid1, k); max2 = ternarymaxsearch(a, mid1 + 1, mid2, k); max3 = ternarymaxsearch(a, mid2 + 1, r, k); return max(max1, max2, max3); return -1; AINF1169 Informatics II 15
Exercise 4 Task 3: Ternary Search: Analysis The recurrence relation describing the ternary search is: T(n) = 3 T(n/3) + 3 What would be the recurrence relation if the algorithm would split the input array into four partitions instead of three? How would this affect the asymptotic complexity of the algorithm? New recurrence relation: T(n) = 4 T(n/4) + 4 No change in asymptotic complexity: remains Ο(n). AINF1169 Informatics II 16
Exercise 4 Task 4: Towers of Hanoi: Example with Four Disks Let s number the disks of the tower from 1 to N, starting with the smallest disk and let s denote the three places as A, B and C. Thus the starting configuration looks like this: 1 2 3 4 place A place B place C We now are given the task to move the four disks from place A to place C without having a disk with a higher number above a disk with a lower number at any point in time. AINF1169 Informatics II 17
Exercise 4 Task 4: Towers of Hanoi: Example with Four Disks The problem of shifting four disks from place A to place C can be solved in three steps as follows: 1 2 3 4 place A place B place C Move the tower consisting of disks 1 to 3 to place B 4 place A 1 2 3 place B place C Move disk 4 to place C place A place B place C 1 2 3 4 Move the tower consisting of disks 1 to 3 to place C place A 1 2 3 place B 4 place C So far so good, but now we need a way to move a tower of three disks from one position to another How can we solve this? By applying the same strategy recursively! AINF1169 Informatics II 18
Exercise 4 Task 4: Towers of Hanoi: Generalized Recursive Approach From this, we have a generalized, recursive solution to move a tower with k disks (i.e. disks from 1 to k) from place A to place C: 1) Move the tower with disks 1 to k 1 from place A to place B 2) Move disk k (biggest disk of the tower of size k) on place C (only remaining valid place) 3) Move the tower with disks 1 to k 1 from place B to place C Steps 1) and 3) are applied recursively. The base case is reached when the tasks has reduced to shifting a tower of size 1, i.e. shifting a single disk. AINF1169 Informatics II 19
Exercise 4 Task 4: Towers of Hanoi: Example with Five Disks Graphical depiction of the recursive calls (recursion tree) for the example of five disks: Move tower with 5 disks Move tower with 4 disks Move disk number 5 Move tower with 4 disks Move tower with 3 disks Move disk number 4 Move tower with 3 disks Move tower with 2 disks Move disk number 3 Move tower with 2 disks Move disk number 1 (base case) Move disk number 1 (base case) AINF1169 Informatics II 20
Exercise 4 Task 4: Towers of Hanoi: Pseudocode AINF1169 Informatics II 21
Exercise 4 Task 4: Towers of Hanoi: C Code Implementation Possible implementation of a program which prints a sequence of movements which will solve the Towers of Hanoi problem: 01 02 03 04 05 06 07 08 09 10 void printhanoioperations(int numberofdisks, char fromplace, char viaplace, char toplace) { if (numberofdisks == 1) { printf("%c -> %c\n", fromplace, toplace); } else { printhanoioperations(numberofdisks - 1, fromplace, toplace, viaplace); printhanoioperations(1, fromplace, viaplace, toplace); printhanoioperations(numberofdisks - 1, viaplace, fromplace, toplace); } } AINF1169 Informatics II 22
Exercise 4 Task 4: Towers of Hanoi: Recurrence Relation Thus, the recursive solution to the Towers of Hanoi problem can be modeled by the following recurrence relation: T(n) = 2 T(n 1) + 1 This recurrence can be solved by repeated backward substitution: Therefore, this recurrence and hence the running time of the algorithm to solve the Towers of Hanoi problem is in Θ(2 n ). AINF1169 Informatics II 23
Exercise 4 Task 6a: Solving a Recurrence Using Recursion Tree The branches of the recursion tree are longer on the very left side (where n gets always divided by 3). To estimate the total work in its nodes, we will assume the tree has the height of the leftmost branch everywhere and this will yield an upper bound for the total work. On the left side, the tree grows until h = log 3 (n). Therefore an upper bound for the total work is: log 3 (n) T(n) < n i = 0 0 0 11 18 i AINF1169 Informatics II 24
Exercise 4 Task 6a: More Detailed Calculation for Guess h The sum 11 is a finite geometric series, i.e. a sum of form 11 0 a where a = and x = h = log 3 (n). 18 k 18 0 i = 0 1 a x+1 We know that the geometric series sums up to. 1 a Thus: T(n) < n With h = log 3 (n) we get: 0 i h i = 0 11 1 = n 18 0 110 1 18 0 h + 1 = n 180 11 T(n) < n 2 h + 1 1 bb = 70 18 30 180 = n 1 11 110 log 0 1 n 3 11 180 18 0 bb = 0 n 7 18 18 0 70 0 0 11 18 h+1 i 0 0 0 0 11 7 x k = 0 11 1 2 18 30 Since log 3 (11/18) 0.45, n 1+log 3(11/18) n 0.55 which is asymptotically smaller than the linear term of the first summand. Therefore, T(n) is in Ο(n). h+1 0 0 18 7 n n log 3 0 0 18 7 110 18 0 n 0 AINF1169 Informatics II 25 h+1 h + 1 bb 0 = 1 110 1 11 18 0 18 0 18 = 0 n 7 h+1 log 3 (n) 0 0 11 7 0 bb 11 1 = 18 0 n 1 + log 3 110 18 0
Exercise 4 Task 6b: Proofing a Recurrence Using Substitution AINF1169 Informatics II 26
Exercise 4 Task 7: Master Method a) T(n) = 16 T(n/4) + 6n 2 a = 16, b = 4, f(n) = 6n 2 x = log b (a) = log 4 (16) = 2; n x = n log₄(16) = n 2 compare n x to f(n): same growth rate case 2 T(n) = Θ(n 2 log(n)) b) T(n) = T(n/2) + 1 a = 1, b = 2, f(n) = 1 x = log b (a) = log 2 (1) = 0; n x = n 0 = 1 compare n x to f(n): same growth rate case 2 T(n) = Θ(log(n)) AINF1169 Informatics II 27
Exercise 4 Task 7: Master Method c) T(n) = 4 T(n/2) + 2n a = 4, b = 2, f(n) = 2n x = log b (a) = log 2 (4) = 2; n x = n log₄(4) = n 2 compare n x to f(n): n x grows faster case 1 T(n) = Θ(n 2 ) d) T(n) = 2 T(n 2) + n master method not applicable: parsing fails applying repeated substitution yields: T(n) = Ο(n sqrt(2) n ) AINF1169 Informatics II 28
Exercise 4 Task 7: Master Method e) T(n) = 2 T(n/4) + n lg(n) a = 2, b = 4, f(n) = n log 2 (n) x = log 4 (2) = log 4 (2) = 0.5; n x = n log₄(2) = n 0.5 = sqrt(n) compare n x to f(n): f(n) grows faster case 3 Regularity check: we need to find a constant c < 0 such that a f(n/b) c f(n) for any n > n 0. a f(n/b) = 2 f(n/4) = 2 n/4 log 2 (n/4) = 0.5 n (log 2 (n) log 2 (4)) = 0.5 n (log 2 (n) 2) = = 0.5 n log 2 (n) n Thus we re looking for a c which fulfills: 0.5 n log 2 (n) n c n log 2 (n) Obviously, the above equation is certainly true for any c 0.5 and any n > 1 = n 0. T(n) = Θ(n log(n)) AINF1169 Informatics II 29
Exercise 4 Task 7: Master Method d) T(n) = 2 T(n 2) + n master method not applicable: parsing fails; repeated substitution yields: T(n) = Ο(n sqrt(2) n ) AINF1169 Informatics II 30
Summations: Arithmetico-Geometric Series By combining the arithmetic series and the geometric series by a term-by-term multiplication, we get a summation of the form n (a 0 + k d) b k = a 0 + (a 0 + d) b + (a 0 + 2 d) b 2 + (a 0 + 3 d) b 3 + + (a 0 n d) b n k = 0 (with 0 < b 1, d = const and an integer n > 0). This is called a (finite) arithmetico-geometric series. As it is written above (i.e. starting at k = 0) sums up to a 0 1 b (a 0 + (n 1) d) b n 1 b + d b (1 bn 1 ) (1 b) 2 The infinite arithmetico-geometric series sums up to: (a 0 + k d) b k = k = 0 a 0 1 b + d b (1 b) 2 AINF1169 Informatics II 31
Midterm Preparation Examples Solved General Principles
Methods for Determining the Number of Executions of for Loops in Asymptotic Running Time Analysis Multiplicative combination of algorithm parts (and using «end start + 1» et al.) «Table method»: write down the evolution of the loop variable in a table to gain an understanding about the number of executions (probably not possible for more than two levels of nesting) Write loop as a nested multiple sum and solve it (requires a bit of knowledge how to manipulate sums). AINF1169 Informatics II 33
Algorithm Running Time Analysis: Another Example Exercise 2.3 from midterm 1 of spring semester 2016: Perform an exact analysis of the running time of the following algorithm and determine its asymptotic complexity. Algorithm: alg(a, n) 1 for i = 1 to n / 2 do 2 min = i 3 max = n i + 1 4 if A[min] > A[max] then 5 exchange A[min] and A[max] 6 for j = i + 1 to n i do 7 if A[j] < A[min] then 8 min = j 9 if A[j] > A[max] then 10 max = j 11 exchange A[i] and A[min] 12 exchange A[n i + 1] and A[max] AINF1169 Informatics II 34
Algorithm Running Time Analysis: «Table Method» Example The following example shows how to analyse the number executions of the body of the inner loop (lines 7 to 10) of the algorithm alg(a, n) using a table to track the evolution of the number of executions depending on the value of the outer loop variable. Outer loop variable value i = Inner loop variable range j = i+1 n i Number of executions of the inner loop body 1 2 n 1 n 2 2 3 n 2 n 4 3 4 n 3 n 6 k k+1 n k n 2 k n/2 n/2 +1 n/2 0 AINF1169 Informatics II 35
Algorithm Running Time Analysis: Yet Another Example Task 1 of Assignment 2 from FS 2018: AINF1169 Informatics II 36
Summation Rules and Examples Decomposing Sums Changing Summation Indexes Double Sums / Multiple Sums / Nested Sums
Summations: Decomposing Sums Indexes of sums can be rewritten / sums can be «taken apart» as in the following example: n x n x x 1 k = k k k = x k = 1 k = 1 sum up «too much», i.e. start summing up from 1 instead of starting from x remove what has been summed up too much, i.e. numbers from 1 to x 1 This is useful in solving sums stemming from nested loops with interacting loop variables like for example: <pseudo code> Im Falle, dass über eine Konstante aufsummiert wird, gilt einfach ENDE ANFANG + 1 AINF1169 Informatics II 38
Recursion Tree Example (Task 5a of assignment 2 from FS 2018) Solve the following recurrence using the tree method: T(n) = T(n/6) + T(n/2) + n T(1) = 1 AINF1169 Informatics II 39
Trees Nomenclature Binary Trees Binary Tree Traversals Binary Search Trees
Trees: Nomenclature height of tree = 3 height (of node) 3 2 1 level 0 1 2 siblings e.g. B and C are siblings root A B C D internal node e.g. D, G, B E F G H parent (ancestor / father) e.g. D is parent of H child (son) e.g. H is child of D 0 3 I J K L M N sub-tree leaf (external node), e.g. K, C, F AINF1169 Informatics II 41
Binary Tree Traversals: Introduction and Overview Tree traversals are rules on how to visit each node of a binary tree exactly once. There are three famous kinds of depth-first traversals and one kind of breadth-first traversal (visiting neighbors first). Depth-first traversals: Inorder: left root right Preorder: root left right Postorder: left right root Breadth-first traversal: Levelorder: from left to right, from root level to leaf level AINF1169 Informatics II 42
Binary Tree Traversals: Remarks Note that the sequence of nodes stemming from a particular traversal method in general does not allow to unambiguously reconstruct the original tree. An exception is the preorder traversal which allows to reconstruct the original tree in an unambiguous and efficient manner; therefore, the preorder traversal can be used (and considered as) as representational structure for trees, e.g. in order to store a tree in a file. AINF1169 Informatics II 43
Binary Tree Traversals Recursively apply the principle stated before to all nodes of the tree by beginning at the root node. For example: inorder traversal: Left Root Right Pseudo code: inorder(tree T) { inorder(left_subtree(t)); visit(root); inorder(right_subtree(t)); } AINF1169 Informatics II 44
Binary Tree Traversals: Example What are the inorder, preorder, postorder and levelorder traversals of the following binary tree? 27 14 35 10 19 31 42 Inorder: 10, 14, 19, 27, 31, 35, 42 Preorder: 27, 14, 10, 19, 35, 31, 42 Postorder: 10, 19, 14, 31, 42, 35, 27 Levelorder: 27, 14, 35, 10, 19, 31, 42 AINF1169 Informatics II 45
Binary Tree Traversals: Visual Help preorder inorder postorder AINF1169 Informatics II 46
Binary Tree Traversals: Iterative vs. Recursive Implementation Instead of applying a recursive algorithm, the depth-first traversals (inorder, preorder, postorder) can also be implemented iteratively using a (explicit) stack. Similarly, a levelorder traversal can be implemented iteratively using a queue. 1 Step 0 1 2 3 4 5 6 7 8 4 2 3 5 Stack content 1 2 3 4 5 3 6 5 3 5 3 7 8 3 8 3 3 6 7 8 Output of function «next» - 1 2 4 6 5 7 8 3 AINF1169 Informatics II 47
Tree Traversals: Applications What are these tree traversal strategies useful for? Preorder: easy and efficient way to store binary search trees with the possibility to reconstruct them unambigously Inorder: retrieving the node values of a binary search tree in ascending order Postorder: important for compiler design (in assembly: operand operand opcode) Levelorder: used in heapsort algorithm AINF1169 Informatics II 48
Binary Search Trees A binary search tree (BST) is a binary tree where all descendent nodes to the left are smaller or equal than the ancestor node and all descendants to the right are bigger or equal than the ancestor node. Example: 19 Subtree with node values less than 19 11 35 7 6 23 Subtree with node values bigger than 19 13 17 AINF1169 Informatics II 49
Heaps Heap Conditions Representation of a Heap as an Array Heap Operations Heapsort
Heap Conditions A heap is a binary tree. All levels except the lowest are fully filled and the lowest level is occupied starting from left without gaps (= «nearly complete tree»). Keys in nodes of tree are ordered: in a max-heap: for all nodes, the key is bigger than or equal to all its children largest element at root in a min-heap: for all nodes, the father node is always smaller than or equal to his sons smallest element at root AINF1169 Informatics II 51
Representation of a Heap as an Array level 0 1 2 3 1 14 0 2 23 1 3 28 2 4 42 3 5 59 4 37 6 5 7 79 6 92 61 84 Indexing starts at 1 1 2 3 4 5 6 7 8 9 10 14 23 28 42 59 37 79 92 61 84 0 1 2 3 4 5 6 7 8 9 Indexing starts at 0 0 1 1 2 2 2 2 3 3 3 Level Note: This kind of tree traversal is called level order. AINF1169 Informatics II 52
Addressing Parent and Child Nodes 1 2 3 4 7 1 1 2 3 0 1 2 3 k n 2 n 1 n 3 6 5 n 3 n 2 n 1 Getting to parent of node with index k Getting to left child of node with index k Getting to right child of node with index k Indexing starts at 1 k / 2 2 k 2 k + 1 Indexing starts at 0 (k 1) / 2 2 k + 1 2 k + 2 AINF1169 Informatics II 53
Heap Conditions: Exercise The following arrays shall each represent a binary heap. Do they fulfill the conditions of a max-heap? A: 7 1 1 2 4 9 3 3 6 5 Not a heap, because the first element is neither the maximum nor the minimum of the elements 9 B: 9 7 5 2 3 7 2 1 7 2 7 5 By drawing the corresponding heap, we can see that elements at indices 5 and 8 (starting from zero) violate the heap condition. Therefore it is not a heap. 2 3 7 2 1 7 2 AINF1169 Informatics II 54
Heaps: Building a Heap How to get to a heap from an unordered array of values? Let s assume that for a certain node, we knew that both its subtrees are already heaps: In this case, we can apply the same procedure which we used before to trickle down the nodes. Thus, we just start at the last node which has at least one child node and then work our way up to the root while making sure for each node, that everything below (its subtrees) are nice heaps. AINF1169 Informatics II 55
Heaps: Sorting Algorithm 1) Transform input data into heap. 2) Successively remove the root element and reorganize the remaining heap. AINF1169 Informatics II 56
Heaps: Demonstration: Removing the Root 14 23 28 42 59 37 79 92 61 84 23 42 28 61 59 37 79 92 84 14 14 28 42 37 61 59 84 79 92 23 14 37 42 79 61 59 84 92 28 23 14 23 28 42 59 79 61 92 84 37 28 23 14 59 61 79 84 92 42 37 28 23 14 42 59 37 79 92 61 84 61 84 79 92 59 42 37 28 23 14 79 84 92 61 59 42 37 28 23 14 84 92 79 61 59 42 37 28 23 14 92 84 79 61 59 42 37 28 23 14 92 84 79 61 59 42 37 28 23 14 AINF1169 Informatics II 57
Heaps: Demonstration: Building a Heap 92 79 84 61 59 42 37 23 14 28 92 79 84 61 59 42 37 23 14 28 92 79 84 61 28 42 37 23 14 59 92 79 84 14 28 42 37 23 61 59 92 79 37 14 28 42 84 23 61 59 92 14 37 23 28 42 84 79 61 59 14 23 37 61 28 42 84 79 92 59 14 23 37 61 28 42 84 79 92 59 AINF1169 Informatics II 58
Heaps: Comprehension Question Why do we iterate from n/2 to 1 and not from 1 to n/2 in the BuildHeap algorithm? (Does this even matter?) AINF1169 Informatics II 59
Heaps, Heapsort: DIY / Game 1) Form groups of two to three people. 2) Get a deck of playing cards. 3) (If necessary: Familiarize yourself with the ordering of the cards.) 4) Shuffle the cards. 5) Lay out 12 cards in form of a binary three before you on the desk. (If you got the Joker, replace it through another card.) 6) Transform the card into a max- or min-heap. 7) Get a sorted set of cards by successively removing the root element. 8) If you feel comfortable with heapsort or if you get bored: Take 8 random cards of your deck and perform bubble sort, selection sort, insertion sort, quicksort, mergesort, on it. AINF1169 Informatics II 60
Preview on Exercise 5
Exercise 5 Task 1: Heap and Heapsort: Implementation in C Look at the respective slides from the lecture: SL04, pp. 5 33. AINF1169 Informatics II 62
Exercise 5 Tasks 2 & 3: Heap Construction and Heapsort by Hand Look at the respective slides from the lecture: SL04, pp. 5 33. AINF1169 Informatics II 63
Exercise 5 Task 4: Quicksort Example by Hand Look at the respective slides from the lecture: SL04, pp. 35 53. AINF1169 Informatics II 64
Wrap-Up Summary Outlook Questions
Outlook Midterm 1: Friday, 22.03.2019, 12:15 13:45 h Last minute Q&A: Friday, 22.03.2019, 09:00 11:00 h Next tutorial: Tuesday, 26.03.2019, BIN 0.B.06 Topics: Review of Exercise 5 Review of Midterm 1 Heapsort Quicksort Preview to Exercise 6 (your wishes) AINF1169 Informatics II 66
Questions? AINF1169 Informatics II 67
Thank you for your attention. AINF1169 Informatics II 68