Tutorial Session 5. Discussion of Exercise 4, Preview on Exercise 5, Preparation for Midterm 1. Running Time Analysis, Asymptotic Complexity

Transcription

1 Tutorial Session 5 Tuesday, 19 th of March 2019 Discussion of Exercise 4, Preview on Exercise 5, Preparation for Midterm 1 Running Time Analysis, Asymptotic Complexity / BIN 0.B.06

2 Agenda Outlook on Midterm 1 Discussion of Exercise 4 Midterm Preparation Preview on Exercise 5 AINF1169 Informatics II 2

3 Outlook on Midterm 1 General Information What to Expect Exam Strategy Remarks on the Cheat Sheet

4 Date and Location Date, time, location: Hints: Friday, March 22 nd, h 90 minutes exam time in HAH-E-03 and HAH-F-01 (both at Häldeliweg 2, at Zentrum) seating plan will be published on OLAT Don t forget your student ID (Legi). Either this midterm or the next midterm will count 25% towards your final grade. Things will probably be significantly more relaxed if you score a good grade now Failure to show up leads to grade 1 in this midterm. AINF1169 Informatics II 4

5 General Information: Topics and Allowed Auxiliary Materials Topics covered: SL01 to SL03 complete Allowed auxiliary materials: one A4 sheet (both-sided), notes can be handwritten or printed or photocopied pocket calculator with no communication capabilities and no storage / memory, e.g. TI-30 XII B/S AINF1169 Informatics II 5

6 What to Expect There might (or might not) be some changes to the style of the midterm when compared to earlier years since the midterms now count towards the final grade and are no longer graded on a pass/fail basis. I expect the passing grade threshold (grade 4.0) to remain at about 50% of reachable points and I also expect the amount of tasks to stay about the same, i.e. a total 60 points, distributed on 3 to 4 exercises worth 10 to 20 points each. Standard tasks of midterm 1 include: Asymptotic complexity and recurrences Running time analysis Recursion / Divide and conquer algorithm Have a look at previous year s midterms which are available on the tutorial web page. Also, old assignments with solutions are available on the tutorial web page and can be used for additional practice. AINF1169 Informatics II 6

7 What to Expect Some data from midterm 1 of previous semesters (numbers without no-shows, maximum 60 points each): Semester Pass Fail Median points FS % FS % FS % % % % Average points Almost full point range was used in previous years. Points to pass Points distribution in midterm 1 of FS 2018: number of participants points reached AINF1169 Informatics II 7

8 Exam Strategy Make sure, you always at least write something. Note down observations and thoughts for example. If you struggle with formal notation, use natural language additionally (or alternatively). Start with tasks which you feel comfortable to solve. Never give up. AINF1169 Informatics II 8

9 Remarks on the Cheat Sheet The cheat sheet can be a nice help. It saves you from learning stupid things by heart. It may protect from «blackout situations» where you suddenly can t remember things that you actually know. Although: The cheat sheet is useless and can t save you, if you don t understand the content and did not practice its application. Preparing the cheat sheet may be a good preparation and repetition. I suggest, though, to not spend too much time into preparing the perfect «God cheat sheet» solving example tasks (e.g. old midterms and assignments) is far more important and rewarding in my opinion. AINF1169 Informatics II 9

10 Last Minute Q&A Session For people who want to ask questions, I will be available this Friday, 22 rd of March from h here in BIN 0.B.06 (until about at the latest) AINF1169 Informatics II 10

11 Review of Exercise 4 Task 1: Ternary Search Visualization Task 2: Ternary Search Implementation Task 3: Ternary Search Analysis Task 4: Towers of Hanoi: Recurrence Relation Task 5: Towers of Hanoi: Implementation Task 6: Recursion Tree & Substitution Method Task 7: Master Method

12 Exercise 4 Task 1: Ternary Search Task Description: «Based on the three way max finder description, draw a tree to illustrate the process of finding the array s maximum value, given A = [24, 7, -18, 9, -7, -6, 15, 2, 1]. Also write the recurrence relation for this three way max finder algorithm.» left boundary right boundary A = AINF1169 Informatics II 12

13 Exercise 4 Task 1: Ternary Search Visualizaion AINF1169 Informatics II 13

14 Exercise 4 Task 2: Ternary Search: Implementation left boundary right boundary current maximum value base case as long as boundaries are valid, i.e. r might not be smaller than l if the left and right boundary have met (single remaining element) compare value of single remaining element to current maximum value calculate boundary indices for subarrays recursive calls on subarrays AINF1169 Informatics II 14

15 Exercise 4 Task 2: Ternary Search: Implementation in C int ternarymaxsearch(int A[], int l, int r, int k) { int mid1, mid2; int val, max1, max2, max3; if (r >= l) { if (l == r) { if (A[l] > k) { return A[l]; } else { return k; } } } } mid1 = l + (r - l) * 1 / 3; mid2 = l + (r - l) * 2 / 3; max1 = ternarymaxsearch(a, l, mid1, k); max2 = ternarymaxsearch(a, mid1 + 1, mid2, k); max3 = ternarymaxsearch(a, mid2 + 1, r, k); return max(max1, max2, max3); return -1; AINF1169 Informatics II 15

16 Exercise 4 Task 3: Ternary Search: Analysis The recurrence relation describing the ternary search is: T(n) = 3 T(n/3) + 3 What would be the recurrence relation if the algorithm would split the input array into four partitions instead of three? How would this affect the asymptotic complexity of the algorithm? New recurrence relation: T(n) = 4 T(n/4) + 4 No change in asymptotic complexity: remains Ο(n). AINF1169 Informatics II 16

17 Exercise 4 Task 4: Towers of Hanoi: Example with Four Disks Let s number the disks of the tower from 1 to N, starting with the smallest disk and let s denote the three places as A, B and C. Thus the starting configuration looks like this: place A place B place C We now are given the task to move the four disks from place A to place C without having a disk with a higher number above a disk with a lower number at any point in time. AINF1169 Informatics II 17

18 Exercise 4 Task 4: Towers of Hanoi: Example with Four Disks The problem of shifting four disks from place A to place C can be solved in three steps as follows: place A place B place C Move the tower consisting of disks 1 to 3 to place B 4 place A place B place C Move disk 4 to place C place A place B place C Move the tower consisting of disks 1 to 3 to place C place A place B 4 place C So far so good, but now we need a way to move a tower of three disks from one position to another How can we solve this? By applying the same strategy recursively! AINF1169 Informatics II 18

19 Exercise 4 Task 4: Towers of Hanoi: Generalized Recursive Approach From this, we have a generalized, recursive solution to move a tower with k disks (i.e. disks from 1 to k) from place A to place C: 1) Move the tower with disks 1 to k 1 from place A to place B 2) Move disk k (biggest disk of the tower of size k) on place C (only remaining valid place) 3) Move the tower with disks 1 to k 1 from place B to place C Steps 1) and 3) are applied recursively. The base case is reached when the tasks has reduced to shifting a tower of size 1, i.e. shifting a single disk. AINF1169 Informatics II 19

20 Exercise 4 Task 4: Towers of Hanoi: Example with Five Disks Graphical depiction of the recursive calls (recursion tree) for the example of five disks: Move tower with 5 disks Move tower with 4 disks Move disk number 5 Move tower with 4 disks Move tower with 3 disks Move disk number 4 Move tower with 3 disks Move tower with 2 disks Move disk number 3 Move tower with 2 disks Move disk number 1 (base case) Move disk number 1 (base case) AINF1169 Informatics II 20

21 Exercise 4 Task 4: Towers of Hanoi: Pseudocode AINF1169 Informatics II 21

22 Exercise 4 Task 4: Towers of Hanoi: C Code Implementation Possible implementation of a program which prints a sequence of movements which will solve the Towers of Hanoi problem: void printhanoioperations(int numberofdisks, char fromplace, char viaplace, char toplace) { if (numberofdisks == 1) { printf("%c -> %c\n", fromplace, toplace); } else { printhanoioperations(numberofdisks - 1, fromplace, toplace, viaplace); printhanoioperations(1, fromplace, viaplace, toplace); printhanoioperations(numberofdisks - 1, viaplace, fromplace, toplace); } } AINF1169 Informatics II 22

23 Exercise 4 Task 4: Towers of Hanoi: Recurrence Relation Thus, the recursive solution to the Towers of Hanoi problem can be modeled by the following recurrence relation: T(n) = 2 T(n 1) + 1 This recurrence can be solved by repeated backward substitution: Therefore, this recurrence and hence the running time of the algorithm to solve the Towers of Hanoi problem is in Θ(2 n ). AINF1169 Informatics II 23

24 Exercise 4 Task 6a: Solving a Recurrence Using Recursion Tree The branches of the recursion tree are longer on the very left side (where n gets always divided by 3). To estimate the total work in its nodes, we will assume the tree has the height of the leftmost branch everywhere and this will yield an upper bound for the total work. On the left side, the tree grows until h = log 3 (n). Therefore an upper bound for the total work is: log 3 (n) T(n) < n i = i AINF1169 Informatics II 24

25 Exercise 4 Task 6a: More Detailed Calculation for Guess h The sum 11 is a finite geometric series, i.e. a sum of form 11 0 a where a = and x = h = log 3 (n). 18 k 18 0 i = 0 1 a x+1 We know that the geometric series sums up to. 1 a Thus: T(n) < n With h = log 3 (n) we get: 0 i h i = = n h + 1 = n T(n) < n 2 h bb = = n log 0 1 n bb = 0 n h+1 i x k = Since log 3 (11/18) 0.45, n 1+log 3(11/18) n 0.55 which is asymptotically smaller than the linear term of the first summand. Therefore, T(n) is in Ο(n). h n n log n 0 AINF1169 Informatics II 25 h+1 h + 1 bb 0 = = 0 n 7 h+1 log 3 (n) bb 11 1 = 18 0 n 1 + log

26 Exercise 4 Task 6b: Proofing a Recurrence Using Substitution AINF1169 Informatics II 26

27 Exercise 4 Task 7: Master Method a) T(n) = 16 T(n/4) + 6n 2 a = 16, b = 4, f(n) = 6n 2 x = log b (a) = log 4 (16) = 2; n x = n log₄(16) = n 2 compare n x to f(n): same growth rate case 2 T(n) = Θ(n 2 log(n)) b) T(n) = T(n/2) + 1 a = 1, b = 2, f(n) = 1 x = log b (a) = log 2 (1) = 0; n x = n 0 = 1 compare n x to f(n): same growth rate case 2 T(n) = Θ(log(n)) AINF1169 Informatics II 27

28 Exercise 4 Task 7: Master Method c) T(n) = 4 T(n/2) + 2n a = 4, b = 2, f(n) = 2n x = log b (a) = log 2 (4) = 2; n x = n log₄(4) = n 2 compare n x to f(n): n x grows faster case 1 T(n) = Θ(n 2 ) d) T(n) = 2 T(n 2) + n master method not applicable: parsing fails applying repeated substitution yields: T(n) = Ο(n sqrt(2) n ) AINF1169 Informatics II 28

29 Exercise 4 Task 7: Master Method e) T(n) = 2 T(n/4) + n lg(n) a = 2, b = 4, f(n) = n log 2 (n) x = log 4 (2) = log 4 (2) = 0.5; n x = n log₄(2) = n 0.5 = sqrt(n) compare n x to f(n): f(n) grows faster case 3 Regularity check: we need to find a constant c < 0 such that a f(n/b) c f(n) for any n > n 0. a f(n/b) = 2 f(n/4) = 2 n/4 log 2 (n/4) = 0.5 n (log 2 (n) log 2 (4)) = 0.5 n (log 2 (n) 2) = = 0.5 n log 2 (n) n Thus we re looking for a c which fulfills: 0.5 n log 2 (n) n c n log 2 (n) Obviously, the above equation is certainly true for any c 0.5 and any n > 1 = n 0. T(n) = Θ(n log(n)) AINF1169 Informatics II 29

30 Exercise 4 Task 7: Master Method d) T(n) = 2 T(n 2) + n master method not applicable: parsing fails; repeated substitution yields: T(n) = Ο(n sqrt(2) n ) AINF1169 Informatics II 30

31 Summations: Arithmetico-Geometric Series By combining the arithmetic series and the geometric series by a term-by-term multiplication, we get a summation of the form n (a 0 + k d) b k = a 0 + (a 0 + d) b + (a d) b 2 + (a d) b (a 0 n d) b n k = 0 (with 0 < b 1, d = const and an integer n > 0). This is called a (finite) arithmetico-geometric series. As it is written above (i.e. starting at k = 0) sums up to a 0 1 b (a 0 + (n 1) d) b n 1 b + d b (1 bn 1 ) (1 b) 2 The infinite arithmetico-geometric series sums up to: (a 0 + k d) b k = k = 0 a 0 1 b + d b (1 b) 2 AINF1169 Informatics II 31

32 Midterm Preparation Examples Solved General Principles

33 Methods for Determining the Number of Executions of for Loops in Asymptotic Running Time Analysis Multiplicative combination of algorithm parts (and using «end start + 1» et al.) «Table method»: write down the evolution of the loop variable in a table to gain an understanding about the number of executions (probably not possible for more than two levels of nesting) Write loop as a nested multiple sum and solve it (requires a bit of knowledge how to manipulate sums). AINF1169 Informatics II 33

34 Algorithm Running Time Analysis: Another Example Exercise 2.3 from midterm 1 of spring semester 2016: Perform an exact analysis of the running time of the following algorithm and determine its asymptotic complexity. Algorithm: alg(a, n) 1 for i = 1 to n / 2 do 2 min = i 3 max = n i if A[min] > A[max] then 5 exchange A[min] and A[max] 6 for j = i + 1 to n i do 7 if A[j] < A[min] then 8 min = j 9 if A[j] > A[max] then 10 max = j 11 exchange A[i] and A[min] 12 exchange A[n i + 1] and A[max] AINF1169 Informatics II 34

35 Algorithm Running Time Analysis: «Table Method» Example The following example shows how to analyse the number executions of the body of the inner loop (lines 7 to 10) of the algorithm alg(a, n) using a table to track the evolution of the number of executions depending on the value of the outer loop variable. Outer loop variable value i = Inner loop variable range j = i+1 n i Number of executions of the inner loop body 1 2 n 1 n n 2 n n 3 n 6 k k+1 n k n 2 k n/2 n/2 +1 n/2 0 AINF1169 Informatics II 35

36 Algorithm Running Time Analysis: Yet Another Example Task 1 of Assignment 2 from FS 2018: AINF1169 Informatics II 36

37 Summation Rules and Examples Decomposing Sums Changing Summation Indexes Double Sums / Multiple Sums / Nested Sums

38 Summations: Decomposing Sums Indexes of sums can be rewritten / sums can be «taken apart» as in the following example: n x n x x 1 k = k k k = x k = 1 k = 1 sum up «too much», i.e. start summing up from 1 instead of starting from x remove what has been summed up too much, i.e. numbers from 1 to x 1 This is useful in solving sums stemming from nested loops with interacting loop variables like for example: <pseudo code> Im Falle, dass über eine Konstante aufsummiert wird, gilt einfach ENDE ANFANG + 1 AINF1169 Informatics II 38

39 Recursion Tree Example (Task 5a of assignment 2 from FS 2018) Solve the following recurrence using the tree method: T(n) = T(n/6) + T(n/2) + n T(1) = 1 AINF1169 Informatics II 39

40 Trees Nomenclature Binary Trees Binary Tree Traversals Binary Search Trees

41 Trees: Nomenclature height of tree = 3 height (of node) level siblings e.g. B and C are siblings root A B C D internal node e.g. D, G, B E F G H parent (ancestor / father) e.g. D is parent of H child (son) e.g. H is child of D 0 3 I J K L M N sub-tree leaf (external node), e.g. K, C, F AINF1169 Informatics II 41

42 Binary Tree Traversals: Introduction and Overview Tree traversals are rules on how to visit each node of a binary tree exactly once. There are three famous kinds of depth-first traversals and one kind of breadth-first traversal (visiting neighbors first). Depth-first traversals: Inorder: left root right Preorder: root left right Postorder: left right root Breadth-first traversal: Levelorder: from left to right, from root level to leaf level AINF1169 Informatics II 42

43 Binary Tree Traversals: Remarks Note that the sequence of nodes stemming from a particular traversal method in general does not allow to unambiguously reconstruct the original tree. An exception is the preorder traversal which allows to reconstruct the original tree in an unambiguous and efficient manner; therefore, the preorder traversal can be used (and considered as) as representational structure for trees, e.g. in order to store a tree in a file. AINF1169 Informatics II 43

44 Binary Tree Traversals Recursively apply the principle stated before to all nodes of the tree by beginning at the root node. For example: inorder traversal: Left Root Right Pseudo code: inorder(tree T) { inorder(left_subtree(t)); visit(root); inorder(right_subtree(t)); } AINF1169 Informatics II 44

45 Binary Tree Traversals: Example What are the inorder, preorder, postorder and levelorder traversals of the following binary tree? Inorder: 10, 14, 19, 27, 31, 35, 42 Preorder: 27, 14, 10, 19, 35, 31, 42 Postorder: 10, 19, 14, 31, 42, 35, 27 Levelorder: 27, 14, 35, 10, 19, 31, 42 AINF1169 Informatics II 45

46 Binary Tree Traversals: Visual Help preorder inorder postorder AINF1169 Informatics II 46

47 Binary Tree Traversals: Iterative vs. Recursive Implementation Instead of applying a recursive algorithm, the depth-first traversals (inorder, preorder, postorder) can also be implemented iteratively using a (explicit) stack. Similarly, a levelorder traversal can be implemented iteratively using a queue. 1 Step Stack content Output of function «next» AINF1169 Informatics II 47

48 Tree Traversals: Applications What are these tree traversal strategies useful for? Preorder: easy and efficient way to store binary search trees with the possibility to reconstruct them unambigously Inorder: retrieving the node values of a binary search tree in ascending order Postorder: important for compiler design (in assembly: operand operand opcode) Levelorder: used in heapsort algorithm AINF1169 Informatics II 48

49 Binary Search Trees A binary search tree (BST) is a binary tree where all descendent nodes to the left are smaller or equal than the ancestor node and all descendants to the right are bigger or equal than the ancestor node. Example: 19 Subtree with node values less than Subtree with node values bigger than AINF1169 Informatics II 49

50 Heaps Heap Conditions Representation of a Heap as an Array Heap Operations Heapsort

51 Heap Conditions A heap is a binary tree. All levels except the lowest are fully filled and the lowest level is occupied starting from left without gaps (= «nearly complete tree»). Keys in nodes of tree are ordered: in a max-heap: for all nodes, the key is bigger than or equal to all its children largest element at root in a min-heap: for all nodes, the father node is always smaller than or equal to his sons smallest element at root AINF1169 Informatics II 51

52 Representation of a Heap as an Array level Indexing starts at Indexing starts at Level Note: This kind of tree traversal is called level order. AINF1169 Informatics II 52

53 Addressing Parent and Child Nodes k n 2 n 1 n n 3 n 2 n 1 Getting to parent of node with index k Getting to left child of node with index k Getting to right child of node with index k Indexing starts at 1 k / 2 2 k 2 k + 1 Indexing starts at 0 (k 1) / 2 2 k k + 2 AINF1169 Informatics II 53

54 Heap Conditions: Exercise The following arrays shall each represent a binary heap. Do they fulfill the conditions of a max-heap? A: Not a heap, because the first element is neither the maximum nor the minimum of the elements 9 B: By drawing the corresponding heap, we can see that elements at indices 5 and 8 (starting from zero) violate the heap condition. Therefore it is not a heap AINF1169 Informatics II 54

55 Heaps: Building a Heap How to get to a heap from an unordered array of values? Let s assume that for a certain node, we knew that both its subtrees are already heaps: In this case, we can apply the same procedure which we used before to trickle down the nodes. Thus, we just start at the last node which has at least one child node and then work our way up to the root while making sure for each node, that everything below (its subtrees) are nice heaps. AINF1169 Informatics II 55

56 Heaps: Sorting Algorithm 1) Transform input data into heap. 2) Successively remove the root element and reorganize the remaining heap. AINF1169 Informatics II 56

57 Heaps: Demonstration: Removing the Root AINF1169 Informatics II 57

58 Heaps: Demonstration: Building a Heap AINF1169 Informatics II 58

59 Heaps: Comprehension Question Why do we iterate from n/2 to 1 and not from 1 to n/2 in the BuildHeap algorithm? (Does this even matter?) AINF1169 Informatics II 59

60 Heaps, Heapsort: DIY / Game 1) Form groups of two to three people. 2) Get a deck of playing cards. 3) (If necessary: Familiarize yourself with the ordering of the cards.) 4) Shuffle the cards. 5) Lay out 12 cards in form of a binary three before you on the desk. (If you got the Joker, replace it through another card.) 6) Transform the card into a max- or min-heap. 7) Get a sorted set of cards by successively removing the root element. 8) If you feel comfortable with heapsort or if you get bored: Take 8 random cards of your deck and perform bubble sort, selection sort, insertion sort, quicksort, mergesort, on it. AINF1169 Informatics II 60

61 Preview on Exercise 5

62 Exercise 5 Task 1: Heap and Heapsort: Implementation in C Look at the respective slides from the lecture: SL04, pp AINF1169 Informatics II 62

63 Exercise 5 Tasks 2 & 3: Heap Construction and Heapsort by Hand Look at the respective slides from the lecture: SL04, pp AINF1169 Informatics II 63

64 Exercise 5 Task 4: Quicksort Example by Hand Look at the respective slides from the lecture: SL04, pp AINF1169 Informatics II 64

65 Wrap-Up Summary Outlook Questions

66 Outlook Midterm 1: Friday, , 12:15 13:45 h Last minute Q&A: Friday, , 09:00 11:00 h Next tutorial: Tuesday, , BIN 0.B.06 Topics: Review of Exercise 5 Review of Midterm 1 Heapsort Quicksort Preview to Exercise 6 (your wishes) AINF1169 Informatics II 66

67 Questions? AINF1169 Informatics II 67

68 Thank you for your attention. AINF1169 Informatics II 68