Data Structures. Outline. Introduction. Andres Mendez-Vazquez. December 3, Data Manipulation Examples

Transcription

1 Data Structures Introduction Andres Mendez-Vazquez December 3, / 53 Outline 1 What the Course is About? Data Manipulation Examples 2 What is a Good Algorithm? Sorting Example A Naive Algorithm Counting Steps About the Worst Case A more realistic step count Definition of Big O Meaning for insertion sort 3 Examples of Complexities Problems Calculating Complexities 2 / 53

2 Introduction Data Manipulation Data structures is concerned with the representation and manipulation of data. Further All programs manipulate data. Thus!!! So, all programs represent data in some way. Data manipulation requires a data structure and an algorithm!!! 4 / 53 So you must go beyond... First Believing that coding is the only think that you need to be a Computer Scientist Coding is only a tool to express our designs, our thoughts!!! Second Assuming that you can get away from mathematics!!! Computer Science is Mathematics at its core!!! 5 / 53

3 Examples Data Bases Using B-trees for fast access to the records!!! 7 / 53 Example Storing Sparse Matrices Sparse Matrix 5x5 Matrix Numeric Elements Empty Elements Why? Reduce the amount memory used for storage!!! 8 / 53

4 Example Shortest Path in a Map 9 / 53 What is the course about? First Understanding Data Structure and Algorithms needed to develop programs for data manipulation. Why? The study of Data Structures and Algorithms is fundamental to Computer Science. 10 / 53

5 In sorting, we want to have the following Rearrange a sequence of numbers 9,3,4,2,1 Into an increasing sequence 1, 2, 3, 4, 9 Or Into a decreasing sequence 9, 4, 3, 2, 1 12 / 53 Notation For now, we will assume that the data It is an array A. Meaning item index Thus If we need to reference elements on the array through an index, we use the following notation A [index] (1) 13 / 53

6 Naive Algorithm What if we do the following 1 Pick an element in the array A. 2 Put that element in the correct position. Thus We will concentrate first in inserting the element in the correct position For this imagine the following We already have an array A = [ ] where - represents null elements. 15 / 53 Thus Where do we start comparing to insert? Ideas? Maybe From the right!!! After all we have space Let us do it with A = [ ] 16 / 53

7 What about the code? What do we use for going through a sequence? Ideas? In addition, we need to make space For the new item. Code p r i v a t e i n t [ ] i n s e r t ( i n t [ ] A, i n t t ){ // E x t r a V a r i a b l e s i n t j ; // I n s e r t i n t o A [ 0 : i 1] f o r ( j = A. l e n g h t 1; j >=0 && t<a [ j ] ; j ){ // s h i f t to t h e r i g h t A [ j +1]=A [ j ] ; } A [ j +1]= t ; r e t u r n A ; } 17 / 53 Ok, We have that...now what? How do we use this piece of code for the insertion sort? Ideas? Problem with the code!! You require that the array where you insert to be sorted!!! Thus How do we solve this problem? Look at this!!! At the Board / 53

8 Then, Insertion Sort works like this First Start with a sequence of size 1. After all an array with one element is sorted!!! Second Repeatedly insert remaining elements Example Sort 7, 3, 5, 6, 1 19 / 53 Final Code Something Notable // S o r t A assume i s f u l l p u b l i c i n t [ ] I n s e r t i o n S o r t ( i n t [ ] A){ // E x t r a Space i n t B [ ] = new i n t [ A. l e n g t h ] ; // I n i t A r r a y B B[0]=A [ 0 ] ; i n t s i z e = 1 ; i n t j, t ; f o r ( i n t i = 1 ; i < A. l e n g t h ; i ++){ t = A [ i ] ; f o r ( j = s i z e 1; j >=0 && t<b [ j ] ; j ) { // s h i f t to t h e r i g h t B [ j +1]=B [ j ] ; } B [ j +1]= t ; s i z e ++; } r e t u r n B ; } 20 / 53

9 Question: Which Complexity You Have with Insertion Sort? After all you have!!! Time Complexity!!! For this, we have Time Complexity Techniques: Count a particular operation Count number of steps Asymptotic complexity 22 / 53 Basically, we will use the second one First Look at this part of the code f o r ( j = s i z e 1; j >=0 && t<b[j] ; j ) B [ j +1]=B [ j ] ; Then How many comparisons are made? 23 / 53

10 The Possible Cases Worst-case count Maximum count Best-case count Minimum count In addition, we have Average Count 25 / 53 Example: Worst-Case For the Code Code, Look at the red compares f o r ( j = s i z e 1; j >=0 && t<b[j] ; j ) B [ j +1]=B [ j ] ; Case I If B = [1, 2, 3, 4] and t = 0 = 4 compares. What about the general case If B = [1, 2, 3,..., i] and t = 0 = i compares. 26 / 53

11 Worst-Case For An Almost Final Version Simplified Version \\Remember s i z e i s t h e s i z e o f B f o r ( i = 1 ; i < A. l e n g t h ; i++ ) f o r ( j = s i z e 1; j >=0 && t<b[j] ; j ) B [ j +1]=B [ j ] ; Worst Case for the Outer Loop 1 Outer loop takes n steps if A.length== n What about the inner loop? Answer Number of steps in the inner loop = size+1 27 / 53 What is the range of j? Very Simple For the first step j = 1 1 = Number of Steps = 2 (0 and -1) For the last step j = n 1 1 = Number of Steps = n Here, we have n 1 passing statements and 1 that gets us out the loop. Total Compares n = n (n + 1) 2 1 (2) 28 / 53

12 A more realistic step count Counting when A.length = n // S o r t A assume i s f u l l p u b l i c i n t [ ] I n s e r t i o n S o r t ( i n t [ ] A){ Step // I n i t i a l V a r i a b l e s 0 i n t B [ ] = new i n t [ A. l e n g t h ] ; 1 i n t s i z e = 1 ; 1 i n t i, j, t ; 1 // I n i t i a l i z e t h e A r r a y B 0 B[0]=A [ 0 ] ; 1 f o r ( i = 1 ; i < A. l e n g t h ; i ++){ n t = A [ i ] ; n 1 f o r ( j=s i z e 1; j >=0&&t<B [ j ] ; j ) i +1 { // s h i f t to t h e r i g h t 0 B [ j +1]=B [ j ] ; } i B [ j +1]= t ; n 1 s i z e ++; n 1 } r e t u r n B ; 1 } 30 / 53 The Result Step count for body of for loop is The summation n 1 n (n 1) + n + (i + 1) + (i) (3) They have the quadratic terms n 2. Complexity Insertion sort complexity is O ( n 2) i=1 j=1 31 / 53

13 Definition of Big O Definition For a given function g(n) O(g(n)) ={f (n) There exists c > 0 and n 0 > 0 s.t. 0 f (n) cg(n) n n 0 } Graphically 33 / 53 What? The term n 0 It tells you when for the following n s you will have that f (n) cg(n) What about the so called c It can be seen a way to increase or decrease the height of the function!!! 34 / 53

14 What does this means for insertion sort? We have n 1 n (n 1) + n + (i + 1) + (i) = n + n (n 1) 2 i=1 j=1 n (n 1) + n 1 + = n + n(n 1) =... n 2 + 4n + 2 n 2 + 4n 2 + 2n 2 Thus n 2 + 4n + 2 7n 2 (4) With T insertion (n) = n 2 + 4n + 2 describing the number of steps for insertion when we have n numbers. 36 / 53 Actually For n 0 = = 14 < = 28 (5) Graphically 37 / 53

15 Meaning First Time or number of operations does not exceed cn 2 for a constant c on any input of size n (n suitably large). Questions Is O(n 2 ) too much time? Is the algorithm practical? For this imagine, we have a machine able to make 10 9 instructions/seconds 38 / 53 Then We have the following n n n log n n 2 n 3 n micros 10 micros 1 milis 1 second 17 minutes 10, micros 130 micros 100 milis 17 minutes 116 days milis 20 milis 17 minutes 32 years years It is much worse n n 10 2 n years years 10,000?????? 10 6?????????? The Reign of the Non Polynomial Algorithms 39 / 53

16 Basically These complexities allow us to compare algorithms You can compare 2 algorithms having different asymptotic complexity For example two types of sorting algorithms with O(n) and O(n 2 ) complexities. However This notation does not account for constant factors. For example For many cases of n, n is much worse than n / 53 A Huge Problem for calculating complexities The memory hierachy: Slower Memories as further you go from the CORE L3 Cache To Main Memory L2 Cache L1i Cache L1i Cache L2 Cache L1d Cache CPU Core 1 CPU Core 2 L1d Cache L1d Cache P-to-P CPU Core 3 CPU Core 4 L1d Cache L2 Cache L1i Cache L1i Cache L2 Cache Note: I will recommend to read What Every Programmer Should Know About Memory by Ulrich Drepper Red Hat, Inc. 42 / 53

17 Examples B-Trees are generalized Binary Search Trees With element in the nodes sorted in increasing order 43 / 53 Example We can improve B-Tree policies to find indexes inside nodes by Binary Search Linear Search 44 / 53

18 Here, a reasonable assumption!!! Classic Analysis Binary Search Faster than Linear Search!!! Node Structure a (i) a (i+1) i However, Core i7 has prefetching Thus, for a certain size of a node, linear searching will be faster than binary search due to prefetching hardware!!! 45 / 53 Main Problem Main problem Our analysis does not account for this difference in memory access times. Thus What do we need? What? A way to measure the time in our Java programs. 46 / 53

19 What do we need? Data 1 Worst-case data 2 Best-case data 3 Average-case data In addition A way to measure the time in a machine and language Problem We require certain degree of accuracy that we do not have 47 / 53 What do we have? Java Instruction System.currentTimeMillis() It returns the current time in milliseconds. The granularity of the value depends on the underlying operating system and may be larger. Yes!!! The accuracy is not great for it!!! 48 / 53

20 Possible Solution What to do? Ideas? What about this logic? 1 Assume that you have a 100 millisecond accuracy in a Linux System. 2 Assume that you want to have a measurement error of 10%. 3 What do we do? 49 / 53 Simple Idea for Accuracy If we want to have a 10% error We need to measure 100% of the total time. In the case of 100 milliseconds We need to measure operations for 1000 milliseconds!!! Or trueelapsedtime = finishtime - starttime +/- Error 50 / 53

21 First Solution Repeat method until you accumulate enough time For this, if you have 100 milliseconds for the code to be measured, then you need a Time 1000 milliseconds to obtain an accuracy of 10%. First we device the timing code // g i v e s time i n m i l l i s e c o n d s s i n c e 1/1/1970 GMT l o n g s t a r t T i m e = System. c u r r e n t T i m e M i l l i s ( ) ; // code to be timed comes h e r e l o n g e l a p s e d T i m e = System. c u r r e n t T i m e M i l l i s () s t a r t T i m e ; 51 / 53 What is the problem with this Solution? PROBLEM!!! l o n g s t a r t T i m e = System. c u r r e n t T i m e M i l l i s ( ) ; l o n g c o u n t e r ; // Put code to i n i t i a l i z e a [ ] h e r e // Go f o r randomized a r r a y do{ c o u n t e r ++; SomeClass. I n s e r t i o n S o r t (A ) ; } w h i l e ( System. c u r r e n t T i m e M i l l i s () s t a r t T i m e < 1000) l o n g e l a p s e d T i m e = System. c u r r e n t T i m e M i l l i s () s t a r t T i m e ; f l o a t timeformethod = ( ( f l o a t ) e l a p s e d T i m e )/ c o u n t e r ; 52 / 53

22 Fix Solution l o n g s t a r t T i m e = System. c u r r e n t T i m e M i l l i s ( ) ; l o n g c o u n t e r ; do{ c o u n t e r ++; // Move t h e code to i n i t i a l i z e a [ ] h e r e // Go f o r randomized a r r a y SomeClass. I n s e r t i o n S o r t (A ) ; } w h i l e ( System. c u r r e n t T i m e M i l l i s () s t a r t T i m e < 1000) l o n g e l a p s e d T i m e = System. c u r r e n t T i m e M i l l i s () s t a r t T i m e ; f l o a t timeformethod = ( ( f l o a t ) e l a p s e d T i m e )/ c o u n t e r ; 53 / 53