Robust Programs with Filtered Iterators

Transcription

1 Robust Programs with Filtered Iterators Jiasi Shen, Martin Rinard MIT EECS & CSAIL 1

2 Standard Scenario Input file Program Output 2

3 Structured Input Units Input Input unit Input unit Input unit unit Program Output 3

4 Request Request Request Request Server 4

5 Video Video Videoframe Videoframe frame frame Video player 5

6 Data analytics 6

7 Unanticipated Corner Cases Input Input unit Input unit Input unit unit Program Output 7

8 Unanticipated Corner Cases Input Input unit Input unit Input unit unit Program Output 8

9 Unanticipated Corner Cases Input Input unit Input unit Input unit unit Program 9

10 Easy to avoid? 10

11 User Study 11

12 Small Programming Task Original image Thumbnail 12

13 Small Programming Task Example input Img Img Img Img Example output Img1 2 Img Img3 Img4 3 13

14 Small Programming Task Original image Img Image Name Scaling factor Height Width Pixels 14

15 Small Programming Task Original image Img Image Name Scaling factor Height Width Pixels

16 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 16

17 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 17

18 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 3 _ ( ) / 4 = 3 18

19 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 3 5 ( ) / 4 = 5 19

20 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels _ ( ) / 4 = 4 20

21 Small Programming Task Original image Thumbnail Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels ( ) / 4 = 3 21

22 Your program should be able to handle arbitrary inputs by skipping malformed images. 22

23 Defects by MIT Participants Defect Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X MP X X X MS X X WP X X WS X X WM X X X X WA X X Total

24 Defect AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X MP X X X MS X X WP X X WS X X WM X X X X WA X X Total s = 0;... while ( c!= '\n' ){... s = s * 10 + c-'0';... c = read(f); } redh = h/s; Illegal input, unanticipated 24

25 Defect AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X MP X X X MS X X WP X X WS X X WM X X X X WA X X Total imgsize = h * w; img = malloc(imgsize);... nh = h / s; nw = w / s; while(i<nh) {... while(j<nw) {... res = res + img[(i*s+ni)* w + (j*s+nj)];... Legal input, extreme cases 25

26 Defect AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X MP X X X MS X X WP X X WS X X WM X X X X WA X X Total fn = malloc(11);... while (i < 11) { c = read(f);... if (c == ' ') { break; }... fn[i] = c; i = i+1; } fn[i] = 0; Legal input, developer mistake 26

27 Input Units and Defects All possible input units Legal input units Illegal input units Extreme cases Developer mistakes Program doesn t crash on these input units Unanticipated 27

28 Bad Input Units Cause Crashes All possible input units Legal input units Illegal input units Program crashes on these bad input units 28

29 Unanticipated Corner Cases Input Input unit Input unit Input unit unit Program 29

30 Fix: Discard and Continue Execution Discard Input Input unit Input unit Input unit unit New program Output 30

31 Fix: Discard and Continue Execution Discard Input Input unit Input unit Input unit unit Continue execution New program Output 31

32 Fix: Discard and Continue Execution As if the bad input unit never existed Input unit Input unit Input unit Continue execution New program Output 32

33 Behavior Appears Repeatedly Applications and input units Wireshark (packets) GIMP (images) Claws Mail (message options) Chromium (CSS attributes) Fixed bugs by conceptually discarding the bad input units and continuing execution Other potential applications Embedded systems (events) Network routers (packets) Other input formats with input units (chunks, files, objects, ) Servers (requests) Data analytics (rows) Video players (frames) Document editors (lines, data sheets) F. Long et al, Automatic Runtime Error Repair and Containment via Recovery Shepherding, PLDI 14 33

34 Goal: Automatically Discard Bad Input Units Discard Input Input unit Input unit Input unit unit Continue execution Program Output 34

35 Goal: Automatically Discard Bad Input Units Program doesn t crash on any input unit Illegal input units Legal input units Automatically skip these input units Process these input units 35

36 Provide the Abstraction as a Language Construct 36

37 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 37

38 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 38

39 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 39

40 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 40

41 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 41

42 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { abort transaction } else{ commit transaction release outputs }}} 42

43 Schema of Filtered Iterators split input into input units iterate over input units { atomic transaction { delay outputs until commit process input unit if unhandled exception or assertion failure { }}} abort transaction } else{ commit transaction release outputs Continue execution as if bad input units never existed 43

44 Input Input unit Input unit Input unit unit split input into input units process input unit Output 44

45 Filter Out Bad Input Units Based on Execution Errors Discard Input Input unit Input unit Input unit unit split input into input units Continue execution process input unit Output 45

46 All possible input units Legal input units Illegal input units Program doesn t crash on these input units 46

47 Achieved: Automatically Recover from Bad Input Units Program doesn t crash on any input unit Illegal input units Legal input units Automatically skip these bad input units process input unit 47

48 Achieved: Automatically Recover from Bad Input Units Discard Input Input unit Input unit Input unit unit split input into input units Continue execution process input unit Output 48

49 Achieved: Automatically Recover from Bad Input Units As if the bad input unit never existed Input unit Input unit Input unit split input into input units Continue execution process input unit Output 49

50 All possible input units Legal input units Illegal input units Program doesn t crash on these input units 50

51 Not A Goal: Discard All Illegal Input Units All possible input units Legal input units Illegal input units Not a goal to discard all illegal input units Program doesn t crash on these input units 51

52 Not A Goal: Produce Correct Outputs for All Processed Input Units All possible input units Legal input units Illegal input units Program doesn t crash on these input units Not a goal to be logically correct 52

53 RIFL (Robust Input Filtering Language) Research Vehicle 53

54 Syntax for Text Files inspectt (e, f, du) { } process input unit e loop condition f input file handle du end-of-unit delimiter 54

55 CSV Example A,B,C 1,2,3 inspectt (!end(f), f, \n ) {... inspectt (..., f,, ) {... }} A,B,C 1,2,3 A B C 55

56 Syntax for Binary Files inspectb (e, f, o, w) { } process input unit e loop condition f input file handle o offset of length field w width of length field 56

57 PCAP Example Packet n 12 bytes 4 bytes n bytes Packet inspectb (true, f, 12, 4) { }... 57

58 Consequences of Filtered Iterators 58

59 User Study Participants: Computer science graduate students and post-docs at MIT RIFL group Control group: RIFL excluding filtered iterators Thumbnail generator task Time: Unlimited (took minutes) 59

60 Images are Input Units Example input Img Img Img Img Example output Img1 2 Img Img3 Img4 3 Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 60

61 Benign Input Units Example input Img Img CharS b CharPix a4 BufOvfVeryLongName Div0S Div0H HeapOvf BufOvfInt Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 61

62 Illegal Input Units Example input Img Img CharS b CharPix a4 BufOvfVeryLongName Div0S Div0H HeapOvf BufOvfInt Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 62

63 Tricky Input Units Example input Img Img CharS b CharPix a4 BufOvfVeryLongName Div0S Div0H HeapOvf BufOvfInt Img Img Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 63

64 Handle arbitrary inputs by skipping malformed images Example input Img Img CharS b CharPix a4 BufOvfVeryLongName Div0S Div0H HeapOvf BufOvfInt Img Img Example output Img1 2 Img Img3 Img4 3 Image Name Scaling factor Height Width Pixels Thumbnail Name Pixels 64

65 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

66 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

67 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

68 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

69 Possible Fatal Defects Defect AWL AWO ARL ARO DS DD NA IL Description Out-of-bounds array write when reading input, triggered by input fields that are longer than an input buffer. Out-of-bounds array write when reading input, triggered by an integer overflow that causes overly small memory allocation. Out-of-bounds array read during computation, triggered by image dimensions that are too large for an input buffer. Out-of-bounds array read during computation, triggered by an integer overflow that causes overly small memory allocation. Division by zero during computation, triggered by a zero scaling factor. Division by zero when checking integer over ow, triggered by a zero dimension. Null array access when reading input, triggered by failed memory allocation. Infinite loop when reading illegal input units. 69

70 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

71 Possible Other Defects Defect MP MS WP WS WM WA Description Memory leak even when processing common legal input units. Memory leak when skipping input units. Wrong behavior from producing partial outputs for illegal input units. Wrong behavior from desynchronization for at least one input unit after illegal input units. Wrong behavior from misusing illegal input units and producing outputs for these illegal input units as if they are legal. Wrong behavior from aborting on illegal input units. 71

72 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

73 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

74 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

75 Data Corruption or Input Desynchronization Defect Control 1 Control 2 Control 3 Control 4 Control 5 Input CharTrail b Img AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X Output CharTrail 2 IL X Fatal MP X X X MS X X Img5 2 WP X X WS X X WM X X X X WA X X Other Total

76 Undesirable Partial Output Defect Control 1 Control 2 Control 3 Control 4 Control 5 Input Short Img AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Output Short2EEEEE 2 2 Fatal MP X X X MS X X WP X X WS X X WM X X X X WA X X Other Total

77 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

78 Fewer Defects with Filtered Iterators Defect RIFL 1 RIFL 2 RIFL 3 RIFL 4 RIFL 5 Control 1 Control 2 Control 3 Control 4 Control 5 AWL X X X AWO ARL X ARO X X X X DS X X X X DD NA X X X X X IL X Fatal MP X X X X X MS X X X WP X X WS X X WM X X X X X X X WA X X Other Total

79 Fatal and Non-Fatal Defects in Control Group All possible input units Legal input units Illegal input units Program doesn t crash on these input units 79

80 Non-Fatal Defects in RIFL Group Program doesn t crash on any input unit Illegal input units Legal input units Automatically skip these bad input units process input unit 80

81 RIFL group (filtered iterators) Control group More Correct Behavior with Filtered Iterators 81

82 Simpler Code with Filtered Iterators 50 Cyclomatic Complexity RIFL group Control group 82

83 Simpler Code with Filtered Iterators 50 Cyclomatic Complexity RIFL group Control group 83

84 Simpler Code with Filtered Iterators 50 Cyclomatic Complexity RIFL group Control group 84

85 Simpler Code with Filtered Iterators 50 Cyclomatic Complexity RIFL group Control group 85

86 Simpler Code with Filtered Iterators Cyclomatic Complexity Lines of Code RIFL group Control group RIFL group Control group Omit unnecessary checks for crashes Check semantic errors with assertions without having to elaborate error recovery Focus on main functionality 86

87 Automatically Handle Errors That Would Otherwise Cause Crashes or Assertion Failures 87

88 Automatically Handle Corner Cases That Developers Don t Want to Handle Exhaustively 88

89 Potential Limitations Debugging Can make unintentional mistakes silent Error log or IDE support Assumptions Structured input units Obtaining partial results is preferable to terminating 89

90 Related Work Exception handling J. B. Goodenough. Exception Handling: Issues and a Proposed Notation. Commun. ACM 1975 Recovery by manipulating execution M. Rinard et al. Enhancing Server Availability and Security Through Failure-oblivious Computing. OSDI 2004 S. Sidiroglou and A. D. Keromytis. Using Execution Transactions To Recover From Buffer Overflow Attacks. Technical Report 2004 Language designs involving transactions A. Shinnar et al. Integrating support for undo with exception handling. Technical Report 2004 B. Demsky and A. Dash. Bristlecone: A Language for Robust Software Systems. ECOOP 2008 A. Warth et al. Worlds: Controlling the Scope of Side Effects. ECOOP

91 Conclusion Filtered iterators Iterate over input units Filter out bad input units when errors occur Atomic rollback of updates Inspired by developer bug fixes Enable more robust and simpler programs Abilities verified by user study 91