Carat: Collaborative Energy Diagnosis on Mobile Devices

Transcription

1 Carat: Collaborative Energy Diagnosis on Mobile Devices Adam J. Oliner Kuro Labs AMP Lab, UC Berkeley Anand P. Iyer and Ion Stoica AMP Lab, UC Berkeley Eemil Lagerspetz and Sasu Tarkoma U of Helsinki

2 Mobile is Hot... sometimes, literally: 2

3 A Day in the Life 3

4 A Day in the Life 3

5 A Day in the Life 3

6 A Day in the Life 3

7 A Day in the Life 3

8 A Day in the Life 3

9 A Day in the Life 3

10 A Day in the Life 3

11 A Day in the Life 3

12 Users Questions Why is my battery draining? Is that normal? What can I do about it? 4

13 Today 1. Carat 2. Sampling to Diagnosis 3. Dealing with Uncertainty 4. Implementation 5. Deployment 5

14 1. Carat

15 Prior Approaches Ad hoc e.g., no-sleep bug Intrusive Generic Kill all background apps Dim the screen 7

16 Prior Approaches Ad hoc e.g., no-sleep bug Intrusive Generic Kill all background apps Dim the screen 7

17 the crowd the cloud big data instrumentation data raw and derived data Spark on EC2 DynamoDB and S3 actions and reports statistical analysis Our Approach

18 the crowd the cloud big data instrumentation data raw and derived data Spark on EC2 DynamoDB and S3 actions and reports statistical analysis First collaborative approach for diagnosing energy problems. Our Approach

19 Why Collaborative? Enables diagnosis Nearly impossible on a single device Normal? Trigger? Severity? Frequency? Distribute instrumentation overhead Compensate for biases and uncertainty 9

20 Design Goals Most invasive method that works on both Android and ios Eligible for Apple s App Store Investigate how far we can take diagnosis 10

21 Carat Mobile app for ios and Android Personalized energy debugging What is misbehaving Whether that is normal What you can do about it How much it will help 11

22 Carat

23 Operating Instructions Install Carat on your ios/android device Open daily+ for about a week Perform the actions it suggests Open occasionally for new actions 13

24 Privacy Concerns Opt-in EULA, prior to installation Nothing personally identifying Sampled data shown within the app 14

25 Open Source Both apps are on github FreeBSD license Analysis code is proprietary carat.cs.berkeley.edu 15

26 2. Sampling to Diagnosis

27 t 1 WiFi? ios 5.1?... Carat Sampling

28 t 1 WiFi? ios 5.1?... t 2 WiFi? ios 5.1?... Carat Sampling

29 t 1 WiFi? ios 5.1?... t 2 WiFi? ios 5.1?... t 3 WiFi? ios 5.1?... Carat Sampling

30 t 1 WiFi? ios 5.1?... t 2 WiFi? ios 5.1?... t 3 WiFi? ios 5.1?... t 4 WiFi? ios 5.1?... Carat Sampling

31 t 1 WiFi? ios 5.1?... t 2 WiFi? ios 5.1?... Computing Rates

32 t 1 WiFi? ios 5.1?... t t 2 WiFi? ios 5.1?... t Computing Rates

33 t 1 WiFi? ios 5.1?... % t 2 WiFi? ios 5.1?... % = discharge rate (%/s) t Computing Rates

34 t 1 WiFi? ios 5.1?... F t 2 WiFi? ios 5.1?... % = discharge rate (%/s) t F Computing Rates

35 F Probability Energy Rate (% / s) Energy Anomalies

36 F Probability Energy Rate (% / s) Energy Anomalies

37 F Probability Energy Rate (% / s) Energy Anomalies

38 energy hog F Probability Energy Rate (% / s) Energy Anomalies

39 Probability Energy Rate (% / s) Energy Anomalies

40 Probability Energy Rate (% / s) Without the crowd, there is no way to Energy Anomalies know whether this use is normal.

41 (Given is not a Hog.) Probability Energy Rate (% / s) Energy Anomalies

42 (Given is not a Hog.) energy bug Probability Energy Rate (% / s) Energy Anomalies

43 Original Distribution Probability Energy Rate (% / s) 21

44 Original Distribution E[reference] E[subject] Probability Energy Rate (% / s) 21

45 Original Distribution E[reference] E[subject] Probability D Energy Rate (% / s) 21

46 Mean Distribution E[subject] E[reference] Probability E[Energy Rate] (% / s) 22

47 Error and Confidence EV Probability E[Energy Rate] (% / s) 23

48 Error and Confidence EV ± e %/s 50% confidence Probability E[Energy Rate] (% / s) 23

49 Error and Confidence EV ± E %/s 95% confidence Probability E[Energy Rate] (% / s) 23

50 Probability Confidence Factors Energy Rate (% / s) EV 24

51 Probability Confidence Factors Energy Rate (% / s) EV 24

52 Probability Confidence Factors Energy Rate (% / s) EV 24

53 Significant Probability μ 2 μ 1 ε 2 d' ε 1 Subject Reference E[Battery Drain (% / s)] 25

54 Probability Probability Probability Classification Energy Rate (% / s) not-f F Energy Rate (% / s) Energy Rate (% / s) 26

55 Probability Probability Probability Classification Energy Rate (% / s) not-f F Energy Rate (% / s) Energy Rate (% / s) 26

56 Probability Probability Probability Classification Energy Rate (% / s) not-f F Energy Rate (% / s) Energy Rate (% / s) 26

57 Diagnosis a not-v x 27

58 Diagnosis a Killing app a will give x±e of battery life not-v (95% confidence), as would upgrading the x OS to version v. 27

59 3. Dealing with Uncertainty

60 Trigger Measured % Actual % BatteryLevelChanged x x else x (x-5, x] Measurement Uncertainty

61 Trigger Measured % Actual % BatteryLevelChanged x x else x (x-5, x] 85% 85% Measurement Uncertainty

62 Trigger Measured % Actual % BatteryLevelChanged x x else x (x-5, x] 85% 85% 85% (80%,85%] Measurement Uncertainty

63 Trigger Measured % Actual % BatteryLevelChanged x x else x (x-5, x] 85% 85% Prior 85% (80%,85%] Measurement Uncertainty

64 Probability Prior Energy Rate (% / s) Measurement Uncertainty

65 Battery % Time actual Probability Prior Energy Rate (% / s) Measurement Uncertainty

66 Battery % x x x x actual samples Time Probability Prior Energy Rate (% / s) Measurement Uncertainty

67 Battery % x x x x actual samples Time Probability Prior Energy Rate (% / s) Measurement Uncertainty

68 Battery % x x x x actual samples x Time y Probability Prior Energy Rate (% / s) Measurement Uncertainty

69 Battery % x x x x actual samples x Time y y Probability x Prior Energy Rate (% / s) Measurement Uncertainty

70 Ground Truth Does sampling match reality? What does sampling cost? 31

71 Ground Truth iphone 4S + Monsoon Power Monitor Galaxy Tab Leyden Energy Usage scripts 32

72 Customer Service

73 Ground Truth Battery indicator agrees with the monitor :00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 Battery level (%) On-Screen Battery Indicator [Redacted] Carat Samples Samples Power Monitor with Carat [Redacted] Samples Power Monitor with Weather Time (HH:MM) 34

74 Ground Truth Good accuracy with low overhead Density Power Monitor (w/ Carat) Power Monitor (w/out Carat) Carat Energy Discharge Rate (%/s) 35

75 Ground Truth Discharge rate estimation Android: %/sec ios: %/sec Overhead 3.5% of the battery less than Weather Less than experimental imprecision 36

76 4. Implementation

77 the crowd the cloud big data instrumentation data raw and derived data Spark on EC2 DynamoDB and S3 actions and reports statistical analysis Carat Infrastructure

78 Spark Cluster computing framework Resilient Distributed Datasets (RDDs) Read-only in-memory data structures Parallel operations (e.g., reduce) Iterative or interactive workloads 39

79 Rates to Distributions Rate RDDs from sample pairs Compute conditional distributions Leverage Spark parallelism r 1 ĉ map c r {0,1} r n ĉ c r {0,1} reduce map c c r... count r count groupby c c {{r,count}}... {{r,count}} 40

80 Server Traffic Bandwidth (bytes/s) Actual 1 byte/sec/client Number of Clients 41

81 Parallelization 42

82 5. Deployment

83 Initial Deployment TestFlight over-the-air beta 100 sign-ups (Apple s max) 75 installs Collected 10,000+ samples 44

84 Initial Results Found 35 apps exhibiting energy bugs e.g., Facebook, Kindle, Flipboard Corroborated with forum posts, news articles, and data correlations Injected three bugs in Wikipedia Detected all of them 45

85 Suddenly...

86 Suddenly...

87 Suddenly...

88 Suddenly...

89 Suddenly...

90 Carat went viral. In 24 hours, there were dozens of articles and we had more than 100,000 users. Suddenly...

91 Carat Today 450,000+ devices 60% ios 40% Android Tens of millions of samples 47

92 Energy Anomalies Hogs 11,256 hogs (9.4%) e.g., Pandora and Skype Bugs 483,354 buggy instances (5.3%) e.g., Kindle, Facebook, and YouTube 48

93 Kindle Bug (ios) E-book reader Bug on 3.9% of clients Forum: WhisperSync Confirmed by our data Turn on WiFi 36m improvement 49

94 Kindle Diagnosis Network Off 12.5 h With Kindle 8.4 h 3G 6.3 h Wi-Fi 6.9 h Without Kindle 7.7 h 50

95 Twitter Bug (Android) Microblogging app Bug on 14.9% of clients MCAD implicates OS WiFi also helps Upgrade to ICS m improvement 51

96 Battery Life +10% after 10 days +30% after 90+ days Relative Battery Life Days Since First Report 52

97 Error Convergence Error bounds converge in practice Relative EV and Error Sample Count Expected Value Error Upper Bound Error Lower Bound 53

98 Prediction Accuracy 95.4% within confidence bounds Empirical Slope Perfect Prediction Within 95% Confidence Bounds Outside 95% Confidence Bounds Projected Slope 54

99 Positive Feedback Carat has given me control of my phone in a way I never had before. Thank you! :-D In closing, I just want to say thank you & keep up the great work w/carat. It has to be one favorite apps on my Droid. I ve been measuring my battery with and without its recommendations for the past month, and I m getting about extra minutes of battery life from my phone just wanted to tell you that I love your app - in fact, it s the only one that I keep running all the time on my iphone 5. I absolutely love the app. Very, very useful. I love this app. Very very very useful. 55

100 the crowd the cloud big data instrumentation data raw and derived data Spark on EC2 DynamoDB and S3 actions and reports statistical analysis Our Approach

101 the crowd the cloud big data link structure A links-to B The Google Model

102 the crowd the cloud big data link structure A links-to B search PageRank The Google Model

103 the crowd the cloud big data A friends-with B social platform The Facebook Model

104 the crowd the cloud big data personal data A friends-with B social platform The Facebook Model

105 the crowd the cloud big data To define and build the layer between the crowd and big data. This layer extracts data and computation from the crowd, and extracts value from big data for the crowd. AMP Lab Vision

106 the crowd the cloud big data instrumentation data raw and derived data dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy actions and reports statistical analysis Kuro Labs

107 the crowd the cloud big data instrumentation data raw and derived data dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy actions and reports statistical analysis Kuro Labs

108 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Kuro Labs

109 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Kuro Labs

110 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Kuro Labs

111 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Kuro Labs

112 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Kuro Labs

113 the crowd the cloud big data instrumentation data raw and derived data X actions and reports statistical analysis dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy dsa gfdg gfdg gfdg gfdw hy Stats as a Service Kuro Labs

114 Next Steps API for developers New application domains Deploy to the crowd; diagnose in the cloud Platform for collaborative diagnosis Statistics as a service kuro-labs.com 61

115 Fin You have reached the end of the presentation. Please turn back. 62