Principles and Design of IoT systems Level 11 course [20 credits] Week 3. Professor D K Arvind dka AT inf.ed.ac.uk

Transcription

1 Principles and Design of IoT systems Level 11 course [20 credits] Week 3 Professor D K Arvind dka AT inf.ed.ac.uk

2 References J.J. Carr, Sensors and Circuits Prentice Hall. Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. Statistician. 1983;32: / Davide Giavarina Understanding Bland Altman analysis Biochem Med (Zagreb) Jun; 25(2): Harris and Taylor Medical Statistics Made Easy 3e Scion Publishing Ltd ISBN

3 Characterisation and Calibration of Sensors 3

4 Why calibrate low-cost sensors? Variation due to manufacturing might yield slightly different readings from sensors from the same production run Differences in sensor design sensors respond differently under similar conditions, especially if they use different methods to measure the same parameter Sensors subjected to shock, humidity, heat and cold during storage, transport, assembly may show changes in response Some sensors age and their response change over time requiring periodic re-calibration In the case of analog sensors, the Analog-to-Digital Converter (ADC) is subject to variability Inertial sensors are sensitive to alignment with the system being measured. 4

5 Relative Humidity Sensors (hygrometer) Sensor measures the moisture and temperature in the air and reports rh as a percentage of the ratio of moisture in the air to the max amount that can be held in the air at that temperature. 3-types : Capacitive, Resistive, Thermal Capacitive Thin strip of metal oxide placed between two electrodes changes its electrical capacitance with the atmosphere s relative humidity Resistive changes in the resistance of the electrodes on either side of the salt medium as the humidity changes Thermal two thermal sensors (encased in dry Nitrogen) and the other exposed to the air, conduct electricity proportional to the ambient humidity 5

6 Comparison of Relative Humidity Sensors 6

7 Characteristics of sensors Range is the maximum and minimum value range over which a sensor works well Accuracy Given an absolute standard, accuracy is the amount of uncertainty in any measurement Precision reproducibility of the measurements Resolution the ratio between the max signal measured to the smallest part that can be resolved with an ADC Sensitivity the smallest change that can be measured Hysterisis - The sensor will tend to read low with an increasing signal and high with a decreasing signal. Linearity the output is directly proportional to the input Response time time for the sensor to change to a new output state 7

8 Accuracy Effects of errors due to gain and offset parameters Offset errors given as a unit of measurement such as volts or ohms and independent of the magnitude of the input signal being measured, regardless of the range or gain settings, e.g. ±1.0 millivolt (mv) Gain errors expressed as a percentage of the magnitude of the input signal, e.g., ±0.1% Total accuracy = ±(0.1% of input +1.0 mv). For input ranging from 0 to 10V : Input voltage 0V 5V 10V Range of readings within accuracy spec. -1 mv to +1 mv 4.994V to 5.006V (±6 mv) 9.989V to V (±11 mv) 8

9 9

10 Precision: Temperature from 5 co-located temperature sensors 10

11 Precision: Relative humidity from 5 co-located sensors 11

12 Resolution Sensor capable of making measurements ±10V range (20V span) using a 16-bit ADC Smallest possible theoretical change detected is 1 part in 2 16 i.e., 20V = 305 uv per ADC count 16 (2 4 ) counts of noise resolves the accuracy down to 12 bits Averaging can reduce noise by the square root of the number of samples, but at the cost of speed. In a system with 3 bits of noise, i.e. 8 (2 3 ) counts of noise averaging 64 samples will reduce noise contribution to 1 count ( 64 = 8: 8 8 = 1) Assumptions: technique cannot deal with effects of nonlinearity and noise must have a Gaussian distribution 12

13 Sensitivity Sensor with ±1.0 volt input range and ±4 counts of noise ADC resolution is (2 4 ) counts of noise resolves the accuracy down to 12 bits Peak-to-peak sensitivity is ±4 counts x (2 4096) or ±1.9mV p-p For a sensor rated for 1000 units with an output voltage of 0-1 volts (V), 1 unit is equivalent to 1 mv Sensitivity is 1.9mV p-p, taking 2 units for input to detect a change 13

14 Hysterisis Hysterisis is a measure of the property that the sensor output changes is independent of the direction of changes of the input parameter. 14

15 Response Times The Response Time (Tr) is the time taken by the sensor output to change state to a final settled value within a tolerance band in response to a change in its input Below-left: response time to a postive step-function change in the sensor input Below-right: shows the decay time (T d ) in response to a negative stepfunction change in the sensor input Time Constant (T) is similar to the time taken for a capacitor to charge through a resistance and is less than the Response Time 15

16 Linearity Measure of the sensor output departing from the ideal curve Affected by temperature, vibration, acoustic noise level, and humidity Nonlinearity (%) = Dmax/Input Value x100 16

17 Dynamic Non-linearity Measure of the sensor s ability to follow rapid changes in the input parameter determined by mplitude distortion characteristics, phase distortion characteristics, and response time F(X) = ax + bx 2 + cx 3 + dx K F(X) is the output; and X is the input and its harmonics; K is an offset constant Signifcant when the error harmonics generated by the sensor action fall into the same frequency bands as the natural harmonics produced by the dynamic action of the input parameter. 17

18 Calibration of Sensors 18

19 Accuracy is a combination of precision, resolution and calibration A sensor that gives repeatable measurements with a good resolution, can be calibrated for accuracy Calibrated against a standard reference a sensor known to be accurate and calibrated against a NIST standard Standard physical reference for some types of sensors» Rangefinders rulers and metre stciks» Temperature Boiling water and Ice-water bath» Accelerometer gravity is 1G on the surface of the earth 19

20 Characteristic Curve: Thermocouple response to input 20

21 Calibration process maps the sensor s response to an ideal response Offset - sensor is constantly higher or lower than ideal response and corrected with a single-point calibration Sensitivity/Slope difference in slope implies sensor output changes at a different rate than the ideal and corrected with a two-point calibration Linearity sensors may have linear response curve over a part of the measurement range Others may require curve-fitting to achieve accurate measurements over the measurement range 21

22 Standard Deviation SD used for data which is normally distributed, how much the data is spread around the mean A range of +/- 1 SD above and below the mean includes 68.2% of the values 2 SD includes 95.4%; 3 SD includes 99.7% Simple test for normal distribution: check Mean 2SD Average stay at resort is 10 days and SD is 8 days, then Mean 2SD is -6 days impossible value - data is not normally distributed 22

23 Mean weight of 80Kg and SD of 5Kg 23

24 Mean weight of 80Kg and SD of 3Kg 24

25 Box and whisker plot Median may be given with its interquartile range: First quartile point has one-quarter of the data below it Third quartile point has three-quarters of the data below it Inter Quartile range (IQR) has the middle half of the 25

26 Count of airborne particles of sizes in the range um encountered in different environments in Edinburgh, Scotland Arvind DK, Mann J, Bates CA, Kotsev K, The AirSpeck family of static and mobile wireless air quality monitors, in Proc.19th Euromicro Conference on Digital System Design, Cyprus, Aug

27 Count of airborne particles of sizes in the range um encountered in different environments in Delhi, India Arvind DK, Mann J, Bates CA, Kotsev K, The AirSpeck family of static and mobile wireless air quality monitors, in Proc.19th Euromicro Conference on Digital System Design, Cyprus, Aug

28 Confidence Interval Instead of the mean value of a sample, CI gives the range that is likely to contain the true population value True value is the mean value if we had the data for the entire population For a sample of 100 patients average systolic BP measurements before treatment is 170 mmhg and dropped by 20 mmhg after treatment 95% CI is implies that we can be 95% confident that the true effect of treatment is to lower the BP by mmhg Study B - 50 patients treated by the same drug also reduced their mean BP by 20 mmhg, but with a wider 95% CI of -5 to +45 (which includes 0, i.e. no change) => more than 5% chance that there was no true change in BP 28

29 Regression line illustrates correlation but not necessarily agreement 29

30 Line of Equality illustrates agreement or lack of it in this case 30

31 Correlation and level of Agreement How well people or tests agree used to look at how accurately a test can be repeated 0 no significant agreement - no more than would have been expected by chance 0.5 good agreement 0.7 very good agreement 1 perfect agreement Continuous data Inter Class Correlation Coefficient (ICC) is a commonly used measure of agreement When data can be put in ordered categories Ordinal data, the kappa statistic is used 31

32 Correlation and level of Agreement The same 3D motion captures of human gait of patients after a knee operation are scored by gait analysts in two clinics Both put them in ordered categories: normal, GA1, GA2, GA3 and No Recovery Kappa of 0.25 implies that there was little agreement between the two clinics Measure of agreement different from correlation Two analysts independently examine and one gives a consistent score of 5 points higher than the other, the correlation will be high, but agreement score will be low 32

33 Bland and Altman A change in scale of measurement does not affect correlation but affects the agreement Data which seem to be in poor agreement can produce quite high correlations Correlation depends on the range of the true quantity in the sample X-axis: mean of the two measurements Y-axis: difference between the two values i.e. What is the difference between the two methods vs the best measure of the true value we have, which is the mean of the two methods. 33

34 34

35 35

36 Regression line between measurements done by Methods A & B 36

37 Plot of differences between method A and method B vs. the mean of the two measurements The bias of units is represented by the gap between the X axis, corresponding to a zero differences, and the parallel line to the X axis 37

38 Bland and Altman This negative bias seems to be due to measurements over 200 units, while for lower value data are closer to each other. We can summarise the lack of agreement by calculating the bias, estimated by the mean difference (d) and the standard deviation of the differences (s). We would expect most of the differences to lie between d - 2s and d + 2s, or more precisely, 95% of differences will be between d s and d s, if the differences are normally distributed (Gaussian). Normal distribution of the differences must always be verified, for example by drawing a histogram. Test for normal distribution - Shapiro-Wilk test, D Agostino-Pearson test, Kolmogorov-Smirnov test can be done, for the hypothesis that the distribution of the observations in the sample is normal (if P < 0.05 then reject normality) 38

39 Probability (P) value P value gives how likely is it that a hypothesis is true Hypothesis is that there is no difference between two treatments, say, call the null hypothesis P value gives the probability of any observed differences happening by chance P = 0.5 : the probability of a difference this large having happened by chance is 0.5 in 1 or 50:50 P = 0.05 : the probability of a difference this large having happened by chance is 0.05 in 1 or 1 in 20 P = 0.01 : is considered to be highly significant can happen by chance 1 in 100 times unlikely by still possible P = : is considered to be very highly significant - can happen by chance 1 in 1000 times, even less likely but still just possible 39

40 Distribution plot of differences between measurement by methods A and B The dotted line represents Normal distribution. Shapiro-Wilk test for normal distribution accepted normality (P = 0.814) 40

41 Bland and Altman After ensuring that differences are normally distributed, use s to define the limits of agreement s = 34.8, so 95% of differences will be d-1.96s = (1.96 x 34.8) = d +1.96s = (1.96 x 34.8) = 41.1 Results measured by method A may be 95.4 units below or 41 above method B 41

42 Odds Ratio compare patients (cases) with a certain condition with patients who do not (control) Odds calculated by dividing the number of times an event has happened by the number of times it does not 1 boy is born in every two births then the Odds is 1 1 in every 100 patients suffers from side effect of drug - Odds 1:99 = Odds ratio is calculated by dividing the odds of being exposed to a risk factor to the odds in the control group 1 no difference in risk > 1 rate of that event in increased in patients exposed to the risk < 1 rate of that event is reduced 42

43 Example 100 patients with knee injuries were matched for age and sex with 100 patients who did not in cases: 40 were skate boarders and 60 were not; in control: 20 were skate boarders and 80 were not Odds of being a skate boarder in cases is 40/ Odds of being a skate boarder in control is 20/80 = 0.25 OR 0.66/0.25 = % CI is Odds of being a skate boarder is 2.64 times in the cases compared to controls as CI does not include 1 (no different in risk) that is statistically significant Conclude that patients with knee injuries are more likely to be skate boarders than those w/o knee injuries 43

44 Risk Ratio Probability that an event will happen Number of events/no of people at risk 1 boy is born every two births - risk is ½ = in every100 patient suffers side-effect of drug 1/100 = 0.01 Risk Ratio is calculated by dividing the risk in the treated (exposed) to the risk in the control or unexposed group = 1 no difference in risk between the two group >1 rate of that event is increased in the exposed group (cf control) <1 rate of that event is reduced Risk ratios are given with 95% CI, if CI does not include 1 (no difference in risk), then it is statistically significant 44