A Methodology for Clock Benchmarking

Transcription

1 A Methodology for Clock Benchmarking Julien Ridoux Darryl Veitch ARC Special Research Centre for Ultra-Broadband Information Networks THE UNIVERSITY OF MELBOURNE NICTA Victoria Research Laboratory Dept. of Electrical & Electronic Engineering THE UNIVERSITY OF MELBOURNE

2 Introduction Higher demand on the network, better clocks Network applications are more and more distributed Users/Providers need higher reactivity and more precision Essential for testbeds and performance evaluations Limitation: the quality of clock synchronisation Need for higher accuracy Need for higher reliability Difficult to benchmark timekeeping systems Against which reference? How to access clock instantaneously? 2

3 Clock and Timestamping errors Timestamps Clock Time Drifting Clock Perfect Clock t k t k Event occurring at true time True Time 3

4 Clock and Timestamping errors Timestamps Clock Time C(t k) Drifting Clock Perfect Clock t k t k Event occurring at true time t k True Time 3

5 Clock and Timestamping errors Timestamps Clock Time C(t k) E(t k ) θ(t k) ξ(t k ) Drifting Clock Perfect Clock Clock error or offset θ(t k ) = C(t k ) t k Timestamping error ξ(t k ) = t k t k t k Total error: t k Event occurring at true time t k True Time E(t k) = C(t k) t k = θ(t k) + ξ(t k ) 3

6 Clock and Timestamping errors In practice, no perfect clock for benchmarking Total relative error: E C1,C 2 (t k ) = C 1 (t k) C 2 (t k) = θ C1 (t k) θ C2 (t k) + ξ C1 (t k ) ξ C2 (t k ) The clock and timestamping errors combine Without a perfect clock; benchmarking a challenge Timestamping error: eliminate / estimate Clock error: relative / absolute Need a strong methodology 4

7 CubinLab Testbed 3 Clocks under study (Linux & FreeBSD) SW-GPS: ntpd + GPS sync. Absolute Clock SW-NTP: ntpd + Net. sync. Absolute Clock TSCclock: Net. sync, Absolute & Difference Clock Internal Monitor Host SW-GPS SW-NTP External Monitor DAG-GPS DAG Card Unix PC UDP Sender & Receiver GPS Receiver NTP Server Stratum 1 SW-GPS TSCclock Hub PPS Synchronization Bi-directional NTP flow Bi-directional UDP flow Time Request 5

8 CubinLab Testbed Kernel timestamping of UDP packets Outgoing / Incoming directions External: DAG Card Internal: Multiple clocks simultaneously Internal Monitor Host SW-GPS SW-NTP External Monitor DAG-GPS DAG Card Unix PC UDP Sender & Receiver GPS Receiver NTP Server Stratum 1 SW-GPS TSCclock Hub PPS Synchronization Bi-directional NTP flow Bi-directional UDP flow Time Request 6

9 Internal comparison Modified kernels: timestamps taken back to back Identical delay accessing the clocks Timestamping errors cancel t k = t k ξ C1 (t k ) = ξ C2 (t k ) Obtain a comparison of the two clocks offsets E C1,C 2 (t k ) θ C1 (t k ) θ C2 (t k ) Free of timestamping error No absolute performance with respect to true time 7

10 External comparison Use of the DAG card Considered best absolute time reference available E C,Dag (t k ) = θ C (t k ) + ξ C (t k ) ξ Dag (t k ) Provides absolute reference But suffers from timestamping error Additional kernel modifications to reduce noise Standard location in kernel for all clocks Improved locations for the TSCclock As close as possible to the last bit transmitted/received Interrupt bottom-half / driver implementation 8

11 Reducing kernel timestamping error Unix PC DAG [ms] Host t a t g a t g f t f d h d h time Standard IN (1) Improved IN (2) Improved OUT (3) Standard OUT (4) (1) Minutes Stand. OUT: med= 268 iqr= 36.1 Stand. IN: med= 177 iqr= (2) Impr. OUT: med= 84.5 iqr= 14.9 Impr. IN: med= 92.1 iqr= 8.1 (3) (4) Outgoing Incoming

12 Reducing kernel timestamping error Standard timestamping location with the TSCclock Outgoing direction noisier: IQR 1 µs larger Asymmetry of 1 µs between Outgoing / Incoming Improved location much better Outgoing Incoming [ms] Stand. OUT: med= 268 iqr= 36.1 Stand. IN: med= 177 iqr= Standard IN (1) Improved IN (2) Improved OUT (3) Standard OUT (4) (1) (2) Impr. OUT: med= 84.5 iqr= 14.9 Impr. IN: med= 92.1 iqr= Minutes (3) (4)

13 Reducing kernel timestamping error Standard timestamping location with the TSCclock Outgoing direction noisier: IQR 1 µs larger Asymmetry of 1 µs between Outgoing / Incoming Improved location much better The same clock in both directions!!! Which direction to trust? Outgoing Incoming [ms] Stand. OUT: med= 268 iqr= 36.1 Stand. IN: med= 177 iqr= Standard IN (1) Improved IN (2) Improved OUT (3) Standard OUT (4) (1) (2) Impr. OUT: med= 84.5 iqr= 14.9 Impr. IN: med= 92.1 iqr= Minutes (3) (4)

14 Host RTT measurement Unix PC DAG If we use both directions Minimum Host RTT: Available timestamps: r h = d h + d h R h = r h + ξ(t f ) ξ(t a ) Host RTT available since measured with the same clock Minimum can be filtered; noise is the width of histogram of R h Host t a t g a t g f t f d h d h time [ms] Standard Improved Standard noise: med= 443 iqr= 36.6 Improved noise: med= 177 iqr= Minutes

15 Recovering one-way measurements Ambiguity due to the asymmetry that we can t evaluate asym = d h d h asym [ r h, r h ] DAG DAG DAG Host time Host time Host time t a t g a t g f t f t a t g a t g f t f t a t g a t g f t f d h = d h d h d h d h d h = One way delays can t be recovered individually Host RTT impact of noise on one-way measurement 2 r h Median is ambiguous but bounded by Histograms are broadened because of IQR( R h ) 11

16 Beware of problematic drivers/nic Two hosts with same OS / hardware FreeBSD 6.1 Pentium-D architecture But different NIC / Driver Maxwell : Broadcom 5157 Gig-E (Brown) Tastiger : 3Com 1/1 Mbps (Black).2 Maxwell Tastiger.4 Tastiger: med= 77.5 iqr= Maxwell: med= 1 iqr= 69.2 [ms] Minutes Choose carefully! R h The quality of measurement drives the accuracy of the methodology! 12

17 Let s get started Now that we have an accurate testbed internal / external timestamping and validation improved kernel timestamping removed problematic hardware... we can start the detective work 13

18 SW-NTP vs. TSCclock Internal comparison large oscillations ± 1ms 1.5 SW NTP TSCclock ockdiff: SW TSC: med= 14.6 iqr= 546 [m.1 [ms] Days 5 5 External comparison SW-NTP responsible Noise: IQR( R h ) = 37µs r h = 1µs (ambiguity = 2µs) Accurate view for SW-NTP, difficult diagnosis for TSCclock [ms] SW NTP TSCclock Days 1 SW NTP: med= 58.6 iqr= TSCClock: med= 43.4 iqr= 22.2 [mus 545µs.8 22µs

19 SW-GPS vs. TSCclock [ms] Internal comparison similar behavior (IQR = 14µs) SW GPS TSCclock.1 5 Days 1 External Comparison ockdiff: SW TSC: med= 22.3 iqr= 14.2 [m Noise: IQR( R h ) = 23µs r h = 1µs (ambiguity = 2µs) One clock may have (IQR = + noise). But can t be verified! TSCclock slightly ahead but which clock is worse? µs [ms] SW GPS TSCclock.1 5 Days SW GPS: med= 13.7 iqr= µs TSCClock: med= 37.4 iqr= µs

20 SW-GPS vs. TSCclock (Zoom) Observe SW-GPS and TSCclock more closely Observe oscillations with a 2mn period.5 SW GPS TSCclock [ms] Hours Both clocks show oscillations Temperature effect (air-conditioning variations) SW GPS TSCclock.1 [ms] Hours

21 SW-GPS and (TSCclock) Difference Clock Measure UDP packets inter-arrivals Compare SW-GPS IAT and TSCclock Difference Clock IAT Final error in a ±1µs band Resolution of SW-GPS is 1µs (struct timeval) Can t interpret errors within this band 1mus Spikes of up to 1µs magnitude By construction, can t be due to the difference clock Small time scale stability of the oscillator Due to the SW-GPS clock!! SW GPS C d (t) W GPS C x 1 3 d (t): med=.298 iqr=.71 [m 5mus 1mus 1mus 5mus 1mus Days

22 Conclusion Clock benchmarking is a challenge requires good quality hardware a strong methodology requires rigour and attention to details Our methodology and testbed highlights the need for kernel modifications presents Internal / External complementary comparisons provides comparison down to the system clock resolution allows to track causes of observed strange behaviors j.ridoux@ee.unimelb.edu.au 18