Performance Analysis of ARQ Protocols using a Theorem Prover

Performance Analysis of ARQ Protocols using a Theorem Prover Osman Hasan Sofiene Tahar Hardware Verification Group Concordia University Montreal, Canada ISPASS 2008

Objectives n Probabilistic Theorem Proving A robust and precise probabilistic analysis technique n What is it? n Why do we need it? n How can we apply it for the performance analysis of ARQ Protocols? O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 2

3 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions

4 Motivation Probabilistic Choice Noise Aging Phenomena Environmental Conditions Unpredictable Inputs n Performance Analysis n Simulation n State-of-the-art n Inaccurate results n Theorem Proving n Proposed Solution

5 Performance Analysis Random Components R andom Variables (Dis crete/ C ontinuous) Hardware Software System Model Probabilistic and Statistical Properties Computer Based Analysis Framework Property Satisfied?

6 Probabilistic Analysis Approaches Random Components Analysis Accuracy Expressiveness No CPU Time Issue Automation Simulation Simulation Approximate Probabilistic random State Machine variable functions good Observing some test cases û ü û ü Model Checking Probabilistic State Machine Exhaustive Verification ü û û ü Formal Methods Theorem Proving Precise random variable functions Mathematical Reasoning ü ü ü û

7 Theorem Prover n A notation (syntax) n A small set of fundamental axioms (facts) n A Boolean variable can be True or False: a.(a = T) (a =F) n A small set of inference (deduction) rules n Equality is transitive: a b c. (a = b) (b = c) (a = c) n Soundness n Every new theorem must be created from n Basic axioms and primitive inference rules n Already proved theorems or inference rules n Theory (collection of verified theorems in a file) n Can be reloaded in theorem provers n Facilitates the instant utilization of already verified theorems

8 Theorem Proving Example n Check if y>x for the given system (x is a natural number) x 2 ( x +1) y 1 y>x Problem statement 2 (x+1) 2 >x Implementation 3 (x+1).(x+1)>x Definition of Square 4 (x+1).x+(x+1).1>x Distributivity 5 x.x+1.x+x.1+1.1>x Distributivity 6 x.x+x+x+1>x Multiplicative Identity 7 x.x+x+1+x>x Additive Commutivity 8 x.x+x+1>0 Addition Cancellation 9 True Natural numbers > 0

9 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions

HOL Theorem Prover n Higher-order logic theorem prover n University of Cambridge, UK n 5 axioms n 8 primitive inference rules n Numerous proof assistants are available n Inbuilt mathematical theories of Boolean, list, set, integers, real analysis, measure, and probability theory O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 10

11 Theorem Proving Based Performance Analysis System Properties (Discrete Random Variables) Discrete Random Variables Probabilistic Properties Statistical Properties System Description System Properties Random Components System Model Probabilistic Analysis Theorems Theorem Prover Formal Proofs of Properties Continuous Random Variables Probabilistic Properties Statistical Properties System Properties (Continuous Random Variables)

12 Formal Verification of Random Variables n Measure Theory n Probability space of Infinite Boolean sequence (B ) B : positive integers Boolean 0 1 2 3 4 5 6 7 T/F T/F T/F T/F T/F T/F T/F T/F n A random variable that n n Accepts : α Returns: β can be modeled in HOL as a function f : α B (β x B )

13 Random Variables in HOL Example n Coin Flip (Head, Tail) B (flip_outcome x B ) n Algorithm flip s = (if (top element of s) then Head else Tail, remaining portion of s) n Probabilistic Properties P {s flip s = Head} = ½

14 Discrete Random Variables in HOL Theorems: Discrete Random Variables Random variable Uniform(m) Bernoulli(p) Geometric(p) HOL Funtions unif_rv bern_rv geom_rv PMF (Pr (X = n)) 1 m p p ( 1 p) n

15 Continuous Random Variables in HOL Theorems: Continuous Random Variables Random Variable HOL Functions CDF (Pr (X x) Exponential(l) Uniform(a,b) Rayleigh(l) exp_rv uniform_rv rayleigh_rv 0, x 0 -lx 1- exp, 0 < x 0, x a x - a, a < x b b - a 1, b < x 0, 1- exp -x 2 2 2l x 0, 0 < x

16 Verification of Statistical Properties Definition: Expectation for Discrete Random Variables Ex[ X ] = i= 1 i Pr( X = i) Theorem: Expectation Properties Ex [ c] = c Ex n i= 1 X i = n i= 1 Ex [ X ] i

Verification of Statistical Properties Theorems: Discrete Random Variables Random variable HOL Function Expectation Variance Uniform(m) unif_rv m 2 ( m + 1) 12 2 1 Bernoulli(p) bern_rv p p( 1 p) Geometric(p) geom_rv 1 p 1 p 2 p O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 17

Probabilistic Theorem Proving Case Studies n Very few examples n Roundoff error analysis of a Digital Processer n Verification of a couple of probabilistic properties n Probabilistic Analysis of Algorithms n Miller Rabin Test n Coupon-Collector s Problem O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 18

19 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions

Automatic Repeat Request (ARQ) n Reliable communication between computers n Transmitter n Repeats transmission of a data frame until it receives an ACK n Receiver n Discards erroneous data frames n Sends Acknowledgment (ACK) for Error-free data frames n Applications n Transmission Control Protocol (TCP) n High-level Data Link Control (HDLC) Standard O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 20

21 ARQ Protocols n Implementation variants of ARQ principle n Stop-and-Wait n Go-Back-N n Selective Repeat n Performance Analysis Metric n Message Delay n Both simulation and state-based formal techniques fail to produce reasonable results n A subtle interaction of a number of distributed components

22 Stop-and-Wait Protocol n Delay (Unsuccessful Transmission Trial) T = t + t n Delay (Successful Transmission Trial) T s = t f + ta + 2( t prop + t proc ) u f out

23 Go-Back-N Protocol n Delay (Unsuccessful Transmission Trial) T = t + t n Delay (Successful Transmission Trial) T = t u s f f out

24 Selective Repeat Protocol n Delay (Unsuccessful Transmission Trial) T = t n Delay (Successful Transmission Trial) T = t u s f f

25 Average Message Delay of ARQ Protocols n p: Bit-error probability of the channel n Average (Message Delay) =? n Step 1: Message Delay (T u,t s,p) n Geometric Random Variable Delay = (G-1)T u + T s n Step 2: Average of the above random variable

Step 1: Message Delay in HOL n Geometric random variable function (geom_rv) n Success probability =? n Error behaviour of single bit: bern_rv(p) Definition: Frame Error n p. f_err 0 p = false f_err (n + 1) p = bern_rv(p) (f_err n p) Definition: Probability of Successful Transmission nf na p. suc_p_arq nf na p = P { (f_err nf p) (f_err na p) = false } O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 26

Step 1: Message Delay in HOL Theorem: Probability of Successful Transmission nf na p. 0 p p 1 suc_p_arq nf na p = (1-p) (nf + na) n Proof n Boolean Logic, Positive Integers, Real Numbers, Set, Probability Definition: ARQ Message Delay nf na p Tu Ts. arq_del = Tu (geom_rv ((1-p) (nf + na) ) 1) + Ts O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 27

28 Step 2: Average Message Delay Theorem: Linearity of Expectation Ex [ ax + b] = ae[ X ] + b n Proof n Already verified Expectation properties Ex [ c] = c Ex n i= 1 X i = n i= 1 Ex [ X ] i n Boolean Logic, Positive Integers, Real Numbers, Set, Probability

29 Average Message Delay in HOL Stop-and-Wait Protocol n Proof: n Definition: Stop-and-Wait Message Delay nf na p tout tprop tproc tf ta. sw_del nf na p tout tprop tproc tf ta = (tf + tout) (geom_rv ((1-p) (nf + na) ) 1) + tf + ta + 2(tproc + tprop) Theorem: Average Stop-and-Wait Message Delay nf na p tout tprop tproc tf ta. (0 p) (p < 1) expec (sw_del nf na p tout tprop tproc tf ta) = (tf + tout) (1 - (1-p) (nf + na) )/((1-p) (nf + na) ) + tf + ta + 2(tproc + tprop) Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable

30 Average Message Delay in HOL Go-Back-N Protocol n Proof: n Definition: Go-Back-N Message Delay nf na p tout tf. gbn_del nf na p tout tf = (tf + tout) (geom_rv ((1-p) (nf + na) ) 1) + tf Theorem: Average Go-Back-N Message Delay nf na p tout tf. (0 p) (p < 1) expec (gbn_del nf na p tout tf) = (tf + tout) (1 - (1-p) (nf + na) )/((1-p) (nf + na) ) +tf Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable

31 Average Message Delay in HOL Selective Repeat Protocol Definition: Stop-and-Wait Message Delay nf na p tf. sr_del nf na p tf = (tf) (geom_rv ((1-p) (nf + na) ) 1) + tf Theorem: Average Stop-and-Wait Message Delay nf na p tf. (0 p) (p < 1) expec (sr_del nf na p tf) = (tf)/((1-p) (nf + na) ) n Proof: n Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable

32 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions

33 Conclusions n Probabilistic Theorem Proving n Model randomness in systems with higher-order-logic random variables n Verify probabilistic and statistical properties in a theorem prover n Exact Answers n Useful for the analysis of Safety critical application n Performance Analysis of ARQ Protocols n Delay Characteristic Higher-order-logic random variable n Verification of Linearity of Expectation Property in HOL n Results exactly match the paper-and-pencil based analysis methods 100% precise

34 Conclusions n Probabilistic Theorem Proving is not a golden solution to all performance analysis problems n Interactive and tedious nature n Less critical sections of the system n Simulation n Critical sections of the system that can be expressed as a Markov Chain n Model Checking n Critical sections of the system that cannot be handled by Model Checking n Thereom Proving

Thank you For more information: http://hvg.ece.concordia.ca Contact: o_hasan@ece.concordia.ca O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 35

Additional Slides O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 36

Performance Analysis Basics Random Variables n Discrete Random Variables n Attain a countable number of values n Examples n n Uniform (countable values in an interval [a,b]) Bernoulli (True, False) n Continuous Random Variables n Attain an uncountable (infinite) number of values n Examples n n Uniform (all real values in an interval [a,b]) Exponential (The time between independent events) O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 37

Performance Analysis Basics Properties of Random Variables n Used to characterize system s behaviour n Probabilistic properties n Probability (Multiplier delay = x) n Statistical properties n Average message delay of a telecommunication protocol n Major decision making criteria in performance analysis O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 38