Performance Analysis of ARQ Protocols using a Theorem Prover
|
|
- Osborne Perry
- 5 years ago
- Views:
Transcription
1 Performance Analysis of ARQ Protocols using a Theorem Prover Osman Hasan Sofiene Tahar Hardware Verification Group Concordia University Montreal, Canada ISPASS 2008
2 Objectives n Probabilistic Theorem Proving A robust and precise probabilistic analysis technique n What is it? n Why do we need it? n How can we apply it for the performance analysis of ARQ Protocols? O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 2
3 3 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions
4 4 Motivation Probabilistic Choice Noise Aging Phenomena Environmental Conditions Unpredictable Inputs n Performance Analysis n Simulation n State-of-the-art n Inaccurate results n Theorem Proving n Proposed Solution
5 5 Performance Analysis Random Components R andom Variables (Dis crete/ C ontinuous) Hardware Software System Model Probabilistic and Statistical Properties Computer Based Analysis Framework Property Satisfied?
6 6 Probabilistic Analysis Approaches Random Components Analysis Accuracy Expressiveness No CPU Time Issue Automation Simulation Simulation Approximate Probabilistic random State Machine variable functions good Observing some test cases û ü û ü Model Checking Probabilistic State Machine Exhaustive Verification ü û û ü Formal Methods Theorem Proving Precise random variable functions Mathematical Reasoning ü ü ü û
7 7 Theorem Prover n A notation (syntax) n A small set of fundamental axioms (facts) n A Boolean variable can be True or False: a.(a = T) (a =F) n A small set of inference (deduction) rules n Equality is transitive: a b c. (a = b) (b = c) (a = c) n Soundness n Every new theorem must be created from n Basic axioms and primitive inference rules n Already proved theorems or inference rules n Theory (collection of verified theorems in a file) n Can be reloaded in theorem provers n Facilitates the instant utilization of already verified theorems
8 8 Theorem Proving Example n Check if y>x for the given system (x is a natural number) x 2 ( x +1) y 1 y>x Problem statement 2 (x+1) 2 >x Implementation 3 (x+1).(x+1)>x Definition of Square 4 (x+1).x+(x+1).1>x Distributivity 5 x.x+1.x+x.1+1.1>x Distributivity 6 x.x+x+x+1>x Multiplicative Identity 7 x.x+x+1+x>x Additive Commutivity 8 x.x+x+1>0 Addition Cancellation 9 True Natural numbers > 0
9 9 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions
10 HOL Theorem Prover n Higher-order logic theorem prover n University of Cambridge, UK n 5 axioms n 8 primitive inference rules n Numerous proof assistants are available n Inbuilt mathematical theories of Boolean, list, set, integers, real analysis, measure, and probability theory O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 10
11 11 Theorem Proving Based Performance Analysis System Properties (Discrete Random Variables) Discrete Random Variables Probabilistic Properties Statistical Properties System Description System Properties Random Components System Model Probabilistic Analysis Theorems Theorem Prover Formal Proofs of Properties Continuous Random Variables Probabilistic Properties Statistical Properties System Properties (Continuous Random Variables)
12 12 Formal Verification of Random Variables n Measure Theory n Probability space of Infinite Boolean sequence (B ) B : positive integers Boolean T/F T/F T/F T/F T/F T/F T/F T/F n A random variable that n n Accepts : α Returns: β can be modeled in HOL as a function f : α B (β x B )
13 13 Random Variables in HOL Example n Coin Flip (Head, Tail) B (flip_outcome x B ) n Algorithm flip s = (if (top element of s) then Head else Tail, remaining portion of s) n Probabilistic Properties P {s flip s = Head} = ½
14 14 Discrete Random Variables in HOL Theorems: Discrete Random Variables Random variable Uniform(m) Bernoulli(p) Geometric(p) HOL Funtions unif_rv bern_rv geom_rv PMF (Pr (X = n)) 1 m p p ( 1 p) n
15 15 Continuous Random Variables in HOL Theorems: Continuous Random Variables Random Variable HOL Functions CDF (Pr (X x) Exponential(l) Uniform(a,b) Rayleigh(l) exp_rv uniform_rv rayleigh_rv 0, x 0 -lx 1- exp, 0 < x 0, x a x - a, a < x b b - a 1, b < x 0, 1- exp -x 2 2 2l x 0, 0 < x
16 16 Verification of Statistical Properties Definition: Expectation for Discrete Random Variables Ex[ X ] = i= 1 i Pr( X = i) Theorem: Expectation Properties Ex [ c] = c Ex n i= 1 X i = n i= 1 Ex [ X ] i
17 Verification of Statistical Properties Theorems: Discrete Random Variables Random variable HOL Function Expectation Variance Uniform(m) unif_rv m 2 ( m + 1) Bernoulli(p) bern_rv p p( 1 p) Geometric(p) geom_rv 1 p 1 p 2 p O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 17
18 Probabilistic Theorem Proving Case Studies n Very few examples n Roundoff error analysis of a Digital Processer n Verification of a couple of probabilistic properties n Probabilistic Analysis of Algorithms n Miller Rabin Test n Coupon-Collector s Problem O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 18
19 19 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions
20 Automatic Repeat Request (ARQ) n Reliable communication between computers n Transmitter n Repeats transmission of a data frame until it receives an ACK n Receiver n Discards erroneous data frames n Sends Acknowledgment (ACK) for Error-free data frames n Applications n Transmission Control Protocol (TCP) n High-level Data Link Control (HDLC) Standard O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 20
21 21 ARQ Protocols n Implementation variants of ARQ principle n Stop-and-Wait n Go-Back-N n Selective Repeat n Performance Analysis Metric n Message Delay n Both simulation and state-based formal techniques fail to produce reasonable results n A subtle interaction of a number of distributed components
22 22 Stop-and-Wait Protocol n Delay (Unsuccessful Transmission Trial) T = t + t n Delay (Successful Transmission Trial) T s = t f + ta + 2( t prop + t proc ) u f out
23 23 Go-Back-N Protocol n Delay (Unsuccessful Transmission Trial) T = t + t n Delay (Successful Transmission Trial) T = t u s f f out
24 24 Selective Repeat Protocol n Delay (Unsuccessful Transmission Trial) T = t n Delay (Successful Transmission Trial) T = t u s f f
25 25 Average Message Delay of ARQ Protocols n p: Bit-error probability of the channel n Average (Message Delay) =? n Step 1: Message Delay (T u,t s,p) n Geometric Random Variable Delay = (G-1)T u + T s n Step 2: Average of the above random variable
26 Step 1: Message Delay in HOL n Geometric random variable function (geom_rv) n Success probability =? n Error behaviour of single bit: bern_rv(p) Definition: Frame Error n p. f_err 0 p = false f_err (n + 1) p = bern_rv(p) (f_err n p) Definition: Probability of Successful Transmission nf na p. suc_p_arq nf na p = P { (f_err nf p) (f_err na p) = false } O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 26
27 Step 1: Message Delay in HOL Theorem: Probability of Successful Transmission nf na p. 0 p p 1 suc_p_arq nf na p = (1-p) (nf + na) n Proof n Boolean Logic, Positive Integers, Real Numbers, Set, Probability Definition: ARQ Message Delay nf na p Tu Ts. arq_del = Tu (geom_rv ((1-p) (nf + na) ) 1) + Ts O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 27
28 28 Step 2: Average Message Delay Theorem: Linearity of Expectation Ex [ ax + b] = ae[ X ] + b n Proof n Already verified Expectation properties Ex [ c] = c Ex n i= 1 X i = n i= 1 Ex [ X ] i n Boolean Logic, Positive Integers, Real Numbers, Set, Probability
29 29 Average Message Delay in HOL Stop-and-Wait Protocol n Proof: n Definition: Stop-and-Wait Message Delay nf na p tout tprop tproc tf ta. sw_del nf na p tout tprop tproc tf ta = (tf + tout) (geom_rv ((1-p) (nf + na) ) 1) + tf + ta + 2(tproc + tprop) Theorem: Average Stop-and-Wait Message Delay nf na p tout tprop tproc tf ta. (0 p) (p < 1) expec (sw_del nf na p tout tprop tproc tf ta) = (tf + tout) (1 - (1-p) (nf + na) )/((1-p) (nf + na) ) + tf + ta + 2(tproc + tprop) Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable
30 30 Average Message Delay in HOL Go-Back-N Protocol n Proof: n Definition: Go-Back-N Message Delay nf na p tout tf. gbn_del nf na p tout tf = (tf + tout) (geom_rv ((1-p) (nf + na) ) 1) + tf Theorem: Average Go-Back-N Message Delay nf na p tout tf. (0 p) (p < 1) expec (gbn_del nf na p tout tf) = (tf + tout) (1 - (1-p) (nf + na) )/((1-p) (nf + na) ) +tf Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable
31 31 Average Message Delay in HOL Selective Repeat Protocol Definition: Stop-and-Wait Message Delay nf na p tf. sr_del nf na p tf = (tf) (geom_rv ((1-p) (nf + na) ) 1) + tf Theorem: Average Stop-and-Wait Message Delay nf na p tf. (0 p) (p < 1) expec (sr_del nf na p tf) = (tf)/((1-p) (nf + na) ) n Proof: n Ex [ ax + b] = ae[ X ] + b n Expectation of Geometric random variable
32 32 Outline n Introduction n Theorem Proving based Performance Analysis n Performance Analysis of ARQ Protocols n Conclusions
33 33 Conclusions n Probabilistic Theorem Proving n Model randomness in systems with higher-order-logic random variables n Verify probabilistic and statistical properties in a theorem prover n Exact Answers n Useful for the analysis of Safety critical application n Performance Analysis of ARQ Protocols n Delay Characteristic Higher-order-logic random variable n Verification of Linearity of Expectation Property in HOL n Results exactly match the paper-and-pencil based analysis methods 100% precise
34 34 Conclusions n Probabilistic Theorem Proving is not a golden solution to all performance analysis problems n Interactive and tedious nature n Less critical sections of the system n Simulation n Critical sections of the system that can be expressed as a Markov Chain n Model Checking n Critical sections of the system that cannot be handled by Model Checking n Thereom Proving
35 Thank you For more information: Contact: O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 35
36 Additional Slides O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 36
37 Performance Analysis Basics Random Variables n Discrete Random Variables n Attain a countable number of values n Examples n n Uniform (countable values in an interval [a,b]) Bernoulli (True, False) n Continuous Random Variables n Attain an uncountable (infinite) number of values n Examples n n Uniform (all real values in an interval [a,b]) Exponential (The time between independent events) O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 37
38 Performance Analysis Basics Properties of Random Variables n Used to characterize system s behaviour n Probabilistic properties n Probability (Multiplier delay = x) n Statistical properties n Average message delay of a telecommunication protocol n Major decision making criteria in performance analysis O. Hasan Performance Analysis of ARQ Protocols using Theorem Proving 38
Formalization of Normal Random Variables
Formalization of Normal Random Variables M. Qasim, O. Hasan, M. Elleuch, S. Tahar Hardware Verification Group ECE Department, Concordia University, Montreal, Canada CICM 16 July 28, 2016 2 Outline n Introduction
More informationCOEN6551: Formal Hardware Verification
COEN6551: Formal Hardware Verification Prof. Sofiène Tahar Hardware Verification Group Electrical and Computer Engineering Concordia University Montréal, Quebec CANADA Accident at Carbide plant, India
More informationFormal Verification of Tail Distribution Bounds in the HOL Theorem Prover
Formal Verification of Tail Distribution Bounds in the HOL Theorem Prover Osman Hasan and Sofiène Tahar Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada Email:
More informationFormal Reliability Analysis of Combinational Circuits using Theorem Proving
Formal Reliability Analysis of Combinational Circuits using Theorem Proving Osman Hasan a,, Jigar Patel b, Sofiène Tahar b a School of Electrical Engineering and Computer Science National University of
More informationModeling and Simulation NETW 707
Modeling and Simulation NETW 707 Lecture 6 ARQ Modeling: Modeling Error/Flow Control Course Instructor: Dr.-Ing. Maggie Mashaly maggie.ezzat@guc.edu.eg C3.220 1 Data Link Layer Data Link Layer provides
More informationAn Informal introduction to Formal Verification
An Informal introduction to Formal Verification Osman Hasan National University of Sciences and Technology (NUST), Islamabad, Pakistan O. Hasan Formal Verification 2 Agenda q Formal Verification Methods,
More informationAccurate Reliability Analysis of Combinational Circuits using Theorem Proving
Accurate Reliability Analysis of Combinational Circuits using Theorem Proving Osman Hasan, Jigar Patel and Sofiène Tahar Department of Electrical and Computer Engineering, Concordia University, Montreal,
More informationReliability Block Diagrams based Analysis: A Survey
Reliability Block Diagrams based Analysis: A Survey O. Hasan 1, W. Ahmed 1 S. Tahar 2 and M.S. Hamdi 3 1 National University of Sciences and Technology, Islamabad Pakistan 2 Concordia University, Montreal,
More informationDiscrete Random Variables
CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary mahanti@cpsc.ucalgary.ca Random Variables A random variable is
More informationFormalization of Continuous Probability Distributions
Formalization of Continuous Probability Distributions Osman Hasan and Sofiène Tahar Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada Email: {o hasan, tahar}@ece.concordia.ca
More informationarxiv: v1 [cs.lo] 7 Dec Department of Electrical and Computer Engineering,
Dynamic Fault Trees Analysis using an Integration of Theorem Proving and Model Checking Yassmeen Elderhalli 1, Osman Hasan 1,2, Waqar Ahmad 2 and Sofiène Tahar 1 arxiv:1712.02872v1 [cs.lo] 7 Dec 2017 1
More informationCSE 123: Computer Networks
CSE 123: Computer Networks Total points: 40 Homework 1 - Solutions Out: 10/4, Due: 10/11 Solutions 1. Two-dimensional parity Given below is a series of 7 7-bit items of data, with an additional bit each
More informationVerifying Probabilistic Programs using the HOL Theorem Prover Joe Hurd p.1/32
Verifying Probabilistic Programs using the HOL Theorem Prover Joe Hurd joe.hurd@cl.cam.ac.uk University of Cambridge Verifying Probabilistic Programs using the HOL Theorem Prover Joe Hurd p.1/32 Contents
More informationLecture 4: Sampling, Tail Inequalities
Lecture 4: Sampling, Tail Inequalities Variance and Covariance Moment and Deviation Concentration and Tail Inequalities Sampling and Estimation c Hung Q. Ngo (SUNY at Buffalo) CSE 694 A Fun Course 1 /
More informationFormal Reasoning about Systems Biology using Theorem Proving
Formal Reasoning about Systems Biology using Theorem Proving Adnan Rashid*, Osman Hasan*, Umair Siddique** and Sofiène Tahar** *School of Electrical Engineering and Computer Science (SEECS), National University
More informationCMPSCI 240: Reasoning Under Uncertainty
CMPSCI 240: Reasoning Under Uncertainty Lecture 5 Prof. Hanna Wallach wallach@cs.umass.edu February 7, 2012 Reminders Pick up a copy of B&T Check the course website: http://www.cs.umass.edu/ ~wallach/courses/s12/cmpsci240/
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationBernoulli and Binomial
Bernoulli and Binomial Will Monroe July 1, 217 image: Antoine Taveneaux with materials by Mehran Sahami and Chris Piech Announcements: Problem Set 2 Due this Wednesday, 7/12, at 12:3pm (before class).
More informationFormal Verification Techniques. Riccardo Sisto, Politecnico di Torino
Formal Verification Techniques Riccardo Sisto, Politecnico di Torino State exploration State Exploration and Theorem Proving Exhaustive exploration => result is certain (correctness or noncorrectness proof)
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationRandomized Algorithms
Randomized Algorithms Prof. Tapio Elomaa tapio.elomaa@tut.fi Course Basics A new 4 credit unit course Part of Theoretical Computer Science courses at the Department of Mathematics There will be 4 hours
More informationFormal Reliability Analysis using Higher-Order Logic Theorem Proving
Formal Reliability Analysis using Higher-Order Logic Theorem Proving Naeem Ahmad Abbasi A Thesis in The Department of Electrical and Computer Engineering Presented in Partial Fulfillment of the Requirements
More informationELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Random Variable Discrete Random
More informationEE 505 Introduction. What do we mean by random with respect to variables and signals?
EE 505 Introduction What do we mean by random with respect to variables and signals? unpredictable from the perspective of the observer information bearing signal (e.g. speech) phenomena that are not under
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More informationDiscrete Random Variables
Chapter 5 Discrete Random Variables Suppose that an experiment and a sample space are given. A random variable is a real-valued function of the outcome of the experiment. In other words, the random variable
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 3 Common Families of Distributions Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 28, 2015 Outline 1 Subjects of Lecture 04
More informationAlan Bundy. Automated Reasoning LTL Model Checking
Automated Reasoning LTL Model Checking Alan Bundy Lecture 9, page 1 Introduction So far we have looked at theorem proving Powerful, especially where good sets of rewrite rules or decision procedures have
More informationLecture Notes 17. Randomness: The verifier can toss coins and is allowed to err with some (small) probability if it is unlucky in its coin tosses.
CS 221: Computational Complexity Prof. Salil Vadhan Lecture Notes 17 March 31, 2010 Scribe: Jonathan Ullman 1 Interactive Proofs ecall the definition of NP: L NP there exists a polynomial-time V and polynomial
More informationRandom Variable. Pr(X = a) = Pr(s)
Random Variable Definition A random variable X on a sample space Ω is a real-valued function on Ω; that is, X : Ω R. A discrete random variable is a random variable that takes on only a finite or countably
More informationBounded Retransmission in Event-B CSP: a Case Study
Available online at www.sciencedirect.com Electronic Notes in Theoretical Computer Science 280 (2011) 69 80 www.elsevier.com/locate/entcs Bounded Retransmission in Event-B CSP: a Case Study Steve Schneider
More informationProbabilistic Model Checking Michaelmas Term Dr. Dave Parker. Department of Computer Science University of Oxford
Probabilistic Model Checking Michaelmas Term 2011 Dr. Dave Parker Department of Computer Science University of Oxford Overview Temporal logic Non-probabilistic temporal logic CTL Probabilistic temporal
More informationChapter 6 Randomization Algorithm Theory WS 2012/13 Fabian Kuhn
Chapter 6 Randomization Algorithm Theory WS 2012/13 Fabian Kuhn Randomization Randomized Algorithm: An algorithm that uses (or can use) random coin flips in order to make decisions We will see: randomization
More informationDigital Systems. Validation, verification. R. Pacalet January 4, 2018
Digital Systems Validation, verification R. Pacalet January 4, 2018 2/98 Simulation Extra design tasks Reference model Simulation environment A simulation cannot be exhaustive Can discover a bug Cannot
More informationFormal Verification of Mathematical Algorithms
Formal Verification of Mathematical Algorithms 1 Formal Verification of Mathematical Algorithms John Harrison Intel Corporation The cost of bugs Formal verification Levels of verification HOL Light Formalizing
More informationLecture 4 Event Systems
Lecture 4 Event Systems This lecture is based on work done with Mark Bickford. Marktoberdorf Summer School, 2003 Formal Methods One of the major research challenges faced by computer science is providing
More informationChapter 1 Statistical Reasoning Why statistics? Section 1.1 Basics of Probability Theory
Chapter 1 Statistical Reasoning Why statistics? Uncertainty of nature (weather, earth movement, etc. ) Uncertainty in observation/sampling/measurement Variability of human operation/error imperfection
More informationAbstractions and Decision Procedures for Effective Software Model Checking
Abstractions and Decision Procedures for Effective Software Model Checking Prof. Natasha Sharygina The University of Lugano, Carnegie Mellon University Microsoft Summer School, Moscow, July 2011 Lecture
More informationMechanizing Elliptic Curve Associativity
Mechanizing Elliptic Curve Associativity Why a Formalized Mathematics Challenge is Useful for Verification of Crypto ARM Machine Code Joe Hurd Computer Laboratory University of Cambridge Galois Connections
More informationLecture Notes: Axiomatic Semantics and Hoare-style Verification
Lecture Notes: Axiomatic Semantics and Hoare-style Verification 17-355/17-665/17-819O: Program Analysis (Spring 2018) Claire Le Goues and Jonathan Aldrich clegoues@cs.cmu.edu, aldrich@cs.cmu.edu It has
More informationThe Underlying Semantics of Transition Systems
The Underlying Semantics of Transition Systems J. M. Crawford D. M. Goldschlag Technical Report 17 December 1987 Computational Logic Inc. 1717 W. 6th St. Suite 290 Austin, Texas 78703 (512) 322-9951 1
More informationChapter Learning Objectives. Random Experiments Dfiii Definition: Dfiii Definition:
Chapter 2: Probability 2-1 Sample Spaces & Events 2-1.1 Random Experiments 2-1.2 Sample Spaces 2-1.3 Events 2-1 1.4 Counting Techniques 2-2 Interpretations & Axioms of Probability 2-3 Addition Rules 2-4
More informationInformal Statement Calculus
FOUNDATIONS OF MATHEMATICS Branches of Logic 1. Theory of Computations (i.e. Recursion Theory). 2. Proof Theory. 3. Model Theory. 4. Set Theory. Informal Statement Calculus STATEMENTS AND CONNECTIVES Example
More informationFirst-Order Theorem Proving and Vampire. Laura Kovács (Chalmers University of Technology) Andrei Voronkov (The University of Manchester)
First-Order Theorem Proving and Vampire Laura Kovács (Chalmers University of Technology) Andrei Voronkov (The University of Manchester) Outline Introduction First-Order Logic and TPTP Inference Systems
More informationFault-Tolerant Computer System Design ECE 60872/CS 590. Topic 2: Discrete Distributions
Fault-Tolerant Computer System Design ECE 60872/CS 590 Topic 2: Discrete Distributions Saurabh Bagchi ECE/CS Purdue University Outline Basic probability Conditional probability Independence of events Series-parallel
More informationChapter 2: Random Variables
ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:
More informationIntroduction to Information Theory
Introduction to Information Theory Impressive slide presentations Radu Trîmbiţaş UBB October 2012 Radu Trîmbiţaş (UBB) Introduction to Information Theory October 2012 1 / 19 Transmission of information
More informationa zoo of (discrete) random variables
discrete uniform random variables A discrete random variable X equally liely to tae any (integer) value between integers a and b, inclusive, is uniform. Notation: X ~ Unif(a,b) a zoo of (discrete) random
More informationEECS150 - Digital Design Lecture 4 - Boolean Algebra I (Representations of Combinational Logic Circuits)
EECS150 - Digital Design Lecture 4 - Boolean Algebra I (Representations of Combinational Logic Circuits) September 5, 2002 John Wawrzynek Fall 2002 EECS150 Lec4-bool1 Page 1, 9/5 9am Outline Review of
More informationUniversity of Surrey. Bounded Retransmission in Event-B CSP: A Case Study. Steve Schneider, Helen Treharne and Heike Wehrheim
University of Surrey Bounded Retransmission in Event-B CSP: A Case Study Department of Computing Steve Schneider, Helen Treharne and Heike Wehrheim March 21 st 2011 Computing Sciences Report CS-11-04 Bounded
More informationTheorem 1.7 [Bayes' Law]: Assume that,,, are mutually disjoint events in the sample space s.t.. Then Pr( )
Theorem 1.7 [Bayes' Law]: Assume that,,, are mutually disjoint events in the sample space s.t.. Then Pr Pr = Pr Pr Pr() Pr Pr. We are given three coins and are told that two of the coins are fair and the
More informationECE-517: Reinforcement Learning in Artificial Intelligence. Lecture 4: Discrete-Time Markov Chains
ECE-517: Reinforcement Learning in Artificial Intelligence Lecture 4: Discrete-Time Markov Chains September 1, 215 Dr. Itamar Arel College of Engineering Department of Electrical Engineering & Computer
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More information2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y.
CS450 Final Review Problems Fall 08 Solutions or worked answers provided Problems -6 are based on the midterm review Identical problems are marked recap] Please consult previous recitations and textbook
More informationOutline. EECS150 - Digital Design Lecture 4 - Boolean Algebra I (Representations of Combinational Logic Circuits) Combinational Logic (CL) Defined
EECS150 - Digital Design Lecture 4 - Boolean Algebra I (Representations of Combinational Logic Circuits) January 30, 2003 John Wawrzynek Outline Review of three representations for combinational logic:
More informationStatistical Preliminaries. Stony Brook University CSE545, Fall 2016
Statistical Preliminaries Stony Brook University CSE545, Fall 2016 Random Variables X: A mapping from Ω to R that describes the question we care about in practice. 2 Random Variables X: A mapping from
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationEE 178 Lecture Notes 0 Course Introduction. About EE178. About Probability. Course Goals. Course Topics. Lecture Notes EE 178
EE 178 Lecture Notes 0 Course Introduction About EE178 About Probability Course Goals Course Topics Lecture Notes EE 178: Course Introduction Page 0 1 EE 178 EE 178 provides an introduction to probabilistic
More informationOrigins of Probability Theory
1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.
More informationLecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability
Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic
More informationEssentials on the Analysis of Randomized Algorithms
Essentials on the Analysis of Randomized Algorithms Dimitris Diochnos Feb 0, 2009 Abstract These notes were written with Monte Carlo algorithms primarily in mind. Topics covered are basic (discrete) random
More informationGuidelines for Solving Probability Problems
Guidelines for Solving Probability Problems CS 1538: Introduction to Simulation 1 Steps for Problem Solving Suggested steps for approaching a problem: 1. Identify the distribution What distribution does
More informationFormalization of Birth-Death and IID Processes in Higher-order Logic
Formalization of Birth-Death and IID Processes in Higher-order Logic Liya Liu, Osman Hasan and Sofiène Tahar Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada Email:
More informationStatistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
More informationIntroduction to Stochastic Processes
Stat251/551 (Spring 2017) Stochastic Processes Lecture: 1 Introduction to Stochastic Processes Lecturer: Sahand Negahban Scribe: Sahand Negahban 1 Organization Issues We will use canvas as the course webpage.
More informationBasic concepts of probability theory
Basic concepts of probability theory Random variable discrete/continuous random variable Transform Z transform, Laplace transform Distribution Geometric, mixed-geometric, Binomial, Poisson, exponential,
More informationComputing and Communications 2. Information Theory -Entropy
1896 1920 1987 2006 Computing and Communications 2. Information Theory -Entropy Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2017, Autumn 1 Outline Entropy Joint entropy
More information1 Proof techniques. CS 224W Linear Algebra, Probability, and Proof Techniques
1 Proof techniques Here we will learn to prove universal mathematical statements, like the square of any odd number is odd. It s easy enough to show that this is true in specific cases for example, 3 2
More informationRandom variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationModel Checking I. What are LTL and CTL? dack. and. dreq. and. q0bar
Model Checking I What are LTL and CTL? q0 or and dack dreq q0bar and 1 View circuit as a transition system (dreq, q0, dack) (dreq, q0, dack ) q0 = dreq and dack = dreq & (q0 + ( q0 & dack)) q0 or and D
More informationMachine Learning using Bayesian Approaches
Machine Learning using Bayesian Approaches Sargur N. Srihari University at Buffalo, State University of New York 1 Outline 1. Progress in ML and PR 2. Fully Bayesian Approach 1. Probability theory Bayes
More informationOverview. Discrete Event Systems Verification of Finite Automata. What can finite automata be used for? What can finite automata be used for?
Computer Engineering and Networks Overview Discrete Event Systems Verification of Finite Automata Lothar Thiele Introduction Binary Decision Diagrams Representation of Boolean Functions Comparing two circuits
More informationLecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes
Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities
More informationProbability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27
Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, 2018 1 / 27 Outline 1 Elements of probability 2 Random variables 3 Multiple
More informationProbability Theory for Machine Learning. Chris Cremer September 2015
Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares
More information02 Propositional Logic
SE 2F03 Fall 2005 02 Propositional Logic Instructor: W. M. Farmer Revised: 25 September 2005 1 What is Propositional Logic? Propositional logic is the study of the truth or falsehood of propositions or
More informationIntroduction to Wireless & Mobile Systems. Chapter 4. Channel Coding and Error Control Cengage Learning Engineering. All Rights Reserved.
Introduction to Wireless & Mobile Systems Chapter 4 Channel Coding and Error Control 1 Outline Introduction Block Codes Cyclic Codes CRC (Cyclic Redundancy Check) Convolutional Codes Interleaving Information
More informationRandom Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping
Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the
More informationSTAT 516: Basic Probability and its Applications
Lecture 4: Random variables Prof. Michael September 15, 2015 What is a random variable? Often, it is hard and/or impossible to enumerate the entire sample space For a coin flip experiment, the sample space
More informationTowards a Mechanised Denotational Semantics for Modelica
Towards a Mechanised Denotational Semantics for Modelica Simon Foster Bernhard Thiele Jim Woodcock Peter Fritzson Department of Computer Science, University of York PELAB, Linköping University 3rd February
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More information1 INFO Sep 05
Events A 1,...A n are said to be mutually independent if for all subsets S {1,..., n}, p( i S A i ) = p(a i ). (For example, flip a coin N times, then the events {A i = i th flip is heads} are mutually
More informationLogic. Combinational. inputs. outputs. the result. system can
Digital Electronics Combinational Logic Functions Digital logic circuits can be classified as either combinational or sequential circuits. A combinational circuit is one where the output at any time depends
More informationContinuing discussion of CRC s, especially looking at two-bit errors
Continuing discussion of CRC s, especially looking at two-bit errors The definition of primitive binary polynomials Brute force checking for primitivity A theorem giving a better test for primitivity Fast
More informationProbabilistic Guarded Commands Mechanized in HOL
Probabilistic Guarded Commands Mechanized in HOL Joe Hurd joe.hurd@comlab.ox.ac.uk Oxford University Joint work with Annabelle McIver (Macquarie University) and Carroll Morgan (University of New South
More informationAn-Najah National University Faculty of Engineering Industrial Engineering Department. Course : Quantitative Methods (65211)
An-Najah National University Faculty of Engineering Industrial Engineering Department Course : Quantitative Methods (65211) Instructor: Eng. Tamer Haddad 2 nd Semester 2009/2010 Chapter 3 Discrete Random
More informationContinuous Probability Spaces
Continuous Probability Spaces Ω is not countable. Outcomes can be any real number or part of an interval of R, e.g. heights, weights and lifetimes. Can not assign probabilities to each outcome and add
More informationSample Spaces, Random Variables
Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted
More informationDiscrete Random Variable
Discrete Random Variable Outcome of a random experiment need not to be a number. We are generally interested in some measurement or numerical attribute of the outcome, rather than the outcome itself. n
More informationCS 237: Probability in Computing
CS 237: Probability in Computing Wayne Snyder Computer Science Department Boston University Lecture 11: Geometric Distribution Poisson Process Poisson Distribution Geometric Distribution The Geometric
More informationTheorem Proving beyond Deduction
Theorem Proving beyond Deduction Specification and Verification with Higher-Order Logic Arnd Poetzsch-Heffter (Slides by Jens Brandt) Software Technology Group Fachbereich Informatik Technische Universität
More informationCS151 Complexity Theory. Lecture 13 May 15, 2017
CS151 Complexity Theory Lecture 13 May 15, 2017 Relationship to other classes To compare to classes of decision problems, usually consider P #P which is a decision class easy: NP, conp P #P easy: P #P
More informationA POMDP Framework for Cognitive MAC Based on Primary Feedback Exploitation
A POMDP Framework for Cognitive MAC Based on Primary Feedback Exploitation Karim G. Seddik and Amr A. El-Sherif 2 Electronics and Communications Engineering Department, American University in Cairo, New
More informationIntroduction to Machine Learning
Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB
More informationEleventh Problem Assignment
EECS April, 27 PROBLEM (2 points) The outcomes of successive flips of a particular coin are dependent and are found to be described fully by the conditional probabilities P(H n+ H n ) = P(T n+ T n ) =
More informationHow many rounds can Random Selection handle?
How many rounds can Random Selection handle? Shengyu Zhang Abstract The construction of zero-knowledge proofs can be greatly simplified if the protocol is only required be secure against the honest verifier.
More informationSFM-11:CONNECT Summer School, Bertinoro, June 2011
SFM-:CONNECT Summer School, Bertinoro, June 20 EU-FP7: CONNECT LSCITS/PSS VERIWARE Part 3 Markov decision processes Overview Lectures and 2: Introduction 2 Discrete-time Markov chains 3 Markov decision
More informationChapter 2. Reductions and NP. 2.1 Reductions Continued The Satisfiability Problem (SAT) SAT 3SAT. CS 573: Algorithms, Fall 2013 August 29, 2013
Chapter 2 Reductions and NP CS 573: Algorithms, Fall 2013 August 29, 2013 2.1 Reductions Continued 2.1.1 The Satisfiability Problem SAT 2.1.1.1 Propositional Formulas Definition 2.1.1. Consider a set of
More informationMethodology for Computer Science Research Lecture 4: Mathematical Modeling
Methodology for Computer Science Research Andrey Lukyanenko Department of Computer Science and Engineering Aalto University, School of Science and Technology andrey.lukyanenko@tkk.fi Definitions and Goals
More information