EnM Probability and Random Processes

Transcription

1 Historical Note: EnM Probability and Random Processes Probability has its roots in games of chance, which have been played since prehistoric time. Games and equipment have been found in Egyptian tombs and Romans had to establish laws to repudiate gaming activities. However, in general, the formal study of probability was provided in the 1600 s by the Chevalier de Mere in a question posed to Pascal: Two people, A and B, agree to play a series of fair games until one person has won six games. They each have wagered the same amount of money, the intention being that the winner will be awarded the entire pot. But suppose, for whatever reason, the series is prematurely terminated, at which point A has won five games and B has won three games. How should the stakes be divided? Pascal posed the problem in probabilistic terms and engaged in extensive correspondence with the French mathematician, Fermat. This began the formal mathematical development of probability theory. The correct answer is that A should receive seven-eights of the total amount wagered. For example, suppose the contest were resumed. The scenarios that lead to A being the first person to win six games are: A wins sixth trial; A loses ninth trial and wins the tenth trial; A loses the ninthe and tenth trial but wins the eleventh trial. The game must end here, since if A loses one more, that means that B has won six games and takes the pot.. So 2 p+ qp+ q p = Prob{A wins game given he won 5 trial when game stopped} = + + = Bernoulli and De Moivre continued the development in the first century, followed by Gauss and La Place in the nineteenth century. In the twentieth century, probability became a major branch of mathematical research and has spread to every corner of scientific research. The role of probability and statistics is extremely prominent in the practice of all phases of engineering and business. Engineers are becoming more concerned with the deviation of practical situations from the hard lined theory. It is the job of the Science of Probability and Statistics not to replace engineering judgement, but to provide the engineer with the best mathematical and logical methods for using his experience and judgment in the solution of complex decision problems. The field of Probability and Statistics includes the following: Ι Theory of Probability ΙΙ Statistical Analysis ΙΙΙ Statistical Decision Theory

2 All three have much in common, but each occupies its own place in the science. The general relationship can be illustrated by considering the following chart:

3 Probability & Statistics Probability Theory Statistical Analysis Statistical Decision Theory Mathematical Probability Models Analysis of Data Making Decisions Under Uncertainty Event Relationships Distributions of Random Variables Continuous Random Variables Normal Log Normal Extreme Value Discrete Random Variables Bernoulli Poisson Stochastic Processes Estimation of Parameters Fitting of Distributions Hypothesis Testing Design of Experiments Bayesian Theory & The Decision Tree Prior Probability Pre-posterior Probability Posterior Probability Value Concepts Personal Probability

4 MEANING OF PROBABILITY: The theory of probability deals mainly with averages of mass phenomenon, occurring sequentially or simultaneously. Such as occurs in the fields of: games of chance polling insurance heredity quality control statistical mechanics queuing theory noise It has been observed that in these and other fields, certain averages approach a constant value or the number of observations increase. This value remains the same if averages are evaluated over a selected sequence or a specific subsequence, i.e. in a coin experiment, the ratio of heads to total tosses approaches ½, or some other constant, and the same ratio is obtained if one considers, say, every 4 th toss. Students are often skeptical about the validity of probabilistic statements. They have been taught that the world evolves according to physical laws that specify exactly its future (determinism) and that probabilistic descriptions are used only for random or chance phenomena, the initial conditions of which are unknown. This deep rooted skepticism about the truth of probabilistic results can be overcome by proper interpretation of the meaning of probability. We will attempt to learn that, like all other scientific discipline, probability is an exact science and all conclusions follow logically from the axioms. It is of course true that the correspondence between theoretical results and the real world is imprecise, however this is characteristic not only of probabilistic conclusion but of all scientific statements. In probabilistic investigations the following steps must be clearly distinguished: Step 1 - (Physical) We determine, by a process that is not and cannot be made exact, the probabilities P(A i ) of various physical events, A i. Step 2 - (Conceptual) We assume that the numbers, P(A i ) satisfy certain axioms, and by deductive logic we determine the probabilities P(B i ) of other events, B i. Step 3 -(Physical). We make physical predictions concerning the events B i based on the number P(B i ) so obtained. Steps 1 and 3 deal with the real world and all statements as inexact. Step 2 deals with the abstract model. This is depicted in the following diagram: S t e p 1 P h y s i c a l S t e p 2 C o n c e p t u a l S t e p 3 P h y s i c a l

5 The term probability has four interpretations: Axiomatic definition (mode concept) Relative Frequency (empirical) Classical (equally likely) Subjective (measure of belief) In our studies we will use only the axiomatic definition as basis of the theory (Step 2). The other three will be used in determination of probabilistic data of real experiments, (Step 1), and application of theoretical results to real experiments, (Step 3). Let us now begin to examine the basic principles of mathematical probabililty Theory of Probability The theory of probability deals with averages and other parameters of mass phenomenon, occurring sequentially or simultaneously. For example: Electron emissions Arrival time for services Noise levels in a network Load levels on a structure It has been observed that these averages approach a constant value as the number of observations increase. The probability of an event, A, can be interpreted in the following sense: Let n = number of times an experiment is performed n a = number of times event A occurs Then the fraction of favorable events is na n = Fraction of favorable events Therefore, P(A) = Probability that event A will occur = lim n na n To present a more formal definition we must understand the concepts of Experiments, Events, and certain element of Set Theory. The number of sample outcomes associated with an experiment need not be finite. Suppose that a coin is tossed until the first tail appears. If the first toss is itself a tail, the outcome of the experiment is T; if the first tail occurs on the second toss, the outcome is HT; and so on. Theoretically, of course, the first tail may never occur, and the infinite nature of S is really apparent: S={T,HT,HHT,HHHT, }

6 We can define: Random Experiment Non-repeatable experiment with more than one outcome. Set - Collection of objects or elements i.e. Heads and tails An apple A vehicle A stress or load level A Number or function, for example {x x 2 +-3x 1 20 =0} set of real numbers φ = empty set {f / f(x) dx < } {φ } = set of empty sets Subset A subset, B, is a set A, is another set whose elements are also elements of A (belongs to A) Field - I - Set of all subsets in S Space, or Universe- S The largest set containing all possible outcomes of a random experiment. Example Consider the following experiment which involves the tossing of a fair die. The Universe is Space = S = {f 1, f 2, f 3,, f 6 } The number of subsets possible are calculated to be 2 6 = 64, namely. 0,{f 1 },{f 2 },.(f 1 f 2 } {f 1 f 2 f 3 }..S Example A woman has her purse stolen by two thieves. She is then shown a police line-up consisting of 5 suspects, including the two perpetuators 1) What is the sample space associated with the experiment Woman picks two suspects out of the line-up? 2) Which outcomes are in the event, A, that she males at least one correct identification? Solution, Let S i = ith suspect, I=1,2,3 T i = ith guilty suspect, I = 1,2 Then, S={(S 1 S 2 )(S 1 S 3 )(S 2 S 3 )(S 1 T 1 )(S 1 T 2 )(S 2 T 1 )(S 2 T 2 )(S 3 T 1 )(S 3 T 2 )(T 1 T 2 )} S = (5 x 4) / 2=10 A = {(S 1 T 1 )(S 1 T 2 )(S 2 T 1 )(S 2 T 2 )(S 3 T 1 )(S 3 T 2 )(T 1 T 2 )} A = 3 x x 1 = 7

7 Set Theory Beginning our review of a few principles of the Set theory, it is convenient to represent sets by closed plane figures called Venn Diagrams S B S = Space or universe A, B, C = Sets f 1, f 2,... = Elements of the sets Subset Notation and Operations Sums (Union) D B - D is a subset of B f i B - f i is an element of B or belongs to B A B - All elements of A, or B, or both. D C A A+ B AB B Note: A + B = B + A (Commutative) (A + B) + C = A + (B + C) (Associative) Products (or Intersection) A B = C = All elements in A and B AB

8 AB = BA (Commutative) (AB)C = A(BC) (Associative) A(B + C) = AB + AC (Distributive) Example Let S = {1, 2, 3, 4, 5} and A = {even} = {2, 4} B = {less than 5} = {1, 2, 3, 4} AB = {even} {less than 5} = {even, less than 5} = {2, 4} Note: Two sets are MUTUALLY exclusive if A B = φ i.e. Complement: = Complement of A = all elements of S that are not in A S A A Note: _ S = φ = null set = empty set = impossible event Difference: A-B = all elements in A but not in B

9 A - B = A = A - AB Also: But: (A-B) + B+ A = A + B (A + A) - A = φ A + (A - A) = A Does not obey associative or commutative law.