Empirical Distributions

Size: px

Start display at page:

Download "Empirical Distributions"

Tyler Eustace Caldwell
5 years ago
Views:

1 Empirical Distributios A empirical distributio is oe for which each possible evet is assiged a probability derived from experimetal observatio. It is assumed that the evets are idepedet ad the sum of the probabilities is. A empirical distributio may represet either a cotiuous or a discrete distributio. If it represets a discrete distributio, the samplig is doe o step. If it represets a cotiuous distributio, the samplig is doe via iterpolatio. The way the table is described usually determies if a empirical distributio is to be hadled discretely or cotiuously; e.g., discrete descriptio cotiuous descriptio value probability value probability To use liear iterpolatio for cotiuous samplig, the discrete poits o the ed of each step eed to be coected by lie segmets. This is represeted i the graph below by the gree lie segmets. The steps are represeted i blue: x.5 I the discrete case, samplig o step is accomplished by accumulatig probabilities from the origial table; e.g., for x =.4, accumulate probabilities util the cumulative probability exceeds.4; is the evet value at the poit this happes (i.e., the cumulative probability is the first to exceed.4, so the value is 35). I the cotiuous case, the ed poits of the probability accumulatio are eeded, i this case x=.5 ad x=.65 which represet the poits (.5,) ad (.65,35) o the graph. From basic college algebra, the slope of the lie segmet is (35-)/(.65-.5) = 5/.4 = The slope = 37.5 = (35-)/(.65-.4) so = 35 - (37.5.5) = = 5.65.

2 Discrete Distributios To put a little historical perspective behid the ames used with these distributios, James Beroulli (654-75) was a Swiss mathematicia whose book Ars Cojectadi (published posthumously i 73) was the first sigificat book o probability; it gathered together the ideas o coutig, ad amog other thigs provided a proof of the biomial theorem. Siméo-Deis Poisso (78-84) was a professor of mathematics at the Faculté des Scieces whose 837 text Recherchés sur la probabilité des jugemets e matière crimielle et e matière civile itroduced the discrete distributio ow called the Poisso distributio. Keep i mid that scholars such as these evolved their theories with the objective of providig sophisticated abstract models of real-world pheomea (a effort which, amog other thigs, gave birth to the calculus as a major modelig tool). I. Beroulli Distributio A Beroulli evet is oe for which the probability the evet occurs is p ad the probability the evet does ot occur is -p; i.e., the evet is has two possible outcomes (usually viewed as success or failure) occurrig with probability p ad -p, respectively. A Beroulli trial is a istatiatio of a Beroulli evet. So log as the probability of success or failure remais the same from trial to trial (i.e., each trial is idepedet of the others), a sequece of Beroulli trials is called a Beroulli process. Amog other coclusios that could be reached, this meas that for trials, the probability of successes is p. A Beroulli distributio is the pair of probabilities of a Beroulli evet, which is too simple to be iterestig. However, it is implicitly used i yeso decisio processes where the choice occurs with the same probability from trial to trial (e.g., the customer chooses to go dow aisle with probability p) ad ca be cast i the same kid of mathematical otatio used to describe more complex distributios: p (-p) - for =, p() = otherwise p() The expected value of the distributio is give by E(X) = (-p) + p = p p -p The stadard deviatio is give by σ = (- p)( - p) + p(- p) = p (-p) While this is otatioal overkill for such a simple distributio, it s costructio i this form will be useful for uderstadig other distributios.

3 Samplig from a discrete distributio, requires a fuctio that correspods to the distributio fuctio of a cotiuous distributio f give by F(x) = This is give by the mass fuctio F(x) of the distributio, which is the step fuctio obtaied from the cumulative (discrete) distributio give by the sequece of partial sums x p() x f()d For the Beroulli distributio, F(x) has the costructio for - x < F(x) = -p for x < for x which is a icreasig fuctio (a so ca be iverted i the same maer as for cotiuous distributios). Graphically, F(x) looks like F(x) p -p which iverted yields the samplig fuctio -p x I other words, for radom value x draw from [,), = if x < -p if -p x < I essece, this demostrates that samplig from a discrete distributio, eve oe as simple as the Beroulli distributio, ca be viewed i the same maer as for cotiuous distributios.

4 II. Biomial Distributio The Beroulli distributio represets the success or failure of a sigle Beroulli trial. The Biomial Distributio represets the umber of successes ad failures i idepedet Beroulli trials for some give value of. For example, if a maufactured item is defective with probability p, the the biomial distributio represets the umber of successes ad failures i a lot of items. I particular, samplig from this distributio gives a cout of the umber of defective items i a sample lot. Aother example is the umber of heads obtaied i tossig a coi times. The biomial distributio gets its ame from the biomial theorem which states that the biomial k -k! ( a + b) = a b where = k k k!( - k)! It is worth poitig out that if a = b =, this becomes ( ) k + = = Yet aother viewpoit is that if S is a set of sie, the umber of k elemet subsets of S is give by! = k!( - k)! k This formula is the result of a simple coutig aalysis: there are! ( -)... ( - k + ) = ( - k)! ordered ways to select k elemets from ( ways to choose the st item, (-) the d, ad so o). Ay give selectio is a permutatio of its k elemets, so the uderlyig subset is couted k! times. Dividig by k! elimiates the duplicates. Note that the expressio for couts the total umber of subsets of a -elemet set. For idepedet Beroulli trials the pdf of the biomial distributio is give by p() = p ( p) otherwise for =,,..., Note that by the biomial theorem, p() = (p + (- p)) =, verifyig that p() is a pdf.

5 Whe choosig items from amog items, the term p ( p) represets the probability that are defective (ad cocomitatly that (-) are ot defective). The biomial theorem is also the key for determiig the expected value E(X) for the radom variable X for the distributio. E(X) is give by E(X) = p( i ) i (the expected value is just the sum of the discrete items weighted by their probabilities, which correspods to a sample s mea value; this is a extesio of the simple average value obtaied by dividig by, which correspods to a weighted sum with each item havig probability /). For the biomial distributio the calculatio of E(X) is accomplished by! term i commo - - E(X) = p ( p) = p ( p)!( - )! (- p) = p (- p) p = p p = p(p + - p) preset i every summad apply the biomial theorem to this (ote that - = (-) - (-) ) This gives the result that E(X) = p for a biomial distributio o items where probability of success is p. It ca be similarly show that the stadard deviatio is p ( p) The biomial distributio with = ad p=.7 appears as follows: = p p().3 mea

6 Its correspodig mass fuctio F(x) is give by F() which provides the samplig fuctio x A typical samplig tactic is to accumulate the sum p() icreasig util the sum's value exceeds the radom value betwee ad draw for x. The fial summatio limit is the sample value. Note that i cotrast to a cotiuous pdf described by some formula, the fuctio for a fiite discrete pdf has to be give i its relatioal form by a table of pairs, which i tur madates the kid of "search" algorithm approach used above to obtai.

7 III. Poisso Distributio (values =,,,...) The Poisso distributio is the limitig case of the biomial distributio where p=λ/ ad. The expected value E(X) = λ. The stadard deviatio is give by λ. The pdf is λ e p() =! This distributio dates back to Poisso's 837 text regardig civil ad crimial matters, i effect scotchig the tale that its first use was for modelig deaths from the kicks of horses i the Prussia army. I additio to modelig the umber of arrivals over some iterval of time (recall the relatioship to the expoetial distributio; a Poisso process has expoetially distributed iterarrival times), the distributio has also bee used to model the umber of defects o a maufactured article. I geeral the Poisso distributio is used for situatios where the probability of a evet occurrig is very small, but the umber of trials is very large (so the evet is expected to actually occur a few times). Graphically, with λ =, it appears as: p() λ mea The samplig fuctio looks like: x

8 IV. Geometric Distributio The geometric distributio gets its ame from the geometric series: various flavors of the geometric series The pdf for the geometric distributio is give by p() = < r, r =, r =, - r (- r) for r ( + ) r = (- r) ( p) - p otherwise for =,,... The geometric distributio is the discrete aalog of the expoetial distributio. Like the expoetial distributio, it is "memoryless"; i.e., P(X > a+b X > a) = P(X > b) (the geometric distributio is the oly discrete distributio with this property just as the expoetial distributio is the oly cotiuous oe behavig i this maer). Its expected value is give by - E(X) = (- p) p = p = (-+ p) p (by applyig the 3 rd form of the geometric series). p The stadard deviatio is give by. p A plot of the geometric distributio with p =.3 is give by p() mea

9 A typical use of the geometric distributio is for modelig the umber of failures before the first success i a sequece of idepedet Beroulli trials. This is a typical sceario for sales. Suppose that the probability of makig a sale is.3. The p() =.3 is the probability of success o the st try, p() = (-p)p =.7.3 =. which is the probability of failig o the st try (with probability -p) ad succeedig o the d (with probability p). p(3) = (-p)(-p)p =.5 is the probability that the sale takes 3 tries, ad so forth. A radom sample from the distributio represets the umber of attempts eeded to make the sale.

Probability and statistics: basic terms

Probability and statistics: basic terms Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample