ECS /1 Part IV.2 Dr.Prapun

Transcription

1 Sirindhorn International Institute of Technology Thammasat University School of Information, Computer and Communication Technology ECS35 4/ Part IV. Dr.Prapun.4 Families of Continuous Random Variables Theorem.4 states that any nonnegative function f() whose integral over the interval (, + ) equals can be regarded as a probability density function of a random variable. In real-world applications, however, special mathematical forms naturally show up. In this section, we introduce a couple families of continuous random variables that frequently appear in practical applications. The probability densities of the members of each family all have the same mathematical form but differ only in one or more parameters..4. Uniform Distribution Definition.35. For a uniform random variable on an interval [a, b], we denote its family by uniform([a, b]) or U([a, b]) or simply U(a, b). Epressions that are synonymous with X is a uniform random variable are X is uniformly distributed, X has a uniform distribution, and X has a uniform density. This family is characterized by {, < a, > b f X () = b a, a b The random variable X is just as likely to be near any value in [a, b] as any other value. 9

2 In MATLAB, (a) use X = a+(b-a)*rand or X = random( Uniform,a,b) to generate the RV, (b) use pdf( Uniform,,a,b) and cdf( Uniform,,a,b) to calculate the pdf and cdf, respectively. {, < a, > b Eercise.36. Show that F X () = a b a, a b 84 Probability theory, random variables and random processes f () F () b a Fig. 3.5 The pdf and cdf for the uniform random variable. Figure 6: The pdf and cdf for the uniform random variable. [6, Fig. 3.5] f () F () Eample.37 (F). Suppose X is uniformly distributed on πσ the interval (, ). (X U(, ).) a b a b (a) Plot the pdf f X () of X. Fig. 3.6 (b) Plot the cdf F X () of X. The pdf and cdf of a Gaussian random variable. μ μ Gaussian (or normal) random variable is described by the following pdf: f () = This is a continuous random variable that.38. The uniform distribution provides a probability model for selecting a point at random from the interval [a, b]. { } ep ( μ) πσ σ, (3.6) Use with caution to model a quantity that is known to vary where μrandomly σ are two between parameters a and whose b but meaning about is described which little later. Itelse is usually is known. denoted as N (μ, σ ). Figure 3.6 shows sketches of the pdf and cdf of a Gaussian random variable. The Gaussian random variable is the most important and frequently encountered random variable in communications. This is because 3 thermal noise, which is the major source of noise in communication systems, has a Gaussian distribution. Gaussian noise and the Gaussian pdf are discussed in more depth at the end of this chapter. The problems eplore other pdf models. Some of these arise when a random variable

3 Eample.39. [9, E. 4. p. 4-4] In coherent radio communications, the phase difference between the transmitter and the receiver, denoted by Θ, is modeled as having a uniform density on [ π, π]. (a) P [Θ ] = (b) P [ Θ π ] = 3 4 Eercise.4. Show that when X U([a, b]), EX = a+b Var X = (b a), and E [ X ] ( = 3 b + ab + a )..4. Gaussian Distribution.4. This is the most widely used model for the distribution of a random variable. When you have many independent random variables, a fundamental result called the central limit theorem (CLT) (informally) says that the sum (technically, the average) of them can often be approimated by normal distribution. Definition.4. Gaussian random variables: (a) Often called normal random variables because they occur so frequently in practice. (b) In MATLAB, use X = random( Normal,m,σ) or X = σ*randn + m. (c) f X () = πσ e ( m σ ). In Ecel, use NORMDIST(,m,σ,FALSE). In MATLAB, use normpdf(,m,σ) or pdf( Normal,,m,σ). Figure 7 displays the famous bell-shaped graph of the Gaussian pdf. This curve is also called the normal curve. 3,

4 84 Probability theory, random variables and random processes f () F () (d) F X () has no closed-form epression. However, see.48. Inb MATLAB, a use normcdf(,m,σ) or cdf( Normal,,m,σ). In Ecel, use NORMDIST(,m,σ,TRUE). Fig. 3.5 a b a (e) We write X N ( m, σ ). The pdf and cdf for the uniform random variable. b πσ f () F () Fig. 3.6 The pdf and cdf of a Gaussian random variable. Figure 7: The pdf and cdf of N (µ, σ ). [6, Fig. 3.6] μ μ Gaussian (or normal) random variable is described by the following pdf:.43. EX = m and Var X = σ. This is a continuous random variable that { } f () = ep ( μ) πσ σ, (3.6).44. Important probabilities: P [ X µ < σ] =.687; P [ X µ > σ] =.373; P [ X µ > σ] =.455; P [ X µ < σ] =.9545 These values are illustrated in Figure. where μ and σ are two parameters whose meaning is described later. It is usually denoted as N (μ, σ ). Figure 3.6 shows sketches of the pdf and cdf of a Gaussian random variable. The Gaussian random variable is the most important and frequently encountered random variable in communications. This is because thermal noise, which is the major source of noise in communication systems, has a Gaussian distribution. Gaussian noise and the Gaussian pdf are discussed in more depth at the end of this chapter. The problems eplore other pdf models. Some of these arise when a random variable is passed through a nonlinearity. How to determine the pdf of the random variable in this case is discussed net. Eample.45. Figure compares several deviation scores and the normal distribution: Functions of a random variable A function of a random variable y = g() is itself a random variable. From the definition, the cdf of y can be written as (a) Standard scores have a mean of zero and a standard deviation of.. F y (y) = P(ω : g((ω)) y). (3.7) (b) Scholastic Aptitude Test scores have a mean of 5 and a standard deviation of. 3

5 9 3.5 The Gaussian random variable and process.6 (a).4 Signal amplitude (V) t (s) (b) Histogram Gaussian fit Laplacian fit f () (/V) (V) Fig. 3.4 (a) A sample skeletal muscle (emg) signal, and (b) its histogram and pdf fits. Figure 8: Electrical activity of a skeletal muscle: (a) A sample skeletal muscle (emg) signal, and (b) its histogram and pdf fits. [6, Fig. 3.4] [ [ ] = f ()d] = K e a d = K = K e a d = = y= 33 y= e ay dy e a( +y ) ddy. (3.3)

6 3.5 The Gaussian random variable and process f () j m jω 3) Fourier transform:.5 ( f ) f ( ) e dt e ω F = = ωσ. X 4) Note that α π e d = α. Fig. 3.5 Plots of the zero-mean Gaussian m pdf for different values of standard deviation, σ. 5) P[ X > ] = Q σ ; [ ] m m P X < = Q = Q. σ σ X Figure 9: Plots of the zero-mean Gaussian pdf for different values of standard deviation, σ X. [6, Fig. 3.5] 6) Q-function: ( ), pictures the comparison of sevation scores and the normal distritandard scores have a mean of d a standard deviation of.. ic Aptitude Test scores have a 5 and a standard deviation of P X μ < σ =.687, P X μ > σ =.373 σ = σ = Range (±kσ ) k = k = k = 3 k = 4 of the pdf are ignorable. μ Indeed σ μ μwhen + σ communication systems are considered later it is the μ σ μ μ + σ presence of these tails that results in bit errors. The probabilities are on the order of 3, very small, but still significant in terms of system performance. It is of interest to Figure : Q zprobability = e d see how far, in terms of z π density corresponds function to P[ X > zof ] where X X N ~ N(µ, (,σ ) ;). σ, one must be from the mean value to have the different levels of that is Q error probabilities. ( z ) is the probability of the tail of N As shall be seen in later chapters (, this ). FIGURE A translates to the required SNR to achieve a specified bit error probability. N (, This ) is also shown in Table 3.. Having considered the single (or univariate) Gaussian random variable, we turn our attention to the case of two jointly Gaussian random variables (or the bivariate case). Again they are described by their joint pdf which, in general, is an eponential whose eponent is a quadratic in the two variables, i.e., f,y (, y) = QKe ( z ) (a +b+cy+dy+ey +f ), where the constants K, a, b, c, d, e, and f are chosen to satisfy the basic properties.5 of a valid joint pdf, namely being always nonnegative ( ), having unit volume, and also that the marginal pdfs, f () = f z,y(, y)dy and f y (y) = % 4% 34% 34% f,y(, 4% y)d, are % valid. Written in standard forma) the Q is joint a decreasing pdf is function with Q ( ) = Q z = Q z Standard Scores Intelligence Scale scores have a and a standard deviation of 6. case there are 34 percent of the tween the mean and one standard b) ( ) ( ), 4 percent between one and c) Qtwo ( Q( z )) = z deviations, and percent beyond π SAT π Scores 4 sin θ ard deviations. sin θ d) Q( ) = e dθ π Q ( 84 ) = e dθ π Binet Intelligence Scale Scores d Q standard score. 6.3 Applications of Normal Distributions The normal distribution can help us to determine probabilities. 6.4 Notation The z notation is critical in the use of normal σ = 5.3 distributions. 6.5 Normal Approimation of the Binomial Binomial probabilities can be estimated by using a normal distribution. al Probability Distributions nce Scores al probability distribution P is X considered μ > σ Table =.455, 3. Influence the P X single of μ σ< on σ different = most.9545 quantities important probatribution. An unlimited number of continuous random variables have either a normal oimately normal distribution. f X P(m kσ < m + ( kσ ) ( ) f X ) all familiar with IQ (intelligence Error probability quotient) scores and/or 3 SAT (Scholastic 4 Aptitude 6 Test) 8 scores have a mean of Distance and a from standard the mean deviation 3.9 of 6. SAT 3.7scores have 4.75 a mean 5.6 of 95% 68% standard deviation of. But did you know that these continuous random variables a normal distribution? k, Applying Psychology: Critical and Creative e) Thinking, ( ) = Figure e ; 6. Q Pictures ( f ( ) ) = the Comparison e of f ( Several ). Deviation the Normal Distribution, Figure 99 Prentice-Hall, : Comparison d Inc. Reproduced of π Several dby permission Deviation of πpearson Scoresd Education, and the Normal Inc. Distribution d ( f ( ) ) d 34

7 (c) Binet Intelligence Scale 43 scores have a mean of and a standard deviation of 6. In each case there are 34 percent of the scores between the mean and one standard deviation, 4 percent between one and two standard deviations, and percent beyond two standard deviations. [Source: Beck, Applying Psychology: Critical and Creative Thinking.].46. N (, ) is the standard Gaussian (normal) distribution. In Ecel, use NORMSINV(RAND()). In MATLAB, use randn. The standard normal cdf is denoted by Φ(z). It inherits all properties of cdf. Moreover, note that Φ( z) = Φ(z)..47. Relationship between N (, ) and N (m, σ ). (a) An arbitrary Gaussian random variable with mean m and variance σ can be represented as σz+m, where Z N (, ). 43 Alfred Binet, who devised the first general aptitude test at the beginning of the th century, defined intelligence as the ability to make adaptations. The general purpose of the test was to determine which children in Paris could benefit from school. Binets test, like its subsequent revisions, consists of a series of progressively more difficult tasks that children of different ages can successfully complete. A child who can solve problems typically solved by children at a particular age level is said to have that mental age. For eample, if a child can successfully do the same tasks that an average 8-year-old can do, he or she is said to have a mental age of 8. The intelligence quotient, or IQ, is defined by the formula: IQ = (Mental Age/Chronological Age) There has been a great deal of controversy in recent years over what intelligence tests measure. Many of the test items depend on either language or other specific cultural eperiences for correct answers. Nevertheless, such tests can rather effectively predict school success. If school requires language and the tests measure language ability at a particular point of time in a childs life, then the test is a better-than-chance predictor of school performance. 35

8 I z <I>(z) I z ~(z) I z <I>(z) I z <I>(z) I z <I>(z) I z <I> (z) I Table 3. The standard nonnal CDF <t>(y).

9 This relationship can be used to generate general Gaussian RV from standard Gaussian RV. (b) If X N ( m, σ ), the random variable Z = X m σ is a standard normal random variable. That is, Z N (, ). Creating a new random variable by this transformation is referred to as standardizing. The standardized variable is called standard score or z-score..48. It is impossible to epress the integral of a Gaussian PDF between non-infinite limits (e.g., ()) as a function that appears on most scientific calculators. An old but still popular technique to find integrals of the Gaussian PDF is to refer to tables that have been obtained by numerical integration. One such table is the table that lists Φ(z) for many values of positive z. For X N ( m, σ ), we can show that the CDF of X can be calculated by ( ) m F X () = Φ. σ Eample.49. Suppose Z N (, ). Evaluate the following probabilities. (a) P [ Z ] 36

10 (b) P [ Z ] Eample.5. Suppose X N (, ). Find P [ X ]..5. Q-function: Q (z) = z N, π e z d corresponds to P [X > z] where X N (, ); that is Q (z) is the probability of the tail of N (, ). The Q function is then a complementary cdf (ccdf). erf ( z ) Q ( z ) N (,).9.8 Q( z ) z z Figure : Q-function (a) Q is a decreasing function with Q () =. (b) Q ( z) = Q (z) = Φ(z).5. Error function (MATLAB): erf (z) = π Q ( z ) (a) It is an odd function of z. z e d = (b) For z, it corresponds to P [ X < z] where X N (, ). (c) lim z erf (z) = 37

11 I z Q(z) I z' Q(z) Iz Q(z) Q(z) Q(z) , Ql.O I z Iz 3, ; Table 3. The standard normal complementary CDF Q(z).

12 k, k odd = = k 35 ( k ) σ, k even k E ( X μ) ( k ) E ( X μ) k k 4 6 ( k ) σ, k odd X μ E = π [Papoulis p ]. k 35 (d) erf ( z) = erf (z) ( k ) σ, k even 4 Var X ( = 4μ σ + ( σ. )) ( ) (e) Φ() = + erf = () erfc, odd 8) For N (,) and k, ( ) k k k E X = k E X = 35 ( k, ) k even (f) The complementary error function: erfc (z) = erf (z) = Q ( z 9) Error function (Matlab): ( ) = z ) = ( π = erf z e d Q z) corresponds to π z e d P X < z where X ~ N,. N, erf ( z ) Q ( z ) a) erf ( z) lim = z Figure 3: erf-function and Q-function b) erf ( z) = erf ( z).4.3 Eponential Distribution Definition.53. The eponential distribution is denoted by E (λ). (a) λ > is a parameter of the distribution, often called the rate parameter. (b) Characterized by { λe f X () = λ, >,, { e F X () = λ, >,, z 38

13 Survival-, survivor-, or reliability-function: (c) MATLAB: X = eprnd(/λ) or random( ep,/λ) f X () = eppdf(,/λ) or pdf( ep,,/λ) F X () = epcdf(,/λ) or cdf( ep,,/λ) Eample.54. Suppose X E(λ), find P [ < X < ]. Eercise.55. Eponential random variable as a continuous version of geometric random variable: Suppose X E (λ). Show that X G (e λ ) and X G (e λ ) Eample.56. The eponential distribution is intimately related to the Poisson process. It is often used as a probability model for the (waiting) time until a rare event occurs. time elapsed until the net earthquake in a certain region decay time of a radioactive particle time between independent events such as arrivals at a service facility or arrivals of customers in a shop. duration of a cell-phone call time it takes a computer network to transmit a message from one node to another. 39

14

15 .57. EX = λ Eample.58. Phone Company A charges $.5 per minute for telephone calls. For any fraction of a minute at the end of a call, they charge for a full minute. Phone Company B also charges $.5 per minute. However, Phone Company B calculates its charge based on the eact duration of a call. If T, the duration of a call in minutes, is eponential with parameter λ = /3, what are the epected revenues per call E [R A ] and E [R B ] for companies A and B? Solution: First, note that ET = λ = 3. Hence, E [R B ] = E [.5 T ] =.5ET = $.45. and E [R A ] = E [.5 T ] =.5E T. ( Now, recall, from Eercise.55, that T G ) e λ. E T = Therefore, e λ E [R A ] =.5E T.59. Hence,.59. Memoryless property: The eponential r.v. is the only continuous 44 r.v. on [, ) that satisfies the memoryless property: P [X > s + X > s] = P [X > ] for all > and all s > [8, p ]. In words, the future is independent of the past. The fact that it hasn t happened yet, tells us nothing about how much longer it will take before it does happen. Imagining that the eponentially distributed random variable X represents the lifetime of an item, the residual life of an item has the same eponential distribution as the original lifetime, 44 For discrete random variable, geometric random variables satisfy the memoryless property. 4

16 regardless of how long the item has been already in use. In other words, there is no deterioration/degradation over time. If it is still currently working after years of use, then today, its condition is just like new. In particular, suppose we define the set B+ to be { + b : b B}. For any > and set B [, ), we have P [X B + X > ] = P [X B] because P [X B + ] P [X > ].6. Summary: = B+ λe λt dt e λ τ=t = B λe λ(τ+) dτ e λ. X Support S X f X () = { Uniform U(a, b) (a, b), a < < b, b a, otherwise. Normal (Gaussian) N (m, σ ) R πσ e ( m { λe Eponential E(λ) (, ) λ, >,, σ ) Table 4: Eamples of probability density functions. Here, λ, σ >. 4