THE POISSON WAITING TIME DISTRIBUTION: ANALYSIS AND APPLICATION

Transcription

1 THE POISSON WAITING TIME DISTRIBUTION: ANALYSIS AND APPLICATION by Kyle Park ECED 6530 Project Report 12/12/2013

2 Table of Contents Summary ii Nomenclature ii List of Figures iii 1 The Waiting Time Distribution The Pedestrian Crossing Problem Formulation of Solution First Principles Number of Interarrival Times Posterior Analysis of Interarrival Times Linear Combination Interpretation Compound Distribution Interpretation Analysis of k summed truncated exponential random variables, R k Properties of the Shaping Function Family, {S k } The Waiting Time Probability Density Function, f T (t) Analysis of Solution Convergence of Summation Properness of the PDF, f T (t) Flatness over the Interval (0,t 0 ] Comparison with Simulated Results 9 2 Pilot Application in Queuing Theory Effective Average Service Rate in a Poisson Server The Disgruntled Bank Teller Problem First Principles Existence of an optimal t Remarks 18 4 References 20 i

3 Summary Summary This report deals with a time-to-first-arrival problem that arises in the conventional Poisson Process. More specifically, the report studies the length of time measured from an arbitrary starting point to the first interarrival time exceeding a particular threshold. The problem of describing this length of time is established from a first principles approach. The probability density function describing the time is determined, analyzed, and compared with empirical (Monte- Carlo simulated) results. A pilot application for the result is proposed and detailed. Finally, remarks are made regarding related areas of study and areas for further work on this problem. Nomenclature : PDF, CDF of some random variable, with index variable. : A function (or PDF) convolved with itself times. : Gaussian Approximation to ). : Independent and Identically Distributed. : Laplace Transform operator; operation specifically over the index. : Discrete random variable with Poisson distribution, over time interval. : Geometric distribution parameters, in this case equal to,, respectively. : Random variable representing sum of truncated exponential random variables. : 'th Shaping function with parameter. : Family of (set of) Shaping functions. : Poisson waiting time threshold. : Point of discontinuity, first point of unity in. : Random variable with Poisson Waiting Time distribution. : Vector containing discrete realizations of. : Arithmetic mean of. : Discrete random variable with Geometric distribution. : Set of Complex Numbers. : Set of Natural Numbers including zero. : Set of Real Numbers. : Global maximum value of. : Defined as. : Support of (interval of where is nonzero). : Shorthand for Geometric distribution with parameter. : Shorthand for Poisson Waiting Time distribution with parameter and threshold. : Shorthand for Poisson distribution with parameter over time interval. : Poisson rate parameter, also average service rate. : Effective average service rate, also written as a function of. : Random variable with exponential distribution. : Random variable with truncated exponential distribution. ii

4 List of Figures List of Figures Figure 1 - Schematic of Pedestrian Crossing Problem. 1 Figure 2 - Monte-Carlo Simulation of Pedestrian Crossing Problem. 9 Figure 3 - Monte-Carlo Histogram vs. Analytical PDF. 10 Figure 4 - Schematic of Disgruntled Teller Problem. 13 Figure 5 - Plot of Effective Average Service Rate. 16 Figure 6 - Plots of select Shaping Functions and their Gaussian Approximations. 19 iii

5 The Poisson Waiting Time Distribution 1 The Poisson Waiting Time Distribution 1.1 The Pedestrian Crossing Problem A pedestrian approaches a road with the intent to safely cross. Cars arrive along the pedestrian's intended path according to a conventional Poisson Process, with rate parameter cars per unit time. Since the arrival of cars on the pedestrian's intended path is a Poisson Process, the time between arrivals of cars is exponentially distributed, with the same rate parameter of cars per unit time. If the time between arrivals of a given pair of cars, or the interarrival time, is large enough, then the pedestrian will safely cross. Suppose the pedestrian will cross in the first interarrival time of or greater. The goal is to characterize the time the pedestrian must wait before crossing safely. To do this, we seek the answer to two questions. First; how long does the pedestrian expect to wait before crossing? Second; what is the probability distribution of this waiting time? 1.2 Formulation of Solution First Principles Since the expected time immediately follows from the probability distribution, the problem is reduced to a single outcome - determining the probability distribution of the Poisson waiting time, or more simply the waiting time. The pedestrian crossing problem is summarized in the schematic diagram below, Figure 1. Cars, arriving at a rate of λ cars per unit time Pedestrian's Path Figure 1 - Schematic of Pedestrian Crossing Problem. From the diagram, the waiting time is described by the random variable, and the interarrival times are described by, the exponentially distributed interarrival times of a conventional Poisson Process. To choose the reference point from which to measure, assume the pedestrian approaches the roadside and begins to wait at exactly. This is a fair assumption due to the memoryless property and mutual independence of the interrarival times. Based on this assumption, the waiting time is equal to the sum of the interarrival times prior to the first of size greater than or equal to the threshold. It may be tempting to 1

6 The Poisson Waiting Time Distribution conclude that the probability distribution of is simply the sum of independent exponential random variables (the Gamma or Erlang distribution), but this is not the case. There are two complications: how many interarrival times are to be summed, and what a posteriori information do we have regarding those interarrival times? Number of Interarrival Times If the pedestrian approaches an empty road, he or she would not have to wait before crossing. In other words, the first interarrival time is at least the threshold time, so the waiting time is identically zero. If the first car arrives in less than but the second is very far away, then the waiting time is equal to the first interarrival time. If two cars pass quickly and the third is far away, then the waiting time is equal to the sum of the first two interarrival times. In general, the sizes of the interarrival times are random but the number of interarrival times before the first of sufficient size is also random. To demonstrate this; the probability that the next interarrival time is smaller than the threshold is given by: (1) The complementary probability threshold is given by: that the next interarrival time is greater than or equal to the (2) Therefore, if counts the number of interarrivals before the first of sufficient size, by the multiplication rule we can write: Hence, follows the geometric distribution including. Specifically, represents the event that the pedestrian can cross immediately, and this event occurs with probability. This raises an apparent paradox; one would expect to be able to assume any nonnegative value in and should therefore have a continuous distribution with an associated PDF and CDF. So;. However, the case demonstrates that. This is a significant result and will be discussed further on. (3) 2

7 The Poisson Waiting Time Distribution Posterior Analysis of Interarrival Times Suppose you are the pedestrian; you have waited a particular amount of time and seen a particular number of cars cross your path when the first sufficiently large interarrival time occurs. At this instant, you look back at the interarrival times before this one; what do you know about those times? By definition, you know those previous interarrival times must have been smaller than ; otherwise, you would have crossed sooner! Instead of being properly exponentially distributed, we know that all of the interarrival times (independent of how many there were) are limited to a domain of. So, by posterior analysis, the interarrival times follow the truncated exponential distribution, given by: The subscript is adopted to describe the shaping done to the PDF. Noticing (1) and employing the unit step function, equation (4) is restated as: Linear Combination Interpretation To formalize an expression for, we could try to express the waiting time as a sum of truncated exponential random variables weighted by the frequency that this result occurs, ], in a linear combination over all. This is effectively a marginalization of over all possible, and can be written as: The additional subscripts denote the interarrival time in the event of such times. The benefit of this notation is that the expectation,, can be easily derived due to the linearity of the expectation operator. Applying this, we can see: (4) (5) (6) This is an elegant formula but not an unexpected one; this is a generic result from the study of compound distributions[1], which will be addressed shortly. Using the convolution formula for sums of random variables, we can rewrite the individual sums of the terms as new random variables, as follows: 3 (7) (8)

8 The Poisson Waiting Time Distribution Where, and, where the superscript denotes the convolution of successive terms. The curious term has also been replaced with using a new random variable, whose PDF corresponds to a value of 0 with probability 1; this is a degenerate distribution, and can be expressed functionally as: Due to the sifting property of the Dirac delta function, other random variables, since: has a consistent definition with the and in general: Equation (9) is herein dubbed the convolution property of developing an explicit form for., and will be useful for (9) Compound Distribution Interpretation By choosing an alternate representation, we can write the form: as a compound distribution of where is the i th interarrival time in a set of interarrival times, though in this case is itself a random number. From the study of compound distributions[1], we can directly write the PDF of as: but this result is exactly consistent with the definition of interpretation of, so we can rewrite the PDF as: from the linear combination (10) Beyond and, if we can determine an explicit expression for, then we have an explicit expression for. 4

9 The Poisson Waiting Time Distribution Analysis of k summed truncated exponential random variables, Rk Explicit forms for already exist, so we motivate our analysis by considering the form of. Using the convolution property (9), we write: After extracting the relevant terms from the integral, what is left is the convolution of a unitheight rectangle of width by an identical rectangle. The well known result of this operation is a triangle, with base and height. Generalizing over repeated convolutions, the PDF of can be written as: In words, is a negative exponential random variable shaped by the function. Hence, the properties of the family of shaping functions,, define the properties of, and therefore the properties Properties of the Shaping Function Family, {Sk} The shaping functions,, are the consequence of truncating the exponential distributions; if we did not truncate the exponential tail of, we would expect the function to collapse into the polynomial factors that characterize the Gamma (Erlang) distribution. We start by defining the first two members by comparison of (11) to the known expressions for and and by presenting the convolution property shared with : (11) This recursively defines all members of the set of shaping functions, though it does not give an explicit formula. Such a formula is possible to achieve for any given, as the result will be a set of piecewise-continuous polynomials, but it is prohibitively time-consuming to do in practice ( is a piece-wise polynomial over 4 intervals, is a piecewise polynomial over 5 intervals, etc., and each is defined through a successive convolution integral). 5

10 The Poisson Waiting Time Distribution To aid with further analysis, additional properties are presented. From the convolution property, the Laplace Transform of is: This transform is valid and, including. For, has compact support: This is verifiable through the Laplace transform or through the convolution formula. The maximum value reached by is: This can be determined by centering in, evaluating the limit, and evaluating the inverse Fourier transform at. Finally, the following inequality holds: This property shows the connection between the Gamma distribution and the shaping functions. Particularly, it can be shown that equality holds over the domain ; granted the trivial case of, consider the convolution formula of over the specified domain: (12) Taking this result and extending it to : So, convolution over this domain exactly corresponds to the anti-derivative of the polynomial term, causing the equality to hold. These terms should be recognized as the polynomial factors that differentiates the Exponential distribution from the Gamma distribution. For times greater than, convolution no longer corresponds to the anti-derivative, so the shaping function begins to tail off and remain less than the specified upper bound. 6

11 The Poisson Waiting Time Distribution The Waiting Time Probability Density Function, ft(t) With the family of shaping functions now well defined, the probability distribution of the Poisson waiting time can be written as: Expanding the term and simplifying the sum, the waiting time PDF is: For brevity, a random variable following this distribution is denoted by. The presence of the term resolves the apparent paradox mentioned previously, since we can write: 1.3 Analysis of Solution Convergence of Summation To perform analysis on equation (13), we must first be sure the solution converges in for any fixed. The sequence that defines the sum,, is:. (13) Using property (12) we can instead consider the sequence such that: Both and are nonnegative for and. Performing the ratio test for fixed, we can see: Therefore, converges because converges Properness of the PDF, ft(t) From its definition, is one-sided, so we can show it is a proper PDF if we can demonstrate: 7

12 The Poisson Waiting Time Distribution Inserting equation (13), we have: Simplifying: The remaining integral is intractable in the time domain, so we should attempt to solve an auxiliary problem. Consider such that: Next, consider the Laplace Transform pair for, or: Using the properties of the Laplace Transform, we can write: Applying the final value theorem, we can write: Hence: So: Therefore, is a proper PDF. Furthermore, the last line demonstrates the necessity of the of term; that is, the delta function at the origin. If it were not included, the integral would evaluate to strictly, and hence would not be a proper PDF. 8

13 The Poisson Waiting Time Distribution Flatness over the Interval (0,t0] A curious property of is its uniformity or flatness over the interval (the origin is omitted only because of the delta function, but at the PDF is flat plus delta). To demonstrate this, we can use the equality in property (12) to write: The sum is now recognizable as the Taylor series of the exponential function, so: (14) Equation (14) now summarizes the two most remarkable features of the graph of delta function at the origin and the flatness over. ; the 1.4 Comparison with Simulated Results To verify the formula for, we can perform a Monte-Carlo simulation of the Pedestrian Crossing Problem and compare the resulting waiting times in a histogram against the analytical PDF. Consider the following computer program, Figure 2. START Supply (λ, t 0) Experiment Generate vector X of independent exponential R.V. realizations, R.V. ~ Exp(λ) Histogram Bins := hist(t, Nbins = 100); Search Find index of first element t 0 i := (index); Rescale Rescale Bins to approximate a PDF, such that Bins numerically integrates to 1. Append t := sum(x 1,..., X i-1); OR t := 0 IF i = 1; T.append(t); Mean Tbar := mean(t); NO size(t) = 30000? YES FINISH Return (Tbar, Bins) Figure 2 - Monte-Carlo Simulation of Pedestrian Crossing Problem. 9

14 The Poisson Waiting Time Distribution In this simulation, the vector contains realizations of the random variable. By implementing this program in MATLAB, we can simulate the problem and compare with the analytical solution for arbitrary choices of and (in this case, ). Apparent convergence is reached by evaluating 600 terms of the sum in the analytical expression. Figure 3 provides the results of the Monte-Carlo simulation: Histogram Analytical PDF Delta Function 0.25 f T (t) X: 3 Y: t Figure 3 - Monte-Carlo Histogram vs. Analytical PDF. By inspection we see the analytical result matches the Monte-Carlo histogram, although the histogram poorly represents the Dirac Delta function (since the first bin includes the probability mass from instead of simply ). Additionally, the simulated average of is: While the analytical mean of from equation (7) is: So there is agreement in the mean up to one decimal place. This is a surprising result; even with a sample size of the simulated mean has a significant amount of variability over repeated runs of the simulation. If fact, if we wanted to analyze the probability of achieving this accuracy (i.e.: ) we could invoke the weak law of large numbers (WLLN). Doing this, we can write: 10

15 The Poisson Waiting Time Distribution The variance[1] of can be written as: (15) Though closed form, equation (15) is not a simple formula to write down, so the variance (using the stated values for and ) is computed to be: From this we can write: so the desired complementary probability is: which means the probability of receiving accuracy in the mean only up to one decimal place could be as low as ~92.5%. To obtain the same worst-case probability in the WLLN sense for accuracy to two decimal places, we would need a sample size of about 3 million. This result corroborates a statement from [1] that describes simulation as a common (and perhaps necessary) technique for working with compound distributions but as"... often ineffective and requires a great capacity of computing power" (2, para. 5). It is simple to demonstrate, through the use of timestamps, that the majority of computation time spent producing the results in Figure 3 was devoted to generating samples of ; the PDF, while itself requiring the approximation of an infinite sum and the approximation of convolution integrals, finishes in less time with more accurate results. Pilot applications (such as the one explored in the next section) that require reliable statistics from the waiting time distribution may be impossible to simulate in reasonable time but can be determined from the analytical results of equation (13). 11

16 Pilot Application in Queuing Theory 2 Pilot Application in Queuing Theory 2.1 Effective Average Service Rate in a Poisson Server Consider a conventional Poisson server process where an uninterrupted stream of customers are served at an average service rate of customers per unit time; the number of customers served over a period of time is denoted, where. If we define an arbitrary length of time,, and wish to consider the server's effective average service rate,, we could define such a quantity as: For a conventional Poisson process,, so this ratio becomes: As expected, the effective average service rate is identically equal to the average service rate. Next, suppose we choose a random epoch duration, independent of the underlying Poisson process. For consistency, we can generalize the definition of to: The generalized formula (16) expresses the effective average service rate over the ensemble of possible epoch times. The conditional expectation is justified since the expectation is computed a posteriori; the effective average service rate is computed after counting for a duration of, then averaged over the ensemble of all possible. Also notice that formula (16) is consistent with the conventional Poisson case. 2.2 The Disgruntled Bank Teller Problem Suppose the teller at a bank serves customers in a process that operates as a conventional Poisson process; customers arrive immediately and are served at an average service rate of customers per unit time. Now suppose the teller is very impatient; if a customer arrives with a request that takes longer to process than a critical threshold time, the teller will stop processing the request and yell at the customer. After a short delay, the disgruntled teller is replaced and the next customer begins to be serviced (the insulted customer leaves in disgust). Curiously, the replacement teller is just as impatient, and will drop the request of (and yell at) the next customer whose request exceeds a time of. (16) 12

17 Pilot Application in Queuing Theory The bank manager fears that scaring away customers is hurting the bank's efficiency (i.e.: effective service rate). Is the bank manager's fear necessarily correct? In more general terms, is it possible (in the presence of a delayed restart) to increase the effective average service rate in a Poisson server by systematically dropping long requests? If so, is there an optimum threshold beyond which requests should be dropped? First Principles The scenario of the disgruntled teller is described in the schematic diagram presented below, Figure 4. t = 0 Dropped Customer's projected service time Served Customer Dropped Customer Delay Period Next Served Customer t Figure 4 - Schematic of Disgruntled Teller Problem. Next Epoch Since the customer service times are assumed to be i.i.d., we can define an epoch to start at the arrival of the first customer and proceed until the end of the first delay period, and determine the effective average service rate of the whole process as the effective average service rate of the ensemble of possible epochs (as we did for the conventional Poisson process). The number of customers successfully served in one epoch is equivalent to the number of service times before the first one exceeding. Applying equation (3), this number is denoted,. The time,, is a random variable consisting of three terms: Where, and and are positive real constants. From formula (16), we can write down the effective service rate as: Determining an explicit form for equation (17) requires examination of the expression. In the conventional Poisson case, this is trivial because the number is essentially independent of how the time interval is chosen. Unlike the conventional Poisson (17) 13

18 Pilot Application in Queuing Theory case, the compound distribution of demonstrates interdependence of and, so the conditioning event cannot be so easily dropped. Consider the auxiliary problem: (18) Using Bayes' Rule, we can write: Rearranging and inserting the known, we have: The denominator term is recognized the CDF of evaluated between and. The numerator term is generally intractable, but in this case we notice the condition states there are exactly k interarrivals that comprise, and this exactly consistent with the definition of given in equation (8). So, we write: In the limit as, the condition on the LHS becomes via squeeze theorem, while the RHS takes the form of a parametric derivative. From the basic properties of differentiation, we have: so the equation can be written in the limit as: This means equation (18) can be rewritten in the limit as: If we consider strictly greater than 0 to remove the delta function from the denominator, inserting the expressions for both and produces the solution to equation (18): 14 (19)

19 Pilot Application in Queuing Theory For compactness, let the expectation of be written as. Combining equations (17) and (19), we have: and simplifying: (20) Equation (20) is now directly comparable to the effective average service rate in the conventional Poisson server case: if the ratio is ever greater than unity then the bank manager's fear is incorrect and the disgruntled tellers are actually increasing the effective average service rate by selectively dismissing long requests Existence of an optimal t0 Examining equation (20), we see that the effective average service rate is a function of three variables;,, and. Using the maximum value property of mentioned in Section 1.2.7, it can be shown that equation (20) is adequately a function of two variables; and for a fixed parameter. By also fixing, we are left with a function of only over which to search for an optimal. The optimality problem for this case is described as follows: does a exist such that is maximized over all possible? In the classical optimality problem, we would try to solve, but this is not a safe approach because the family } has issues with both continuity and differentiability; is discontinuous at, and is not differentiable at or. Therefore, the most effective way to proceed may be to choose values for and and demonstrate that can be maximized for that particular case. Consider the case where (this can be done without a loss of generality in (20) if and defined relative to ), and that ( is twice the length of the average service time in the equivalent conventional Poisson process). This reduces equation (20) to: By avoiding the direct use of differentiation to argue the existence of an optimal, we are left using an argument from continuity and the mean value theorem. Hence, we must determine the interval(s) of where is at least continuous. Inspecting the range of the summations, the only discontinuous member of we are concerned with is, (21) 15

20 Pilot Application in Queuing Theory which is known to be discontinuous at. Since the summation deals with, the point of discontinuity,, is a fixed point of, or: Two solutions exist, but one is trivial at numerically, and is computed to be:. The non-trivial solution can only be found By evaluating the Taylor series of the equation near its two solutions the following intervals can be determined: By extension, the intervals of continuity of are: Over the first interval, the equality of property (12) holds, and (21) can be reduced to: (22) which is always less than unity within the valid interval of. For the remaining interval, analysis becomes prohibitively difficult, so we switch to a numerical approach. Evaluating equation (21), the following graph is generated, Figure (t ) X: 1.26 Y: X: 1.25 Y: t Figure 5 - Plot of Effective Average Service Rate. 16

21 Pilot Application in Queuing Theory The portion of the graph circled in red indicates a breakdown of the numerical solution; for values of beyond this point the graph dies linearly towards 0. The inability to reliably compute beyond this point is due to two confounding factors. First, the functions are being directly computed using convolution (at great computational cost) for accuracy. Second, has exponential-order growth in ; since only a finite number of 's can be computed in realistic time and each is supported over, there are fewer and fewer values of that satisfy as grows, so the solution will eventually become unstable. Since Figure 5 shows there are values of such that, the effective average service rate can be increased beyond, at least in this particular case. All that remains to be shown is the existence of an optimal, where reaches its global maximum. Given continuity over the interval and having demonstrated that there is a such that ; if we can show that has a horizontal asymptote, then we can conclude a global maximum (and therefore a corresponding optimal ) exists. Using property (12) and the reduction from equation (22), we can write: However, property (12) ensures that equality holds over the interval, which becomes when written in the limit. This is now a superset of the domain of, so we can rewrite the limit as a strict equality, or: which is the desired horizontal asymptote. Given this asymptote, the existence of a such that, and the continuity for, we can conclude via the mean value theorem that a global maximum of must exist inside the interval. So, while we cannot say that this is true for all choices of and, we can conclude that an optimal does exist for some choices of and, and that similar approach can be performed for any particular case that may arise. 17

22 Remarks 3 Remarks The CDF,, of the waiting time distribution is not easily expressed beyond ; although on the interval, the CDF is simply the function of a line,, owing to the flatness property. The most promising approach appears to be finding the CDF of : if we make a clever choice of indicator function, we have: The RHS can now be rewritten as: This expresses the CDF of in terms of the Laplace Transform of, which is known in explicit terms of elementary functions. However the term does not yet possess a convenient Laplace Transform pair. There may be some approach that exploits the symmetry of about its midpoint,, that can be used to complete the expression. The compound distribution approach using a geometrically distributed counting variable and truncated PDF may be applicable to the more general Renewal Process; that is, a counting process similar to the Poisson Process where the interarrival times (or in the parlance of Renewal Theory, the holding times) are not necessarily exponentially distributed. The procedure would be similar; find and, determine the truncated PDF, and write down an expression for that PDF convolved with itself an arbitrary number of times. The difficulty may present itself in this last step, since the convolution integral is generally hard to deal with (though in this case, properties of the exponential function greatly reduced the complexity of the integral). The WLLN was employed in Section 1.4 to place a worst-case probability bound on the numerical accuracy of simulated waiting-time statistics. A better bound may be possible; since the Laplace Transform of is well defined and the function is one-sided in time, an exponential bound can be written down in terms of the Moment Generating Function of. Finding the optimal shape parameter to make this a Chernoff bound, however, would likely involve the sum of a complicated alternating series. 18

23 Remarks Finally, to aid numerical computation of, convolution may be replaceable with a Gaussian Approximation to ; in an argument nearly identical to the Central Limit Theorem, one can say that successive members of the family are more Gaussian than previous ones (in fact, the sum of fair dice rolls follows a progression that is practically identical to and is certainly Gaussian in the limit). By starting with the ansatz: and choosing such that the squared error in area between and is minimized, the Gaussian approximation can be written down as: One notable issue with this approximation is its lack of compact support. To further correct the approximation; we could truncate, vertically shift, and rescale to artificially impose compact support, but the shaping parameter would no longer be valid. The advantage of the approximation is that it can be computed directly using elementary functions for any ; even though contains, this expression has a closed (albeit long) form that can be computed much faster than a convolution integral or approximation thereof. The disadvantage is the weak argument used to derive the approximation and the lack of understanding of how the approximation converges in. Nevertheless, for and values of as small as 4, is a good approximation to. The and cases are presented as evidence in the graphs below, Figure 6. 6 S k (t) 70 S k (t) 5 G k (t) 60 G k (t) k = 4 3 k= t t Figure 6 - Plots of select Shaping Functions and their Gaussian Approximations. 19

24 References 4 References [1] Lin, X. S. (2006). Compound Distributions. Encyclopedia of Actuarial Science. Retrieved from 20