We automate the bivariate change-of-variables technique for bivariate continuous random variables with

Transcription

1 INFORMS Jornal on Compting Vol. 4, No., Winter 0, pp. 9 ISSN (print) ISSN (online) 0 INFORMS Atomating Biariate Transformations Jeff X. Yang, John H. Drew, Lawrence M. Leemis Department of Mathematics, College of William & Mary, Williamsbrg, Virginia 387 {jxyang@gmail.com, jhdrew@math.wm.ed, leemis@math.wm.ed} We atomate the biariate change-of-ariables techniqe for biariate continos random ariables with arbitrary distribtions. This extends the algorithm for niariate change-of-ariables deised by Glen et al. [Glen, A. G., L. M. Leemis, J. H. Drew A generalized niariate change-of-ariable transformation techniqe. INFORMS J. Compt. 9(3) 88 95]. Or transformation procedre handles one to one, k to one, and piecewise k to one transformations for both independent and dependent random ariables. We also present other procedres that operate on biariate random ariables (e.g., calclating correlation and marginal distribtions). Key words: comptational probability; compter algebra systems; continos random ariables; transformation of random ariables History: Accepted by Winfried Grassmann, Area Editor for Comptational Probability and Analysis; receied March 009; reised Noember 009, Jly 00; accepted Jly 00. Pblished online in Articles in Adance December, 00.. Introdction Probability theory addresses a ariety of problems that are both tedios and impractical to work by hand. The adent of compter algebra systems, sch as Maple and Mathematica, has facilitated the deelopment of probability packages. These packages are implementations of algorithms that can qickly perform the reqired tedios calclations. One sch package, A Probability Programming Langage (APPL), coded in Maple, performs standard probability operations on niariate random ariables (Glen et al. 00). Many problems that wold hae taken hors to sole by hand can be qickly compted in APPL ia a few lines of code. Consider the following example. Let X X X 0 be independent and identically distribted (iid) U0 random ariables. Find ( P 4 < 0 i= ) X i < 6 The distribtion of the sm of 0 iid U0 random ariables can be calclated by hand, bt the process wold be both tedios and time consming. APPL soles this problem with the commands > X := UniformRV(0, ); > Y := ConoltionIID(X, 0); > CDF(Y, 6) - CDF(Y, 4); which yield the exact soltion Althogh the soltion to this problem can be approximated by the central limit theorem or Monte Carlo simlation, APPL comptes the exact soltion in jst three lines of code. APPL contains a niariate transformation of ariables procedre that takes the probability density fnction (PDF) of a continos random ariable X and the transformation Y = gx as inpt and retrns the PDF of Y as otpt. APPL crrently lacks procedres to handle biariate distribtions. The most releant software we fond to atomate biariate transformations is Mathematica s statistical package, mathstatica (Rose and Smith 00). Their examples appear to be limited to one to one transformations and independent random ariables X and Y. We present an addendm to APPL that handles one to one transformations, k to one transformations, and piecewise k to one transformations for both dependent and independent continos random ariables X and Y. The biariate transformation techniqe calclates the distribtion of U = gx Y from the joint distribtion of two continos random ariables X and Y with joint PDF f X Y x y defined on the spport set in the xy-plane. In the simplest case, an axiliary dmmy fnction V = hx Y can be fond so that the two fnctions U = gx Y and V = hx Y define a one to one transformation from the set in the xyplane onto a set B in the -plane. Denote the inerse transformation from B to by X = ru V and Y = su V. The formla (Hogg et al. 005) f U V = f X Y r s J gies the joint PDF of U and V, where dx dx J = d d dy dy d d for B

2 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS is the Jacobian. The marginal PDF of U is then calclated by integrating ot of the joint PDF f U = f U V d = f U V d for where = B and is the spport of U. Next, we present a biariate transformation algorithm sed to handle sch one to one transformations, and, more generally, k to one transformations and piecewise k to one transformations for the joint continos random ariables X and Y. The algorithm is modeled after the theorem by Glen et al. (997) for niariate transformations.. Algorithm The prpose of the algorithm is to find the PDF of U = gx Y, where X and Y are biariate continos random ariables with joint PDF f X Y x y. Let the spport of X and Y in the xy-plane be denoted by, which is partitioned into a finite nmber of disjoint components. Denote the ith component of by i and the associated PDF on i by f i x y for i = n. The prpose of this partitioning might be to accommodate a distribtion that is defined piecewise, a transformation that is defined piecewise, a transformation that is many-to-one, or any combination of these sitations. Becase the i might inclde some of their bondary points on which the transformation cold be badly behaed, we consider the interior of each i, denoted by i, which is an open set. Let U V = wx Y = gx Y hx Y be a fnction from to whose domain incldes n i= i. The following assmptions mst hold:. The PDF f i x y is continos for all x y i for i = n.. The fnction w is a one to one transformation from i onto a set B i in the -plane, for i = n. Denoting the inerse of this transformation from i onto B i by X Y = wi U V = r i U V s i U V, we reqire that the Jacobian r i r i J i = s i s i is nonanishing on B i for i = n and that the partial deriaties are eerywhere defined and continos. Becase the B i s are not necessarily pairwise disjoint, we find the contribtion to the joint PDF of U and V oer the component B i that reslts from i, which is (Hogg et al. 005) f i U V = f i r i s i J i for B i for i = n. Then, f U V = f i U V, where the sm is taken oer all those is for which B i. To calclate the marginal PDF f U, we define the following additional notation. For those ales of that can occr for points in B i, denote the contribtion of the component i to the marginal PDF of U by f Ui = f i U V d = f i U V d i where i = B i. To atomate the integration oer i, we hae made the additional assmption that, for each ale of realizable in the interior of B i, i is a single line segment (that is, each ertical line segment spanning B i is entirely contained in B i ) and that the end points of the closre of i are determined by two different constraints. For some transformations defined on, these assmptions can only be satisfied by a jdicios partitioning of. Becase the bondary of B i typically consists of many distinct constraint cres, and the limits of integration for depend on which constraint cres are releant for a particlar ale of, the fnction f Ui is defined in a piecewise fashion with m i pieces, where m i is a positie integer. Then proceed as follows to find the PDF of U. For j = m i, denote the jth piece of f Ui by f Uij, which is defined on an interal denoted by ij < < ij+. Let U = n i= ij j = m i +. Order the elements of U withot repeats and relabel them sing the notation i so that < < < l+, where l = U, and denotes cardinality. Let I k = i j ij k and k+ ij+ for k = l. Then for k k+ the PDF of U is gien by for k = l. f U = i j I k f Uij 3. Data Strctre To implement the algorithm, we will se a data strctre for the distribtion of the biariate random ariable that expands on the list-of-sblists format sed in APPL. The distribtion of the biariate random ariable is presented in a list-of-three-sblists format. The first sblist contains the ordered two-ariable PDF expressions f i x y of the distribtion, corresponding to the components i into which the spport of X and Y has been partitioned. The second sblist describes

3 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS 3 each component of i by sing a list of constraints for that component. The third sblist contains the strings "Continos" and "PDF" to specify the type of distribtion and meaning of the fnction in the first sblist. For proper implementation of or algorithms, each component described in the second sblist mst be a simply connected set (i.e., contains no holes) whose bondary can be formed by consectie segments of cres that satisfy eqations of the form px y = 0. In addition, the constraints listed in the second sblist for each component mst satisfy the following conditions: The constraints mst be entered in adjacent order; clockwise or conterclockwise is acceptable. The constraints mst completely enclose a region. The constraints mst be entered as strict ineqalities. Except for constraints of the form x < a or x > a, each constraint, when written as an eqality, mst pass the ertical line test; i.e., only a single ale of y corresponds to each ale of x. For example, se y < x or y > x, rather than x + y <. If the component is nbonded, a less than or greater than constraint mst be inclded. See Online Spplement A (aailable at for frther discssion of this data strctre and for examples illstrating its se. 4. Implementation The biariate transformation procedre BiTransform(Z, g, h) takes p to three argments as inpt. The first two argments are reqired and the third is optional. The first reqired argment is the joint distribtion of X and Y, which is gien in the listof-three-sblists format. If U = gx Y is a k to one transformation or defined in a piecewise fashion, or both, then the ser mst partition the spport of X and Y sch that gx Y is one to one and is not defined in a piecewise fashion on each component of the partition. The second reqired argment is the transformation of interest U = gx Y, gien in the Maple fnction format [(x, y) g(x, y)]. If only one transformation is proided, then g(x, y) is applied to all components of the spport of X and Y. Otherwise, the ser mst spply a nmber of transformations eqal to the nmber of components, where the first transformation corresponds to the first component, the second transformation corresponds to the second component, and so on. The third optional argment is the transformation V = hx Y, which is described sing the same Maple fnction format and conentions sed when describing U. The defalt transformation is the dmmy transformation V = hx Y = Y for all ales in. We gie detail in this paragraph concerning the atomatic determination of the inerse. Each component i of the spport of X and Y is associated with a corresponding transformation U = g i X Y and V = h i X Y for i = n. The algorithm adapts to the case in which the inerse transformation X Y = r i U V s i U V fond by the Maple sole procedre retrns mltiple soltions, and the correct inerse mst be chosen. For example, the two to one fnction U V = X Y yields inerses X Y = U V and X Y = U V. The BiTransform procedre finds the correct inerse by selecting the inerse that satisfies x i = r i g i x i y i h i x i y i and y i = s i g i x i y i h i x i y i where x i y i i. An algorithm for determining sch a point x i y i in i follows. Let W i denote the set of x-ales for all intersections of adjacent constraint eqalities that define i. Let t i be the minimm of W i, and let T i be the maximm of W i. If W i contains more than two elements, let ˆt i be the second-smallest element of W i, and let T i be the second-largest element of W i.. If t i = and T i =, then x i = 0 and y i is the aerage of the y-ales for the two constraint eqalities associated with x i = 0 that maximize the difference of the y-ales.. If t i = and T i is finite, then x i = T i + T i / and y i is the aerage of the two y-ales that correspond to x i on the two constraint eqalities associated with x i. 3. If t i is finite, then x i = t i + ˆt i / and y i is the aerage of the two y-ales that correspond to x i on the two constraint eqalities associated with x i. Each constraint that defines i is an ineqality of the form px y < 0, where p is a real-aled continos fnction. The corresponding constraint for B i is fond by sbstitting the appropriate inerse transformations determined by the algorithm described in the preios paragraph to get pr i s i < 0. One difficlty arises when the nmber of constraints for i is greater than the nmber of constraints for B i. In other words, at least one of the corresponding constraints for B i has been deleted as redndant. The following example illstrates this problem. Consider the biariate random ariables X and Y with spport on the open nit sqare 0 < x <, 0 < y <. The transformation U = XY with dmmy transformation V = Y is a one to one transformation from to B (shown in Figre where the bondaries of and B are not a part of the spport) with inerse X = U /V, Y = V. Sbstitting X = U /V and Y = V into the linear constraints x > 0, y > 0, x <, y < gies the linear constraints > 0, > 0, <, <. The second constraint, > 0, is redndant and

4 4 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS.0 y B Figre Transformation from to B Gien by U V = XY Y Note. The constraint eqality y = 0 of is mapped to a point on the bondary of B. shold not be inclded in the description of the spport of UV. The spport of X and Y is bonded by for linear constraints whose corresponding eqalities intersect at the adjacent points 0 0, 0,, 0. These points on the bondary of are mapped to 0 0, 0 0,, 0, respectiely, on the bondary of B. The BiTransform procedre checks for eqality between adjacent intersections for B and deletes redndant constraints. In this case, the first and second intersections for map to the same point, 0 0, for B, so > 0 is deleted. There are cases where Maple fnctions do not operate as desired. The most troblesome of these fnctions is sole. In particlar, the inpt sole({x = a, y = infinity}) retrns NULL for a = 0 bt retrns {x = a, y =} for all a 0. The sole procedre is sed primarily to find the intersection points between adjacent constraints associated with the spport. Becase joint distribtions constrained to the first qadrant are fairly common, if statements in the code handle the a = 0 case. Howeer, it is impossible to add conditional statements for all problems of this type (e.g., sole({x = y y, y = infinity}) also retrns NULL). Ths, the capability of or transformation procedre, BiTransform, is limited in handling distribtions with infinite spport. The scope of problems that can be soled by BiTransform is limited to those whose adjacent constraint eqalities can be soled analytically by the sole fnction. 5. Examples This section illstrates the se of the biariate transformation procedre BiTransform for a ariety of distribtions and transformations. Example illstrates the se of BiTransform on independent random ariables. Example illstrates transforming dependent x random ariables. Example 3 illstrates piecewise transformations. Example 4 illstrates k to one transformations. Example 5 illstrates piecewise k to one transformations. Example 6 illstrates a method for working arond a spport that is not a simply connected set. Example (Independent X and Y ). The joint PDF of the independent continos random ariables X U0 and Y U0 is f X Y x y = f X xf Y y = 0 < x < 0 < y < The distribtion of U = XY is fond sing the transformation techniqe as follows: = gx y = xy x = r = / = hx y = y y = s = / / J = 0 = / f U V = = 0 < < f U = d = ln = ln 0 < < The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > XY := [[(x, y) -> ], [[y > 0, x > 0, y <, x < ]], ["Continos", "PDF"]]; > g := [(x, y) -> x y]; > h := [(x, y) -> y]; which retrn ln 0 Continos, PDF

5 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS 5 as the PDF of U as expected. Leaing ot the third line of code and the third argment in BiTransform reslts in identical otpt. Example (Dependent X and Y ). The joint PDF of dependent continos random ariables X and Y is f X Y x y = x > 0 y > 0 x + y < The distribtion of U = X + Y is fond sing the transformation techniqe as follows: = gx y = x + y = hx y = x y J = f U V = = f U = x = r = + y = s = = 0 < < < < d = 0 < < The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > XY := [[(x, y) -> ], [[x > 0, y > 0, x + y < ]], ["Continos", "PDF"]]; > g := [(x, y) -> x + y]; > h := [(x, y) -> x - y]; which correctly retrn 0 Continos, PDF as the PDF of U. Example 3 (Piecewise Transformation). The joint PDF of the independent continos random ariables X U0 and Y U0 is f X Y x y = f X xf Y y =.0 y 0 < x < 0 < y < Let U be a piecewise transformation with U = X + Y for 0 < x < /, 0 < y <, and U = XY for / < x <, 0 < y <. The transformation is illstrated in Figre, sing V = X Y on 0 < X < /, 0 < Y < and V = Y on / < X <, 0 < Y <. For with 0 < x < /, 0 < y < = g x y = x + y x = r = + = h x y = x y y = s = J = = f U V = =/ 0<+ < 0< < / d 0 < < / f U = / d / < <.0 B / d < < 3/ 0 < < / = / / < < 3 / < < 3/ For with = g x y = xy, = h x y = y we hae J = / from Example and f U V = = / < < 0 < < / d = ln 0 < < / f U = / d = ln / < < B.0 x Figre Piecewise Transformation from to B and to B

6 6 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS Using the algorithm for biariate transformations, + ln 0 < < / f U = f U + f U = / ln / < < 3 / < < 3/ The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > XY := [[(x, y) ->, (x, y) -> ], [[y > 0, x < 0, y <, x < /], [y > 0, x > /, y <, x < ]], ["Continos", "PDF"]]; > g := [(x, y) -> x + y, (x, y)-> x y]; > h := [(x, y) -> x - y, (x, y)-> y]; which retrn the correct PDF of U in the APPL listof-sblists format as + ln ln + / + 3/ 0 / 3/ Continos, PDF Example 4 (Two to One Transformation). The joint PDF of the independent continos random ariables X and Y is f X Y x y = /4 < x < < y < The distribtion of U nder the two to one transformation U = X Y and V = Y is fond sing the algorithm described in. Partition the spport of X and Y sch that the transformation is one to one in each component. Let be the component bond by < x < 0 < y < ; and let be the component bond by 0 < x < < y <. Both components map onto B nder the transformation U = X Y and V = Y, as illstrated in Figre 3. y For, = g x y = x y.0 x = r = + / = h x y = y y = s = + / + / J = 0 = + / For, f U V = 4 + / = 8 + / f U = < < < < + 8 / d < < / d 0 < < + 8 / d < < + /4 < < 0 = /4 0 < < /4 + /4 < < = g x y = x y x = r = + / = h x y = y y = s = J = + / + / 0 = + / f U V = 4 + / = 8 + / f U = < < < < + 8 / d < < / d 0 < < + 8 / d < < 0 B 0 x Figre 3 Two to One Transformation from = to B

7 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS 7 + /4 < < 0 = /4 0 < < /4 + /4 < < Using the algorithm for biariate transformations, f U = f U + f U + / < < 0 = / 0 < < / + / < < The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > XY := [[(x, y) -> /4, (x, y) -> /4], [[y > -, x > -, y <, x < 0], [y > -, x > 0, y <, x < ]], ["Continos", "PDF"]]; > g := [(x, y) -> x - y]; > h := [(x, y) -> y]; which retrn the correct PDF of U in the APPL listof-sblists format as + / / / + / 0 Continos PDF Example 5 (Piecewise Two to One Transformation). The joint PDF of independent continos random ariables X and Y is f X Y x y = /6 < x < < y < The transformation U = X Y and V = Y is two to one on < x <, < y < and one to one on < x <, < y <. Partition the spport of X and Y sch that the transformation U = X Y and.0 y V = Y is one to one in each component. Let be the component bond by < x < 0 < y < ; and let be the component bond by 0 < x < < y <, as illstrated in Figre 4. The marginal PDF of U is.0 + /3 + /6 + /6 f U = /3 + /6 + + /6 /6 + + /6 /6 + /3 B B < < 0 0 < < < < < < 3 3 < < 5 The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > XY := [[(x, y) -> /6, (x, y) -> /6], [[y > -, x > -, y <, x < 0], [y > -, x > 0, y <, x < ]], ["Continos", "PDF"]]; > g := [(x, y) -> x - y, (x, y) -> x - y]; > h := [(x, y) -> y, (x, y) -> y]; Note that g and h each contain two fnctions instead of one as in the preios example. Declaring the same transformation for each component achiees the same reslt as declaring the transformation once. The correct PDF of U is retrned in the APPL list-of-sblists format as + /3 + /6 + /6 /3 + /6 + + /6 /6 + + /6 /6 + / Continos PDF x Figre 4 Piecewise Two to One Transformation Note. The region maps onto B B, whereas only maps to B.

8 8 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS Example 6 (Nonsimply Connected Spport for U and V ). Let X and Y be niformly distribted on with transformations U = X cos Y The joint density of U and V is f U V = + V = X sin Y < + < 4 and the marginal PDF of U is sinh 4 < < f U = sinh 4 sinh < The commands to assign data strctres and apply the biariate transformation techniqe are as follows: > X := UniformRV(, ); > Y := UniformRV(-Pi, Pi); > XY := JointPDF(X, Y); > g := [(x, y) -> x cos(y)]; > h := [(x, y) -> x sin(y)]; > BiTransform(XY, g, h); which displays the joint density of U V as [[ ] / + [[ < + + < ( arctan + ) ]] < + ] Continos, PDF The marginal PDF is calclated by manally diiding the spport of U V into six regions sing =, = 0, and =, as illstrated in Figre 5. The Maple commands > UV := [[(, ) -> / ( Pi sqrt( + )), (, ) -> / ( Pi sqrt( + )), (, ) -> / ( Pi sqrt( + )), (, ) -> / ( Pi sqrt( + )), (, ) -> / ( Pi sqrt( + )), (, ) -> / ( Pi sqrt( + ))], [[ < sqrt(4 - ), < -, > 0], [ < sqrt(4 - ), > -, > sqrt( - ), < ], [ < sqrt(4 - ), >, > 0], [ > -sqrt(4 - ), < -, < 0], [ > -sqrt(4 - ), > -, < -sqrt( - ), < ], [ > -sqrt(4 - ), >, < 0]], ["Continos", "PDF"]]; > MarginalPDF(UV, ); Figre 5 0 correctly retrn 0 Spport of U and V [[ /ln +4++/ln +4+ /ln ++ /ln +4+ +/ln +4+ /ln ++ /ln +4++/ln ] +4+ ] Continos, PDF as the marginal PDF of U. [Recall that sinh x = lnx + x +.] 6. Conclsions and Frther Research Calclating the distribtion of a transformation of two random ariables by hand can be tedios, especially when working with piecewise joint distribtions or piecewise transformations. An atomated method for compting biariate transformations saes the ser time and preents calclation errors. The algorithm and BiTransform procedre we deeloped allows for one to one, k to one, and piecewise k to one transformations for both independent and dependent continos random ariables. In the ftre, we hope to see compter-based algorithms for biariate discrete transformations, as well as mltiariate transformations inoling three or more random ariables. There are nmeros applications of BiTransform in stochastic operations research (one from qeing theory is gien in Online Spplement F). The algorithm described in this paper has been implemented in approximately,300 lines of code and is aailable at leemis/bivarappl.txt. The APPL code can be downloaded at

9 INFORMS Jornal on Compting 4(), pp. 9, 0 INFORMS 9 Electronic Companion An electronic companion to this paper is aailable as part of the online ersion that can be fond at Acknowledgments The athors thank Area Editor Winfried Grassmann, an associate editor, and two referees for their helpfl comments and for sggesting two examples that are inclded in the paper. References Glen, A. G., D. L. Eans, L. M. Leemis. 00. APPL: A probability programming langage. Amer. Statistician 55() Glen, A. G., L. M. Leemis, J. H. Drew A generalized niariate change-of-ariable transformation techniqe. INFORMS J. Compt. 9(3) Hogg, R. V., J. W. McKean, A. T. Craig Introdction to Mathematical Statistics, 6th ed. Prentice Hall, Upper Saddle Rier, NJ. Rose, C., M. D. Smith. 00. Mathematical Statistics with Mathematica. Springer-Verlag, New York.