12 TRANSFORMING BIVARIATE DENSITY FUNCTIONS

Transcription

1 1 TRANSFORMING BIVARIATE DENSITY FUNCTIONS Hving seen how to trnsform the probbility density functions ssocited with single rndom vrible, the next logicl step is to see how to trnsform bivrite probbility density functions. Integrtion with two Independent Vribles Consider f(x 1,x ), function of two independent vribles. Using crtesin coordintes, f(x 1,x ) might be represented by surfce bove the x 1 x plne; the vlue of f(x 1,x ) t ny point (x 1,x ) corresponds to the height bove the point. With two independent vribles, integrtion is normlly expressed with double-integrl sign nd integrtion is over some rnge R, specified re in the x 1 x plne: f(x 1,x ) dx 1 dx R Such n integrl represents volume. If R is circle then this integrl corresponds to the volume of cylinder stnding on R whose upper end is cut by the surfce f(x 1,x ). The following integrl gives the volume of cone whose height is h nd whose bse is circle of rdius centred on the origin (,), this circle being the region R: ( ) x h x dx 1 dx R Note tht t the centre of the circle x 1 + x = nd the vlue of the integrnd is h. At the edge of the circle x 1 + x = nd the vlue of the integrnd is. In principle, two integrtions re crried out in turn: [ ( ] x x 4 h x )dx 1 dx (1.1) In this rerrngement, integrtion is over one qudrnt of the circle nd the result is multiplied by 4. For given vlue of x integrtion is long strip one end of which is t x 1 = nd the other end of which is t x 1 = x. This ccounts for the limits on the inner integrtion. Alredy, seemingly simple exmple of integrtion with two independent vribles is beginning to become uncomfortbly hrd! There re better wys of determining the volume of cone but by judicious substitution of both independent vribles even the present pproch cn be gretly simplified. Integrtion by Substitution of two new Vribles The generl formul for integrtion by substitution of new vrible ws given s (11.1): b f(x) dx = y(b) y() f ( x(y) ) dx dy dy The trnsformtion function is y(x) nd its inverse is x(y). 1.1

2 The equivlent formul when there re two independent vribles is: f(x 1,x ) dx 1 dx = f ( x 1 (y 1,y ),x (y 1,y ) ) R x R y (y 1,y ) dy 1 dy (1.) There re two trnsformtion functions, y 1 (x 1,x ) nd y (x 1,x ), nd their inverses re x 1 (y 1,y ) nd x 1 (y 1,y ). The regions R x nd R y re identicl subject to the first being specified in the x 1 x plne nd the second being specified in the y 1 y plne. The item (x 1,x ) is clled Jcobin nd is defined s: (y 1,y ) (y 1,y ) = x 1 x 1 = x 1 x 1 To simplify (1.1) bove, use the trnsformtion functions: y 1 = x 1 + ( x y = tn 1 x ) x 1 nd their inverses x 1 = y 1 cos y x = y 1 siny Note tht: (y 1,y ) = cos y y 1 siny sin y y 1 cos y = y 1 Using (1.), the integrtion in (1.1) becomes: 4 π [ ( h 1 y 1 )y 1 dy 1 ] This is, of course, simply trnsformtion from crtesin coordintes to polr coordintes. In the first, smll element of re is δx 1.δx wheres in the second smll element of re is δy 1.y 1 δy. Integrtion is gin over one qudrnt of the circle. The inner integrtion is long rdius nd, in the outer integrtion, this rdius is swept through n ngle of 9. Continuing: ( ) π 4 h 3 dy = 4 3 π dy h 6 dy = 4h 6 The result is the fmilir formul for the volume of cone. 1. π = π h 3

3 Appliction to Bivrite Probbility Density Functions Formul (1.) hs direct ppliction to the process of trnsforming bivrite probbility density functions... Suppose X 1 nd X re two rndom vribles whose bivrite probbility density function is f(x 1,x ). It is common prctice to represent given pir of vlues of the two rndom vribles X 1 nd X s point in the x 1 x plne. By definition: P(X 1,X lies in specified region R x ) = f(x 1,x ) dx 1 dx R x (1.3) Any function of rndom vrible (or indeed of two or more rndom vribles) is itself rndom vrible. If y 1 nd y re tken s trnsformtion functions, both y 1 (X 1,X ) nd y (X 1,X ) will be derived rndom vribles. Let Y 1 = y 1 (X 1,X ) nd Y = y (X 1,X ). Tke R y s identicl to the region R x but specified in the y 1 y plne. Necessrily: P(Y 1,Y lies in specified region R y ) = P(X 1,X lies in specified region R x ) From this nd by (1.3) nd (1.): P(Y 1,Y lies in specified region R y ) = f ( x 1 (y 1,y ),x (y 1,y ) ) R y (y 1,y ) dy 1 dy Notice tht the integrnd is expressed wholly in terms of y 1 nd y. Clling this integrnd g(y 1,y ): P(Y 1,Y lies in specified region R y ) = g(y 1,y ) dy 1 dy R y This demonstrtes tht g(y 1,y ) is the probbility density function ssocited with the two rndom vribles Y 1 nd Y. The requirements for f nd g to be single vlued nd non-negtive re just s in the onevrible cse nd it is customry for the reltionship between probbility density function f(x 1,x ), the inverses x 1 (y 1,y ) nd x (y 1,y ) of pir of trnsformtion functions, nd the derived probbility density function g(y 1,y ) to be written: g(y 1,y ) = f ( x 1 (y 1,y ),x (y 1,y ) ) (y 1,y ) This is directly nlogous to reltionship (11.4) given for the trnsformtion of single rndom vrible into nother. 1.3

4 Summry Single Vrible nd Bivrite Trnsformtions In this section, summry of the single vrible cse nd summry of the bivrite cse re presented together so tht the correspondence between the two cn redily be seen. Trnsformtion of single rndom vrible: Strt with rndom vrible X. Assume the ssocited probbility density function is f(x). Choose trnsformtion function y(x). Let the derived rndom vrible be Y = y(x). Assume the ssocited probbility density function is g(y). Assume the inverse of the trnsformtion function is x(y). The reltionship between f(x) nd g(y) is: g(y) = f ( x(y) ) dx dy As specil cse, if f(x) corresponds to uniform distribution, the reltionship is: g(y) = dx dy Trnsformtion of pir of rndom vribles: Strt with two rndom vribles X 1 nd X. Assume the ssocited bivrite probbility density function is f(x 1,x ). Choose two trnsformtion functions y 1 (x 1,x ) nd y (x 1,x ). Let the derived rndom vribles be Y 1 = y 1 (X 1,X ) nd Y = y (X 1,X ). Assume the ssocited bivrite probbility density function is g(y 1,y ). Assume the inverses of the two trnsformtion functions re x 1 (y 1,y ) nd x (y 1,y ). The reltionship between f(x 1,x ) nd g(y 1,y ) is: g(y 1,y ) = f ( x 1 (y 1,y ),x (y 1,y ) ) (y 1,y ) As specil cse, if f(x 1,x ) corresponds to uniform distribution, the reltionship is: g(y 1,y ) = (y 1,y ) (1.4) 1.4

5 Exmple The Box Muller Trnsformtion An erlier ttempt to trnsform uniform distribution into norml distribution proved unsuccessful. Fortuntely the difficulties cn be overcome by strting with the bivrite equivlent of the uniform distribution. Suppose X 1 nd X re two independent rndom vribles ech distributed Uniform(,1). Bringing these together leds to the following bivrite probbility density function: { 1, if x1,x < 1 f(x 1,x ) =, otherwise Informlly, the function f = 1 when (x 1,x ) lies in unit squre which hs one corner t the origin but f = if (x 1,x ) lies outside this squre. This is the uniform distribution ssumed in reltionship (1.4). Suppose tht the trnsformtion functions re: y 1 = ln(x 1 ) cos(πx ) nd y = ln(x 1 ) sin(πx ) (1.5) First, derive the inverse functions: x 1 = e 1 (y 1 +y ) nd x = 1 π tn 1( y y 1 ) Next, evlute the Jcobin: x 1 x 1 = y 1 e 1 (y 1 +y ) y e 1 (y 1 +y ) 1 π y /y (y /y 1 ) 1 π 1/y (y /y 1 ) = 1 π [ ] e 1 (y 1 +y ) ( y ) 1 + (y /y 1 ) 1+ y 1 From (1.4): g(y 1,y ) = (y 1,y ) = 1 e 1 1 y 1 e 1 y π π Recll (1.3) nd note tht this bivrite probbility density function corresponds to two independent rndom vribles Y 1 nd Y which re ech distributed Norml(,1). It is now cler how to trnsform uniform distribution into norml distribution: Strt with two independent rndom vribles X 1 nd X which re ech distributed Uniform(,1). From the trnsformtion functions y 1 nd y derive two new rndom vribles being Y 1 = y 1 (X 1,X ) nd Y = y (X 1,X ). The derived rndom vribles will ech independently be distributed Norml(,1). This process is known s the Box Muller trnsformtion. 1.5

6 Box Muller Refinement The following procedure, written in hypotheticl progrmming lnguge, mkes use of the Box Muller trnsformtion; repeted clls of this procedure will return rndom numbers which re distributed Norml(,1): PROCEDURE norml X1 = uniform(,1) X = uniform(,1) Y1 = sqrt(-*ln(x1))*cos(*pi*x) Y = sqrt(-*ln(x1))*sin(*pi*x) RETURN Y1 END It is ssumed tht uniform, sqrt, ln, cos, nd sin re librry procedures which hve the obvious effects. In prticulr, repeted clls of uniform(,1) will return rndom numbers which re distributed Uniform(,1). Mthemticlly the procedure is fine but it is not ltogether stisfctory from Computer Science point of view. Most obviously, the vlue Y is computed but never used. It would be better to rrnge for the procedure to hve two sttes. In one stte, both Y1 nd Y would be evluted nd the vlue of Y1 returned. The vlue of Y would be retined so tht it could be returned on the next cll when the procedure would be in the lternte stte in which no evlution would be necessry. The procedure lso mkes two clls of sqrt, two clls of ln nd one ech of cos nd sin. All six of these clls re quite expensive in computer time nd it is possible to be much more efficient. Insted of strting with the two rndom vribles X 1 nd X which re ech distributed Uniform(,1) better pproch is to begin with two other rndom vribles W 1 nd W whose vlues represent the (crtesin) coordintes of point in unit circle centred on the origin. All points in the circle re eqully likely, just s in the rindrops nd pond exmple discussed erlier. Assuming the vlues of W 1 nd W re w 1 nd w respectively, the rndom vribles X 1 nd X re then given vlues: x 1 = w 1 + w nd x = 1 π tn 1( w w 1 ) (1.6) It will be demonstrted shortly tht these derived rndom vribles X 1 nd X re both distributed Uniform(,1) nd cn therefore be used s before. At this stge, the introduction of the two rndom vribles W 1 nd W hrdly seems to hve led to n improvement but it will be shown tht, by their use, the number of expensive procedure clls cn be gretly reduced. To pprecite how this revised pproch works nd why it leds to greter efficiency, it is necessry to revisit the circulr pond

7 In the figure below, the coordintes of the point D re shown s (w 1,w ), these being the vlues of the rndom vribles W 1 nd W : 1 D r θ w 1 w From the figure, r is the distnce of point D from the centre nd r = w 1 + w but, from (1.6), x 1 = w 1 + w. Hence: x 1 = r or r = x 1 (1.7) The vlue of the derived rndom vrible X 1 is therefore the squre of the distnce r of D from the centre nd, from the experience of the rindrops nd pond exmple, it is distributed Uniform(,1). ( ) From the figure, θ = tn 1 w but, from (1.6), x w = 1 ( ) w 1 π tn 1. w 1 Hence: x = θ or θ = π x (1.8) π Assuming two-rgument inverse-tngent function is used (such s ATAN in Excel), θ will be uniformly distributed over the rnge to π. This ensures tht the vlue of the derived rndom vrible X is distributed Uniform(,1). It is now cler tht both X 1 nd X re distributed Uniform(,1). From the figure nd from (1.7) nd (1.8): w 1 = r cos θ = x 1 cos(πx ) so cos(πx ) = w 1 x1 nd: w = r sinθ = x 1 sin(πx ) so sin(πx ) = w x1 The trnsformtion functions (1.5) cn therefore be rewritten: y 1 = ln(x 1 ) ln(x 1 ) w 1 nd y = w (1.9) x 1 x 1 1.7

8 The procedure written in the hypotheticl progrmming lnguge cn now be modified to ccommodte the revised pproch: PROCEDURE norml REPEAT W1 = uniform(-1,+1) W = uniform(-1,+1) X1 = W1*W1+W*W UNTIL X1<1 FACTOR = sqrt(-*ln(x1)/x1) Y1 = FACTOR*W1 Y = FACTOR*W RETURN Y1 END The first two ssignment sttements in the REPEAT UNTIL loop give vlues to the rndom vribles W 1 nd W but these vlues re ech in the rnge 1 to +1. The coordintes (w 1,w ) represent point which is gurnteed to lie inside squre centred on the origin but is not gurnteed to lie inside the unit circle. A preliminry vlue w 1 + w is ssigned to the derived rndom vrible X 1 ; this is the squre of the distnce from the origin. This vlue is cceptble if it is less thn one. If not, the loop is repeted nd new vlues re determined for W 1 nd W nd the derived rndom vrible X 1. The vlue ssigned to the identifier FACTOR is the vlue of the fctor common to both expressions in (1.9). Multiplying this fctor by w 1 nd w respectively provides vlues for the derived rndom vribles Y 1 nd Y which re both distributed Norml(,1). Notice tht no vlue is computed for the derived rndom vrible X since x does not feture in the expressions in (1.9). This procedure is more efficient tht its predecessor nd mkes only single cll of sqrt nd single cll of ln nd there re no clls of cos or sin. Nevertheless, the procedure still mkes no use of Y. A little extr progrmming could sve the vlue of Y for the next cll of the procedure. Another improvement would be to enhnce the procedure so tht it hd two rguments MEAN nd STDEV nd returned vlue which is distributed Norml(MEAN,STDEV ) insted of Norml(,1). Nervous reders might be lrmed t wht ppers to be negtive rgument for the sqrt function. Remember tht x 1 is in the rnge to 1 so ln(x 1 ) is gurnteed to be negtive which ensures tht ln(x 1 ) is positive. There is more serious cuse for concern in tht x 1, the rgument of ln, could in principle be zero. This possibility cn be trpped by modifying the condition fter UNTIL to <X1<1 so tht x 1 hs to be strictly greter thn zero. 1.8