Multivariate Probability Estimation for Categorical Variables from Marginal Distribution Constraints

Transcription

1 Paper 06, CCG Aual Report, 009 ( 009) Multvarate Probablty Estmato for Categorcal Varables from Margal Dstrbuto Costrats Yupeg L ad Clayto V. Deutsch I geostatstcs, the smulato of categorcal varables such as faces or rock types requres a codtoal dstrbuto based o the formato from earby data locatos. The codtoal probablty ca easly be calculated from the multvarate dstrbuto of the usampled value ad all data values. I ths paper, a ew multvarate probablty estmato algorthm s proposed. The multvarate probablty s estmated from lower order margal probabltes that are kow from the avalable data. The requred codtoal probablty s the obtaed drectly by Bayes law. I ths algorthm, the bvarate margal probabltes are mposed to a tal multvarate probablty as the costrats ad satsfed by teratvely modfyg the tal multvarate probablty. The bvarate margal probabltes are ferred from sampled locatos, a trag mage or profles alog drll holes or wells. A sparse matrx s used to calculate the margal probabltes from the multvarate probablty, whch sgfcatly saves CPU tme ad makes ths estmato algorthm practcal. Ths algorthm ca be exteded to hgher order (m>) margal probablty costrats. The theoretcal framework s developed ad llustrated wth a umber of realstc examples. Itroducto I reservor maagemet, a umercal reservor model s always requred for resource evaluato ad reservor flow smulato put for producto forecastg ad recovery calculatos. May factors wll have a effect o umercal reservor model. Most geostatstcal practtoers agree that faces are oe of the most mportat reservor heterogeetes. I geology, a faces s a dstctve rock ut that forms uder certa codtos of sedmetato, reflectg a partcular process or evromet (e.g. rver chaels, delta systems, submare fas, reefs). Usually, faces are rock uts that are somehow statstcally homogeeous. Although, the faces for each usampled locato s uque, t s vewed as a radom varable that could take a set of possble values or states. The radom varables are locato depedet ad doated wth the locato vectoru. The possble outcome of the radom varable for the locato s deoted as k ad k {,,..., K} whch are mutually exclusve ad exhaustve. Practcally, the total umber of faces K s less tha 5. We ca thk of evet beg true. u= kas a evet ad P( u = k) Multvarate probablty: A specfc spatal arragemet of radom varables{ u, =,..., } locatos s deoted as{ u = k ; =,..., } of or ot a state k {,,... K} probabltes s the probablty of ths partcular dfferet locatos wll defe a set of. A state of the faces combato outcomes at each locato for a set. The ucertaty (our lack of kowledge) of whether exsts at locato u s characterzed by a set of dscrete multvarate P( u, u,..., u ; k, k,..., k ) wrtte as: P( u,..., u ; k,..., k ) = Prob{ u = k, u = k,..., u = k } The multvarate probablty has the followg propertes: = Prob{ = k, =,..., } u () k {,,... K}; =,.., 0 P( u, u,..., u ; k, k,..., k ) P( u, u,..., u; k, k,..., k ) = For coveece, the otato for the multvarate probablty wll be smplfed to P( u,, =,..., ; k, =,..., ). Oe characterstc of ths dscrete multvarate probablty s ts 06-

2 Paper 06, CCG Aual Report, 009 ( 009) huge state space. The umber of all possble states wll be K, ad these K states have dfferet probabltes to exst. Cosder there are 3 faces the doma, ad 0 sampled locatos, the the multvarate probablty state space wll be 3 0 = 3,486,784,40. Multvarate probablty margalzato: Gve the -varate multvarate probablty P( u l = k l, l =,..., ), dfferet orders of margal probablty ca be calculated based o the law of total probablty. Frst order margal probablty P ( ; ) k u as a uvarate probablty of each faces k for the locato us defed as: P( u ; k ) = P( u l, l =,..., ; k l, l =,..., ) () Alll wth u = k The uvarate margal probablty satsfy 0 P ( u; k) ad P ( u ; k ) =. Secod order margal or bvarate margal probablty P (, ;, ) u u k k multvarate probablty as: Alll wth u = k & u j = k j ; j K k = j j ca be calculated from the P ( u, u j; k, k j ) = P( u l, l =,..., ; k l, l =,..., ) (3) The bvarate margal probablty satsfy K K u u j j. k = k j = 0 P ( u, u j; k, k j ) ad P (, ; k, k ) = Followg ths logc, ay m order margal probablty of the -varates multvarate probablty P(,..., ; k,..., k ) u u m m smplfed as m P ( k,..., k ) ca be calculated as: m Pm ( u,..., um; k,..., km) = P( u l, l =,..., ; k l, l =,..., ) (4) Alll wth u = k,..., u = k ; m m I expresso(4), the m-varate margal probablty ca be ay m arbtrary subgroup of the radom varables. Codtoal probablty: we are ofte terested estmatg the probablty that faces k prevals at locato u based o the eghborg data. The usampled locato s deoted as u (could be ay locato the doma, just for dex coveece) these eghborg locatos { u = k, =,..., }. The probablty of faces k for usampled locato u wll be: P( u = k u = k,..., u = k ) ad ca be calculated from the multvarate probablty based o Bayes law: P( u = k, u = k,..., u = k ) P( u = k u = k,..., u = k ) = (5) P( u = k,..., u = k ) I ths equato, the umerator s a -varates multvarate probablty, whle the deomator s the (- )-varates margal probablty of ths -varates multvarate probablty. Usg the mult-varates margal probablty calculato as expressed (4), the codtoal probablty (5) wll be calculated as: m 06-

3 Paper 06, CCG Aual Report, 009 ( 009) P( u = k u = k,..., u = k ) = = P( u = k, u = k,..., u = k) P( u = k,..., u = k ) k = K k = P( u = k, u = k,..., u = k ) P( u = k, u = k,..., u = k ) As stated Equato(6), the ferece of ths codtoal probablty dstrbuto s the cetral problem of geostatstcs. There are several establshed ad ew approaches to fer the codtoal probablty drectly or drectly, such as dcator krgg or multple pot geostatstcs. I dcator approach (Goovaerts, 994; Jourel, 983), the codtoal probablty multvarate probablty s calculated as a lear combato of dcator data: P( u = k u = k,..., u = k ) = λ P( u = k u = k ) (7) = The weghts characterze the depedece betwee the sampled locatos uad the formato from sampled locatos to usampled locato. I the multpot statstcs approach, the requred -varate multvarate probablty s costructed from a trag mage. The trag mage represets the heterogeety characterstcs that the geologst expects to see the study area. The, the requred codtoal probablty expresso (6) ca be calculated by just coutg the relatvely states umber occurred the trag mage: Cout( u = k, u = k,..., u = k ) P( u = k u = k,..., u = k} = (8) Cout( u = k,..., u = k ) Those two approaches are reasoable, but the lear probablty combato approach of dcator approach may ot result models that show the approprate level of spatal detal. The statoary assumpto multpot geostatstcs may cause some dffculty. Furthermore, because of the huge state space, t s dffcult to get eough replcato for hgher order multvarate probablty from a sgle trag mage. Drectly estmatg the multvarate probablty from lower margal probabltes s proposed here to fer the multvarate probablty expresso (6) drectly. The lower margal probabltes wll have mmal statoary requremet ad the olear combato wll be more approprate may cases. (6) Multvarate probablty estmato from margal costrats Whe the radom varables { u = k ; =,..., } are depedet, the multvarate probablty s gve by: P( u,..., u ; k,..., k ) = p( u ; k ), k =,..., K (9) = I most spatal statstcs case, the radom varables are depedet. The data formato redudacy should be cosdered the estmato process. Lower margal probabltes are used as a tool to characterze the formato ad the redudacy betwee the sampled locatos ad the usampled locato. I ths approach, the lower margal probablty s used as costrats to teratvely modfy the estmated multvarate probablty utl they are all satsfed. The drect multvarate probablty estmato algorthm (DMPE) llustrated pseudo-program laguage geeral case s: Beg Iput: the uvarate probablty p ad the kow m order margal probablty Geerate a tal multvarate probablty Repeat Target Ρm * P wth the depedet assumpto: P * = p 06-3

4 Paper 06, CCG Aual Report, 009 ( 009) - Use the * P as the tal multvarate probablty estmato - calculate the curret margal Calc Pm δ P ; from the curret estmated multvarate probablty Target δ P ; - calculate a modfy factor vector F = Pm wth the kow margal probablty ad the curret Calc Pm estmated margal probablty; δ δ - calculate a ew updated multvarate probablty wth the modfy factor: p + = ( F p ) C δ + δ - gve P to P for ext terato; Utl: the chage of the multvarate probablty s stable: O Output: the fal estmated multvarate probablty Ed est P I the modfcato process, C s used to do ormalzato to make Small Numercal Example δ + δ ε Wth O = p p δ + P a lct probablty. grd, three To llustrate the DMPE process, cosder the followg spatal smulato problem. Oe a 3 categores may exst at ay oe of them. I ths small case ( = 3, K = 3), the total states umber wll be3 3 = 7, all the possble states are show Fgure. Each of the possble states wll have a specfc probablty to exst whch s characterzed by a multvarate probablty. I order to store ad dex all the probabltes more effcetly, the multvarate probabltes are ordered a oe dmeso array ad gve a uque dex to each of them. The K states dexed from to K wll be calculated usg the followg dex fucto: I Equato(0), k =,,..., l f ( k,,..., ) ( k ) K K l l l = = + l (0) l=. It s obtaed by orderg ad codg all the categores to a teger set accordg to the order, but how to order the categores does ot matter. Table s the dex calculato example for ths case. I ths small example, the total umber of secod order combatos three grds wll be C =, ad each of them may have 9 outcomes, the outcomes of grd ad 3 are show Fgure. 3 3 The smlar outcomes for combato of grd ad, or grd ad 3, so totally there wll be 7 outcomes. Geerally, for bvarate margal probablty, the total state umber wll be K C. Deote the P = [ p, p..., p ] T multvarate probablty s 7 P = [ b, b,..., b ] T whch the probablty for each state to exst ad also deote the bvarate probablty as 7. From equato(3), ths case, each bvarate probablty ca be calculated from the multvarate probablty as: b b = b p p p7 06-4

5 Paper 06, CCG Aual Report, 009 ( 009) As showg above matrx equato, each bvarate probablty assgmet s the sum of the subset comg from the full table of the multvarate probablty. Ther relatoshp s characterzed by the dcator dex matrx I, ad t ca be wrtte matrx form as: P = I P () As the umber of locatos or grd odes crease, the dmeso of matrx Icreases dramatcally. If there are 0 locatos ad 3 faces, the dmeso of matrx Iwll be Ad the o-zero 0 elemets umber each row s K = 3 = 656whch compose a sparse matrx. The colum umbers for the ozero value the sparse matrx deped o whch par of bvarate margal s calculated. The margalzato computato wll be a lear sparse matrx computato whch s very fast. If the bvarate probablty for each two locatos was gve, t wll be show ext that the multvarate probablty ca be estmated. Note, the above (DMPE) algorthm, the lower order margal probablty ca be ay hgher order tha bvarate margal probablty. I geeral, deote Pm as m order margal probablty vector, t ca be calculated from the multvarate probablty as: Pm = Im P () The matrx Im s the dcator dex matrx, the dmeso s m K K wth! =. m m m!( m)! Assume the multvarate probablty for the state outcome s already kow as Table. From the kow multvarate probablty, the bvarate probablty ca also be calculated ad lsted Table. The put for DMPE algorthm s the uvarate probablty ad bvarate probablty. The estmato result s also lsted Table whch s show that the kow multvarate probabltes are reproduced farly well. The DMPE algorthm coceved the effort of geostatstcs faces modelg algorthm mprovemet s cocded wth oe multvarate probablty approxmato formato theory research proposed by Ku ad Kullback (969). I ther approach, based o the maxmum etropy theory, the multvarate probablty s obtaed from the lower margal probablty by a straghtforward teratve algorthm whch s very smlar to the proposed DMPE algorthm ths paper. I Ku ad Kullback approach, for a multvarate probablty wth N varables K dscrete categores, the par wse costrats wll requre K N /K =K N- terms from the full multvarate terms. Ths operato wll be very CPU tesve for large N (depedece o K). May authors recogzed ths approach as a fatastc approach but oted that the computatoal complex s a major barrer(freema, 97; Saeres ad Fouss, 004) ad thus few practcal applcatos. Whle DMPE approach, each sgle state of the full states set ca be easly dexed. Also, usg the dex dcator trasform, the margalzato computato ca be doe a very fast lear operato style wth a small storage requremet by takg advatage of the sparse matrx. Ths hgher effcet dex ad sparse matrx computato the teratve scheme successfully make ths powerful estmato more applcable practcal problems. Table 3 s the comparso betwee these three methods usg the same data as those used by Ku ad Kullback (969). Reservor Example The above DMPE algorthm ca be used the tradtoal cell-based sequetal smulato approach (Deutsch ad Jourel, 998) where all smulato grd cells are vsted oly oce alog a radom path ad smulated cell values become codtog data for cells vsted later the sequece. Ay usampled grd cell uvsted alog the radom path s smulated as follows:. Look for the codtog data (orgal well data or prevously smulated cell values) closest to u;. Based o the dstace betwee every two locatos, retreve bvarate probablty from the expermetal bvarate probablty dagram, see the comg explaato; 3. Usg the DMPE algorthm to estmate the multvarate probablty; 3. The estmated codtoal probablty of each faces at us computed usg the Bayes law based o the faces at each codtog data locato, see equato(6); 06-5

6 Paper 06, CCG Aual Report, 009 ( 009) 4. Draw a smulated faces value from the resultg local probablty dstrbuto usg Mote-Carlo samplg, ad assg that value to the grd cell u ad go to the ext usampled locato utl all the usampled locato are vsted. The eeded margal probablty ths DMPE approach ca be obtaed from a trag mage or geologcal aalyss. The most easly obtaed margal s the uvarate ad bvarate margal probabltes. The uvarate margal s the proporto of all the faces at each usampled locato. They could be the global proporto or locally varyg proportos. I DMPE algorthm, the uvarate probablty wll be used to calculate the tal estmato. The bvarate margal probablty for two locatos s defed as P (, ;, ) u u k k whch ca be obtaed from the trag mage, well vertcal j j profles or from outcrops. From the trag mage Fgure 3, wthout thkg the dfferet drecto, as locato ( u, u ) apart from each other to a further dstace, the sotropc bvarate probablty betwee j faces, ad 3 wll compose a dagram as show Fgure 4. The bvarate probablty dagram s close to the trasto probablty dagram (L, 007) that used the ew developed markov trasto probablty based geostatstcs (Carle, 000; Carle ad Fogg, 996; L ad Zhag, 006). Cocluso Dscrete multvarate probabltes ca be drectly estmated from the lower order margal probablty dstrbuto costrats. The estmated multvarate probablty works well wth reasoable CPU tme whch promses practcal use the future. The use of m-varate (m>) margal costrats ca be eforced the terato process gve hgher order margal probabltes ca be obtaed. Practcal 3D smulato ad tegratg sedmetary patters from sequece stratgraphy wll be the subject of future research. Refereces CARLE, S.F., 000. Use of a Trasto Probablty/Markov Approach to Improve Geostatstcal of Faces Archtecture. I., 50 Klobytes pages ; PDFN. CARLE, S.F. ad FOGG, G.E., 996. Trasto probablty-based dcator geostatstcs. Mathematcal Geology, 8, DEUTSCH, C.V. ad JOURNEL, A.G., 998. GSLIB: Geostatstcal software Lbrary ad User s Gude. New York, Oxford Uversty Press. FREEMAN, J.J., 97. APPROXIMATING DISCRETE PROBABILITY DISTRIBUTIONS. IEEE Trasactos o Iformato Theory, 7, GOOVAERTS, P., 994. Comparatve performace of dcator algorthms for modelg codtoal probablty dstrbuto fuctos. Mathematcal Geology, 6, JOURNEL, A., 983. Noparametrc estmato of spatal dstrbutos. Mathematcal Geology, 5, KU, H. ad KULLBACK, S., 969. Approxmatg dscrete probablty dstrbutos. IEEE Trasactos o Iformato Theory, 5, LI, W., 007. Trasograms for characterzg spatal varablty of sol classes. Sol Scece Socety of Amerca Joural 7, LI, W. ad ZHANG, C., 006. A Geeralzed Markov Cha Approach for Codtoal Smulato of Categorcal Varables from Grd Samples. 0, SAERENS, M. ad FOUSS, F., 004. Yet Aother Method for Combg Classfers Outputs: A Maxmum Etropy Approach. I: Multple Classfer Systems

7 Paper 06, CCG Aual Report, 009 ( 009) k k 3 Table : The oe dmesoal dces of three varables ad three faces k l + ( k ) K l l= f ( k, =,..., ) +(-)*3 (-) +(-)*3 (-) + (-)*3 (3-) +(-)*3 (-) +(-)*3 (-) + (-)*3 (3-) (-)*3 (-) +(3-)*3 (-) + (3-)*3 (3-) (3-)*3 (-) +(3-)*3 (-) + (3-)*3 (3-) 7 Table oe umercal example of multvarate probablty estmato from the DMPE algorthm MV dex TRUE MV probablty Bvarate Margal estmated MV probablty Table 3 the multvarate probablty estmato results from three dfferet approaches True MV probablty Ku-ad-Kullback estmato DMPE estmato l l 06-7

8 Paper 06, CCG Aual Report, 009 ( 009) Fgure all the states outcomes for 3 categores 3 grds Fgure all the states outcomes for oe combato of two grds out of three Fgure 3 the trag mage ad the ferred bvarate probablty dagram Fgure 4 the DMPE smulato results ad tradtoal SISIM smulato results comparso 06-8