Data Mining in Petroleum Upstream The Use of Regression and Classification Algorithms

Transcription

1 Data Mnng n Petroleum Upstream The Use of Regresson and Classfcaton Algorthms Dawe L #, Guangren Sh Research Insttute of Petroleum Exploraton & Development, PetroChna, Bejng 00083, Chna Emal: leedw@petrochna.com.cn Abstract Scentfc Journal of Earth Scence June 205, Volume 5, Issue 2, PP Data mnng (DM) technque has seen enormous successes n some felds, but ts applcaton to petroleum upstream (PUP) s stll at ntal stage, as PUP s qute dfferent from the other felds n many aspects. The most popular DM algorthms n PUP are regresson and classfcaton. Through many DM applcatons to PUP, we have found that: a) the preferable algorthm for regresson s back-propagaton neural network (BPNN), the next are regresson of support vector machne (R-SVM) and multple regresson analyss (MRA), and the preferable algorthm for classfcaton s classfcaton of support vector machne (C-SVM), the next s Bayesan successve dscrmnaton (BAYSD); b) C-SVM can also be appled n data cleanng; c) both MRA and BAYSD can also be appled n dmenson-reducton, and BAYSD s preferable one; d) R-mode cluster analyss (RCA) can be appled n dmenson-reducton, whle Q-mode cluster analyss (QCA) can be appled n sample-reducton. A case study n PUP ndcates that R-SVM, BPNN and MRA are not applcable for regresson, whereas C-SVM s applcable for classfcaton n ths case. Keywords: Data Mnng; Regresson; Classfcaton; Data Cleanng; Dmenson-reducton; Sample-reducton; Ol Layer INTRODUCTION Petroleum upstream (PUP) has already entered bg data epoch for many years. For example, there are about 50 large nformaton systems that have been bult n PetroChna, and there are about 590TB PUP data and wells structured data n only one of them. At present, the major applcatons of PUP data n these nformaton systems are only nformaton storage and nqury, whch s far away from fully utlzng the values of these data assets. Data mnng (DM) s a preferable soluton for ths problem. DM technque has seen enormous successes n some felds of busness and scences, but ts applcaton to PUP s stll at ntal stage. Ths s because that PUP s dfferent from the other felds n many aspects, such as mscellaneous data types, huge quantty, dfferent measurng precson, and lots of uncertantes to data mnng results. DM can be dvded nto two classes: small data mnng (SDM), whch deals wth several thousand samples at most; and bg data mnng (BDM), whch deals wth ten thousand samples at least. In PUP, the sesmc, remote sensng and well log data are potental applcatons of BDM, e.g. Zhu and Sh (203) and Sh et al. (204) used BDM n well log[][2]; but the other data are potental applcatons of SDM, e.g. Sh (203) used SDM n 36 case studes[3], and Sh (205) used SDM n 8 case studes of petroleum geology[4]. In DM, the most popular algorthms are regresson and classfcaton. There manly exst three regresson and fve classfcaton algorthms[3]: the multple regresson analyss (MRA), the error back-propagaton neural network (BPNN), the regresson of support vector machne (R-SVM), the classfcaton of support vector machne (C-SVM), the decson trees (DTR), the Naïve Bayesan (NBAY), the Bayesan dscrmnaton (BAYD), and the Bayesan successve dscrmnaton (BAYSD). These eght algorthms use the same known parameters, and also share the same unknown that s predcted. The only dfference between them s the method and calculaton results. It s well known that MRA, BPNN and R-SVM are regresson algorthms wth real number results, whle C-SVM, DTR, NBAY, BAYD and BAYSD are classfcaton algorthms wth nteger number results. In the eght algorthms, only MRA s a lnear algorthm whereas the other seven are nonlnear algorthms, ths s due to the fact that MRA

2 constructs a lnear functon whereas the other seven algorthms construct nonlnear functons, respectvely. Snce the usage of DTR s qute complcated and BAYSD s superor to BAYD, these two classfcaton algorthms (DTR and BAYD) are not ntroduced n ths paper. 2 REGRESSION AND CLASSIFICATION ALGORITHMS Assume that there are n learnng samples, each assocated wth m+ numbers (x, x 2,, x m, y ) and a set of observed values (x, x 2,, x m, y ), wth =, 2,, n for these numbers. In prncple, n>m, but n actual practce n>>m. The n samples assocated wth m+ numbers are defned as n vectors: x = (x, x 2,, x m, y ) (=, 2,, n) () where, n s the number of learnng samples; m s the number of ndependent varables n samples; x s the th learnng sample vector; x j s the value of the j th ndependent varable n the th learnng sample, j=, 2,, m; and y s the value of the th learnng sample, the observed value. Equaton () s the expresson of learnng samples. Let x 0 be the general form of a vector of (x, x 2,, x m ). The prncples of MRA, BPNN, NBAY and BAYSD are the same,.e. try to construct an expresson, y=y(x 0 ), such that Equaton (2) s mnmzed. Certanly, these three dfferent algorthms use dfferent approaches and result n dfferent accuracy of calculaton results. n y = ( x ) 0 y where, y( x 0 ) s the calculaton result of the dependent varable n the th learnng sample; and the other symbols have been defned n Equaton (). However, the prncples of R-SVM and C-SVM algorthms are to try to construct an expresson, y=y(x 0 ), n order to maxmze the margn based on support vector ponts to obtan the optmal separatng lne. Ths y=y(x 0 ) s called the fttng formula obtaned n the learnng process. The fttng formulas of dfferent algorthms are dfferent. In ths paper, y s defned as a sngle varable. The flowchart s as follows: the st step s the learnng process, usng n learnng samples to obtan a fttng formula; the 2 nd step s the learnng valdaton, substtutng n learnng samples nto the fttng formula to get predcton values (y, y 2,, y n ), respectvely, so as to verfy the ftness of an algorthm; and the 3 rd step s the predcton process, substtutng k predcton samples expressed wth Equaton (3) nto the fttng formula to get predcton values (y n+, y n+2,, y n+k ), respectvely. 2 x = (x, x 2,, x m ) (=n+, n+2,, n+k) (3) where, k s the number of predcton samples; x s the th predcton sample vector; and the other symbols have been defned n Equaton (). Equaton (3) s the expresson of predcton samples. 2. Error Analyss of Calculaton Results To express the calculaton accuracy of the predcton varable y for learnng and predcton samples when the aforementoned sx algorsms (MRA, BPNN, R-SVM, C-SVM, NBAY, and BAYSD) are used, the absolute relatve resdual R(%), the mean absolute relatve resdual R(%) and the total mean absolute relatve resdual R (%) are adopted. The absolute relatve resdual for each sample, R(%), s defned as R(%) = ( y - y ) / y 00 (4) where, y s the calculaton result of the dependent varable n the th learnng sample; and the other symbols have (2)

3 been defned n Equatons () and (3). It s noted that zero must not be taken as a value of y to avod floatng-pont overflow. Therefore, for regresson algorthm, delete the sample f ts y =0; and for classfcaton algorthm, postve nteger s taken as values of y. The mean absolute relatve resdual for all learnng samples or predcton samples, R(%), s defned as Ns R R(%) / Ns (5) (%)= = where, N s =n for learnng samples whle N s = k for predcton samples; and the other symbols have been defned n Equatons () and (3). For learnng samples, R(%) and R(%) are called the fttng resdual to express the ftness of learnng process, and here R(%) s desgnated as R(%); and for predcton samples, R(%) and R(%) are called the predcton resdual to express the accuracy of predcton process, and here R(%) s desgnated as R 2(%). The total mean absolute relatve resdual for all samples, R (%), s defned as R (%) = [ R(%)+ R 2 (%)]/2 (6) When there are no predcton samples, R (%)= R(%). 2.2 Nonlnearty and Soluton Accuracy of Studed Problem Snce MRA s a lnear algorthm, ts R (%) for a studed problem expresses the nonlnearty of y=y(x) to be solved,.e. the nonlnearty of the studed problem. Whether lnear algorthm (MRA) or nonlnear algorthms (BPNN, R-SVM, C-SVM, NBAY and BAYSD), ther R (%) of a studed problem expresses the accuracy of y=y(x) obtaned by each algorthm,.e. soluton accuracy of the studed problem solved by each algorthm. y = y(x) created by BPNN s an mplct expresson (t cannot be expressed as a usual mathematcal formula); whereas that of the other four algorthms are explct expressons (they are expressed as a usual mathematcal formula). 3 DATA PREPROCESSING The methods of data preprocessng are varous (such as data cleanng, data ntegraton, data transformaton and data reducton etc.). These methods wll be used before data mnng. Mamon and Rokach (200) made more detaled summary to data preprocessng[5], Han and Kamber (2006) specfed more detaled methods related to data preprocessng[6]. Ths paper only dscusses the aforementoned sx algorthms (MRA, BPNN, R-SVM, C-SVM, NBAY and BAYSD) used n data cleanng and data reducton. 3. Data Cleanng Realstc data are often nosy, mperfect and nconsstent. The man job for data cleanng s to fll up the mssed data, make nosy data smooth, dentfy or elmnate abnormal data as well as solve nconsstent problems. For example, n the case study of Zhu and Sh (203) and Sh et al. (204), C-SVM was employed to 2242 leanng samples and t was found that R(%) of 27 leanng samples were not zero. By correctng y of the 27 samples, C- SVM run agan resultng R(%)=0[][2]. 3.2 Data Reducton It usually takes lots of tme to make analyss of the complcated data to the large scaled DB content, whch often makes ths analyss unrealstc and unfeasble, partcularly nteractve DM s needed. Data reducton technque s

4 used to obtan a smplfed data set from the orgnal huge data set, and keep ths smplfed data set ntegraton as the orgnal data set. In ths way, t s obvously more effcent to perform DM on the smplfed data set, and the mned results are almost the same as the results from the orgnal data set. The man strateges to do data reducton are: a) data aggregaton, such as buld up data cube aggregaton, ths aggregaton s manly used for the constructon of data cube data warehouse operaton, b) data reducton s manly used for the detecton and elmnaton of those unrelated, weak correlated or redundant attrbutes or dmensons, c) data compresson, to compress data set by usng codng technque (e.g. mnmum codng length or wavelet); and d) numerosty reducton, to use smpler data expresson mode (e.g. parameter model, non-parameter model clusterng, samplng and hstogram ) to dsplace the orgnal data. Here we only dscuss the dmenson-reducton whch can reduce the number of ndependent varables and samplereducton whch can reduce the number of samples. MRA and BAYSD can serve as poneerng dmenson-reducton tools[3], because the two algorthms both can gve the dependence of the predcted value (y) on ndependent varables (x, x 2,, x m ), n decreasng order. However, because MRA belongs to data analyss n lnear correlaton whereas BAYSD s n nonlnear correlaton, n the applcatons wth very strong nonlnearty the ablty of dmenson-reducton of BAYSD s hgher than that of MRA. For example, the case study of Zhu and Sh (203) and Sh et al. (204) s a 6-D problem (x, x 2,, x 5, y)[][2]. For MRA, x s the mnmum dependence of y, so we tred to delete x and run C-SVM, the results shows R(%)=0.036 and R 2(%)=8.3, but the results wthout ths deleton are R(%)=0 and R 2(%)=6.30, whch ndcates ths dmenson-reducton s faled. For BAYSD, however, though t runs n the condton wthout x 3, x 7, x 6, x 4 x 2, x 3, x 2 and x 9, ts R (%) s 6.8, whch shows the dmenson of ths studed problem can be reduced from 6-D to 8- D. Why s the dmenson-reducton of BAYSD successful but that of MRA faled? The reason s that the nonlnearty of the studed problem s strong due to R (%) of MRA s 52.4, and BAYSD s a nonlnear algorthm whle MRA s a lnear algorthm. In addton, R-mode cluster analyss (RCA) can serve as a poneerng dmenson-reducton tool, whle Q-mode cluster analyss (QCA) can serve as a poneerng sample-reducton tool[3]. It s noted that both RCA and QCA are lnear algorthms. The called poneerng tool s whether t succeeds or not needs nonlnear tool (BPNN for regresson problem, C- SVM for classfcaton problem) for the second valdaton, so as to determne how many samples and how many ndependent varables can be reduced. Why does t need second valdaton? Because of the complextes of geoscences rules, the correlatons between dfferent classes of geoscences data are nonlnear n most cases. 4 CASE STUDY: OIL LAYER EVALUATION 4. Studed Problem The objectve of ths case study s to conduct the predcton of formaton flow capacty, whch has practcal value when ol test data are less lmted. Usng data of 4 samples from the Tahe Olfeld n the Tarm Basn, Chna, and each sample contans 7 ndependent varables (x = ol vscosty, x 2 = ol outcome, x 3 = choke sze, x 4 = ol pressure, x 5 = ol densty, x 6 = gas/ol rato, and x 7 = water cut) and an ol test result (y = formaton flow capacty), Kang et al. (2007) adopted BPNN for the predcton of formaton flow capacty[7]. Actually, they only adopted 6 ndependent varables wthout x (ol vscosty) for BPNN. In our case study, among these 4 samples, 2 are taken as learnng samples and one as predcton sample wth 7 ndependent varables (Table ). 4.2 Input Known Parameters They are the values of the known varables x (=, 2, 3, 4, 5, 6, 7) for 2 learnng samples and one predcton sample, and the value of the predcton varable y for 2 learnng samples (Table ). Note: y = formaton flow capacty for regresson calculaton; y = ol layer classfcaton for classfcaton calculaton

5 Sample type Learnng samples Predcton sample TABLE INPUT DATA FOR FORMATION FLOW CAPACITY OF THE TAHE OILFIELD Sample No. Well No. Relatve parameters for formaton flow capacty a y x x 2 x 3 x 4 x 5 x 6 x 7 FFC b OLC c TK S TK TK TK S S TK TK TK TK TK TK (4.80) (3) a x = ol vscosty, mpa s; x 2 = ol outcome, t/d; x 3 = choke sze, mm; x 4 = ol pressure, MPa; x 5 = ol densty, g/cm 3 ; x 6 = gas/ol rato; and x 7 = water cut, %. b y = formaton-flow-capacty (FFC, /(m µm 2 )) or ol-layer-classfcaton (OLC) determned by the ol test; number n parenthess s not nput data, but s used for calculatng R(%). c when y =OLC, t s ol layer classfcaton ( hgh-productvty ol layer, 2 ntermedate-productvty ol layer, 3 low-productvty ol layer, 4 dry layer) determned by the ol test (Table 2). TABLE 2 OIL LAYER CLASSIFICATION BASED ON FORMATION FLOW CAPACITY Ol layer classfcaton Formaton flow capacty (/(m µm 2 )) Hgh-productvty ol layer >00 Intermedate-productvty ol layer (25, 00] 2 Low-productvty ol layer [4, 25] Regresson Calculaton R-SVM, BPNN and MRA are adopted for regresson calculaton. ) Learnng Process Usng the 2 learnng samples (Table ) and by R-SVM, BPNN and MRA, the three functons of formaton flow capacty (y) wth respect to 7 ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ) have been constructed. Usng R-SVM[8][3], BPNN[3] and MRA[3], the result s fttng formula, respectvely. Substtutng the values of x, x 2, x 3, x 4, x 5, x 6, and x 7 gven by the 2 learnng samples (Table ) n the fttng formula of R-SVM, BPNN and MRA, respectvely, the formaton flow capacty (y) of each learnng sample s obtaned. Table 3 shows the results of learnng process by R-SVM, BPNN and MRA. 2) Predcton Process Substtutng the values of x, x 2, x 3, x 4, x 5, x 6, and x 7 gven by the one predcton sample (Table ) n the fttng formula of R-SVM, BPNN and MRA, respectvely, the formaton flow capacty (y) of the predcton sample s obtaned. Table 3 shows the results of predcton process by R-SVM, BPNN and MRA. It can been seen from R (%)=56.65 of MRA (Table 4) that the nonlnearty of the relatonshp between the predcted value y and ts relatve ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ) s very strong. The soluton accuracy of R-SVM and MRA are very low, and the soluton accuracy of BPNN s moderate. Snce ths case study s a very strong nonlnear problem, R-SVM, BPNN and MRA are not applcable y

6 Sample type Learnng samples Predcton sample TABLE 3 PREDICTION RESULTS OF FORMATION FLOW CAPACITY OF THE TAHE OILFIELD Formaton flow capacty (/(m µm 2 )) Sample Regresson algorthm Ol test No. R-SVM BPNN MRA y y R(%) y R(%) y R(%) TABLE 4 Algorthm R-SVM BPNN MRA COMPARISON AMONG THE APPLICATIONS OF THREE REGRESSION ALGORITHMS TO FORMATION FLOW CAPACITY OF THE TAHE OILFIELD Fttng formula Nonlnear, explct Nonlnear, mplct Lnear, explct 4.4 Classfcaton Calculaton Mean absolute relatve resdual R (%) R 2(%) R (%) Dependence of the predcted value (y) on ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ), n decreasng order Tme consumng on PC (Intel Core 2) Soluton accuracy N/A 3 s Very low N/A 30 s Moderate x 4, x 3, x 5, x 6, x, x 2, x 7 < s Very low C-SVM, NBAY, BAYSD and MRA are adopted for classfcaton calculaton. ) Learnng Process Usng the 2 learnng samples (Table ) and by C-SVM, NBAY, BAYSD and MRA, the four functons of ol-layerclassfcaton (y) wth respect to 7 ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ) have been constructed. Usng C-SVM[8][3], NBAY[3], BAYSD[3] and MRA[3], the result s fttng formula, respectvely. Substtutng the values of x, x 2, x 3, x 4, x 5, x 6, and x 7 gven by the 2 learnng samples (Table ) n the fttng formula of C-SVM, NBAY, BAYSD and MRA, respectvely, the ol-layer-classfcaton (y) of each learnng sample s obtaned. Table 5 shows the results of learnng process by C-SVM, NBAY, BAYSD and MRA. 2) Predcton Process Substtutng the values of x, x 2, x 3, x 4, x 5, x 6, and x 7 gven by the one predcton sample (Table ) n the fttng formula of C-SVM, NBAY, BAYSD and MRA respectvely, the ol-layer-classfcaton (y) of the predcton sample s obtaned. Table 5 shows the results of predcton process by C-SVM, NBAY, BAYSD and MRA. It can been seen from R (%)=8.8 of MRA (Table 6) that the nonlnearty of the relatonshp between the predcted value y and ts relatve ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ) s strong. The results of C-SVM are very hgh,.e. not only the R(%)=0, but also R 2 (%)=0, and thus the total mean absolute relatve resdual R (%)=0, whch concdes wth practcalty. Snce ths case study s a strong nonlnear problem, only C-SVM s applcable, but NBAY and BAYSD are not applcable due to very low soluton accuracy

7 Sample type Learnng samples Predcton sample TABLE 5 PREDICTION RESULTS FROM OIL LAYER CLASSIFICATION OF THE TAHE OILFIELD Ol layer classfcaton Sample Ol test a Classfcaton algorthm No. C-SVM NBAY BAYSD MRA b y y R(%) y R(%) y R(%) y R(%) a y = ol-layer-classfcaton ( hgh-productvty ol layer, 2 ntermedate-productvty ol layer, 3 low-productvty ol layer, 4 dry layer) determned by the ol test (Table 2). b Here y calculated by MRA are converted from real number to nteger number by usng round rule. TABLE 6 COMPARISON AMONG THE APPLICATIONS OF FOUR CLASSIFICATION ALGORITHMS TO OIL LAYER CLASSIFICATION OF THE TAHE OILFIELD Algorthm C-SVM NBAY BAYSD MRA 4.5 Summary Fttng formula Nonlnear, explct Nonlnear, explct Nonlnear, explct Lnear, explct Mean absolute relatve resdual R (%) R 2(%) R (%) Dependence of the predcted value (y) on ndependent varables (x, x 2, x 3, x 4, x 5, x 6, x 7 ), n decreasng order Tme consumng on PC (Intel Core 2) Soluton accuracy N/A 5 s Very hgh N/A < s Very low x, x 7, x 4, x 3, x 5, x 6, x 2 s Very low x 2, x 4, x, x 5, x 3, x 6, x 7 < s Strong nonlnearty Summarly, usng data for the formaton flow capacty and ol layer classfcaton n the Tahe Olfeld based on seven ndependent varables (ol vscosty, ol outcome, glb, ol pressure, ol densty, gas/ol rato, and water cut) and an ol test result of 3 samples, of whch 2 are taken as learnng samples and one s taken as predcton sample, regresson algorthms (R-SVM, BPNN, MRA) are adopted for the formaton flow capacty, and classfcaton algorthms (C-SVM, NBAY and BAYSD) are used for the ol layer classfcaton. It s found that a) snce the formaton flow capacty s a very strong nonlnear problem, R-SVM, BPNN and MRA are not applcable; and b) snce the ol layer classfcaton s a strong nonlnear problem, only C-SVM s applcable, but NBAY and BAYSD are not applcable due to very low soluton accuracy. 5 CONCLUSIONS DM s a useful technque for utlzng the values of data asset n PUP. Through many DM applcatons to PUP[][2] [3][4], we have found that: a) the preferable algorthm for regresson problems s BPNN, the next s R-SVM and MRA, and the preferable algorthm for classfcaton problems s C-SVM, the next s BAYSD; b) C-SVM also can be appled n data cleanng; c) both MRA and BAYSD also can be appled n dmenson-reducton, and BAYSD s preferable one; and d) R-mode cluster analyss (RCA) can be appled n dmenson-reducton, whle Q-mode cluster analyss (QCA) can be appled n sample-reducton. It s noted that both RCA and QCA are lnear algorthms. Fnally, a case study (ol layer evaluaton) ndcates that n ths case R-SVM, BPNN and MRA are not applcable for

8 regresson, whereas C-SVM s applcable for classfcaton. REFERENCES [] Zhu, Y., Sh, G. Identfcaton of lthologc characterstcs of volcanc rocks by support vector machne. Acta Petrole Snca, 34 (2), , 203(n Chnese) [2] Sh, G., Zhu, Y., M, S., Ma J., Wan J. A Bg Data Mnng n Petroleum Exploraton and Development. CSCanada, 7(2), -8, 204 [3] Sh, G. Data Mnng and Knowledge Dscovery for Geoscentsts. Elsever Inc., USA, 203 [4] Sh G. Optmal predcton n petroleum geology by regresson and classfcaton methods. Sc J Inf Eng 5(2), 4-32, 205 [5] Mamon, O., Rokach, L. The Data Mnng and Knowledge Dscovery Handbook, 2 nd edton. Sprnger, New York, NY, USA,200 [6] Han, J.W., Kamber, M. Data Mnng: Concepts and Technques, 2 nd edton. Morgan Kaufmann, San Francsco, CA, USA, 2006 [7] Kang, Z., Guo, C., Wu, W. Technque of dynamc descrptons to the crack and cave carbonate rock reservor n the Tahe ol feld, Xnjang, Chna. Journal of Chengdu Unversty of Technology (Scence & Technology Edton), 34 (2), 43-46, 2007(n Chnese) [8] Chang, C., Ln, C. LIBSVM: a lbrary for support vector machnes, Verson 3.. Retreved from ~cjln/lbsvm/, 20 AUTHORS Dawe L s a senor engneer of PetroChna, born n Hebe Provnce, Chna, on May 5th, 969. He got doctoral degree of petroleum geology from Chna Unversty of Geoscences, Bejng, Chna n 996. Snce then, he has worked as an engneer of geophyscs, petroleum geology and IT n dfferent departments (such as Bureau of Geophyscal Prospectng, Research Insttute of Petroleum Exploraton and Development) of PetroChna. He has publshed several books and more than 40 artcles. Two of the major publshed books are Tectonc types of Ol and Gas Basns n Chna (Bejng: Petroleum Industry Press, 2003) and Neotectonsm and Hydrocarbon Accumulaton n Huanghua Depresson, Chna (n Chnese) (Wuhan, Hube Provnce: Chna Unversty of Geoscences Press, 2006). Two of hs mportant publshed artcles are: Ten crtcal factors for the evaluaton of enterprse nformatzaton benefts (n Chnese, 204), Dscusson on nformatzaton n PetroChna (n Chnese, 2009). Hs present job s to buld and manage PUP IM and DM systems for PetroChna. Dr. L s a member of Chna Geologcal Socety and Chna Petroleum Socety. 2 Guangren Sh s a professor of PetroChna, born n Shangha, Chna n February, 940. Hs research contans two felds of basn modelng (petroleum system) and data mnng for geoscences. He has publshed 3 books and about 2 artcles on data mnng