ENVIRONMENTAL DATA MAPPING

Size: px

Start display at page:

Download "ENVIRONMENTAL DATA MAPPING"

Nicholas Frederick Walters
6 years ago
Views:

1 ENVIRONMENTAL DATA MAPPING WITH SUPPORT VECTOR REGRESSION AND GEOSTATISTICS Mhal Kaevs Patrc Wog Stephae Cau IDIAP-RR-00-0 Jue 000 Submtted to ICONIP 000 Uversty of New South Wales, Sydey, NSW 05, Australa, INSA; Place Emle Blodel, 763 Mot-Sat-Aga, Frace

3 Evrometal Data Mappg wth Suppo rt Vector Regresso ad Geostatstcs M. Kaevs IDIAP Dalle Molle Isttute of Perceptual Artfcal Itellgece CP 59, 90 Martgy, Swtzerlad P.M. Wog Petroleum Egeerg U. of New South Wales Sydey, NSW 05, Australa S. Cau Isttut Natoal des Sceces Applquées Place Emle Blodel, B.P Mot-Sat-Aga Cédex, Frace Abstract The paper presets decso-oreted mappg of polluto usg hybrd models based o statstcal learg theory (support vector regresso or SVR) ad spatal statstcs (geostatstcs). Adaptve ad robust SVR approach s used to model o-lear large scale treds the rego ad geostatstcal models spatal predctos ad spatal smulatos are used to prepare decsooreted maps: predcto maps alog wth maps of error varace ad equprobable dgtal models of the polluto based o codtoal stochastc smulatos. The qualty of the proposed approach s tested wth the valdato data set ot used for the model developmet. Real data o sol cotamato by Cherobyl radoucldes Russa s used as a case study.. Itroducto I ths paper, the deas of hybrd models based o mache learg (data-drve) approach ad geostatstcs frstly preseted (Kaevs et al., 996) are developed usg recet developmets statstcal learg theory (Vap, 998) support vector regresso ad geostatstcs. The model s appled to real data o sol cotamato by Cherobyl radoucldes the most cotamated rego of Russa (Bras rego). Exploratory data aalyss ad especally exploratory varography ad varogram modellg are wdely used for better uderstadg of data ad the results.. Support Vector Regresso The frst publcato o the adaptato of support vector regresso (SVR) to spatal data ca be foud at Kaevs ad Cau (000). Detaled explaatos o SVM ca be foud a umber of recet publcatos (e.g. Chrsta ad Shave-Taylor, 000). I the preset paper oly basc equatos are preseted. Let us assumg f s a predcto fucto (.e. a fucto used to predct the value of Z owg the geographcal co-ordates (x,. We defe the cost of choosg ths partcular fucto for a gve decso process. Frst, for a gve observato (x,y,z), we defe the ε-sestve cost fucto: f ( x, Z ε f f ( x, Z > ε C{( x,, Z, ε, f } 0 otherwse where ε characterzes some acceptable error. Now, for all possble observatos, we defe the global or geeralsato error also ow as the tegrated predcto error IPE: IPE( f ) E ( C(( x, y,) Z, ε, f )) ω( x, dxdy Z where ω(x, s some ecoomcal measure, dcatg the relatve mportace of a mstae at pot (x,. Usually ω(x,, so that all postos are assumed to be equally mportat. Now we are gog to defe where to loo for the soluto of the problem of mmsg the tegrated predcto error: w m f ( x, w ϕ ( x, + ω The complexty of the soluto ca be tued through m w (Vap, 998). Thus, a relevat strategy to mmse IPE s to mmse the emprcal error together wth matag w small. Ths ca be obtaed by mmsg the followg cost fucto: mmze w subect to f ( x, y ) - Z ε, for,..., But, ufortuately, some data may le outsde of ths epslo tube due to ose or outlers mag these costrats too strog ad mpossble to fulfl. I ths case Vap suggests to troduce so called slac varables ( ξ, ξ ). These varables measure the dstace betwee the observato ad the ε tube. The dstace betwee the 0

4 observato ad the ε ad ( ξ, ξ ) s llustrated by the followg example: mage you have a great cofdece your measuremet process, but the varace of the measured pheomea s large. I ths case, ε has to be chose a pror very small whle the slac varables ( ξ, ξ ) are optmsed ad thus ca be large. Remember that sde the epslo tube [ f ( x, ε, f ( x, + ε ] cost fucto s zero. Note that by troducg the couple ( ξ, ξ ) the problem has ow uow varables. But these varables are led sce oe of the two values s ecessary equals to zero. Ether the slac s postve ( ξ 0) or egatve ( ξ 0). Thus: Z [ f ( x ε ξ, f ( x, + ε + ξ ], Now, we are loog for a soluto mmsg at the same tme ts complexty (measured by w ) ad ts predcto error (represeted by max ( ξ, ξ ) ξ + ξ ). I ths case, let us troduce a user specfed trade off parameter C betwee these two cotradctory obectves. That leads us to the followg problem: mmse ω + C ( ξ + ξ ) w η η ( α α ) ϕ ( x, y ) C α C α for,..., for,..., for,...m These varables ca be removed from the orgal formulato of the mmsato problem to get the dual formulato of the problem: maxmse- subect to ( α ) α m ε ( α + α) + ( α α) K 0 α, α C ϕ ( x, y ) ϕ ( x, y ) ( α α ) Z ( α α ) ( x, y ) 0 for K,..., m for,..., Ths problem s utractable because of fuctos ϕ. Now we are gog to solve the optmzato problem wthout specfyg fuctos ϕ. To do so t s ecessary to choose ϕ such that: m ϕ ( x, y ) ϕ ( x, y ) G ( x, y ),( x, y )) subect to f ( x, y ) Z ε ξ f ( x, y ) + Z ε ξ ξ, ξ 0 for,..., Ths s the case reproducg erel Hlbert space, where G s the reproducg erel. Fuctos ϕ are the ege fuctos of G. I ths case the soluto ca be formulated the followg form: A classcal way to reformulate a costrat based mmsato problem s to loo for the saddle pot of Lagraga L: L( w, ξ, ξ α) w + C( ξ + ξ ) α ( α ( f ( x, y ) Z + ε + ξ ) ( Z f ( x, y ) + ε + ξ ) ηξ + η ξ ) where α, α, η, η are Lagraga multplers assocated wth the costrats. They ca be roughly terpreted as a measure of the fluece of the costrats the soluto. A soluto wth α α 0 ca be terpreted as the correspodg data pot has o fluece o ths soluto. At the mmum the dervatve of the Lagraga equals to zero (Kuh-Tacer codtos): f ( x, v G(( x,,( x, y )) + b wth v ( α α ). Note that fucto ϕ has dsappeared. Ths soluto oly depeds o the erel fucto G. Note also that here at least oe of alphas s equalled to zero depedg of the observed value Z, above or uder the ε-tube. The ma dffculty of ths QP problem les ts dmeso. For 000 data pots the problem to be solved s of dmeso 000 that maes t tractable for most of the commercal optmsato software. Equalty costrats are ot too complex sce they are very few. Box costrats are also rather smple but there are may of them (4). Ths suggests to use a specfc algorthm tag to accout the specfcty of the box costrats.

5 A typcal practcal choce for the erel s the Gaussa Kerel: ( x x ) + ( y y ) G(( x,,( x, y )) exp σ where σ deotes the badwdth of the erel. The hyperparameters of the SVR ca be tued usg splttg of the orgal data to trag, testg ad valdato sets. 3. Ordary Krgg Detals o the geostatstcal spatal predctos ca be foud a umber of recet boos (e.g. Goovaerts, 997; Deutsch ad Jourel, 997). I the preset study, so-called ordary rgg model (vald uder the hypotheses of secod order statoarty ad trsc radom fucto) s used for the spatal predctos of the resduals. 4. Case study Case study s based o a real data o sol cotamato by Cherobyl radoucldes. Data demostrates varablty at several spatal scales (spatally o-statoary data). I ths case tradtoal geostatstcal models based o a hypothess of secod order statoarty or trsc hypothess ca ot be used drectly. I the preset research hybrd models based o SVR adaptve modelg of large scale treds ad aalyss ad modelg of the resduals wth geostatstcal model (ordary rgg) s appled. The approach follows the deas preseted (Kaevs et al., 996) where artfcal eural etwors were used for the large scale de-tredg. The SVRRK/SVRRSIMM Support Vector Regresso Resdual Krgg/Support Vector Regresso Resdual Smulatos - models follow the deas of the NNRK approach ad cosst of several ma phases: Exploratory data aalyss, Tred aalyss, Exploratory varography ad modelg, Semvarogram/varogram s the basc tool of the spatal structural aalyss ad varography. Theoretcal formula (uder the trsc hypotheses, Deutsch ad Jourel 997) { } ( ) Emprcal estmate of the varogram (expermetal varogram) s followg here N(h) s a umber of pars separated b y a vector h. { } γ(,) xh Var Z( x) Z( x+ h) E Z( x) Z( x+ h) γ( h) N ( h ) γ ( h) + N ( h) ( Z( x) Z( x h) ) SVR tred modelg (de-tredg), Comprehesve aalyss of the resduals, Exploratory varography ad modellg of the resduals, Valdato of the results ad fal predctos. The ma outputs of the SVRRK model are preseted below. SVR detredg Let us preset the results of large scale modellg usg Support Vector Regresso approach. I order to model large scale tred the rego, erel badwdth of the SVR was selected equal to the scale of the rego. The results are preseted Fgure. Fgure. Support Vector Regresso tred modelg. Fgure. Omdrectoal varograms: trag data, tred model, SVR resduals. The geeral spatal correlato structures ca be uderstood from the omdrectoal varograms, preseted Fgure. Raw trag data represet varablty at dfferet scales. Varogram of tred model represets smooth fucto behavour, ad the varogram of the resduals represets small scale varablty (secod order statoary radom fucto). Geostatstcal modelg of the resduals I order to use geostatstcal models asotropc

measures of spatal cotuty (varograms) have to be aalyzed ad modeled (Goovaerts ). Drectoal varogram modelg of the resduals usg Geostat Offce software (Kaevs et al 999b) s preseted Fgure.

The results of the resduals modelg wth ordary rgg are preseted Fgure 3. Fgure 5. Support Vector Regresso resdual Krgg Model Mappg.

6 measures of spatal cotuty (varograms) have to be aalyzed ad modeled (Goovaerts ). Drectoal varogram modelg of the resduals usg Geostat Offce software (Kaevs et al 999b) s preseted Fgure. Predcto mappg wth SVRRK model The results of the Support Vector Mache Resdual Krgg Model cosst of tred modelg wth SVR ad resdual predctos wth ordary rgg. The results of the resduals modelg wth ordary rgg are preseted Fgure 3. Fgure 5. Support Vector Regresso resdual Krgg Model Mappg. Valdato of SVRRK model The model was used to predct depedet (ot used for trag ad tug of the parameters) valdato data set. The results are preseted Fgure 7. Fgure 3. Varogram modellg of the SVR resduals. Fgure 6. Valdato of the SVRRK model wth depedet valdato data set. Fgure 4. Mappg of SVR resduals. Small scale structures ca be recogzed. The ma result s preseted Fgure 4. Let us remd that SVRRK model s a exact model: at the measuremet pots outputs of the model equals to the measuremet data. Support Vector Regresso Resduals Smulato Model All regresso models, by defto, gve some averages of data ad do ot represet spatal varablty ad ucertaty. Usually, reproducg of varablty s a target of smulato models. I case of spatal data there have bee developed several approaches ad models for the codtoal (to the orgal data) spatal smulatos preservg basc spatal statstcal characterstcs: hstograms of declustered data, varograms (see detals Goovaerts 997): sequetal gaussa smulatos, dcator smulatos, smulated aealg, etc. Most of the smulato models are based o so-called secod order statoary radom fuctos, whe codtoal mea value the rego s costat ad spatal covarace fucto depeds oly o the separato vector betwee pots. Oe of the possblty how to avod problems of secod order statoarty s based o the same deas as SVRRK model. Treds (large scale structures) are modeled wth SVR ad spatally correlated resduals are

7 used for smulatos. Smulatos dffers from ay terpolato model. There are two maor dffereces betwee estmatos ad smulatos: The ma obectves of the terpolators are to provde best local estmates z(u) of each usampled value z(u) wthout specfc regard to the resultg spatal statstcs of the estmates. I case of smulatos the resultg global features ad statstcs (the same frst two expermetally foud momets - mea ad covarace or varogram, as well as the hstogram) of the smulated values tae precedece over local accuracy. Stochastc smulato s a process of preparg alteratve, equally probable, hgh resoluto models of the spatal dstrbuto of z(u). The varable ca be categorcal, dcatg presece or absece of a partcular characterstc, or t ca be cotuous. Krgg, for example, provdes a sgle umercal model, whch s the best some local sese. Smulatos provde may alteratve umercal models z l (u), each of whch s a good represetato of the realty some global sese. The dfferece betwee these alteratve models or realzatos provdes a measure of ot spatal ucertaty. I geeral, the obectves of smulato ad estmato are ot compatble. Smulatos reproduce spatal varablty ad ca tae to accout dfferet sources of formato (data tegrato). I the preset paper sequetal gaussa smulatos are cosdered. Gaussa radom fucto models are wdely used statstcs ad smulatos due to ther aalytcal smplcty, they are well uderstood, they are a lmt dstrbutos of may theoretcal results ad were successfully appled may cases. I ths wor we shall use algorthm ow as a Sequetal Gaussa Smulatos. Sequetal Gaussa smulato methodology cossts of several steps:. Determe the uvarate cdf (cumulatve dstrbuto fucto) F Z(z) represetatve of the etre study area ad ot oly of the z-sample data avalable. Declusterg may be eeded.. Usg the cdf F Z(z), perform the ormal score trasform of z-data to y-data wth a stadard ormal cdf. 3. Chec for bvaraty ormalty of the ormal score y- data. 4. If a multvarate Gaussa ormalty radom fucto model ca be adopted for the y-varable, local codtoal dstrbuto s ormal wth mea ad varace obtaed by smple rgg. The statoarty requres that smple rgg (SK) wth zero mea should be used. If there are eough codtog data to cosder ferece of a ostatoary radom fucto model t s possble to use mowg wdow estmatos wth ordary rgg (OK) wth the re-estmato of the mea. But ay case SK varace should be used for the varace of the Gaussa codtoal cumulatve dstrbuto fucto f there are eough codtog data t mght be possble to eep the tred as t s. 5. Start wth sequetal Gaussa smulatos: Defe a radom path, that vsts each ode of the grd (ot ecessarly regular) oce. At each ode u, reta a specfed umber of eghborg codtog data cludg both orgal y-data ad prevously smulated grd ode y-values. Use smple rgg wth the ormal score varogram model to determe the parameters (mea ad varace) of the ccdf (codtoal cumulatve dstrbuto fucto) of the radom fucto Y(u) at locato u. Draw a smulated value y l (u) from that ccdf. Add the smulated value y l (u) to the data set. Proceed to the ext ode, ad loop utl all odes are smulated. 6. Bac trasform the smulated ormal values y l (u) to smulated values for the orgal varable z l (u). A mportat phase of sequetal gaussa smulatos deals wth varography of ormal score values (trasformato form orgal data to uvarate gaussa dstrbuto N(0,). Some results of the sequetal gaussa codtoal smulatos are preseted Fgures 7-9. These fgures are much varable space. The smlarty ad dssmlarty betwee dgtal models of the realty descrbes spatal varablty ad ucertaty. The ext step deals wth the probablstc mappg: mappg to be Above some predefed decso level. Ths s a topc of aother research related to decso oreted mappg of cotamated terrtores. Fgure 7. Codtoal Sequetal Gaussa Smulatos. Realzato #.

Fgure 8. Codtoal Sequetal Gaussa Smulatos. Realzato #3. Fgure 9. Codtoal Sequetal Gaussa Smulatos. Realzato # 76. Usually hudreds of smulated models (realzatos) are geerated.

8 Fgure 8. Codtoal Sequetal Gaussa Smulatos. Realzato #3. Fgure 9. Codtoal Sequetal Gaussa Smulatos. Realzato # 76. Usually hudreds of smulated models (realzatos) are geerated. The smlarty ad dssmlarty betwee dfferet equprobable realzatos of the realty (usg data ad avalable owledge) descrbes spatal varablty ad ucertaty of data. By developg may of equprobable realzatos probablstc/rs mappg s possble as well: mappg of probablty to be above/below some predefed decso/regulato levels. Detaled descrpto of SVRRSm models ad ther applcato to the decso oreted mappg s uder study ad wll be publshed elsewhere. 5. Cocluso The results of spatal data mappg wth hybrd model SVRRK are promsg. Applcato of Support Vector Regresso data drve ad robust approach allowed to develop o-lear large scale model (tred model) the rego uder study. The remag resduals descrbg small scale varablty of polluto were effcetly modelled usg ordary rgg model of geostatstcs. There s a mutual relatoshp betwee models SVRRK: from oe sde geostatstcal approach help to uderstad how much spatally structured formato descrbed by varograms was extracted from data by SVR, ad from aother sde SVR ca be used as a effcet tool for spatal data detredg case of spatally o statoary data. Applcato of the codtoal stochastc smulato models for the SVR resduals s uder study. The results of codtoal smulatos seems to be promsg. After SVR detredg score trasformato of the resduals demostrates that secod order statoary model ca be accepted. I cocluso, whe worg wth spatally dstrbuted data, self-cosstet hybrd models usg mache learg algorthms ad geostatstcs ca brg mutual beeft for the both data drve ad model depedet approaches. Acowledgmets The research was supported part by Europea INTAS grats 376 ad Refereces Deutsch C.V. ad A.G. Jourel. GSLIB: Geostatstcal Software Lbrary ad User s Gude. Oxford Uversty Press, New Yor, 997. Goovaerts P. Geostatstcs for Natural Resources Evaluato. Oxford Uversty Press, New Yor, 997. Kaevs M., Arutyuya R., Bolshov L., Demyaov V., Maga M. Artfcal eural etwors ad spatal estmatos of Cherobyl fallout. Geoformatcs. Vol.7, No.-, 996, pp.5-. Kaevs M., N. Glard, M. Maga, E. Mayoraz. Evrometal Spatal Data Classfcato wth Support Vector Maches. IDIAP Research Report. IDIAP-RR-99-07, 4 p., 999a. ( Kaevs M, V. Demyaov, S. Cherov, E. Saveleva, A. Serov, V. Tmo, M. Maga. Geostat Offce for Evrometal ad Polluto Spatal Data Aalyss. Mathematsche Geologe, N3, Aprl 999b, pp Kaevs M., S. Cau. Evrometal ad Polluto Data Mappg wth Support Vector Regresso. IDIAP Research Report RR Smola A.J., ad B. Scholopf. A Tutoral o Support Vector Regresso. NeuroColt techcal Reports Seres, NC-TR , October 998. Vap V. Statstcal Learg Theory. Joh Wley & Sos, 998.

Summary of the lecture in Biostatistics

Summary of the lecture in Biostatistics Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the