Probabilistic Graphical Models for Climate Data Analysis

Transcription

1 Probablstc Graphcal Models for Clmate Data Analyss Arndam Banerjee Dept of Computer Scence & Engneerng Unversty of Mnnesota, Twn Ctes Aug 15, 013

2 Clmate Data Analyss Key Challenges Hgh-dmensonal dependent data, small sample sze Spatal and temporal dependences, temporal lags Oscllatons wth frequency and phase varatons Important varables are unrelable, e.g., precptaton Several others: Nonlnearty, heavy tals, Potental Opportuntes Mult-model ensembles: Regonal sklls vs global performance Statstcal Downscalng: Coarse to fne scale, capture dependences Understandng tals: Extreme precptaton, mega-droughts, heat waves, etc. Understandng dependences: Statstcal dependences, not correlaton Several others: Predctve modelng, uncertanty quantfcaton, Source: Overpeck et al., Scence, (011)

3 Graphcal Models Graphcal models Dependences between (random) varables, avod I.I.D. assumptons Closer to realty, learnng/nference s much more dffcult Basc nomenclature Node = Random Varable, Edge = Statstcal Dependency Drected Graphs A drected graph between random varables Example: Bayesan networks, Hdden Markov Models Jont dstrbuton s a product of P(chld parents) Undrected Graphs An undrected graph between random varables Example: Markov/Condtonal random felds Jont dstrbuton n terms of potental functons X 1 X 3 X X 4 X 5

4 Graphcal Models: Key Problems Structure Learnng o Gven: Samples o Problem: Learn the Structure Parameter Estmaton o Gven: Samples and Structure o Problem: Estmate Parameters Inference o Gven: Structure, Parameters, and some varables (part of a Sample) o Problem: Fnd other varables (part of a Sample)

5 Global Clmate Models (GCMs) Source: SCIDAC Source: UCAR/NCAR 5

6 Combnng GCM Outputs Several ways to combne the model outputs Average: Equal weghtage to all models (IPCC AR4 007, Refen and Toum 009) Superensemble: Least Squares (Krshnamurt et al., 00) REA: Relablty based ensemble averagng (Gorg et al., 00) Bayesan: Probablstc estmates of clmate varables (Tebald et al., 005, Smth et al., 011) Onlne Learnng: Trackng clmate models (Monteleon et al., 011) Our work Hypothess: Certan models do well n certan clmatc condtons Goal: Clmate model combnaton Dfferent weghs at dfferent locatons Smlar clmatc condtons should get smlar weghts Bulds on superensemble and probablstc approaches 6

7 Models Smooth Model Combnaton (SMC) X Perod 1 Perod Perod T mn 1 X y j Locatons GCM output y X mn 1 X mn 1 V Pror to ensure smoothness y θ j ~ N 0, A 1 Sparse precson matrx A = Σ -1 p(θ j A -1 ) exp(- θ j T A θ j ) X a j y y j Tr( A T 7 )

8 SMC: Error Term mn 1 V X y Tr( L T ) Error term updates locally

9 SMC: Smoothness Term mn 1 V X y Tr( L T ) Smoothness term updates based on neghborhoods

10 SMC: Graphcal Model Perspectve Generatve Model Pror on rows: θ j ~ N(0,A -1 ) Condtonal: y ~ N(X θ, σ ) A θ j M Precson matrx specfcaton Gaussan Markov random feld (GMRF) Precson A = L = D W, the dscrete graph Laplacan Intrnsc Condtonally Autoregressve Model (ICAR) Spatal statstcs lterature (Dggle et al., 1998, Besag et al., 1995, Banerjee et al., 004, Rue et al., 005) Estmaton of precson matrx Estmate whch locatons are smlar Estmated precson s full rank but sparse y X L 10

11 Data Set and Methodology GCM output: Monthly average surface temperature Target varable: Temperature from Clmatc Research Unt (CRU) Error/accuracy measures: RMSE and MAE Smoothness measures: Kendall τ Spearman ρ Sorted order of the regresson coeffcents GCM1 GCM GCM3 GCM4 j GCM GCM1 GCM3 GCM4 11

12 Spatal Error Profle: AVE vs SMC AVE SMC SMC has lower compared to AVE: lower by ~ 0.5⁰C Errors (vsbly) reduced n many regons Afrca, Greenland, Southeast Asa, Sbera Hgh errors n some regons Northern Europe/Russa, Chna/Tbet, West South Amerca, North Amerca 1

13 Error vs Smoothness SMC 13

14 Mega-Droughts Mega-Droughts Persstent over space and tme Catastrophc consequences Examples Late 1906s Sahel drought 1930s North Amercan Dust Bowl Dscrete Markov Random Feld (MRF) Each node x s wet or dry Observatons: Precptaton Smoothness n space and tme Most lkely state assgnments Each (lat,long,tme) gets wet or dry Advanced analyss x 0 x 0 x 0 x 0 x 5 x 5 x 5 x 5 x 4 x 4 x 4 x 4 Sol mosture, hydrology/watershed models Multple states based on severty, e.g., lower quantles x 6 x 6 x 6 x 6 x 3 x 3 x 3 x 3 x 1 x 1 x 1 x 1 x 7 x 7 x 7 x 7 x x x

15 Results: Droughts startng n s Drought n central Canada n the 190s Drought n Eastern Chna n the 190s The Dustbowl n the 1930s Drought n northwest Amerca n the 190s Drought n southern Afrca n the 190s

16 Results: Droughts startng n s The prolonged drought n Sahel n the 1970s Drought n Inda and Bangladesh n the 1960s

17 Major Droughts:

18 Learnng dependences x 5 x 0 x 6 x 7 x 1 Dependences n graphcal models a j = 0 x x j x, j Example: x 0 x 5 x 1,x,x 3,x 4,x 6,x 7 Condtonal ndependence x 4 a 34 x Gaussan model x ~ N(0,Σ) Precson A = Σ -1 s sparse x 3 p x 0, A 1 exp a j x xj y x 0 x 1,j a j = 0 x xj x, j x 5 x 6 x 7 x Estmatng dependency structure One-vs-rest sparse regresson Lasso for multvarate Gaussans n x 4 y h β T x h + λ n β 1 x 3 h=1 y = x 0, z = (x 1,, x 7 ) 18

19 Sparse Regresson, Structure Learnng x 0 x 1 x 6 x 5 x 7 x x 4 x 3 Source: Delsole et al., 00, 006, 009 Gaussan Copula: (f 1 (x 1 ),,f (x ),,f p (x p )) ~ N(0, Σ), precson A = Σ -1 Not (x 1,,x,,x p ) ~ N(0, Σ), f are monotonc transformatons Sparse A can be consstently estmated (H. Lu et al., 01) Lnear Programmng (LP) based estmator (CLIME) (T. Ca et al., 011) LP estmator scales to hgh-dmensonal copulas (Our work) Mllons of varables, trllons of edges 19

20 References K. Subban and A. Banerjee, Clmate Mult-model Regresson Usng Spatal Smoothng, SIAM Data Mnng (SDM), 013. [Best Applcaton Paper Award] Q. Fu, A. Banerjee, S. Less, and P. Snyder. Drought Detecton for the Last Century: A MRF-based Approach, SIAM Data Mnng (SDM), 01. Q. Fu, H. Wang, and A. Banerjee, Bethe-ADMM for Tree Decomposton based Parallel MAP nference, Uncertanty n Artfcal Intellgence (UAI), 013. C. Jn, Q. Fu, H. Wang, A. Agrawal, W. Hendrx, W.-K. Lao, M. A. Patwary, A. Banerjee, and A. Choudhary, Solvng Combnatoral Optmzaton Problems usng Relaxed Lnear Programmng: A Hgh Performance Computng Perspectve, BgMne workshop (KDD), 013. [Best Paper Award] C. Hseh, I. Dhllon, P. Ravkumar, A. Banerjee, A Dvde-and-Conquer Method for Sparse Inverse Covarance Estmaton, Advances n Neural Informaton Processng Systems (NIPS), 01. S. Chatterjee, K. Stenhaeuser, A. Banerjee, S. Chatterjee, and A. Ganguly, Sparse Group Lasso: Consstency and Clmate Applcatons, SIAM Data Mnng (SDM), 01. [Best Student Paper Award]