A Gaussian Mixture Model Approach to

Transcription

1 A Gaussian Mixture Model Approach to A Parametric, Feature-Based Method Valliappa 1,2 John Kain 2 1 Cooperative Institute of Mesoscale Meteorological Studies University of Oklahoma 2 Radar Research and Development Division National Severe Storms Laboratory Sep. 2009

2 Traditional methods don t work well Can simply compare the forecast and observed images pixel by pixel, but: 1 Small-scale structure differences lead to large errors 2 Position errors lead to double-penalty 3 Difficult to diagnose errors from MSE, POD/FAR/CSI, Brier score, etc. Four proposed solutions to these issues.

3 1. Neighborhood methods Instead of comparing pixel-by-pixel, evaluate forecasts within a local neighborhood: compute fractional coverage and Brier score for different box sizes (Roberts and Lean, 2008) Addresses only problem of small-scale structure differences.

4 2. Scale decomposition Instead of computing errors for different box sizes, decompose the images into images using wavelets and parcel out the error at different scales and intensity thresholds (Casati et al. 2004) Addresses problems of small-scale structure differences and double penalty.

5 3. Find deformations Find deformations needed to make forecast field resemble observed field and compute error from deformation matrix (Gilleland 2009, Marzban 2009) Addresses only double penalty issue.

6 4. Verify based on objects Identify objects based on thresholding the spatial fields and then associate objects to find object attributes (Davis, 2009) Addresses all three issues, but object identification and assocation can be problematic.

7 What is a GMM? Intuitively: find an optimal way to place Gaussian functions at various points in the image such that the sum of these Gaussians mimics the input gridded field. Original 5 Gaussians

8 Number of Gaussians The accuracy of fit gets better as you increase number of Gaussians. Original 10 Gaussians

9 Diminishing Returns 20 Gaussians 50 Gaussians At some point, the benefits of a parametric model are lost and you might as well just use the pixel values.

10 with GMM The GMM captures the key features in an image (subject to Gaussian approximations). Compare the Gaussians parameters to gain insight into how two images differ: Parameter Meaning Implication µ x,µ y Center point Translation error σx,σ 2 y,σ 2 xy Variance, covariance Rotation, aspect ratio, size π k Amplitude of Gaussian Intensity error

11 Example : Geometric Geometric dataset from [Ahijevych et al., 2009]. Chose 3 Gaussians to demonstrate that exact number is not critical. geom000 geom003

12 Example : Geometric µ x µ y σx 2 σ xy σy 2 π k 0 Original pts right pts right pts right too big

13 Example : Geometric µ x µ y σx 2 σ xy σy 2 π k pts right turned pts right huge

14 Yes, but... Three questions: 1 How does this compare to Method X...? 2 How do you fit a Gaussian Mixture Model to data? 3 How well does it do on real (not synthetic) data? 4 How do you associate Gaussians? Compute error?

15 Advantages of GMM approach 1 Splits, merges happen automatically if needed for optimal fit. 2 The Gaussian is a parametric function: a highly compressed view of the information in the data 3 Number of Gaussians used a good measure of the scale at which the image is being represented. 4 Transformations of Gaussians correspond to easily identifiable changes in their parameters.

16 How the concept compares to... Technique Similarity Difference Filtering Naturally incorporates Compare model scale parameters not pixel values Object Compare features No thresholds, split/merge problems, etc.

17 How the concept compares to... Wavelets Wavelets provide multiresolution i.e. images at different scales; GMM provides objects at different scales Field deformation Optical flow approaches are non-parametric; you get an answer at each grid point.

18 Why consider a GMM approach? Multi-scale like filtering-based methods Transformation-detecting like object-based methods Simple to implement Mathematically elegant

19 How to fit a GMM 1 Initialize the GMM. 2 Carry out Expectation-Minimization (EM) algorithm to iteratively tune the GMM. 3 Store the parameters of each Gaussian component of the GMM.

20 The GMM The GMM is defined as a weighted sum of K two-dimensional Gaussians: 1 f (x, y) = 2π Σ xy Σ xy is: G(x, y) = K π k f k (x, y) (1) k=1 1 )(y µy ))Σ e ((x µx xy ((x µ x )(y µ y )) T /2 ( ) σ 2 x σ xy σ xy σ 2 y (2) (3)

21 The Expecation-Minimization (EM) method Given a set of points x i, y i, it is possible to fit these points to a GMM, G(x, y), by following an iterative method known as the expectation-minimization (EM) method. Assume that an initial choice of parameters µ xk, µ yk, Σ xy k exists for each of the K components. P(x i, y i θ) = K π k f k (x i, y i µ xk, µ yk, Σ xy k ) (4) k=1

22 E-M method E-step: M-step: ( P(k x i, y i, θ) = π kf k (x i, y i µ xk, µ yk, Σ xy k ) P(x i, y i θ) N i=1 µ x = E(x) = (P k(x i, y i )x i ) N i=1 P k(x i, y i ) E((x µ x ) 2 ) E((x µ x )(y µ y )) E((x µ x )(y µ y )) E((y µ y ) 2 ) ) (5) (6) (7) π k = 1 N N P k (x i, y i ) (8) i=1

23 The EM method is problematic The EM process has to be bootstrapped with some initial guess at a GMM. The EM process will start at that point and slowly climb towards the local maximum in likelihood space. Only promises a local maximum

24 Initializing the GMM A significant amount of spatial coherence in weather images that we can take advantage of to place the initial mixture components: 1 Group pixels into contiguous regions 2 Randomize the pixels within a region (to remove order-dependence) 3 Arrange regions in order (maybe in order of top-left point, or of centroid, or K-means cluster centroids) 4 Break pixel list into K equal parts 5 If a pixel falls into the kth group, the initial weight is one for the kth component and zero for all other components.

25 Intensity? Recall that the GMM was defined so as to sum to 1, and that the EM method optimized the likelihood of the parameters given the positions of the pixels (and not the intensity). Two minor changes: 1 The total intensity associated with all the pixels in the image is used to scale the GMM 2 More intensive locations are repeated several (m) times: m = 1 + round( CDF(I xy) freq(i mode ) ) I xy < I mode (9)

26 Number of components? Traditional way to estimate number of components: BIC = 2l(θ) 6Klog(N) (10) l(θ) = N log(p(x i, y i )) (11) i=1 Doesn t work: hundreds of components. We just used 3.

27 Perturbed dataset 2km WRF from CAPS perturbed (See [Ahijevych et al., 2009]). fake000 fake003 fake007

28 GMM parameters for perturbed cases µ x µ y σx 2 σ xy σy 2 π k Original pts. right pts. down pts. right pts. down pts. right pts. down

29 GMM parameters for perturbed cases Description µ x µ y σx 2 σ xy σy 2 π k 24 pts. right pts. down pts. right pts. down pts. right pts. down times pts. right pts. down minus 2 mm

30 June 1, 2005 Three quite different systems: Plains, NW, SE 3-member GMM fit does not capture these three events. Instead, NW ignored. As pointed out by [Wernli et al., 2009], it would be advantageous to carry out this analysis on smaller domains where only one type of of meteorological system predominates. Higher order GMM fits do capture all these systems. We chose to use only a 3rd order fit so as to keep the analysis of member parameters tractable.

31 The GMM Observed 2CAPS 4NCAR 4NCEP

32 GMM Parameters µ x µ y σx 2 σ xy σy 2 π k Obs CAPS NCAR NCEP

33 First Gaussian Component 1st Gaussian component corresponds to the Northern Great Plains. µ x µ y σx 2 σ xy σy 2 π k Observed CAPS NCAR NCEP All 3 forecasts displaced to the north and west; 2CAPS is the least displaced. 4NCAR underestimates the precip; 2CAPS overestimates; 4NCEP gets it correct. 2CAPS gets shape wrong; 4NCAR, 4NCEP get north-south extent correct but overestimate east-west extent.

34 Second Gaussian Component Corresponds to Southern Great Plains into Texas. µ x µ y σx 2 σ xy σy 2 π k Obs CAPS NCAR NCEP All forecasts displaced to north; with 2CAPS again the best. Shape: 4NCEP, 4NCAR overly vertical; 2CAPS also wrong orientation, but better. Intensity: 2CAPS is closest; 4NCAR, 4NCEP significant overestimates

35 Third Gaussian Component Corresponds to Southeastern US µ x µ y σx 2 σ xy σy 2 π k Obs CAPS NCAR NCEP All get intensity and orientation correct but displaced to the east; 4NCEP also displaced to north. 4NCEP too large in north-south direction: precipitation even in correct in aggregate is spread over too large an area.

36 Translation error: Rotation error: e tr = (µ xf µ xo ) 2 + (µ yf µ yo ) 2 (12) e rot = 180 π cos 1 (v f.v o ) (13) v f and v o are the maximum-variance eigen vectors of the covariance matrices (Σ) Scaling error: e sc = π k f π ko (14)

37 Associating Gaussians For each Gaussian in the forecast field, compute the overall error with each Gaussian of observed field and pick the one with the lowest error. e = 0.3 min( e tr 100, 1)+ 0.2 min(e rot, 180 e rot )/ (max(e sc, 1/e sc ) 1) This can also be used to rank model forecasts. The weights are subjective. (15)

38 Synthetic images: Rank Description e tr e rot e sc e 1 50 pts. right pts. right pts. right, too big pts. right, wrong orient pts. right, huge Rotation error on circular objects is undefined. Ranking: geom001, geom002, geom004, geom003 and finally geom005.

39 Spring 2009 cases: rank Overall, the rank of the models is CAPS (0.34), NCAR (0.49) and finally NCEP (0.50). At the extremely coarse scale at which the forecasts have been compared, the 2CAPS forecast exhibits the least translation, orientation and scaling errors. If we were to increase the number of Gaussians, it would be possible to perform the comparison at a finer detail.

40 Areas for further exploration 1 Number of components An information criterion more amenable to the forecast verification problem needs to be developed. 2 error For example, an error metric based on size could be defined as: e size = σ xf σ yf σ xo σ yo σ xo σ yo (16)

41 Areas for further exploration 1 Association or Deformation? Matching Gaussian components across images may become hard with more than 3 components. An alternative approach: start the E-M on the forecast field with the GMM that corresponds to the observed field and observe how the GMM components get deformed. 2 Initialization of EM into other algorithms for initializing the EM process may prove beneficial. 3 Low intensity regions should not be ignored. Perhaps break up large spatial areas into smaller areas and then fit GMMs to them...

42 Acknowledgements Funding for this research was provided under NOAA-OU Cooperative Agreement NA17RJ1227. The GMM fitting technique described in this paper has been implemented within the Warning Decision Support System Integrated Information (WDSSII; [ et al., 2007]) as part of the w2smooth process. It is available for download at

43 References Ahijevych, D., Gilleland, E., Brown, B., and Ebert, E. (2009). Application of spatial verification methods to idealized and NWP gridded precipitation forecasts. Weather and ing, 0(0):InPress., V., Smith, T., Stumpf, G. J., and Hondl, K. (2007). The warning decision support system integrated information. Weather and ing, 22(3): Wernli, H., Hofmann, C., and Zimmer, M. (2009). forecast verification methods intercomparison project application of the SAL technique. Weather and ing, 0(2):DOI: /2009WAF