Correcting biased observation model error in data assimilation

Transcription

1 Correcting biased observation model error in data assimilation Tyrus Berry Dept. of Mathematical Sciences, GMU PSU-UMD DA Workshop June 27, 217 Joint work with John Harlim, PSU

2 BIAS IN OBSERVATION MODELS Consider the standard filtering problem, x i = f (x i 1 ) + ω i 1 y i = h(x i ) + η i We assume the true observation function h(x) is unknown An approximate model is available h(x) so that y i = h(x i ) + η i = h(x i ) + b i + η i Where b i h(x i ) h(x i ) is called the bias

3 EXAMPLE 1: LORENZ-96 Consider the standard 4-dimensional Lorenz-96, ẋ j = x j 1 (x j+1 x j 2 ) x j + 8 We observe 2 of the 4 variables We draw ξ i U(, 1) and let the observations be, { xk ξ h(x k ) = i >.8 β k x k 8 else β k N (.5, 1/5). h is applied to 7 randomly chosen variables Remaining 13 are directly observed

4 EXAMPLE 1: LORENZ-96 The result is a bimodal distribution, cloudy/clear Obs Model Error = True Obs - h(true State)

5 CORRECTING THE BIAS Our goal is to find p(b i y i ) We can then adjust the filter by defining a new innovation Where ˆb i = E p(bi y i )[b i ] ˆɛ i = ɛ i + ˆb i = y i h(x f i ) + ˆb i We also inflate the obs covariance by R i = R o + ˆR bi Where ˆR bi = E p(bi y i )[(b i ˆb i )(b i ˆb i ) ]

6 CORRECTING THE BIAS If we can estimate p(b i y i ) we can fix the obs We will use Bayes to find p(b i y i ) = p(b i )p(y i b i ) We will use a simple prior p(b i ) = N (ɛ i, P y i ) ɛi = y i h(xi f ) is the innovation is the innovation covariance estimate P y i The real challenge is to estimate p(y i b i ) We will learn p(y i b i ) from training data using the kernel estimation of conditional distributions

7 CORRECTING THE BIAS Below plots have y i 4 Left is clear, right is cloudy Notice bimodal likelihood Probability Prior, p(bl) Likelihood, p(yi bl) Posterior, p(bl yi) Bias Prior, ˆbi Bias Posterior, ˆµbi True Model Error, bi Probability Prior, p(bl) Likelihood, p(yi bl) Posterior, p(bl yi) Bias Prior, ˆbi Bias Posterior, ˆµbi True Model Error, bi Obs Model Error Obs Model Error

8 LEARNING THE CONDITIONAL DISTRIBUTION Given training data (y i, b i ) our goal is to learn p(y i b i ) For a kernel K (α, β) = e α β 2 δ 2 we define Hilbert spaces { N } { N } H y = a i K (y i, ) : a R N, H b = a i K (b i, ) : a R N i=1 i=1 For example the kernel density estimate (KDE) ˆq is in H y ˆq(y) = 1 m N N K (y i, y) Eigenvectors φ l of K ij = K (y i, y j ) form an orthonormal basis for H y. Similarly ϕ k are a basis for H b. i=1

9 LEARNING THE CONDITIONAL DISTRIBUTION We assume that p(y b) can be approximated in H y H b Let C yb ij = φ i, ϕ j and C bb ij = ϕ i, ϕ j then define ( ) 1 C y b = C yb C bb + λi We can then define a consistent estimator of p(y b) by ˆp(y b) = N i,j=1 C y b i,j φ i (y)ϕ j (b)ˆq(y) We define eigenfunctions with Nystöm extension ϕ j (b) = λ 1 j N ϕ j (b i )K (b i, b) i=1

10 OVERVIEW Learning Phase: Given training data set (x i, y i ) Compute the biases b i = y i h(x i ) Learn the conditional distribution p(y b) Filtering: Forecast x f i Use prior p(b) = N (ɛ i, P y i ) innovation ɛ i = y i h(x f i ) Combine with conditional to find p(b y i ) = p(b)p(y i b) Estimate conditional mean ˆb i and covariance ˆR bi Adjust innovation ˆɛ i = ɛ i + ˆb i and R i = R o + ˆR bi Apply Kalman update, continue to the next filter step

11 OVERVIEW Primary Filter Prior Posterior p(x i ) p(x i y i ) Secondary Filter Error Prior Error Posterior p(b) p(b y i ) +Training Data Observation Likelihood y i p(y i b)

12 LORENZ-96 RESULTS Obs True State EnKF Time -1

13 LORENZ-96 RESULTS Works well with small measurement noise Observations need to be precise, but not accurate EnKF, Unobstructed Obs EnKF, Obstructed Obs, Obstructed Obs EnKF, Unobstructed Obs EnKF, Obstructed Obs, Obstructed Obs RMSE RMSE Measurement Noise Standard Devation, R o Observation Time

14 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Consider a 7-dim l model for a column of atmosphere Baroclinic anomaly potential temperatures, θ 1 and θ 2 Boundary layer anomaly potential temperature, θ eb Vertically averaged water vapor content, q Cloud fractions: congestus f c, deep f d, and stratiform f s Extrapolate anomaly potential temperature at height z T (z) = θ 1 sin( zπ Z T ) + 2θ 2 sin( 2zπ Z T ), z [, 16] Khouider, B., J. Biello, and A. J. Majda, 21: A stochastic multicloud model for tropical convection.

15 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Extrapolate anomaly potential temperature at height z T (z) = θ 1 sin( zπ Z T ) + 2θ 2 sin( 2zπ Z T ), z [, 16] Brightness temperature-like quantity at wavenumber-ν [ h ν (x, f ) = (1 f d f s ) (1 f c ) ( θ eb T ν () + + f c T (z c )T ν (z c ) + zd z c + (f d + f s )T (z d )T ν (z d ) + zc T (z) T ] ν z (z) dz Setting f = is the clear sky model z d T (z) T ν (z) dz, z T (z) T ν (z) dz) z (1)

16 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Weighting functions define RTM at different wavenumbers height (z) weighting function ( Tν z )

17 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Biases at the 16 observed wavenumbers

18 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Multimodal likelihood functions Probability Prior, p(bl) Likelihood, p(yi =.184 bl) Posterior, p(bl yi =.184) True Model Error, bi Obs Model Error Probability Prior, p(bl) Likelihood, p(yi =.195 bl) Posterior, p(bl yi =.195) True Model Error, bi Obs Model Error

19 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Truth Truth θ Time Truth θ Time Truth θ eb q Time Time

20 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS Truth fc fd Time.2 Truth fs Truth Time Time

21 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS θ 1, MSE (percent of variance) θ eb, MSE (percent of variance) Measurement Noise (percent of obs variance) Measurement Noise (percent of obs variance) θ 2, MSE (percent of variance) q, MSE (percent of variance) Measurement Noise (percent of obs variance) Measurement Noise (percent of obs variance)

22 EXAMPLE 2: MULTI-CLOUD SATELLITE-LIKE OBS fd, MSE (percent of variance) fc, MSE (percent of variance) Measurement Noise (percent of obs variance) Measurement Noise (percent of obs variance) fs, MSE (percent of variance) Measurement Noise (percent of obs variance)