State Space Models for Wind Forecast Correction

Similar documents
A State Space Model for Wind Forecast Correction

A Gaussian state-space model for wind fields in the North-East Atlantic

Hidden Markov Models for precipitation

The analog data assimilation: method, applications and implementation

Lecture 2: From Linear Regression to Kalman Filter and Beyond

EnKF-based particle filters

Gaussian Process Approximations of Stochastic Differential Equations

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Gaussian Process Approximations of Stochastic Differential Equations

Dropout as a Bayesian Approximation: Insights and Applications

Ensemble Kalman Filter

Lagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model. David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC

Condensed Table of Contents for Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control by J. C.

The Kalman Filter ImPr Talk

Data assimilation in high dimensions

Approximate Bayesian computation: an application to weak-lensing peak counts

Sequential Monte Carlo Methods for Bayesian Computation

Combining analog method and ensemble data assimilation: application to the Lorenz-63 chaotic system

An Efficient Ensemble Data Assimilation Approach To Deal With Range Limited Observation

Lecture 6: Bayesian Inference in SDE Models

Smoothers: Types and Benchmarks

Lagrangian Data Assimilation and Its Application to Geophysical Fluid Flows

Data assimilation Schrödinger s perspective

Modelling and forecasting of offshore wind power fluctuations with Markov-Switching models

Correcting biased observation model error in data assimilation

University of Athens School of Physics Atmospheric Modeling and Weather Forecasting Group

Probabilistic Machine Learning

State and Parameter Estimation in Stochastic Dynamical Models

EnKF Review. P.L. Houtekamer 7th EnKF workshop Introduction to the EnKF. Challenges. The ultimate global EnKF algorithm

Data assimilation with and without a model

Autonomous Navigation for Flying Robots

Localization and Correlation in Ensemble Kalman Filters

State-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Financial Econometrics / 49

DATA ASSIMILATION OF VIS / NIR REFLECTANCE INTO THE DETAILED SNOWPACK MODEL SURFEX/ISBA- CROCUS

A data-driven method for improving the correlation estimation in serial ensemble Kalman filter

Bayesian Networks BY: MOHAMAD ALSABBAGH

Convergence of Particle Filtering Method for Nonlinear Estimation of Vortex Dynamics

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

Stochastic Calculus. Kevin Sinclair. August 2, 2016

Introduction to Machine Learning

Objective localization of ensemble covariances: theory and applications

Estimating model evidence using data assimilation

Estimation of positive sum-to-one constrained parameters with ensemble-based Kalman filters: application to an ocean ecosystem model

What do we know about EnKF?

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Forecasting and data assimilation

Probabilistic Graphical Models

Cross-validation methods for quality control, cloud screening, etc.

Superparameterization and Dynamic Stochastic Superresolution (DSS) for Filtering Sparse Geophysical Flows

State-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53

Variational Autoencoder

Data assimilation with and without a model

GL Garrad Hassan Short term power forecasts for large offshore wind turbine arrays

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning

output dimension input dimension Gaussian evidence Gaussian Gaussian evidence evidence from t +1 inputs and outputs at time t x t+2 x t-1 x t+1

Improved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter

Stochastic contraction BACS Workshop Chamonix, January 14, 2008

X t = a t + r t, (7.1)

Learning Relational Kalman Filtering

Asymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½

Relative Merits of 4D-Var and Ensemble Kalman Filter

Introduction to the regression problem. Luca Martino

Lecture 6: Gaussian Mixture Models (GMM)

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Automated Likelihood Based Inference for Stochastic Volatility Models using AD Model Builder. Oxford, November 24th 2008 Hans J.

Nonlinear and non-gaussian state-space modelling by means of hidden Markov models

4. DATA ASSIMILATION FUNDAMENTALS

Comparison of 3D-Var and LETKF in an Atmospheric GCM: SPEEDY

Fundamentals of Data Assimilation

The Ensemble Kalman Filter:

The Ensemble Kalman Filter:

The Use of a Self-Evolving Additive Inflation in the CNMCA Ensemble Data Assimilation System

Seminar: Data Assimilation

Practical Aspects of Ensemble-based Kalman Filters

Fundamentals of Data Assimila1on

Can hybrid-4denvar match hybrid-4dvar?

Modeling and state estimation Examples State estimation Probabilities Bayes filter Particle filter. Modeling. CSC752 Autonomous Robotic Systems

Construction of an Informative Hierarchical Prior Distribution: Application to Electricity Load Forecasting

A new Hierarchical Bayes approach to ensemble-variational data assimilation

A new iterated filtering algorithm

Comparing the SEKF with the DEnKF on a land surface model

Parameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets

EnKF and filter divergence

UNDERSTANDING DATA ASSIMILATION APPLICATIONS TO HIGH-LATITUDE IONOSPHERIC ELECTRODYNAMICS

Ensemble-based Data Assimilation of TRMM/GPM Precipitation Measurements

EM-algorithm for Training of State-space Models with Application to Time Series Prediction

Data assimilation as an optimal control problem and applications to UQ

Variational inference

CPSC 540: Machine Learning

Dynamic linear models (aka state-space models) 1

Nowcasting techniques in use for severe weather operation in NMC/CMA

STA414/2104. Lecture 11: Gaussian Processes. Department of Statistics

Convergence of the Ensemble Kalman Filter in Hilbert Space

Superparameterization and Dynamic Stochastic Superresolution (DSS) for Filtering Sparse Geophysical Flows

The Kalman Filter. Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience. Sarah Dance

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

4DEnVar. Four-Dimensional Ensemble-Variational Data Assimilation. Colloque National sur l'assimilation de données

Forecasting Wind Ramps

Model error and parameter estimation

Environment Canada s Regional Ensemble Kalman Filter

Transcription:

for Wind Forecast Correction Valérie 1 Pierre Ailliot 2 Anne Cuzol 1 1 Université de Bretagne Sud 2 Université de Brest MAS - 2008/28/08

Outline 1 2 Linear Model : an adaptive bias correction Non Linear Model : bias and location correction 3 4

Outline 1 2 Linear Model : an adaptive bias correction Non Linear Model : bias and location correction 3 4

Accuracy of wind forecast Wind Energy Management Security and Rescue Forecast errors : intensity and position Satellite Forecast (same date) 2008/5/24 5h55 60 58 56 54 52 50 48 46 44 42 40 16 14 12 10 8 6 4 60 58 56 54 52 50 48 46 44 42 40 16 14 12 10 8 6 4

Forecast and observations 2008/05/25-00 :00 50 2008/05/25-12 :00 50 49 49 48 48 47 47 46 46 45 353 354 355 356 357 358 359 360 45 353 354 355 356 357 358 359 360

Weather forecast correction (state of art) Local Numerical Weather models Purely stochastic Y t = f(y t 1,, Y t k ; θ) + σɛ t Combined models Regression [Lange et al. (2006), von Bremen et al. (2007)] or Y obs t Y obs t = f(y for t = f(y for t, Z for t ; θ) + σɛ t, Y obs t 1,..., Y obs t k ; θ) + σɛ t Data Assimilation [Dee and da Silva (1998), Galanis et al. (2002)] based on state space models

Outline Linear Model Non Linear Model 1 2 Linear Model : an adaptive bias correction Non Linear Model : bias and location correction 3 4

First step : single location Linear Model Non Linear Model Position error phase error 20 15 10 5 observation forecast 0 02/03 02/04 02/05 42 observed : -, forecast : -+ 10 8 6 4 2 52 51 50 49 48 47 46 45 44 43

A first model Linear Model Non Linear Model Linear Gaussian spate-space model db t = α(b t µ)dt + σdw t Y true t Y obs t Error (hidden) = Y for t + B t "True" Y t (hidden) = Y true t + σ obs ɛ t Observed Y t W t : standard brownian motion, ɛ t Gaussian white noise Y t : observed at time t 1,, t K Inference Kalman filter : weighted mean between the predicted state and the observation

Need another model Linear Model Non Linear Model Location or phase error 2008/5/24 5h55 60 58 56 54 52 50 48 46 44 42 40 16 14 12 10 8 6 4 60 58 56 54 52 50 48 46 44 42 40 16 14 12 10 8 6 4

Introduction of a phase correction Linear Model Non Linear Model Model d t = α ( t µ )dt + σ dv t db t = α B (B t µ B )dt + σ B dw t Y true t Y obs t = Y for t+ t + B t Phase error (hidden) Intensity error (hidden) "True" Y t (hidden) = Y true t + σ W ɛ(t) Observed Y t V t, W t : standard brownian motions, ɛ t Gaussian white noise Y t : observed at time t 1,, t K Inference Parameter estimation : EM algorithm (need smoothing) or maximum likelihood by a gradient algorithm [Robbins and Monroe, 1951] θ k = θ k 1 + γ k θ L T (θ) Monte Carlo approximation of L T (θ) and θ L T (θ) based on particular filtering [Coquelin et al., 2007] Breaks in the data due to the assimilations of the observations in the numerical weather model

Outline 1 2 Linear Model : an adaptive bias correction Non Linear Model : bias and location correction 3 4

Wind forecast correction Example of correction for Brest 20 15 10 Obs For Nonlinear Phase Linear Reg Lin 5 0 02/03 02/04 02/05

Comparison Root Mean Square Errors EQM 9 8 7 6 5 4 Linear Nonlinear Phase Numerical Stochastic Reg Lin 3 2 1 0 0 0.5 1 1.5 2 time (day)

Outline 1 2 Linear Model : an adaptive bias correction Non Linear Model : bias and location correction 3 4

Correction of weather forecast by bias and phase correction with dynamic Phase correction does not improve the bias correction rmse Mean error Local phenomena Perspectives : Spatial model Local weather