Radiance Data Assimilation. Andrew Collard Environmental Modeling Center NCEP/NWS/NOAA With input from: John Derber

Transcription

1 Radiance Data Assimilation Andrew Collard Environmental Modeling Center NCEP/NWS/NOAA With input from: John Derber

2 Outline Introduction Different types of satellite data. Basic theory of satellite observations Radiative Transfer Spectroscopy Basic concept of a retrieval Assimilating satellite radiances. Data assimilation equation Quality control. Bias correction. Observation errors. Thinning Monitoring. Thinning Future of satellite data assimilation.

3 Introduction Satellites instruments do not directly measure the atmospheric state. Instead they measure radiation emitted by and/or transmitted by the atmosphere. This presentation describes the relationship between the atmospheric state and the observed radiation. And how the information contained therein is exploited through assimilation into the NWP model.

4 Different Types of Satellite Data

5 Meterological Satellite Constellation (from WMO)

6 Different Types of Satellite Data Active (bouncing a signal off something) Wind Lidar SAR Cloud radar Scatterometry

7 Different Types of Satellite Data Occultation (signal passing through atmosphere) GPSRO HALOE SAGE SCIAMACHY

8 Different Types of Satellite Data Passive (receiving radiative signal from source) Visible instruments IR instruments Microwave instruments

9 Passive Instruments This talk will focus on passive infrared and microwave instruments as they are the most common and biggest contributors to Numerical Weather Prediction

10 Geometry: Limb vs Nadir Sounding Limb sounding: Viewing the Earth s atmosphere tangentially Higher vertical resolution Lower horizontal resolution Most often used for observing the stratosphere and above. Not often used in NWP

11 Geometry: Limb vs Nadir Sounding Nadir sounding: Viewing towards the Earth s surface Lower vertical resolution Higher horizontal resolution Most often used in NWP

12 Meterological Satellite Constellation (from WMO) Currently operationally Assimilate radiances at NCEP

13 Basic theory of satellite observations Radiative Transfer

14 Radiative Transfer : Definitions Radiance, I ν, is the radiant energy emitted per unit time, per unit frequency interval, per unit area, and per unit solid angle in a specified direction at a given frequency, ν. The units for radiance are Wm -2 sr -1 (cm -1 ) -1 or equivalent. Radiance is often expressed as the temperature that would produce the equivalent black-body radiance: the Brightness Temperature. This is not to be confused with Irradiance or Flux Density which is the total power per unit frequency interval, crossing perpendicular to a unit area which has units Wm -2 (cm -1 ) -1.

15 Radiative Transfer Absorption of radiance in a volume element Consider monochromatic radiation of frequency ν passing through a volume element of length ds and cross-sectional area da. It contains a gas with n molecules per unit volume, each with an absorption coefficient of k ν. I ν ds I ν +di ν da If we ignore scattering, the change in radiance across the volume due to absorption is given by: (n k ν ds) is the absorptivity of the volume element.

16 Radiative Transfer Emission of radiance in a volume element If we continue to ignore scattering, the change in radiance across the volume due to emission from the same volume element is given by: I ν ds I ν +di ν da

17 Radiative Transfer Schwarzchild Equation of Radiative Transfer Combining the terms for emission and absorption gives

18 Radiative Transfer Schwarzchild Equation in a scattering atmosphere Extinction term (absorption+scattering) Emission term Radiation scattered from all directions, Ω, into the beam We are not going to be considering scattering in this talk.

19 Radiative Transfer Solution to Schwarzchild Equation The general solution to the Schwarzchild equation is: Radiation at observer Radiation at lower Boundary (usually the surface for terrestrial planets) Emission of radiation from the atmosphere itself including reabsorption of this radiation attenuated by the intervening atmosphere

20 Radiative Transfer Transmission and Weighting Functions The general solution to the Schwarzchild equation is: where Which we can transform into pressure, p, coordinates: is the historical definition of the weighting function as it is the weight given to the source function at each level in the solution.

21 Radiative Transfer Weighting Functions and Jacobians

22 Illustration of Jacobian or Weighting Function The form of the Jacobian or weighting function is highly frequency dependent as the atmospheric absorption properties vary with frequency and pressure in the atmosphere. Weighting functions and Jacobians can peak at any level of the atmosphere including the surface.

23 Basic theory of satellite observations Spectroscopy

24 So where does atmospheric absorption come from? Molecules in the atmosphere have energy stored as rotational, vibrational and electronic components The energy states are quantised and may be transformed through emission or absorption of electromagnetic radiation. This results in discrete spectral emission/absorption features in the spectrum. ν 1 ν 2 ν 3 Vibrational Modes for CO 2 In the microwave these are due to rotational transitions In the infrared these are rotational and vibrational transitions Electronic transitions manifest themselves in the visible and ultraviolet

25 Vibration-Rotation Spectrum Ground ν2 transition for CO2 Q-branch ΔJ=0 P-branch ΔJ=-1 R-branch ΔJ=+1 An example of a vibration-rotation band in the infrared CO 2 spectrum. Due to considerations of angular momentum, only changes in the rotational quantum number, J, of -1,0 or 1 are optically active, producing the characteristic three branch structure to the band (some linear molecules have the Q-branch missing).

26 Line Broadening Spectral lines will be broadened through one of the following three processes: Natural broadening: The finite time of the quantum transition corresponds to an uncertainty in the energy through the uncertainty principle. Doppler broadening: Thermal motion of the molecules along the line of sight result in apparent uncertainty in the frequency through the Doppler effect. Collisional (or pressure) broadening: Collisions between molecules during emission and absorption results in modification of energy levels and hence broadening of the spectral line. In the lower atmosphere, collisional broadening dominates, while Doppler broadening is more important in the upper atmosphere.

27 Other Sources of Atmospheric Opacity Continuum absorption: Mostly the combined effect of the far wings of lines from collisional broadening or collisionally induced absorption bands (O 2, N 2, CO 2 on Venus!). Possible effect from dimers of H 2 O. In the 8-12m infrared window, the H 2 O continuum is the dominant source of opacity (except for clouds). Absorption and scattering from cloud particles. Mie theory describes liquid water cloud RT well. Ice crystals are more complicated. Particle size distributions can be important to characterize. Absorption and scattering from aerosols. Similar problem to cloud with added complication of varying compositions. All the above tend to have much broader spectral features than gaseous absorption.

28 Optical Depth Microwave Spectrum Frequency (GHz)

29 Radiance (mw/cm 2 /sr/cm -1 ) An Infrared Spectrum Dashed lines are Planck curves for 200K-300K at 20K intervals CO 2 O 3 Shortwave window H 2 O CO 2 Wavenumber (cm -1 )

30 Brightness Temperature (K) Shortwave window An IASI Spectrum (with solar contribution) Longwave window H 2 O O 3 CO 2 CO 2 An Infrared Spectrum Q-branch Wavelength (μm)

31 Brightness Temperature (K) Shortwave window An IASI Spectrum (with solar contribution) Longwave window H 2 O CO Window channels see the surface in 2 clear skies but are also Omost 3 affected by clouds which may reduce the brightness temperatures by over 100K. An Infrared Q-branch CO Spectrum 2 Wavelength (μm)

32 IASI vs HIRS: The Thermal InfraRed

33 AIRS & HIRS Jacobians in the 15μm CO 2 band 100 hpa HIRS-4 HIRS-5 HIRS-6 Selected AIRS Channels: 82(blue)-914(yellow) HIRS hpa HIRS-8

34 We also need to know the surface emissivity Over ocean we usually have models, e.g.: ISEM (infrared) FASTEM (microwave) Over land we often use atlases, either of the emissivities themselves or of the land type. Emissivities can also be retrieved from the observations themselves.

35 Surface Emissivity Infrared

36 Surface Emissivity Microwave Emissivities over the ocean are very low in the microwave.

37 Forward models To exploit these radiances, it is important to have an accurate way of simulating them from the atmospheric state. Line-by-line (LBL) models use state-of-the-art spectroscopic databases to make these calcuations at high spectral resolution. These monochromatic calculations are then combined using the instruments spectral response functions (ISRFs) to simulate what the instrument observes. This can be very slow. Too slow for operational radiance assimilation.

38 Fast Forward models To allow radiances to be operationally assimilated, fast radiative transfer models, which use regression schemes to simulate the output from LBL models, have been produced. The two main fastmodels used operationally in NWP centers are RTTOV (developed by the EUMETSAT NWPSAF) and CRTM (JCSDA). The errors in the fastmodel are not usually a significant component of the total error budget. Most importantly, fastmodels allow the Jacobians (and the model adjoint) to be calculated efficiently.

39 Basic theory of satellite observations Basic Concept of a Retrieval

40 So we have observations of the radiation emitted from the atmosphere at various frequencies corresponding to: Emission and absorption at various levels Emission and absorption by various gases/clouds/aerosols Now what?

41 Unless we can infer the temperature profile we won t be able to do much else. To do this we need to choose frequencies where we know the absorption profiles already We choose gases with a constant distribution to do this. For the infrared we use CO 2 For the microwave we use O 2 These are hence known as temperature sounding bands. But all bands are sensitive to temperature, often as in the case of H 2 O with sharper Jacobians. Once we have a good temperature profile we can use that to infer molecular abundances of variable species using appropriate frequencies. This is actually performed simultaneously with the temperature estimation when we do data assimilation

42 The Jacobians give the sensitivity to the vertical profiles of temperature/gases/clouds etc. If we sum the contribution of each channel, we can get a very accurate estimate of the mean atmospheric temperature (with very low vertical) resolution. Obtaining vertical profiles If we take differences between each of the channels we can infer the profile with high vertical resolution, but the result will be very noisy. When we assimilate the radiance observations we are effectively producing a minimum variance solution to the problem: which is a compromise between these two extremes

43 Assimilating satellite radiances Data Assimilation Equation

44 J = J b + J o + J c Atmospheric Analysis Problem J = (x-x b ) T B x -1 (x-x b ) + (y-k(x)) T (E+F) -1 (y-k(x)) + J C J = Fit to background + Fit to observations + constraints x x b = Analysis = Background (usually a short-range forecast from the previous cycle) B x = Background error covariance K = Forward model (nonlinear) O = Observations E+F = R = Instrument error + Representativeness error J C = Constraint term

45 Atmospheric Analysis Problem J = J b + J o + J c J = (x-x b ) T B x -1 (x-x b ) + (y-k(x)) T (E+F) -1 (y-k(x)) + J C J = Fit to The background difference between + Fit to observations the observations + constraints and the background transformed into x = Analysis x b = Background model space, the first guess departure, is B x = Background an important error measure. covariance It is often the K = Forward basis of quality model (nonlinear) control procedures. O = Observations E+F = R = Instrument error + Representativeness error J C = Constraint term

46 Assimilating satellite radiances Quality Control

47 Quality Control Procedures The quality control step may be the most important aspect of satellite data assimilation. Data which has gross errors or which cannot be properly simulated by forward model must be removed. Most problems with satellite data come from 4 sources: Instrument problems. Clouds and precipitation simulation errors. Surface emissivity simulation errors. Processing errors (e.g., wrong height assignment, incorrect tracking, etc).

48 Quality Control Procedures IR cannot see through most clouds. Cloud height difficult to determine especially with mixed FOVs. Since deep layers not many channels completely above clouds. Microwave impacted by clouds and precipitation but signal is smaller from thinner clouds. Surface emissivity and temperature characteristics not well known for land/snow/ice. Also makes detection of clouds/precip. more difficult over these surfaces. Error distribution may be asymmetric due to clouds and processing errors.

49 AMSU-A Ch 2 Un-bias-corrected First Guess Departure (K) Cloud detection in the microwave Cloud sensitivity results in a gradient greater than 1.0 Used Obs deemed Cloudy Used Obs deemed Clear (unused obs are small dots) Water sensitivity results in a gradient around 0.5 AMSU-A Ch 1 Un-bias-corrected First Guess Departure (K) 49

50 pressure (hpa) obs-calc (K) Cloud detection in the infrared A non-linear pattern recognition algorithm is applied to departures of the observed radiance spectra from a computed clear-sky background spectra. AIRS channel 226 at 13.5micron (peak about 600hPa) Vertically ranked channel index This identifies the characteristic signal of cloud in the data and allows contaminated channels to be rejected unaffected channels assimilated AIRS channel 787 at 11.0 micron (surface sensing window channel) CLOUD contaminated channels rejected temperature jacobian (K) From ECMWF

51 Assimilating satellite radiances Bias Correction

52 Bias Correction The differences between simulated and observed observations can show significant biases. The source of the bias can come from: Inadequacies in the characterization of the instruments. Deficiencies in the forward models. Errors in processing data. Biases in the background. Except when the bias is due to the background, we would like to remove these biases.

53 Bias Correction Currently bias correction only applied to a few data sets: Radiances. Radiosonde data (radiation correction and moisture). Aircraft data. For radiances, biases can be much larger than signal. Essential to bias correct the data. NCEP currently uses a 2-step process for radiances (other centers are similar). Angle correction (very slowly evolving different correction for each scan position). Air Mass correction (slowly evolving based on predictors).

54 Scan dependent biases for AMSU

55 Satellite radiance observations Bias correction

56 NOAA 18 AMSU-A No Bias Correction

57 NOAA 18 AMSU-A Bias Corrected

58 Observation - Background Histogram 19V 19H 22V 37V 37H 85V 85H DMSP15 July2004 : 1month before bias correction after bias correction

59 Application of NWP Bias Correction for SSMIS F18 T (K) O-B Before Bias Correction O-B Before Bias Correction Global Using Met Office SSMIS Bias Correction Predictors Dsc Asc Unbias & Bias Corrected O-B T (K) O-B After Bias Correction O-B After Bias Correction Global Ascending Node Descending Node Latitude Asc Dsc T (K) 59

60 Bias Correction and QC Interact Bias Correction Observations are bias-corrected after quality control Quality Control Quality control usually uses bias-corrected observations

61 Assimilating satellite radiances Observation Errors

62 Observational Errors Observation errors specified based on instrument errors and statistics Generally for satellite data, variances are specified a bit large since the correlated errors (from RT and instrument errors) are not well known. Observation errors are also generally specified as being uncorrelated spectrally, but efforts are being made to determine the off-diagonal components of the observation error covariance matrix.

63 IASI Observation Errors in ECMWF System

64 IASI Inter-Channel Correlations

65 Specifying Errors in the GSI: Satinfo file Is channel assimilated? -1=Passive, 1=Active (there are other options) The assigned observation error (K) See tomorrow s talk Used in VarQC The maximum allowed departure to pass quality control (K)

66 Assimilating satellite radiances Thinning

67 Thinning or Superobbing Thinning Reducing spatial or spectral resolution by selecting a reduced set of locations or channels. Can include intelligent thinning to use better observation. Superobbing Reducing spatial or spectral resolution by combining locations or channels. Can reduce noise. Includes reconstructed radiances. Can include higher moments contained in data Purser et al., Can be done with obs or departures, but should be done after QC. Both can be used to address 3 problems: Redundancy in data. Reduce correlated error. Reduce computational expense.

68 Count (Millions) Satellite Data Ingest Daily Satellite & Radar Observation Count Level 2 Radar 210 M obs 100% Daily Percentage of Data Ingested into Models *2005 Data 239.5M 125 M obs 100 M obs % 2% Received Data Selected Data Assimilated Data 17.3M 5.2M Five Order of Magnitude Increases in Satellite Data Over Fifteen Years ( ) Received = All observations received operationally from providers Selected = Observations selected as suitable for use Assimilated = Observations actually used by models

69 Assimilating satellite radiances Data Monitoring

70 Data Monitoring It is essential to have good data monitoring. Usually the NWP centres see problems with instruments prior to notification by provider (Met Office especially). The data monitoring can also show problems with assimilation systems. Needs to be ongoing/real time. Monitoring reports from most major NWP centers at:

71 Quality Monitoring of Satellite Data AIRS Channel March 2007 Increase in SD Fits to Guess

72 Overall Comments Satellite data must be treated carefully. Important to be aware of instrument characteristics before attempting to use data. No current component of observing system is used perfectly or as well as possible. Computational expense plays important role in design of system.

73 Questions and/or Comments Please

74 Useful References (1/2) Auligne T.; McNally A. P.; Dee D. P., 207: Adaptive bias correction for satellite data in a numerical weather prediction system, QJRMS, 133, Bormann, N., A. Collard, and P. Bauer, Observation Errors and their error correlations for satellite radiances, ECMWF Newsletter No. 128, p available at Collard, A., F. Hilton, M. Forsythe and B. Candy (2011). From Observations to Forecasts Part 8: The use of satellite observations in numerical weather prediction. Weather, 66: CRTM ftp site: ftp://ftp.emc.ncep.noaa.gov/jcsda/crtm/

75 Useful References (2/2) Dee, D. P., Uppala S., 2009: Variational Bias correction of satellite radiance data in the ERA-Interim reanalysis, QJRMS, 135, Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, Desroziers, G., L. Berre, B. Chapnik & P. Poli, 2005: Diagnosis of observation background and analysis-errorstatistics in observation space. QJRMS., 131, McNally, A.P., J.C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1b radiances in the NCEP SSI analysis system. Q.J.R.M.S., 126, RTTOV homepage.: Rizzi, R., and R.W. Saunders. Principles of remote sensing of atmospheric parameters from space. ECMWF Training Course. N/REMOTE_SENSING Rodgers, C.D. (2000). Inverse Methods for Atmospheric Sounding: Theory and Practice. World Scientific Pub Co Inc.

76 Backup Slides