Sensor Calibration as an Application of Optimal Sequential Estimation toward Maintaining the Space Object Catalog

Transcription

1 Sensor Calibration as an Application of Optimal Sequential Estimation toward Maintaining the Space Object Catalog John H. Seago James W. Woodburn Page 1 of 27

2 Overview In our paper: We propose the application of a sequential filter to the problem of sensor calibration at the Space Control Centers (SCC) We discuss why this is important to the future of the space-object catalog, and some limitations with the current methods. We discuss the outcome a reported demonstration of this technology, conducted for the US SCC more than twenty years ago, using real tracking. We show some results of a limited simulation that appears to support the conclusions of the former study. We mention a theorem that is not well known, but may help the operational use of sequential (recurrent) estimators using real data. Page 2

3 Sensor Calibration is Important for the Catalogs Sensor calibration should help provide for the future: Corrected observations of highest accuracy needed to detect and maintain orbits of small objects measures of uncertainty required to confidently associate uncertain measurements with many uncertain orbits the means to check and correct new operational systems deployed to detect ever smaller objects. Page 3

4 US Space Surveillance Network US SSN sites report angles, range, and sometimes range rate No uncertainty information is provided by sensors It is not requested! AN/FPS-126 Phased Array 344,345,346 Fylingdales Moor AN/FPS-108 Phased Array 393 COBRA DANE US SSN sensors are largely responsible for correcting their own error themselves, since they are experts about their own systems MSSS AMOS / MOTIF / GEODSS Optical 231,232,233,951,952 Maui - Mechanically Steered Radar - Electronically Steered Radar - Optical Imager MSX Midcourse Space Experiment Optical 504 Space-based ALTAIR Mechanical Tracker 334 Kwajalein US Space Control Center is primarily interested in removing an average bias from the measurement, and to give the measurement an uncertainty (weight). Page 4 This is the so-called static weight and bias. - Air Force Surveillance Fence For the US Space Control Center, calibration has meant estimating an average bias and uncertainty from measurement residuals.

5 Space Surveillance Network (~25 stations) Internal Calibration - Twenty+ Years Ago Two decades ago, the AFSPC Space Control Center built reference ephemerides, using: a select number of well-behaved satellites best available sensor data best available theory at Space Control Center A significant discrepancy between forecast bias and observed bias indicates that a sensor must be recalibrated. Optical These ephemerides were used to created residuals from which static weights and biases were estimated. Numbers from this process were independently checked every year by substituting US Navy Transit navigation satellite ephemerides in place of best SSN-derived ephemerides. Satellite Ephemerides Observation Simulation SSN obs error estimates SSN observations µ - 1st moment (bias) - 2nd central moment (weight) Mechanical Tracker Phased Array Page 5

6 Real-Time Calibration Proposal (c.1987) In the mid-1980s, Applied Technology Associates (ATA) received a contract from AFSPC to demonstrate and develop a sequential (recurrent) orbit determination algorithm. In October 1987, ATA published initial findings of a study based on: One year of SSN tracking of US military weather satellites, Static weights and biases supplied from previous internal calibration by the US SCC. ATA found that some supplied static weights and biases were highly erroneous: Some heavily contributing sensors had very large ranging biases that had gone undiscovered Some of the more accurate sensors that contributed occasionally were not weighted enough ATA found it could get better and faster results by estimating the range biases, together with the ballistic coefficients and orbital parameters, using a Kalman filter with a special gravity process-noise model. Process noise is a mathematical representation for the uncertainty in the dynamical system model. The ATA findings were quite unexpected and caused significant turmoil between various branches of AFSPC and radar operators. AFSPC quickly cancelled development ATA went on to develop a real-time orbit determination software (RTOD) as a commercial product. Page 6

7 External Calibration - Today MOBLAS-6, Greenbelt Tahiti Geodetic Observatory International SLR Network (~40 stations) SLR observations Observation Simulation Satellite Ephemerides SSN obs error estimates SSN observations AFSPC successfully addressed the problem by expanding external calibration to something more frequent and more accurate than yearly checks against the US Navy Transit system. MLRS UT McDonald Observatory Zimmerwald Observatory Switzerland µ - 1st moment (bias) - 2nd central moment (weight) Special Perturbations Orbit Determination Satellite State & Covariance Estimate By the mid-1990 s, AFSPC was using satellite laser ranging (SLR) and GPS to estimate truth ephemerides and avoiding internal calibrations. Page 7

8 Some Concerns with External Calibration Identification of observation biases away from the sensor usually requires subjective decision making by an intervening expert analyst. Lack of automation or strongly objective criterion Identification starts with suspected changes in historical plots of weekly averages? It may take a month or more for analyst to confidently recognize that a changepoint has occurred. This is due to the limited numbers of external SLR and GPS data available. Estimation uncertainty in the bias estimate itself is usually not a factor in determining whether a sensor is out of calibration.? ? Estimation uncertainty of the reference ephemeris is usually not considered either. The static weight and bias model may be an insufficient description of actual sensor error phenomenology Page 8

9 Inaccurate Covariance = Track Weighting? ˆ n (A T WA ) l 1 l 1 n (A T Wy ) l 1 l ˆ j k i (A T WA ) l 1 l / k i 1 i 1 j k i i 1 (A T Wy ) l 1 l / k i Estimates of uncertainty are often optimistic True not just for orbit determination, but in many natural sciences Track Weighting is operationally used at Space Control Center to make special perturbations covariance more realistic numerically shown to inflate covariance closer to error levels observed in population error studies Figure adapted from Seago et al. (2003) Paper AAS : CHARACTERIZATION OF SPACE SURVEILLANCE SENSORS USING NORMAL PLACES Precise causes are likely due to mis-modeled or uncorrected physical effects at the sensor over which the Space Control Center has no control: e.g., ionospheric or tropospheric refraction, unknown hardware problems, etc. Page 9

10 Are Sensor Errors Non-stationary? Barker et al. (1999) AAS : Track Weighting is consistent with an assumption of high error autocorrelation (serial correlation) Alfriend and Wilkins (1999) AAS : Track Weighting accommodates random, track-specific biases that vary slowly Seago et al. (2003) AAS , AAS : Error estimates in SSN tracks exhibit nonrandom, directional tendencies, such as integrated moving averages. Track Weighted variance may behave like that of a random walk. Most of these descriptions seem to agree that the need for Track Weighting implies a non-stationary sensor error behavior Figure adapted from Seago et al. (2003) Paper AAS : CHARACTERIZATION OF SPACE SURVEILLANCE SENSORS USING NORMAL PLACES Violations of stationarity assumptions would suggest the need to account for something beyond a static weight and bias. Page 10

11 Real Data Errors Move During a Track 14, 200 Points (490 Tracks, each up to up to 600 sec.) 399-Elgin Range Error (t i ) (t i ) - (t 0 ) Estimated Range Error (t i ) v. Time into Track Estimated Range Error (t i ) Relative to First Error in Track (t 0 ) v. Time into Track Figures adapted from Seago et al. (2003) Paper AAS : MORE CHARACTERIZATION OF SPACE SURVEILLANCE SENSORS USING NORMAL PLACES Ensemble errors estimates appear to move away from where they start. Page 11

12 Sequential (Recurrent) Filtering Optimal sequential filters were first developed for space navigation near the beginning of the Space Age, but batch least-square differential correctors have been used exclusively at the Space Control Center for routine catalog maintenance Operators might be avoiding sequential (recurrent) filters if they believe: require too much tuning or manual intervention? tracking data are too sparse to keep converged? too hard to initialize or too much CPU usage? no operational benefits over existing least-squares differential correction? At least two primary advantages to a sequential (recurrent) filter: The ability to directly estimate time-varying errors or biases, The ability to provide more accurate covariance. Page 12

13 Stochastic (Random) Process The following sequential (recurrent) model may be used to represent time-varying errors in the observations, as well as other system errors: x t k 1 t k 1,t k x t k 1 2 t k 1,t k w t k 1 ; k 0,1,2, ; w ~ N 0, w 2 Properties of the model Page 13 w 2 is the variance of white Gaussian noise First term has an exponentially fading autocorrelation function Similarity to the Ornstein-Uhlenbeck solution to Langevin equation, and the exponentially weighted moving average estimator. Stationary function that decays to zero if no observations are present Non-stationary in the presence of observations Sequential Gauss-Markov t k 1,t k e t k 1 t k This was the original ATA error model used by to calibrate radar ranges.

14 Stochastic (Random) Process (cont.) In practice, an analyst can prescribe an exponential half-life interval 1/2 of the process, instead of : 1/2 = ln(1/2) / The half-life interval 1/2 regulates how quickly a predicted bias value is allowed to change or forget its former value. Its value will be reduced by half over the time interval 1/2 providing that no observations occur. The uncertainty of x(t) is forecast part of the state covariance. Requires an initial value and initial uncertainty of x(t). x t k 1 t k 1,t k x t k 1 2 t k 1,t k t k 1,t k e t k 1 t k w t k 1 ; k 0,1,2, ; w ~ N 0, w 2 *Wright (2002) Paper AAS Optimal Orbit Determination. Page 14

15 Would Real-Time Calibration REALLY Work? To verify decades-old claims that the SSN could be calibrated in real time, the authors ran a limited simulation demonstration using Orbit Determination Tool Kit (ODTK): Original ATA study estimated orbital elements, ballistic coefficients, and range biases. Simulation was necessary as we did not have access to any significant amount of data. Simulation was limited (since we are not full-time analysts). Satellite Tool Kit was used with a simulated population of 29 small, non-maneuvering objects, (~1 kg cubesats ) to determine and limit tracking intervals: The cubesat two-line orbital elements were provided courtesy of Celestrak / T.S. Kelso, CSSI. Radar tracks were limited to 4 minutes per horizon crossing, observations 10 seconds apart (no outages). QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture. Optical tracks were limited to 30 seconds per horizon crossing, observations 10 seconds apart (no outages). Page 15

16 Two Sigmas (m) Hours Simulator Configuration 29 satellites and 23 SSN sensors were simulated and estimated concurrently measurements were simulated and processed over a seven (7) day span. Simulation was done by first estimating only range only, then range with angles. range bias and angle bias estimates Live data Simulator Help GUI Utilities Estimation IOD Least squares Filter Output Archive Ephemeris Intrack Position Uncertainty (0.95P) ~45 minutes to process on a PC 355 parameters estimated simultaneously Scripts Auto Smoother range bias estimates only ODTK ~35 minutes to process on a PC 280 parameters estimated simultaneously No significant change observed in behavior of estimated range bias between these two cases. Page 16

17 Overview of Satellite Force Modeling Force Modeling Modeled? Simulated? Estimated? Earth gravity field Degree & Order 21 w/ Process Noise - Ocean tide & solid earth tide perturbations No - - Third body gravity (Solar, lunar) Yes - - Atmospheric density (air drag) Jacchia, 1971 Half- life = 3 hours Time-Varying Drag coefficient Jacchia, 1971 Half- life = 7 days Time-Varying Solar-radiation pressure Perfectly absorbing sphere Half- life = 2 days Time-Varying Earth albedo No No No Maneuvers No No No Page 17

18 AFSSS Radar Optical Measurement Error Model Range Range Rate Azimuth Elevation Noise Bias 1/2 Life Noise Bias 1/2 Life Noise Bias 1/2 Life Noise Bias 1/2 Life I.D. Name (m) (m) (min) (cm/s) (cm/s) (min) (") (") (min) (") (") (min) 211/2/3 Stallion * /2/3 Maui * /2/3 Diego Garcia * 260 MOSS * AMOS * ALCOR ALTAIR TRADEX /5/6 Fylingdales Moor Millstone Hill 382/3 Clear /7 Cape Cod /9 Beale Shemya /5 Thule Cavalier Eglin San Diego Elephant Butte 743 Silver Lake Tattnall Red River Hawkinsville * natively report in right ascension and declination Noise values adapted from Seago et al. (2002) Paper AAS : MORE RESULTS OF NAVAL SPACE SURVEILLANCE SYSTEM CALIBRATION USING SATELLITE LASER RANGING Page 18

19 Simulation Results for TRADEX Range Bias Simulation run for seven (7) days First three (3) days shown No bias was introduced into this tracking data. Results appear normal. Things to notice The tracking data are fit. The residuals are randomized. In the absence of measurements, the computed 2 error bounds for the measurement bias grows and the bias decays to zero. Page 19

20 Simulation Results for Eglin Radar Range Bias Simulation run for seven (7) days First three (3) days shown A positive ~2 error was introduced into this tracking data at the end of Day 2. Results clearly show the estimator moving onto the bias. Things to notice: The tracking data are fit. The residuals appear randomized. The bias estimate regularly exceeds the 2 error bound threshold, signaling an analyst that the sensor is out of specification. Page 20

21 Simulation Results for Kaena Point Range Bias Simulation run for seven (7) days First three (3) days shown A negative ~2 error was introduced into the tracking data at the end of Day 2. Results clearly show the estimator moving onto the bias. Things to notice: The tracking data are fit. The residuals appear randomized. The bias estimate regularly exceeds the 2 error bound threshold, signaling the analyst that the sensor is out of specification. Page 21

22 Filters versus Smoothers A filter is a sequential (recurrent) estimator The resulting estimate is conditioned or influenced by prior observations available up to the time of the result. A smoother is also sequential (recurrent) estimator The resulting estimate is conditioned or influenced by observation information both before and after the time of the result. Both estimators provide estimates of uncertainty via their covariance. The smoother estimate uncertainty is always smaller than the filter estimate uncertainty because the smoother estimate is based on more (two-sided) information. The two estimates are not independent of each other. Page 22

23 Smoother Smoothers often do not get as much attention as filters because: They are not real-time. They are more complicated. Maneuvers Solve for impulsive maneuvers without additional states* Calibration of finite-time maneuvers Refines filter state & covariance but does not actually need to reprocess the observations Uses by-products and information from filter execution Has same state space as filter Provides post-fit reconstruction of states Improved orbit accuracy and smooth ephemeris Realistic smoother error covariance Provides a metric for IV&V Filter/smoother consistency can be used as a measure of the validity of the modeling We use a non-linear extension of the fixed-interval smoother due to Meditch. Improved bias calibration * Woodburn, Wright, Paper AAS , Estimation of Instantaneous Maneuvers Using a Fixed Interval Smoother. Page 23

24 Filter / Smoother Consistency Test Simulations are nice, but how can an analyst know that his measurement modeling, force modeling, calibration, etc., are sufficiently correct once he starts using real data? If n is the size of the total estimated state and the state errors can be assumed distributed as normal (Gaussian), McReynolds (1984) proved that the difference between the filter state and the smoother state is normally (Gaussian) distributed in n dimensions. Also, the covariance of this difference at any point in time is equal to the difference between the covariance of the filter state and the covariance of the smoother state. The McReynold s test statistic is a scalar distance that can be tested for multi-dimensional normality (Gaussianess) of distribution over time. Failure of the McReynolds test implies that the filter and the smoother do not give the same answer (within the stated uncertainty of both), and we must reject the hypothesis that the total modeling is adequate. Page 24

25 Filter / Smoother Consistency Test Filter-Smoother Consistency Theorem: Let the array x f (t) be an n x 1 filtered estimate at time t having n x n error-covariance P f (t), and let the array x s (t) be its smoothed estimate at time t having error-covariance P s (t). Then, assuming the state-estimate errors of x f (t) and x s (t) are multivariate normal: The n x 1 statistic x (s-f) (t) = x s (t) - x f (t) is multivariate normal at time t, and has n x n covariance P (fs)(t) = P f (t) - P s (t). The time sequence of z(t) = [x (s-f) (t)] T [P (f-s) (t)] -1 [x (s-f) (t)], t = {t 0, t 1, t 2 } provides an (auto-correlated) sample population over the estimation interval, upon which the null hypothesis of multivariate normality can be tested. Filter-Smoother Consistency Test: If the sequence z(t) supports the null hypothesis of multivariate normality, then the hypothesis of total model sufficiency between the filter and smoother models is accepted. If the sequence z(t) does not support the null hypothesis of multivariate normality, then the hypothesis of total model sufficiency between the filter and smoother models is rejected. Page 25

26 Filter / Smoother Consistency Test It is actually very difficult to pass McReynold s filter / smoother consistency test for all time using real data if the measurement noise level, process noise level, and state parameterization are not physically realistic, or are in some other way deficient. A graph of individual standardized state differences is often preferred over a multivariate statistic (see example). In this authors opinion, the diagnostic information provided by McReynold s filter / smoother consistency test statistic is a critical missing link in making use of sequential filters more widespread. Page 26

27 Summary Limited simulation seems to support the decades-old claims that larger radar ranging biases can be successfully detected and estimated at the SCC using a sequential filter close to real time. Other large biases are likely detectable as well. Real-time internal sensor calibration would be an ideal task for sequential filters for processing operational SSN tracking at the Space Control Centers: Provides beneficial supplementary information to present-day external calibration. Allows SCC analysts to develop skills using sequential (recurrent) estimation. Has the potential to improve orbits and covariance estimation on the satellites processed. Has the potential to create a self-calibrating sub-catalog Begin with small, non-maneuvering calibration targets Verify initial modeling and parameterization with filter-smoother consistency Slowly scale up by adding more satellites, resulting in a new real-time catalog with real-time covariance Page 27

28 Thank you for your attention. Page 28