SDSS Data Management and Photometric Quality Assessment Željko Ivezić Princeton University / University of Washington (and SDSS Collaboration) Thinkshop Robotic Astronomy, Potsdam, July 12-15, 2004 1
Outline 1. Overview of SDSS 2. SDSS Data Flow 3. Automated Data Quality Assessment 2
Overview of SDSS Imaging Survey 10,000 deg 2 (1/4 of the full sky), V < 22.5 100,000,000 stars and 100,000,000 galaxies 5 bands (ugriz: UV-IR), 0.02 mag photometric accuracy < 0.1 arcsec astrometric accuracy 3
SDSS Telescope (2.5m) 4
5
Overview of SDSS Imaging Survey 10,000 deg 2 (1/4 of the full sky), V < 22.5 over 100,000,000 stars and 100,000,000 galaxies 5 bands (ugriz: UV-IR), 0.02 mag photometric accuracy < 0.1 arcsec astrometric accuracy Spectroscopic Survey 1,000,000 galaxies 100,000 quasars 100,000 stars 6
Overview of SDSS Imaging Survey and Spectroscopic Survey (about 2/3 finished) Sophisticated data acquisition, processing and distribution systems ( 1 million lines of code): Science Factory: over 1,000 papers based on SDSS data, or referring to SDSS, already published (in 5 years since the first light) SDSS-II: more emphasis on Galactic structure and time domain You can join, too! (for a small fee) 7
SDSS Data Flow 1. Data Acquisition at Apache Point Observatory Peak data rate: 20 GB/hr 2. DLTs mailed to Fermilab 3. Pipelines run in successive order: Serial Stamp Collecting (SSC) Pipeline (bookkeeping) Postage Stamp Pipeline (PSP) (flatfields, PSF, etc) astrom (astrometric calibration) Frames (with PSP, aka Photo) most complex nfcalib (photometric calibration) 8
4. Stuffing operational database, resolving. 5. Spectroscopic target pipelines 6. Plate drilling at the University of Washington 7. Spectroscopic Data Acquisition at APO 8. DLTs mailed to Fermilab 9. Spectroscopic processing (spectro2d and spectro1d) 10. Stuffing database 11. All data eventually loaded into Catalog Archive Server
Web Interfaces to SDSS DR2 (www.sdss.org) Catalog Archive Server (CAS): search tools for querying the imaging and spectro catalogs from SDSS. Spectro Query Server: search spectra by position, or by spectral or photometric parameters. Retrieve survey files. Imaging Query Server: search photometry catalog by position, or by photometric parameters. Retrieve survey files. SpecList: upload plate,mjd,fiber list as part of a SQL query Imaging cross-id: find SDSS neighbors for a list of positions Navigate, Finding charts, : Point and click on images, finding charts, etc. 9
Automated Data Quality Assessment The quality of data is of paramount importance for their scientific impact; e.g. should you trust SDSS photometry (both measurements and errors)? The quantitative data quality assessment is a difficult problem, and automated quantitative assessment is even harder SDSS automated quantitative data quality assessment: the matchqa and runqa pipelines 10
SDSS Automated Data Quality Assessment 1. matchqa pipeline: compares two observations of the same objects magnitudes (aperture, psf, model, fiber, Petrosian) positions shapes star/galaxy separation 11
12
SDSS Automated Data Quality Assessment 1. matchqa pipeline: compares two observations of the same objects 2. runqa pipeline: estimate data quality from single observations the quality of PSF photometry the photometric zeropoint errors 13
Point Spread Function Magnitudes The PSF flux is computed using the PSF as a weighting function: the PSF must be known exquisitely well for required photometric accuracy ( 0.01 mag) 14
SDSS Imaging Point Spread Function 15
Point Spread Function Modeling The PSF is determined heuristically using Karhunen-Loeve transform there is no assumption on the PSF functional form (e.g. Gaussians): a set of N stellar images is expressed as a linear combination of N eigenimages, and the first 3 terms are retained Typical variation in the effective width across a frame is 10%. Modeled by expanding eigencoefficients in terms of x a y b with a + b <= 2. Greatly improves photometric accuracy (rms from 0.05 mag to 0.02 mag, with better outlier behavior) Use the difference between the aperture and PSF magnitudes (for bright stars) to recognize bad PSF models 16
17
Photometric Calibration Imaging data are photometrically calibrated using a network of calibration stars obtained in 2 degree large patches by the Photometric Telescope Patches are separated by of order an hour of scanning time any changes in atmospheric transparency, or other conditions affecting the photometric sensitivity, may not be recognized on shorter timescales. Problem: a dense network of calibration stars across the sky, accurate to 0.01 mag, in five SDSS bands, which could be used for an independent verification of SDSS photometric calibration, does not yet exist. The position of the stellar locus in color-color diagrams is stable can be used to estimate errors in photometric zeropoints 18
19
2 14 1.5 1 0.5 P2 P1 16 18 0 20-0.5 0 1 2 22-0.4-0.2 0 0.2 0.4 14 20 16 15 18 10 20 5 22-0.2 0 0.2 0-0.1 0 0.1 20
21
u z g 22
Photometric Calibration The position of the stellar locus in color-color diagrams can be used to estimate errors in photometric zeropoints to <0.01 mag in patches as small 0.03 deg 2 The width of the stellar locus can be used to automatically recognize substandard photometry Typical errors for SDSS photometric zeropoints are 0.01 mag in g, r, and i, 0.02 mag in z, and 0.03 mag in u (upper limits!) 23
Conclusions SDSS a science factory sophisticated data acquisition, processing and distribution systems The tracking of the position and the width of the stellar locus in color-color diagrams offers a robust automated method, accurate to 0.01 mag, for estimating the photometric accuracy of optical surveys (at least for b > 10 ) SDSS provides encouragement that even more ambitious surveys, such as LSST (20 TB/night!), may also be successful endeavors. 24
Update on Large Synoptic Survey Telescope LSST: a deeper SDSS (V < 24 per epoch, V < 26 coadded), every 3 days, for 10 years! Celestial Cinematography We are incorporated! (Founding partners: NOAO, Research Corporation, University of Arizona, University of Washington) New institutions are joining: the DoE group (2 Gigapixel camera), NCSC, UC Davis, Harvard, Google,... Mirror (8.4m) contract signed with the University of Arizona Mirror Lab The first light in December of 2011 25