Astronomical image reduction using the Tractor

the Tractor DECaLS Fin Astronomical image reduction using the Tractor Dustin Lang McWilliams Postdoc Fellow Carnegie Mellon University visiting University of Waterloo UW / 2015-03-31 1

Astronomical image reduction 2

Astronomical image reduction 3

The Tractor context the traditional or algorithmic approach: estimators; arithmetic operations on the pixels m = f(pixels) model-fitting approach: write down a scalar objective function g and optimize it: m = arg max g(pixels, µ) µ generative model-fitting: working in pixel space: g = χ 2 ( model(µ)i pixel m = arg min i µ σ i pixels i ) 2 4

The Traditional, or Algorithmic, Approach What is the center of this source? 5

The Traditional, or Algorithmic, Approach First smooth by a Gaussian the size of the PSF 6

The Traditional, or Algorithmic, Approach Find peak, and look at 3x3 window around the peak 7

The Traditional, or Algorithmic, Approach Find peak, and look at 3x3 window around the peak y x 8

The Traditional, or Algorithmic, Approach Fit columns as parabolas y x 9

The Traditional, or Algorithmic, Approach Fit columns as parabolas y x 10

The Traditional, or Algorithmic, Approach Fit columns as parabolas y x 11

The Traditional, or Algorithmic, Approach Peak lies along this locus y x 12

The Traditional, or Algorithmic, Approach Fit rows as parabolas y x 13

The Traditional, or Algorithmic, Approach Fit rows as parabolas y x 14

The Traditional, or Algorithmic, Approach Fit rows as parabolas y x 15

The Traditional, or Algorithmic, Approach Peak lies along this locus y x 16

The Traditional, or Algorithmic, Approach Peak lies at the intersection y x 17

The Traditional, or Algorithmic, Approach The essence: Make some assumptions or approximations about the data Come up with an estimator that would produce the right answer if those assumptions held Write an algorithm to implement that estimator Remarks: Works when the assumptions hold Can be hard to find estimators Usually single image Tends to produce pipelines: measure A, set it in stone; measure B, set it in stone;... 18

The Generative Modelling Approach What is the center of this source? 19

The Generative Modelling Approach What center does a model of this source need to have to best match this image? 20

The Generative Modelling Approach Assume it s a point source; assume we have a model of the PSF. Use heuristics to initialize. Render initial model image Image Model Residuals 21

The Generative Modelling Approach Optimize model parameters (eg, with linearized least squares) d(model)/dx d(model)/dy d(model)/dflux 22

The Generative Modelling Approach Optimize model parameters (eg, with linearized least squares) Image Model Residuals 23

The Generative Modelling Approach Optimize model parameters (eg, with linearized least squares) Image Model Residuals 24

The Generative Modelling Approach Optimize model parameters (eg, with linearized least squares) Image Model Residuals 25

The Generative Modelling Approach Optimize model parameters (eg, with linearized least squares) Image Model Residuals 26

The Generative Modelling Approach The essence: Assume we have calibration info about the image (eg, PSF model) Assume we can parameterize the appearance of sources in our images Seek maximum likelihood parameters: it s an optimization problem Remarks: Heuristics to get into the ballpark In theory, very clean In practice, optimization can be tough! Enables iterative solution: fit sources with PSF held fixed; then fit PSF with sources held fixed 27

What is the Tractor? (in a picture) catalog RA, Dec Mags Galaxies: profile shape Stars: motion variability image calibration Photometric, WCS, PSF, Sky model image 28

What is the Tractor? (in a picture) catalog RA, Dec Mags Galaxies: profile shape Stars: motion variability image calibration Photometric, WCS, PSF, Sky model image - observed image per-pixel error = chi 2 likelihood 29

Benefits of the Tractor approach handling multiple exposures, multiple bands, multiple instruments is easy handling masked pixels (cosmic rays, bleed trails) is easy avoids deblending possible to go back and re-fit calibration parameters easy to add priors or external information possible to sample possible to use physical models 32

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (18.00, 18.00) with Flux: 500 shape EllipseE: re=1, e1=0, e2=0 33

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (18.30, 18.30) with Flux: 432.5 shape EllipseE: re=3.74, e1=0.07, e2=0.98 34

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (18.33, 18.32) with Flux: 441.7 shape EllipseE: re=3.85, e1=0.03, e2=0.46 35

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (19.27, 19.29) with Flux: 837.4 shape EllipseE: re=8.16, e1=0.002, e2=0.70 36

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (20.12, 20.06) with Flux: 1094.5 shape EllipseE: re=8.89, e1=-0.013, e2=0.54 37

Example: Galaxy fitting Image Model Residual dx dy dflux dre de1 de2 ExpGalaxy at pixel (19.91, 19.85) with Flux: 1082.7 shape EllipseE: re=8.92, e1=-0.018, e2=0.54 38

Example: Two galaxy fitting Image Model Residual 39

Example: Two galaxy fitting Image Model Residual 40

Example: Two galaxy fitting Image Model Residual 41

Example: Two galaxy fitting Image Model Residual 42

Example: Two galaxy fitting Image Model Residual 43

Example: Two galaxy fitting Image Model Residual 44

Example: Two galaxy sampling skip 45

the Tractor DECaLS Fin Context the Tractor Examples Tricks s1 ee2 s1 ee1 s1 logre s1 Flux s1 y s1 x s0 ee2 s0 ee1 s0 logre s0 Flux s0 y Example: Two galaxy sampling s0 x s0 y s0 Flux s0 logre s0 ee1 s0 ee2 s1 x s1 y s1 Flux s1 logre s1 ee1 s1 ee2 46

Example: Two galaxy sampling e2 B e1 B e2 A e1 A e2 A e1 B e2 B 47

Example: Two galaxy sampling radius B flux B radius A flux A radius A flux B radius B 48

Example: Moving Source Model with no proper motion 49

Example: Moving Source Model with proper motion 50

Example: Moving Source Model with proper motion 51

Example: Moving Source 52

Example: Forced photometry SDSS WISE Model for one source Full Model 53

Tricks the Tractor DECaLS Fin Context the Tractor Examples Tricks central operation: render pixel models & derivatives galaxies: convolution of galaxy profile by PSF (can t do naive convolution) we use two tricks, both using Mixtures of Gaussians approximations of galaxy profiles (G s (r) e kr1/s ) lux / M lux = 6 / ξmax = 4 / badness = 4.64 10 6 lux / M lux = 6 / ξmax = 4 / badness = 4.64 10 6 8 intensity (relative to intensity at ξ = 1) 6 4 2 intensity (relative to intensity at ξ = 1) 10 0 10 1 10 2 10 3 10 4 0 0 2 4 6 8 dimensionless angular radius ξ 10 5 10 3 10 2 10 1 10 0 10 1 dimensionless angular radius ξ 54

Tricks the Tractor DECaLS Fin Context the Tractor Examples Tricks central operation: render pixel models & derivatives galaxies: convolution of galaxy profile by PSF (can t do naive convolution) we use two tricks, both using Mixtures of Gaussians approximations of galaxy profiles (G s (r) e kr1/s ) luv / M luv = 8 / ξmax = 8 / badness = 8.42 10 6 8 luv / M luv = 8 / ξmax = 8 / badness = 8.42 10 6 10 2 intensity (relative to intensity at ξ = 1) 6 4 2 0 intensity (relative to intensity at ξ = 1) 10 1 10 0 10 1 10 2 10 3 0 2 4 6 8 dimensionless angular radius ξ 10 3 10 2 10 1 10 0 10 1 dimensionless angular radius ξ 55

Convolution Trick #1 approximate galaxy profile as mixture of Gaussians (galaxy shape and sky-to-pixel are affine transformations) approximate PSF as mixture of Gaussians convolution is analytic! N psf N galaxy Gaussian components in the convolution 25 20 15 10 5 25 20 15 10 5 0 0 0 25 5 10 15 20 250 25 5 10 15 20 25 20 15 10 20 15 10 5 5 0 0 0 5 10 15 20 250 5 10 15 20 25 57

Convolution Trick #2 pixelized PSF model; take Fourier transform Gaussians have analytic Fourier transforms; F (g) = G evaluate F (galaxy) directly multiply and Inverse FFT to get the convolved profile 14 12 10 8 6 4 2 14 12 10 8 6 4 2 0 0 0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14 58

the Tractor DECaLS Fin Survey Results the DECam Legacy Survey (DECaLS) I I I A public imaging survey using DECam: 64 nights g,r,z over 6,000 square degrees About 2 mags deeper than SDSS (g 24.7, r 23.9, z 23) SDSS DECaLS 59

the Tractor DECaLS Fin Survey Results the DECam Legacy Survey (DECaLS) 29 Depth (mag) 28 27 26 25 24 23 22 HSC g r i z y DES g r i DECaLS z y g r z 21 i z 20 0 5000 10000 15000 20000 Area (square degrees) g SDSS r u LSST full g z LSST single g r i i z y r u y u 60

the Tractor DECaLS Fin Survey Results the DECam Legacy Survey (DECaLS) What s it for? Overlaps the SDSS/BOSS/eBOSS footprints: millions of spectra (Dec 20 to +30) Galaxy and AGN feedback & evolution Milky Way halo & satellites Things we haven t thought of... 90 60 120 150 KIDS 30 ATLAS ATLAS 180 210 0 30 KIDS 240 270 30 300 330 0 Galactic Plane 60 90 60 DES 90 61

the Tractor DECaLS Fin Survey Results the DECam Legacy Survey (DECaLS) What s it really for? Overlaps the SDSS/BOSS/eBOSS footprints: millions of spectra (Dec 20 to +30) Galaxy and AGN feedback & evolution Milky Way halo & satellites Things we haven t thought of... DESI targeting 90 60 120 150 KIDS 30 ATLAS ATLAS 180 210 0 30 KIDS 240 270 30 300 330 0 Galactic Plane 60 90 60 DES 90 62

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 63

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 64

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 65

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 66

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 67

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 68

the Tractor DECaLS Fin Survey Results DECaLS observations 12 nights 2014B, 14 nights 2015A, 8 nights 2015B,... Three-pass tiling: ideally 2 photometric and < 1.3 seeing CCD-sized dithers 69

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data DECam Community Pipeline calibrated images (with tweaked photometric & astrometric calibration) PsfEx PSF models SDSS catalogs for the bright end SED-matched detections for the faint end Run a brick at a time (0.25 0.25 degrees; 3600 3600 pixels) 70

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data SDSS DECam Coadd 71

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data SDSS sources DECam Coadd 72

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data New sources DECam Coadd 73

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Blobs DECam Coadd 74

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data DECam Coadd DECam Coadd 75

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Model + Noise DECam Coadd 76

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Model DECam Coadd 77

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Residuals DECam Coadd 78

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Model DECam Coadd 79

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data Model DECam Coadd 80

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data 81

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data 82

the Tractor DECaLS Fin Survey Results Applying the Tractor to DECaLS data 83

the Tractor DECaLS Fin Survey Results Tractor catalog results 3.0 2.5 SDSS stars galaxies 2.0 r - z (mag) 1.5 1.0 0.5 0.0 0.5 0.5 0.0 0.5 1.0 1.5 2.0 2.5 g - r (mag) 84

the Tractor DECaLS Fin Survey Results Tractor catalog results 3.0 2.5 2.0 DECaLS stars galaxies faint r - z (mag) 1.5 1.0 0.5 0.0 0.5 0.5 0.0 0.5 1.0 1.5 2.0 2.5 g - r (mag) 85

the Tractor DECaLS Fin Thanks! Sample questions what about star/galaxy separation? what about brightness-dependent PSFs ( brighter fatter )? what was that SED-match detection thing you mentioned? can we see that two-galaxy sampling example? what s DESI? what about cosmic rays, satellites, and transients? what about color gradients in galaxies? is linearized least squares the best you can do? could you run this using data from a very different instrument? is the code public? can I run it on my images? will it ever run out of the box like SourceExtractor? 86

the Tractor DECaLS Fin Dark Energy Spectroscopic Instrument (DESI) 5000-fiber spectrograph, 3 FOV, on the 4-m Mayall On-sky 2018, 5-year survey. 9k to 14k sq deg 4M Luminous Red Galaxies (LRGs), z [0.4, 1] 18M Emission Line Galaxies (ELGs), z [0.6, 1.6] 2M tracer QSOs, z < 2.1 plus 0.7M Ly-α QSOs, z > 2.1 87

the Tractor DECaLS Fin Dark Energy Spectroscopic Instrument (DESI) BAO in many redshift slices to nail down the growth history Before DESI DESI 88

the Tractor DECaLS Fin WISE Satellite Wide-field Infrared Survey Explorer Mid-infrared: W1 (3.4µ), W2 (4.6µ), W3 (12µ), W4 (22µ) Primary survey: 2010-2011; reactivated for more in 2014! W1/W2 median of 33 exposures all-sky Scans from Ecliptic pole-to-pole 5,000 exposures at the poles Pixel scale: 2.75 arcsec/pixel: (7 times bigger than SDSS) 89

the Tractor DECaLS Fin WISE photometry SDSS WISE 1984 1984 1982 1982 1980 1980 1978 1978 1976 1976 1974 1974 1972 1678 1680 16821684 1686 1688 Model for one source 1984 1982 1980 1978 1976 1974 1972 1678 1680 16821684 1686 1688 1972 1678 1680 16821684 1686 1688 Full Model 1984 1982 1980 1978 1976 1974 1972 1678 1680 16821684 1686 1688 (Targeted in BOSS W3 ancillary; quasar at z = 2.71) 90

the Tractor DECaLS Fin WISE Tractor photometry: depth We measure sources below the WISE catalog 5-sigma threshold 10 6 Number of sources 10 5 10 4 10 3 10 2 W1 (Tractor) W1 (WISE catalog) 10 12 14 16 18 20 22 24 W1 mag (Vega) 91

the Tractor DECaLS Fin WISE Tractor photometry: accuracy The Tractor measurements are consistent with the WISE catalog Tractor W1 mag 10 11 12 13 14 15 16 17 18 18 17 16 15 14 13 WISE W1 mag 12 11 10 10000 1000 100 10 1 0 92

the Tractor DECaLS Fin WISE Tractor photometry: depth The Tractor measurements of faint objects make sense astrophysically 12 Catalog matches 12 Tractor photometry 14 16 1000 14 16 1000 r (mag) 18 20 22 24 4 2 0 2 4 6 8 10 r - W1 (mag) 100 10 1 0 r (mag) 18 20 22 24 4 2 0 2 4 6 8 10 r - W1 (mag) 100 10 1 0 93

the Tractor DECaLS Fin Multi-Band (SED-matched) Detection Example: g,r,i measurements Assume flat SED ( Vega ); estimate r flux g flux is an estimate of r flux; ˆr g = g i flux is an estimate of r flux; ˆr i = i Three predictions of r flux, with Gaussian errors inverse-variance weighting Assume red spectrum (colors g r = r i = 1): g flux is an estimate of r flux; ˆr g = g 2.5 (with variance 2.5 2 larger) i flux is an estimate of r flux; ˆr i = i/2.5 (with variance 2.5 2 smaller) 94

the Tractor DECaLS Fin SED-matched detection 95

the Tractor DECaLS Fin SED-matched detection 96

the Tractor DECaLS Fin SED-matched detection 97

the Tractor DECaLS Fin SED-matched detection 98

the Tractor DECaLS Fin SED-matched detection 99

the Tractor DECaLS Fin SED-matched detection 100

the Tractor DECaLS Fin SED-matched detection 101

the Tractor DECaLS Fin SED-matched detection 102

the Tractor DECaLS Fin SED-matched detection 103

the Tractor DECaLS Fin SED-matched detection 104

the Tractor DECaLS Fin SED-matched detection 2.0 1.5 1.0 Color-color space locations of strongest detections Flat Red S1 S2 S3 S4 r - i 0.5 0.0 0.0 0.5 1.0 1.5 2.0 2.5 g - r 105

the Tractor DECaLS Fin SED-matched detection 1.8 1.6 Red Blue Strength of SED-matched detections 1.4 Relative S/N vs Flat 1.2 1.0 0.8 0.6 0.4 1 0 1 2 3 4 SDSS catalog color (g-i) 106

the Tractor DECaLS Fin SED-matched detection 107