The Herschel-SPIRE Point Source Catalog Feasibility Study Report

Transcription

1 1 The Herschel-SPIRE Point Source Catalog Feasibility Study Report Bernhard Schulz 1, Gabor Marton 2, Kevin Xu 1, Nanyao Lu 1, David Shupe 1, Ivan Valtchanov 3, Babar Ali 1, Chris Pearson 4, John Rector 1, Tanya Lim 3 1) NHSC-IPAC Caltech, 2) Konkoly Observatory, 3) HSC-ESAC ESA, 4) RAL-STFC (NHSC) 07-Aug-2014 V0.2 THE HERSCHEL-SPIRE POINT SOURCE CATALOG... 1 GOAL AND EXPECTED BENEFITS... 2 PLAN... 3 GLOBAL DATASET PARAMETERS... 4 SCAN MAPS AND COVERAGE... 5 ASTROMETRY... 7 SOURCE EXTRACTION... 8 COMPARISON OF ALGORITHMS... 9 The Test Data... 9 Detection... 9 Photometry... 9 BASELINE EXTRACTION ALGORITHM OPTIMAL PARAMETERS Sussextractor Parameters Timeline Fitter Parameters RELIABILITY COMPLETENESS BACKGROUND CONFUSION SEMI EXTENDED SOURCES TIME ESTIMATES THE CONSOLIDATION PHASE THE DATABASE CATALOG ENTRIES MANPOWER AND SCHEDULE SUMMARY APPENDIX THE ALGORITHMS MISCELLANEOUS REJECTED IDEAS Band Merging Moving and Variable Sources... 25

2 2 High Resolution Maps REFERENCES: Goal and Expected Benefits Generating a source catalog out of the almost seven thousand scan map observations that SPIRE made appears to be a natural follow-up activity after the extraordinary stability of the instrument and its relatively simple design allowed to produce such high quality maps in an automatic way. To keep the task within achievable limits, we aim to build a homogeneous SPIRE Point Source Catalog (SPSC) from all SPIRE scan map observations, geared primarily at the extragalactic community, that can serve as a pathfinder for ALMA and other Submm and Far-IR facilities for many years to come. Photometric treatment of slightly extended sources and crossidentification/association of sources between the three filter bands (band-merging), are currently considered to be outside of the baseline scope of this endeavor, but could be investigated for future projects at another time. From our preliminary study we expect to find a list of several million sources out of observations that cover about 11% of the entire sky if we include turn-around data and calibration observations. Turn-around data cover non-nominal parts of the observation. While the telescope moves from the end of the previous scan leg to the start of the next one, the detectors keep taking data. This data is not easily accessible at the moment and would be included as a bonus. Such a catalog would provide the best possible photometry derived by instrument experts with optimized extraction algorithms. The extraction would be performed in a homogeneous way out of well characterized celestial environments with associated figures for reliability, completeness, photometric, and positional accuracies. The availability of the planned catalog will fill in SED photometry of astronomer s favorite objects without the need for them to invest significant time into establishing their own Herschel data reduction processing and the associated learning curve. The homogeneous source extraction enables a systematic and unbiased comparison of sensitivity across the different SPIRE fields that single programs will generally not be able to provide. The homogeneous coverage of larger fields will provide more data for studies of surface density and clustering of galaxies. Catalog homogeneity is further helped by the fact that SPIRE scan maps are already confusion limited after the first two repetitions.

3 3 Further benefits of such a catalog include statistical studies of galaxy dust mass and emissivity, diffuse dust emission along the Hubble sequence, sub-millimeter galaxy counts, and luminosity functions when combined with already existing databases. The work on the feasibility study in itself has already provided a number of lessons about how certain source extractors work. It has led to the derivation of better operating parameters, and better ways of running the standard pipeline. Even if an SPSC may eventually not be realized, the study activity now already benefits greatly our work within the SPIRE ICC and thus eventually the quality of the legacy archive products and our support of the astronomical community. Many aspects of the program can also be seen as standalone projects that offer their individual benefits independent of the SPSC itself. An example would be the planned characterization of reliability and completeness dependent on a background confusion classification that could be served out of a Herschel archive on its own. Plan The original plan to realize the SPSC was to start with a limited feasibility study, followed by a production phase in case of a positive outcome of the former. The feasibility study would assess the data volume and general parameter space of the project. It would determine major work packages and already build prototype software for basic algorithms and tools. The study needed to touch upon all foreseeable technical and scientific issues and establish a baseline approach for the production phase. We achieved the major goals of the study and identified a number of items that needed to be added to the original plan. Specifically it turned out that an intermediate source database (SpireCat) was needed to collect the source extraction results and we started implementation of a prototype. We found that unless we wanted to wait two more years, we needed to re-process the observations ourselves to allow for our best knowledge about data processing to be included and for turnaround data to be available as well. We also realized that the regionally varying foreground confusion by Galactic Cirrus could not be sufficiently treated by just excluding the Galactic Plane and the alternative plan to establish Confusion Class Maps and a related study of the impact of these classes on reliability and completeness was begun. The production phase will consist of three parts, first a source extraction phase where observations are reprocessed as necessary and source lists are extracted from Level 2 or Level 2.5 maps and the associated Level 1 timelines. A schematic diagram is shown in Fig. 1. The source lists are ingested into a relational database. Once all observations are processed in this way, a consolidation phase will follow, that will cross identify multiple detections of the same object, identify and flag semi-extended

4 4 sources, and extract the final catalog with additional flags and qualifiers. No cross wavelength matching is foreseen, although the previously mentioned multiple detection identification will also provide the necessary tools for a future endeavor like this. The third phase will consist of systematic quality checking of the final catalog and preparing an explanatory supplement or similar publication. Figure 1: Schematic depicting the processing steps necessary to build the SPSC. Observations from the Herschel Science Archive (HSA) are re-processed and saved in a Local Pool database. Source tables are extracted and ingested into a relational source database (SpireCat). Confusion maps that are also created from the reprocessed observations together with the source database are then used to produce the final SPIRE Point Source Catalog. Note that re-processing and source extraction are designed to be parallel processes that can run on several independent machines. Global Dataset Parameters The SPIRE photometer could be operated in five different mapping modes, Small Scan Map, Large Scan Map, Parallel, Point Jiggle, and Small Jiggle Mode. Only the first three of these modes have ever been used for science observations. The Small Jiggle Mode, also known as 64-Point Jiggle Mode, was never released, while the Point Source Jiggle Mode was immediately replaced by the Small Scan Mode. Thus any jiggle maps can safely be excluded for our purposes. The archive contains a total of 6917 scan map observations (5825 excluding calibration observations) that split into 4204 Small Scan Maps, 1880 Large Scan

5 5 Maps, and 833 Parallel Mode Maps (3594, 1475, 756 respectively without calibration). The center locations of all SPIRE scan maps are plotted in Figure 2, with color distinguishing the three remaining modes. Note the chain of red dots along the Galactic Plane that belong to the two degree wide strip observed by the HIGAL program. Figure 2: Positions of all SPIRE Scan map observation on the sky. Scan Maps and Coverage To obtain a rough estimate of the number of point sources we can expect, we needed to make an estimate of the area actually covered by the SPIRE maps. Such an estimate requires some more detailed consideration of the scan patterns performed by the telescope. SPIRE scan maps are observed in a back-and-forth pattern as illustrated in Figure 3. The 4 x8 detector arrays of hexagonally arranged feed horns scan the sky at an angle of 42.4 degrees w.r.t. the Z-axis to reduce the distance of scan tracks between adjacent feed horns and to achieve Nyquist sampling or better. The sky is normally covered twice in almost perpendicular scan directions to suppress 1/f noise in the image reconstruction. Only very few Small- or Large Scan observations exist without a cross-scan direction. Parallel Mode observations have only one scan direction and normally come in pairs that are combined into a Level 2.5 map.

6 6 During the periods where the telescope moves from the end of a nominal scan leg to the start of a new one, the detectors keep taking data. These so-called turnaround data exist in the Herschel archive, but are currently stripped away by the standard pipeline as their quality was initially unclear. No negative effects in these datasets have been found meanwhile and including these will boost the area coverage of the SPIRE maps. Figure 3: In the left column a flux map (above) and its coverage map (below) are shown as currently produced by standard pipeline processing. To the right the same maps are shown including turn-around data. The thin blue arrows indicate nominal scans, the green arrows indicate turn-around scans. The Thick blue arrows show where additional sky coverage was achieved. Using only the nominal length of the scan maps to determine map sizes underestimates the covered area because of i) the scan length refers only to the center of the detector array although it has an extent of 4 x6, and ii) the additional area covered during the turnaround phase. We correct for both effects in a rough geometrical way with special consideration for the Single Scan Maps and we correct for the duplicate coverage on Parallel Mode maps. This results in an estimated total of 4723 square degrees of sky coverage or about 11.4% of the entire sky. A break-down by observing mode is shown in Table 1.

7 7 All scan maps squ. deg fraction of all sky SpirePacsParallel % 6.1% SpirePhotoLargeScan % 4.8% SpirePhotoSmallScan 208 4% 0.5% Total % 11.4% Table 1: The areas observed in the different scan map modes by SPIRE are expressed in square degrees, percentage of the total area covered, and in percentage of the entire sky. Removing the turnaround data would reduce the covered sky area to 10.8% of the entire sky, equivalent to a decrease by 6% of the sky actually covered by SPIRE. The calibration data cover a similar area and would reduce the covered sky to 10.9%. Removing any observation within +/- 2 deg from the Galactic Plane to avoid problems with confusion would reduce the overall coverage by about 20%. Depending on the detection threshold, the coverage area, and the source density, we estimate to find a total of about 3 Million point sources if we limit the flux to 5 times the confusion limit ~ 30 mjy (Nguyen et al. 2010). Astrometry Out to just before the first Airy ring the SPIRE beams are very well approximated by Gaussian profiles with FWHM of 18.2, 24.9, 36.3 for PSW, PMW, and PLW respectively. With a currently assumed reconstructed pointing accuracy of about 2 the deviations are already substantially smaller than the beam profile and generally not a big issue for SPIRE maps. The efforts to improve the Herschel pointing further and bring it to below 1 by using the information from the spacecraft gyroscopes are quite successful but have not been implemented yet fully into the HIPE pointing product generation. We consider the relative astrometry of the three SPIRE bands to be known sufficiently well that we only need to derive the astrometry shift for the 250 µm map and apply that also to the 350 and 500 µm maps. We have developed a prototype script to estimate the overall astrometry shifts in SPIRE maps using the all sky WISE point source catalog. The WISE band W4 (22 µm) detections have a good correspondence to SPIRE 250 µm

8 8 sources. The script obtains the WISE all sky catalog entries for a given map area through the Gator interface of IRSA. Then it stacks small SPIRE 250 µm maps (usually 7x7 or 11x11 pixels) at the positions of the 22 µm detections. Centroiding of the stacked image, using a Gaussian fit, provides an average astrometry shift in RA and Dec. At this point it is still to be decided whether to provide the correction as additional information in the catalog or whether to provide corrected source positions only. The decision will also depend on whether the WISE catalog is always able to provide enough sources for a reliable correction. The information will first be stored in the source database where it will allow a statistical assessment of the pointing accuracy. Source Extraction A number of different algorithms has been tested for source detection and for photometric accuracy. The exercise brought both, a better understanding of the available algorithms and resulted in several corrective actions on the HIPE software and extraction parameters. The algorithms considered are listed in Table 2. More details about the algorithms are given in the Appendix. Table 2: The point source extraction algorithms compared in this exercise. The performance results are based on artificial source injection trials performed with SPIRE timeline and map data of the GOODS-N field.

9 9 Comparison of Algorithms The Test Data For their source detection capabilities we tested only Sussextractor, Getsources and Starfinder. We used a very deep observation of the GOODS-North Field that was split into 7 partial maps of 4 repeats each. This field shows no significant Galactic Cirrus but as being confusion limited shows a multitude of point sources that are generally representing the colder dust in background galaxies. We injected Gaussian point sources at timeline level (Level 1 data), reconstructed the maps, and ran the different algorithms to extract all sources. The flux levels with 20, 25, 30, 35, 40, 50, 60 mjy were grouped close to the expected turnover due to confusion (~30mJy). Then we compared how many sources were recovered at a given flux level and compared extracted with injected photometry. Detection The results for source detection in all three detector arrays are shown in Figure 4 in the top row. The y-axis shows the number of recovered sources and the total of all injected sources, while the x-axis shows the flux of the injected sources. Initially Sussextractor picked up more sources at the faint end than the other two. However, after discovering that we didn t use Getsources in the right way and working with the creator of the algorithm Alexander Men shchikov, we didn t find substantial differences in the detection rate anymore. The Starfinder may also not have been used correctly, but a substantially improved performance didn t appear likely with improved parameters, so we didn t pursue the case further. The diagram still shows the initial results. Photometry The photometric comparison is shown in the row below on Figure 4. In this case the Timeline Fitter reproduces the injected fluxes much better than any of the competing algorithms. This confirms results from similar tests performed by the SPIRE-ICC before. Note that the diagram still shows the initial results for Getsources and Starfinder, which are better with correct operation. Nevertheless, the photometric performance of Timeline Fitter is unlikely to be achieved. This study showed small dependencies on the number of sources injected and whether we avoided existing sources for the source injections or not. The latter certainly introduces a statistical bias and avoidance radii should not be used. Apart from that the study may still suffer from low number statistics. However, the observed variations didn t change the overall outcome, identifying Sussextractor as a very good source extractor and the Timeline Fitter as superior photometry extractor. The only caveat we would need to make is that a sky region with Galactic Cirrus confusion was not tested yet and would probably show somewhat different results.

10 Comparison&to&simula/ons&9& source&recovery #"of"detected"sources"over" the"#"of"injected"sources With"avoidance: SPSC Milestone Meeting, 28 Feb 2014 Comparison&to&simula/ons&9& source&recovery Measured"flux"over"the" injected"flux Without"avoidance: Radius"used"to"match"sources:"FWHM Radius"used"to"match"sources:"FWHM/2 Figure 4: Comparison of three algorithms for their abilities to detect injected sources of different fluxes around the confusion limit of the GOODS-N field and six algorithms for their ability to recover the correct flux. Baseline Extraction Algorithm Based on the results from the source injection exercise we chose Sussextractor for the source detection and Timeline Fitter for photometry as a project baseline. Both did very well in the tests and both are available within HIPE, avoiding additional, potentially tedious interfacing effort with the Herschel archive and SPIRE data reduction. Timeline Fitter requires more effort as it starts with Level 1 timeline data. However, it also offers special options to discriminate extended emission and we think its superior photometric accuracy merits the added inconvenience. Figure 5 shows a flow diagram of the baseline point source extraction procedure. It caters for the special case that parallel mode observations generally require two observations linked together. Since the archive doesn t provide combined timeline products of linked parallel mode observations, this case requires special treatment. Note that the Timeline Fitter is run twice, once with a fixed, circular PSF model, and a second time with a variable size, elliptical PSF model that also allows for tilted backgrounds. The parameters harvested from the initial Sussextractor source detection run together with the results from the two Timeline Fitter runs will allow for 10

11 11 discrimination of extended sources and potentially also spurious source detections due to residual strong glitches. Optimal Parameters Exercising the extraction algorithms led to a deeper understanding thereof and initiated some investigations that resulted in improved operating parameters. Sussextractor Parameters Sussextractor used a PRF model that was only 5x5 pixels by default. It was realized that a 13x13 pixel PRF improved the detection rate but also changed the photometry. Further tests using simulated data with a single injected point source showed that the photometry changed slightly for 3x3, 5x5, and 7x7 pixel PRFs. From sizes of 9x9 pixels and above the brightness derived by Sussextractor remains constant. To minimize the influence of other sources in the vicinity, we chose a 9x9 pixel PRF. Figure 5: Flow diagram of the baseline source extraction algorithm. Sussextractor functions as source detector and Timeline Fitter that works on signal timelines derives the photometry in two subsequent runs. The signal timelines of Parallel Mode observations must be combined before using Timeline Fitter on them. These simulations showed also that Sussextractor doesn t work without at least a small amount of noise in the timeline data. We found that the derived brightness is slightly dependent on the background noise. Since the SPIRE noise maps are calculated from the

12 12 scatter of the readouts within the respective pixels, the noise maps will also have higher values in regions with high flux gradient because of the finite size of a map pixel. Sussextractor takes S/N into account, so the steep flanks of a point source profile will contribute less if the background noise is lower. In practice this is fortunately not an issue as long as the point sources are not too bright, but we will consider Sussextractor photometry unreliable for brighter sources. Figure 6: Brightness values returned by Sussextractor for the same simulated 1Jy source using different size Point Response Functions (PRF) for the three filters. The 5x5 pixel PRF coincidentally returns 1 for PSW and PMW. Through these simulations it was also found that the photometry of Sussextractor can vary by at least +/- 2% depending on where the source is placed w.r.t. the pixel center. We didn t simulate a full raster of positions, so the result is not fully quantified yet. Nevertheless it shows that the photometry of Sussextractor as implemented in HIPE at this time has limits that are still detectable given the SPIRE detector uncertainties. Timeline Fitter Parameters A closer inspection of the default parameters of Timeline Fitter revealed that, with an inner radius of 300 and an outer radius of 350, it was using an exceedingly large annulus to determine the source background leading to a substantial penalty in execution time. The area for the annulus and also the amount of data used for the background determination was around 18 to 67 times larger than the core area that contains the source, depending on the filter band. The area covered by the annulus could be reduced to a much smaller size that is more similar to the core area, without significantly changing the photometry. Based on the known radial structure of the PRF we determined different radii for the three SPIRE bands (see Figure 7). Their areas are only slightly larger than the respective core area, resulting in good photometry with substantially reduced execution times. In addition the

13 13 annuli are located outside the second Airy ring of the PRF to reduce contamination by the source. The values are listed in Table 3. Figure 7: The SPIRE average PRFs plotted against radius in arcsec. From left to right the linked arrows indicate the core radius, and inner and outer radii of the background annuli. The colors black, orange and green stand for the detector arrays PSW, PMW, and PLW respectively. PSW PMW PLW Core radius [ ] Inner radius [ ] Outer radius [ ] Table 3: Parameters optimized for use with Timeline Fitter, where the area of the background annuli with inner and outer radius was reduced to be similar to the area of the core region. Reliability Reliability and completeness are very important characteristic parameters for an astronomical catalog when its data is used statistically. The SPIRE data is somewhat special in that respect as effectively all maps are confusion limited and instrument noise plays a less important role.

14 14 We determine the reliability function as the ratio of spurious detections and the maximum number of possible detections within a given flux-bin. For this study we determined the reliability function from the same deep observation of the GOODS-North field that was also used in the source injection trials described earlier. However, in this case we didn t use injected sources, but all detected point sources in the image. The dataset was divided into 7 maps, constructed from 4 repeats each. The baseline point source extraction was run on all 7 maps. For each source appearing in a map, we determined the nearest counterparts in all other 6 maps within 0.5*the FWHM of the PRF. Such a set of detections was listed as a candidate object and all detections belonging to this candidate object were flagged so they could not be used a second time. Plots for all three arrays of the number of map detections versus the average flux of a given candidate object are shown in Figure 8 in the left column. Since this is no simulation we have to guess what a "real" object is. We set a tentative threshold of n detections with 1<n<7 that are at least necessary to count a given candidate object in our list as a real object. All detections of candidate objects with less than n detections are considered spurious detections PSW TML (real source at least 4 detections) flux [mjy] PMW TML (real source at least 4 detections) flux [mjy] PLW TML (real source at least 4 detections) flux [mjy] reliability reliability reliability PSW TML flux [mjy] PMW TML flux [mjy] PLW TML flux [mjy] Figure 8: Determining reliability of source detections. On the left the number of maps a given source was detected in, out of 7 maps total is plotted versus flux. On the right the detections are translated into a reliability factor depending on source flux. See text for details.

15 15 We split the flux range between 0 and 160 mjy into 11 bins. For each flux bin we determined the reliability function as (1 - n spurious /n max ) where n spurious is the number of spurious detections and n max is the maximum possible number of detections, i.e. the number of real objects times the number of maps (7 in this case). We also determine an alternative reliability function out of the non-detections of real objects. It is derived as (1- n fail /n max ) where n fail is the total of all differences between the number of maps and the number of detections for all real objects, e.g. an object with 5 detections would contribute a value of 7-5=2 to the sum that gives n fail. We analysed four different thresholds (n=3,4,5,6) and the use of source fluxes from either Timeline Fitter (TML) or Sussextractor (SUS). The most obvious difference between the fluxes from both methods is the threshold where SUS fluxes never drop below, while TML fluxes don t show that. This threshold is already at so low fluxes and reliabilities that it is not important for our purposes. There is a balance between the two alternative reliability functions, depending on the setting of the threshold. They become similar for a threshold of 4 detections. We adopt this threshold and the Timeline Fitter fluxes and find the fluxes with 99% reliability are around 60 mjy for normal scan speed maps with 4 repeats. These numbers don t vary much for other acceptable thresholds of n=3 or 5. The resulting reliability functions for the three arrays are shown in Figure 8 on the right, connected by a solid line. The alternative functions derived from the failures to detect a real object are plotted with a dashed line. Figure 8 shows only the plots for n=4 and TML fluxes. This analysis still needs to be repeated for different observation configurations, i.e. other numbers of repetitions and different scan speeds and data rates. The approach taken by Smith et al is still under investigation and is expected to yield similar results. Completeness We define the completeness function as the percentage of times an existing source is detected, depending on the source flux. The source injection experiments with the GOODS-North field data, as described above, determine completeness as a function for a map of an empty region at standard scan speed and with 4 repetitions. The diagrams in Figure 4 indicate already very similar flux levels for completeness as what was found for reliability. The similarity is likely due to the fact that the maps are confusion limited and that spurious detections are very few compared to non-detections with the former governed by instrument noise and the latter by background confusion. The source injection tests made so far still have issues with statistical bias and will require a substantial increase in realizations to eliminate effects due to small number

16 16 statistics. In addition, as with reliability tests, these should be conducted for different observational parameters as well as fields with different Galactic Cirrus confusion. Background Confusion Although the SPSC extraction is expected to be homogeneous, the celestial background, especially at SPIRE wavelengths, is not. Parameters like reliability and completeness most likely depend on the strength of background confusion and are expected to change not only with observation parameters, but also with background structure, specifically that caused by Galactic Cirrus and structure within large resolved nearby galaxies. Similarly the accuracy of extracted photometry could also be affected. To improve the statistical usefulness of the SPSC, we plan to develop a classification of confusion in SPIRE maps and divide the maps into corresponding regions. For that we have determined a suitable, statistically meaningful subset of 160 SPIRE scan map observations. They are selected based on four parameters, their RA and DEC, the duration of the observation, and the mapped area. We used the statistical package R and a model-based clustering algorithm to select this representative sample. According to this, the overall ensemble of scan map observations consists of three major groups, i) short observations with a small footprint and an even distribution on the sky (3825 total), ii) HiGal like observations with a small to medium footprint, distributed close to the Galactic Plane (1527 total), iii) all other observations, specifically those with the longest durations and largest coverage regions (451 total). Out of these groups a representative sample of 160 observations was drawn to conduct our analysis of confusion classes. The investigation will be conducted by: i) first identifying tools for classification, ii) comparing identified classes with visual classification and refining the tools, iii) determining reliability and completeness for each distinguished class, iv) integrating the classification into the overall SPSC source extraction scheme and generating maps that identify regions of different confusion levels. These will be used to assign a corresponding flag to the sources in the catalog. Such a classification can also stand alone and be useful to astronomers in an archive, even if no SPSC is eventually produced. Semi Extended Sources This project aims to produce a catalog of point sources, where the footprint on the sky is indistinguishable from the known beam profile. However, there is a class of semiextended sources that have a slightly larger FWHM but will still be picked up by the detection algorithm. The extension is typically due to the source s relative proximity, extended emission from dust or gas, or sources that are so close together that they are hard to distinguish at the SPIRE spatial scales.

17 17 To provide additional data that allow discrimination and flagging of these sources, the baseline source extraction procedure performs two passes of the Timeline Fitter: One with a round fixed-width PRF assuming a flat background, and another one with more free parameters, i.e. an elliptical variable-width PRF with tilted background. While the first run provides the photometry for point sources, the second run will identify inconsistencies when compared to the first. We will use statistics derived from the data accumulated in the relational source database to determine good thresholds and criteria to flag semi-extended sources. Time Estimates The times involved into carrying out the different phases of the project depend on the final implementation, so we built prototypes at least for the data re-reduction phase at the beginning, and the subsequent source extraction phase. Table 4: Execution times for baseline source extraction routine applied to the first 83 observations in the HSA. The times are set in relation to each other and to the time it actually took the Herschel spacecraft to perform them. The time of the first test is longer as it includes download times from HSA to IPAC. The first tests were performed using the baseline source extraction procedure with the first 83 SPIRE scan-map observations in the Herschel Science Archive (HSA). The results are listed in Table 4. The tests were performed on a dedicated SPIRE computer at IPAC under HIPE that uses local data storage (Local Pools). Thus the first test took substantially longer than the subsequent tests, because all datasets, especially the Level 1 datasets, were transferred from the HSA and kept in the Local Pool. All first three tests contained one source detection stage with Sussextractor and a subsequent photometry

18 18 stage with only one Timeline Fitter run. This was changed to the current baseline configuration with two Timeline Fitter runs for Test 4. The table sets the execution times in relation to the time it took originally for the observation to commence. The total observing time Herschel spent on SPIRE scan map observations is days. Assuming the conditions of Test 4, it would take for our test machine alone about 38 days to extract the point sources. With more machines running in parallel, this time would be cut down proportionally. The real bottleneck seems to be the data download to IPAC. There are several options that need to be considered before sources can be extracted. 1. Download data from the HSA to IPAC and perform processing and extraction at IPAC using dedicated real and virtual machines. 2. Use Rsync Database of SPIRE data already at IPAC that has not been updated since the end of the mission a year ago and perform processing and extraction at IPAC. 3. Use SPIRE account at ESAC and do processing and source extraction on the grid at the HSC. 4. Make source extraction procedure part of the pipeline and produce a standard source list product that is fast to transfer and can be fed into the relational source database. This would also move the work needed for reprocessing require to wait until HIPE 13.1 is released and the scan map data is reprocessed. Closer inspection of Option 2 showed problems with missing products in the rsync database, discarding it from the list. Option 3 has the advantage that no data transfer needs to take place as all data is located nearby. The option of using the HSC grid offers additional speed and allows a well-defined task sharing between local groups in the team, but comes at the cost of a more difficult interfacing back to the relational point-source database at IPAC. Because of general needs of the HNSC-SPIRE team and as a fall-back solution we downloaded the Level 0 data of all SPIRE scan maps during the month of July that was processed from raw telementry with HIPE 11. This approach insures that data reprocessing including turnaround data and using the newest techniques (Two-Pass Pipeline) can also be performed at the NHSC. Options 3 and 4 would move the reprocessing to the HSC site and reduce download times considerably. The last option would use standard reprocessed data of HIPE 13 but would be a long way in the future. For the time being we pursue looking further into Options 3 and 4 but meanwhile continue with Option 1. The Consolidation Phase Once the source tables are extracted and ingested into a database, the consolidation phase can begin. At this point detections of identical sources in the same filter by different observations need to be found and consolidated. Furthermore source detections that are

19 19 obviously false or unreliable need to be identified as well as detections of semi-extended sources. The multitude of parameters produced by the three main source extraction algorithms will have to be statistically analyzed and the final catalog entries have to be produced. Statistical analysis will for instance produce better thresholds to discriminate semi-extended sources. The main activities during this phase will be: 1. Determine where same source has been observed multiple times in the same filter. 2. Find best position of object and uncertainties. 3. Find the most likely brightness of object and uncertainties. 4. Flag extended sources, confusion level, and any other issues. 5. Produce final catalog table. The Database To examine the details of the consolidation phase and its interface to the preceding source extraction phase, we designed a relational database that would fit our requirements. In particular the need for high performance coordinate search and the tabular character of our datasets, excluded an object-oriented database as used in the Herschel Project. We opted for a freely available Postgres relational database that can be set up with very fast Q3C two-dimensional indexing to support the matching step for multiple observations of the same sources. The basic schema is illustrated in Figure 9. The parameters of the 6917 observations are stored in the same number of rows in the Observation Table with the observation ID acting as primary key. Source extraction processes will run on each of these entries and their results stored in the Source Table. This table will also be able to accommodate different versions of source detections in case more than one extraction run must be performed on a given observation in case of anomalies. These processing runs are recorded in the Processing Run Table with history, error code, processing parameters and comments that may be needed during the consolidation phase. The Run List Table coordinates the distribution of jobs among all source extraction processes that run in parallel, to ensure that all jobs are done exactly once. If certain observations need to be processed again because of a bug, the respective flags in the Run List Table would be reset to ready and the extraction processes are restarted. The last table is the Group Table, which effectively is the final catalog. Each row is linked to by one or more entries in the source table that all refer to the same group of sources that have nearly identical positions and are most likely the same object at a given wavelength. The remaining, still to be defined, columns will be derived during the Consolidation Phase, taking into account the data of all source entries in the Source table that are associated with a given object in the Group Table.

20 20 This design may still change in a few details but provides the necessary functionality and flexibility for our project. The roughly expected 3 Million sources represent a reasonable size that poses no problem for this type of database. Figure 9: General layout of the relational source database. It functions as the scheduler and staging area for the source extraction phase, and as the starting point of the coordination phase. Catalog Entries We envision the SPSC to come in three parts, one for each SPIRE filter band. The first column will be a unique source identifier. Once source positions are consolidated, effective positions in RA and DEC (J2000) can be derived including uncertainties. The positional offsets derived from WISE source stacking may be immediately added, but could also be added as separate columns to maintain consistency with the SPIRE maps in the HSA. This will be decided during Consolidation Phase. Similarly source fluxes with uncertainties will be derived from all linked data available in the Source Table. Further columns that we can identify today will be: i) Number of repeats in a map that is correlated with the expected instrumental noise, ii) the number of separate detections of an object, which usually happens in overlaps of maps and if portions of the sky were observed multiple times, iii) flags indicating background confusion noise and associated

21 21 reliability and completeness, FWHM indicating a semi extended source, map quality, and others, iv) and finally the approximate time of the observation which will aid TNO and Asteroid hunters. This short list of entries is preliminary at this point but covers the main components that an astronomer would expect. Manpower and Schedule A project like this requires a certain base of expertise, but also manpower that is not taken away easily by outside tasks that appear occasionally in the team with urgency. The current members of our team are listed in Table 5 below. The expertise regarding SPIRE photometry and mapping present in this group is unparalleled as it includes several members that are associated with the SPIRE project from as far back as instrument component testing. The list is divided into workforce and advisors depending on the amount of contributions that are likely to be available from the individuals, and depending on their respective projected contractual situation over the next year. Unfortunately none of the participants is currently able to spend a major fraction of their time on this project, and many results in this report have originated from other work that fell under the responsibility of the ICC. In fact a number of investigations, like the photometric accuracy of Sussextractor and Timeline Fitter, were triggered by work for the SPSC feasibility study and are good examples for the work on the catalog benefitting the overall understanding of the instrument and ultimately the astronomer and the product quality in the archive. It appears that about two FTEs could complete this task within a year. Considering the actually available manpower fractions within the current team, this goal could only be achieved within that time, if at least some members were able to increase their contribution substantially. In this case a timeline as given in Table 5 could be feasible. Otherwise we estimate a 2 year timeframe as more realistic for completion.

22 22 Workforce Bernhard Schulz (lead) NHSC SPIRE team lead coordination, algorithms, source extraction Kevin Xu NHSC SPIRE photometer scientist reliability, completeness Babar Ali (temporary March July 2014) NHSC SPIRE photometer scientist confusion classification Gabor Marton Postdoc at Konkoly observatory (Budapest) source extraction, reliability, completeness Ivan Valtchanov ESA SPIRE spectrometer scientist (Madrid) Astrometry John Rector NHSC System Architekt Database Tanya Lim ESA SPIRE photometer scientist (Madrid) Photometry Advisors Nanyao Lu NHSC SPIRE spectrometer scientist scientific advisor Dave Shupe NHSC PACS instrument scientist scientific advisor Chris Pearson ICC SPIRE manager, calibration scientist (Oxford) scientific advisor Table 5: A list of current team members split into workforce and advisors, with their respective specializations. Date Milestone Description 31-Jul-2014 Transfer of Level 0 data products to NHSC 31-Aug-2014 Baseline source extraction procedure tested and ready 30-Sept-2014 Relational source database tested and ready 30-Sept-2014 Confusion class maps ready 31-Oct-2014 All data reprocessed to Level 2 or 2.5 (maps) 31-Dec-2014 Source extraction complete, start of consolidation 28-Feb-2015 First catalog table 31-May-2015 Final catalog verified Table 6: Preliminary list of major milestones of the project.

23 23 Summary We have investigated the major ingredients needed to build a Herschel SPIRE Point Source Catalog based on the relatively mature understanding of the scan map data of the SPIRE instrument. In a Feasibility study that ran over four months, we developed and refined a plan for implementation consisting of a data re-reduction and source extraction phase, and a consolidation phase. We expect the catalog to cover about 11% of the sky, containing approximately 3 million sources in all three filters with a flux cutoff at 5 times the confusion noise level. Based on tests of different source extraction algorithms we selected and coded a prototype baseline source extraction algorithm and optimized some of its parameters. We started investigating background and foreground confusion and expect to generate maps indicating the confusion class for every location. We found that a dominant factor in processing time is the actual transfer of data from the HSA to IPAC. Several mitigation strategies can be pursued that should allow the first phase to complete in a few weeks. The benefits of such a catalog to the astronomical community is not only an expert derived homogeneous, and well characterized catalog of point sources in the Submm, but also additional knowledge about SPIRE data, that will benefit SPIRE standard pipeline processing and the SPIRE legacy products in the Herschel archive. The study didn t find any major technical obstacles and most of the key abilities are already developed. The main uncertainty lies in the available manpower for the project. Currently we seem to have just barely enough contributors to make the project possible. An increase of manpower that brings the available total to about 2 FTEs within the next year would alleviate those concerns.

24 24 Appendix The Algorithms We compared six algorithms that are given in the first column of Table 2. Some exist within the HIPE software environment, others are implemented in IDL or Fortran. Sussextractor is a source detector and photometry extractor based on Bayesian model selection and the Bayesian information criterion (Savage & Oliver 2007). The algorithm is provided with a source model and uses maximum likelihood map and threshold to identify sources, and determine source brightness and background. The method was tested as Java implementation in the HIPE environment although also IDL implementations exist. Simultaneous Extractor is a photometry extractor that needs a list of source positions as input. It measures point source photometry of multiple sources simultaneously using a linear inversion method (Roseboom et al. 2010, HCSS User s Reference Manual). Daophot was tested in its Java implementation in HIPE that is based on the FIND and APER procedures of the IDL Astronomy User s Library. The method smoothes the image with a convolution kernel to find source positions like in Sussextractor and then derives fluxes using aperture photometry (Landsman 1995, HIPE Data Analysis Guide). Timeline Fitter is a photometry extractor that fits a 2D Gaussian PSF model to destriped SPIRE Level 1 detector timelines. The original prototype by Bendo (Bendo et al. 2013) was an IDL implementation that was later implemented in Java within the HIPE environment (SPIRE Pipeline Specification Manual). This tool is particular as it was used exclusively in deriving the SPIRE photometric calibration using Neptune. Getsources is described as a multi-scale, multi-wavelength source extraction algorithm (Men shchikov et al. 2012). It was specifically developed for use with far-infrared surveys of Galactic start-forming regions with Herschel. It analyzes fine spatial decompositions of original images across a wide range of scales and across all wavebands. It was tested in its Linux based Bash and Fortran implementation. Starfinder is an IDL based point source extractor designed to analyze Adaptive Optics (AO) images of crowded stellar fields (Diolaiti et al 2000). Flux and position of a given point source are estimated by matching its image with a scaled and shifted copy of the PSF. Individual PSFs are re-inserted into the residual image to perform aperture photometry and minimize uncertainty due to neighboring sources.

25 25 Miscellaneous Rejected Ideas Band Merging The telescope mirror of Herschel is large by spacecraft standards, but the highly sensitive SPIRE instrument is still confusion limited due to the rather large PRFs produced by the telescope at the SPIRE wavelengths. The different PRF sizes in the SPIRE filters that vary by a factor of two in FWHM make it difficult to determine whether two faint point sources found at different wavelengths at the same position originate from the same object. Often single sources at 500µm split up into double or even multiple sources at 250µm. Usually only more detailed studies and additional data from other wavelengths, taking into account the nature of the sources involved, can answer these questions. For this endeavor we consider the implementation of an automatic band-merging impractical with the means we have available. Maybe this can be considered again for future work. Moving and Variable Sources Moving and variable objects are not the prime concern of this catalog, but a potential need for specialized scientific projects is recognized. Specifically the identification of moving objects within the Solar System constitutes a separate science project that we see outside of the scope of this project. However, we plan to support such projects by providing the observation times associated with a specific source. High Resolution Maps We considered the potential benefits of using enhanced maps where the spatial resolution was increased by treatment with the HiRes algorithm. Initial experiments with cluster fields did not look promising and we abandoned this idea. References: The Spectral and Photometric Imaging Receiver (SPIRE) Handbook, HERSCHEL-DOC- 0798, version 2.5, March 17, 2014 Nguyen, H., Schulz, B., Levenson, L. et al. 2010, A&A 518, L5 Smith, A.J. et al. 2012, MNRAS 419, 377 Savage & Oliver 2007, ApJ 661, 1339 Roseboom et al. 2010, MNRAS 409, 48 HCSS User s Reference Manual, Herschel Data Processing Version 10.0, Doc#: HERSCHEL-HSC-DOC-0935, 23 Feb 2014 Landsman 1995, ADASS IV, ASP Conf. Ser. Vol. 77, Payne, Hayes, eds., p 437

26 26 HIPE Data Analysis Guide Version 11.1, Doc #: HERSCHEL-HSC-DOC-1199 SPIRE Pipeline Specification Manual, Version 2.3, Doc #: SPIRE-RAL-DOC , 02 June 2013 Men shchikov et al. 2012, A&A 542, A81 Diolaiti 2000, A&A Sup. 147, 335 Bendo, G. et al. MNRAS, 433, 3062 (2013)