SPIRE Spectrometer data analysis Rosalind Hopwood SPIRE Instrument and Calibration Scientist 26 th October 2016 Exploiting the Herschel Science Archive
Outline Documentation & literature Examining spectra Identifying problems Understanding the route cause of issues Problem case 1: point-like source in high Galactic cirrus Problem case 2: point-like source with a pointing offset Problem case 3: partially extended source Problem case summary Analysing the corrected data Spectral line fitting Estimating the spectral noise Mapping observations R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 2
http://herschel.esac.esa.int/twiki/pub/public/ SpireCalibrationWeb/FTS-flyer.pdf The flyer THE FTS ICC# Spe The SPIRE Fourier Transform Spectrometer SSW c Herschel Pho tro to SPATIAL DOMAIN: INTERFEROGRAM SLW SLW KEY LITERATURE SPIRE Handbook, SPIRE Data Reduction Guide Swinyard et al., 2014 Hopwood et al., 2015 WAVELENGTH COVERAGE SSW 447-1018 GHz 671-294 µm 944-1568 GHz 318-191 µm INSTRUMENTAL 970-440 kms-1 450-280 kms-1 LINE WIDTH HR 1.2 GHz 1.2 GHz Frequency dependent beam CALIBRATION ACCURACY Point-source : 6% Extended-source : 1% sparse mode & 7% mapping mode AVG. CONTINUUM OFFSET (additive): Point-source calibration Extended calibration MAPPING MODES 2.6 FoV Sparse: single pointing Intermediate: 4 point jiggle Full: 16 point jiggle SPARSE OR MAPPING RASTER PATTEREN User defined FoV Fringing Glitches SLW SSW 0.4 Jy 0.3 Jy 9.4x10-20 2.3x10-19 Wm-2Hz-1sr-1! Wm-2Hz-1sr-1! SLW CUBE SSW CUBE Small-scale shape in the spectrum Beats : strong CO ladder in the spectrum SPECTRAL DOMAIN: SPECTRUM SLW LINE PROFILE J=12-11 J=11-10 J=10-9 HR J=9-8 J=8-7 J=7-6 J=5-4 J=6-5 SSW SPECTRAL RESOLUTION HIGH (HR) 1.2 GHz, LOW (LR) 25 GHz Herschel Science Archive Level-2 SPARSE naming example for HR Point-source calibrated Spectra! HR_spectrum_point Extended calibrated! HR_spectrum_ext MAPPING naming example for HR SLW Spectral cubes! HR_SLW_cube / HR_SLW_cube_convol spectra! HR_SLW_spectrum2d R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 3
Reference material SPIRE DRG: Chapter 7 for the Spectrometer SPIRE Handbook: http://herschel.esac.esa.int/docs/spire/spire_handbook.pdf FTS calibration: Swinyard et al. 2014 (arxiv:1403.1107) FTS calibration program: Hopwood et al. 2015 (arxiv:1502.05717) LR calibration: Marchili et al. 2016 (arxiv:1610.02862) Extended-source calibration: Valtchanov et al. (in preparation) Spectrometer pipeline: Fulton et al. 2016 (arxiv:1603.00219) FTS Mapping: Benielli et al. 2014 (arxiv:1401.2040)* Experimental Astronomer, Volume 37, Issue 2, July 2014 Five FTS papers (*including the mapping paper above) Ø Ø Ø Ø Fulton et al. FTS RSRFs arxiv:1401.2049 Hopwood et al. Telescope model correction arxiv:1401.2047 Lu et al. Bright source mode arxiv:1401.2045 Valtchanov et al. Relative pointing offsets arxiv:1401.2043 SECT: Wu et al. 2013 (arxiv:1306.5780) Beam profile: Makiwa et al. 2013 (Applied Optics, Vol. 52, Issue 16, p. 3864-3875) R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 4
This analysis is HIPE based All Spectrometer data products are provided as FITS files and can be accessed using your preferred software See the SPIRE Spectrometer pipeline products presentation for the archive tar file folder structure and FITS file structure for Level-2 products R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 5
Useful scripts Within HIPE there is a set of Useful scripts These are designed to: help facilitate FTS data processing/analysis be easily understood be easy to edit to compliment each other These scripts are written in Jython and some could be easily transferable to Python Suggested workflow for Spectrometer data R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 6
Useful scripts A few more details Compares with a photometer map Subtracts extended emission using the off-axis detectors. HPDPs will be in the HSA soon. Ask the helpdesk if there is one for your observation in the interim POCT: estimates and corrects for pointing offset R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 7
Identifying problems Looking for problem -> documentation is your first port of call SPIRE Data Reduction Guide (SDRG): Chapter 7 for the Spectrometer Relevant sections for this session are: 7.4 Pointing considerations 7.5 Recipes for faint and medium strength sources 7.6 Recipes for semi-extended sources 7.11 Spectral Analysis 7.6.2 Does my spectrum need correcting? R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 8
Identifying problems 1 Point- source embedded in extended emission 2 Point- source observed with a significant poin8ng offset 3 A source with par8ally extended morphology 7.6.2 Does my spectrum need correcting? All three issues lead to a jump between SLW and SSW: Separa8ng the cause of the jump is necessary This may be difficult and some8mes not possible R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 9
Inspecting your data Inspecting an observation in the HSA Check out the summary information provided in the search results Click on the Postcard R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 10
Inspecting your data Inspecting an observation in the HSA Check out the summary information provided in the search results This is the Browse Product image For a sparse single pointed observation this is the spectra from the centre detectors R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 11
Inspecting your data Inspecting an observation in the HSA Check out the summary information provided in the search results This is the Browse Product image For a sparse single pointed observation this is the spectra from the centre detectors ANY PROBLEMS? Not for this one R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 12
Problem case 1 1 Point-source embedded in extended emission AFGL4106: post-red supergiant binary; medium source (> 10Jy) ANY PROBLEMS? R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 13
Problem case 1 1 Point-source embedded in extended emission AFGL4106: post-red supergiant binary; medium source (> 10Jy) ANY PROBLEMS? Yes, there s a jump between the bands R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 14
Problem case 1 1 Point-source embedded in extended emission Useful script: Spectrometer array footprint plot Plots the FTS footprint over a SPIRE photometer map If no photometer map inputted then photobsids used (if there are any) Photometer map should be FIR/ submm Use to identify a background/ foreground Use to check for contaminating neighbours Hopwood et al. 2015 Script edited for plot colours R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 15
SPIRE spectrometer data analysis Problem case 1 ① Point-source embedded in extended emission Useful script: Spectrometer array footprint plot Point-like Point-like sitting in Galactic Cirrus (IRAS 100 µm) Adapted from Hopwood et al. 2015 figure 2 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 16 Partially extended morphology
Problem case 1 1 Point-source embedded in extended emission AFGL4106: post-red supergiant binary; medium source (> 10Jy) ANY PROBLEMS? The extended background is pointsource calibrated leading to a poor continuum shape and extra fringing SLW is more strongly affected due to larger beam size (~2 SSW) Large-scale continuum shape can be corrected SLWC3 Poor shape < 750 GHz Fringing > 850 GHz for SLW SSWD4 Jump between the bands R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 17
Problem case 1 1 Point-source embedded in extended emission AFGL4106: post-red supergiant binary; medium source (> 10Jy) ANY PROBLEMS? The extended background is pointsource calibrated leading to a poor continuum shape and extra fringing SLW is more strongly affected due to larger beam size (~2 SSW) Large-scale continuum shape can be corrected SLWC3 Poor shape < 750 GHz Fringing > 850 GHz for SLW SSWD4 Jump between the bands The background subtracted HPDP could be your solution if not R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 18
Problem case 1 1 Point-source embedded in extended emission SLW 1 st ring Background subtraction script results Shape improved < 750 GHz Fringing > 850 GHz for SLW still there SSW 2 nd ring Jump between the bands is corrected R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 19
Problem case 1 1 Point-source embedded in extended emission Background subtraction using the off-axis detectors A word of caution There is real flux in the detector rings See Hopwood et al. 2015 for more details For a point source (Uranus/Neptune) bang on the centre there is around 1.9% in SLW 1 st ring Only 0.1% in SSW 2 nd ring (1.4% SSW 1 st ring: not used) Point source calibration accounts for this missing bit Lots more flux for semi-extended sources For AFGL4106 the results are worth the loss: fitting is easier SLW 1 st ring SSW 1 st ring SSW 2 nd ring BGS script results Jump is corrected Hopwood et al. 2015 AFGL4106 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 20
Problem case 2 2 Point-source observed with a significant pointing offset AFGL2688: point-like source See the DRG 7.4. Pointing Considerations R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 21
Problem case 2 2 Point-source observed with a significant pointing offset AFGL2688: point-like source Jump between the bands See the DRG 7.4. Pointing Considerations R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 22
Problem case 2 2 Point-source observed with a significant pointing offset AFGL2688: point-like source Check the metadata Jump between the bands See the DRG 7.4. Pointing Considerations R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 23
Problem case 2 2 Point-source observed with a significant pointing offset AFGL2688: point-like source radecoffset The offset between the commanded position and the actual reconstructed pointing, which includes any systematic BSM offset (bsmoffset), but not the APE. Jump bsmoffset BSM offset position (0.0 or 1.72 between arcsec), which is corrected for Check the metadata the bands by the point-source calibration. This systematic offset is included in the radecoffset value. radecoffset and bsmoffset give info on KNOWN pointing See the DRG offset. The APE is still associated to the pointing. 7.4. Pointing Considerations R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 24
Problem case 2 2 Point-source observed with a significant pointing offset Useful script: Pointing Offset Corrector (POCT) Assesses the jump between SLW and SSW bands to deduce and correct pointing offset For point-source calibrated spectra only For the centre detectors only Only use POCT to correct point-like sources The SPIRE calibration is required The method: SECT is used to create a grid of overlap ratios by offsetting the source model on a grid of pointing offsets The pointing offset is derived by interpolating the initial overlap ratio from the grid The spectra are corrected for pointing offset with SECT Input your observation ID at the top of the script and let POCT run it can take a while R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 25
Problem case 2 2 Point-source observed with a significant pointing offset Useful script: Pointing Offset Corrector (POCT) For this observation POCT finds an offset to the requested pointing of 6.08 ± 0.52 -> known (relative method) offset = 6.21 ± 0.32 Corrected results are the green and brown spectra Optimal offset Jump is corrected R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 26
Problem case 3 3 A source with partially extended morphology NGC6302: the Butterfly nebula with FWHM 18 See the DRG 7.6. Recipes for semi-extended sources R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 27
Problem case 3 3 A source with partially extended morphology NGC6302: the Butterfly nebula with FWHM 18 Large jump between the bands See the DRG 7.6. Recipes for semi-extended sources R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 28
Problem case 3 3 A source with partially extended morphology NGC6302: the Butterfly nebula with FWHM 18 Check the metadata See the DRG 7.6. Recipes for semi-extended sources - radecoffset is minimal (0.25 ) - OD is > 1011 so bsmoffset is 0 So what s the issue? R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 29
SPIRE spectrometer data analysis Problem case 3 ③ A source with partially extended morphology SPIRE Spectrometer calibration Point-like Semi-extended Extended Point-like Semi-extended Extended See the DRG 7.6. Recipes for semi-extended sources R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 30
SPIRE spectrometer data analysis Problem case 3 ③ A source with partially extended morphology SPIRE Spectrometer calibration Point-like Semi-extended Extended Point-like Semi-extended Extended NGC6302 See the DRG Point-source calibrated spectra 7.6. Recipes for of a semi-extended source semi-extended sources R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 31
SPIRE spectrometer data analysis Problem case 1 ① Point-source embedded in extended emission Useful script: Spectrometer array footprint plot Point-like Point-like Adapted from Hopwood et al. 2015 figure 2 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 32 Partially extended morphology
Problem case 3 3 see Wu et al. 2013 DRG Section 7.6 & the Handbook A source with partially extended morphology SECT: correcting the data for extent The SECT algorithm is based on the forward coupling efficiency to the source distribution: so a good knowledge of the source distribution is key Here we use the default SECT settings with: - a basic elliptical Gaussian - FWHM of 18 - Eccentricity of 0.2 - No x- or y-offset And setting the source model diameter to be optimised (optimisediameter=true) HOWEVER the correction efficiency (η c ) is assumed to be 1, which may not be true and can introduce extra uncertainties R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 33
Problem case 3 3 A source with partially extended morphology NGC6302: the Butterfly nebula with FWHM 18 Optimum diameter calculated as 18.5 SECT plots the uncorrected and corrected spectra Jump is corrected see Wu et al. 2013 DRG Section 7.6 & the Handbook R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 34 There is room for improvement: Needs better knowledge of the source model
Summary Three key problems that SPIRE Spectra can suffer have been addressed 1. High extended background: background subtracted HPDPs will be available in the HSA soon for this issue soon contact the Helpdesk to see if there is such an HPDP already created for your observation subtracting the off-axis detectors subtracts some real signal weigh up the benefit relative to this 2. Unknown pointing offset: the offset can be estimated using the overlap region not easy to correct in this way for semi-extended sources, unless you have good knowledge of the source distribution 3. Partial source extent: the SECT method should be used a good knowledge of the source distribution is needed η c may not be 1 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 35
Workflow on corrected data The corrected spectra are ready for scientific analysis Spectral line fitting: there is a useful script for line fitting, but if using a fitter outside of HIPE, a few things to remember: The line shape is well represented by a sinc profile Due to asymmetry in the line profile, there is a 2.6% shortfall in line flux calculated from the fitted sinc parameters See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 36
Workflow on corrected data The corrected spectra are ready for scientific analysis Spectral line fitting: there is a useful script for line fitting, but if using a fitter outside of HIPE, a few things to remember: Apodized data may help when searching for faint lines, but fit and take measurement from the standard spectra See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 37
Workflow on corrected data The corrected spectra are ready for scientific analysis Spectral line fitting: there is a useful script for line fitting, but if using a fitter outside of HIPE, a few things to remember: Fit to the lines and continuum simultaneously to achieve the best fit See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 38
Analysing the corrected data The corrected spectra are ready for scientific analysis Estimating the spectral noise: there is a useful script for noise estimation, but the method is straightforward and easy to perform outside of HIPE, a few things to remember: Lines should be removed before taking the noise the sinc wings cause ringing across the frequency range, which is the dominant noise source for spectra with strong features See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 39
Analysing the corrected data The corrected spectra are ready for scientific analysis Estimating the spectral noise: there is a useful script for noise estimation, but the method is straightforward and easy to perform outside of HIPE, a few things to remember: Lines should be removed before taking the noise the sinc wings cause ringing across the frequency range, which is the dominant noise source for spectra with strong features Correcting semi-extended data or correcting for pointing offset will increase the noise estimate because the data are corrected for lost flux by a multiplicative factor See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 40
Analysing the corrected data The corrected spectra are ready for scientific analysis Estimating the spectral noise: there is a useful script for noise estimation, but the method is straightforward and easy to perform outside of HIPE, a few things to remember: Lines should be removed before taking the noise the sinc wings cause ringing across the frequency range, which is the dominant noise source for spectra with strong features Correcting semi-extended data or correcting for pointing offset will increase the noise estimate because the data are corrected for lost flux by a multiplicative factor The next two slides illustrate these points for problem case 3 NGC6302 See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 41
Analysing the corrected data The corrected spectra are ready for scientific analysis Estimating the spectral noise for problem case 3 NGC6302 (Method: standard deviation of baseline subtracted data in frequency bins) Green dashed is noise estimate for the data straight from the archive Blue is the noise estimate for SECT corrected. Which goes up after data is corrected up Lines dominate the noise NGC6302 See DRG Section 7.11 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 42
Analysing the corrected data The corrected spectra are ready for scientific analysis Estimating the spectral noise for problem case 3 NGC6302 (Method: standard deviation of baseline subtracted data in frequency bins) Blue is the noise estimate for SECT corrected The noise is above HSpot due to SECT Orange is after first fitting to the strongest lines Red is noise estimate for the SECT corrected and line subtracted data NGC6302 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 43
Mapping: spatial resolution From the SPIRE instrument introduction presentation 4-jiggle pattern 16-jiggle pattern R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 44
Mapping: coverage SLW inter, 35 SSW inter, 19 SLW full, 17.5 SSW full, 9.5 Cube Image Error Coverage (naïve cube) R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 45
Mapping SSW cubes for NGC 7023 (1342189896) LR, fully sampled, special calibration raster from OD 261 Naïve cube mean of spectra within a pixel CP cube Gaussian weighted mean of spectra within kernel Integrated maps over 1250-1270 GHz CP cube is smoother (lower noise) Convolved spatially, so no difference in line shape Naïve cubes suffer from holes : no coverage CP cubes can have partial spectra at the edges: low coverage They have the same WCS See DRG Sections 7.8 & 7.12 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 46
Mapping SSW cubes for the Orion Bar (1342228734) LR, fully sampled, special calibration raster from OD 261 There are a number of tools & scripts in HIPE that are geared for Herschel cube analysis Integrated line flux plot from the Spectrometer Cube Fitting script See DRG Sections 7.8 & 7.12 R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 47
Q&A Not all of the useful scripts available in HIPE were covered If you have any questions on these, or the ones that were, or any other Spectrometer problems, send them to the helpdesk for the afternoon Q&A Thanks for attending! R. Hopwood Exploiting the Herschel Science Archive: SPIRE Spectrometer data analysis ESAC 26/10/2016 48