HIFI webinar: News, Calibrations, and Data Processing Updates

Size: px

Start display at page:

Download "HIFI webinar: News, Calibrations, and Data Processing Updates"

Stephany McDonald
5 years ago
Views:

1 NHSC HIFI Webinar December 17, 2012 HIFI webinar: News, Calibrations, and Data Processing Updates Pat Morris, Adwin Boogert, Colin Borys with contributions from D. Teyssier & C. McCoey

2 Outline 1. Observatory and HIFI Instrument update 2. New level 2.5 products (HIPE 9 &10) 3. Improved pointing products for data taken between OD320 and 761 (HIPE 9.2) 4. HEB standing wave removal script 5. Sneak peek at new features in HIPE 10 - HSPOT Parameters - DBS reference contamination estimator - Row Flag summary Table - Improved FlagTool - Bright Line flag usage in deconvolution There is a formal Q&A afterwards, but feel free to ask questions at any point during the presentations.

3 Observatory News

4 Observatory+News+ Today+we+are+1313+days+into+the+mission.+ All+systems+are+func<oning+normally.+ End+of+cryogen+expected+in+early+March years+~8+months+of+science+ (accoun<ng+for+the+commissioning+phase).++ Project+will+conduct+~10+days+of+engineering+ tests+ajer+he3lboil.+ + The+Mission+Opera<ons+team+will+then+move+the+ telescope+out+of+l2,+and+put+it+into+a+heliocentric+ orbit+for+re<rement.+ The+telescope+will+not+be+steered+into+the+Moon+for+ sublsurface+water+inves<ga<ons+due+to+costs.+

5 Program+Comple<on+ By+the+end+of+OD#1303+(7+December+2012),+the+ approximate+comple<on+percentages+are:++ >99%+KPGT,+>99%+KPOT,++ >99%+GT1,+96%+GT2++ 98%+OT1p1,+87%+OT2p1+ 24%/4.6%/0.4%+OT1p2+OT2p2+Rank+1/2/3.++ Approximately+10+weeks+(70+ODs,+21h+per+OD)+ remaining+in+herschel s+science+mission.+ We+have+15+ODs+of+released+Priority+1+AORs+available+ (based+on+21h+science+per+od).++ A+further+209+ODs+of+Priority+2+AORs+are+released.+

6 What+has+been+scheduled+so+far+ Science+AORs Science+Hours HIFI PACS SPIRE All To OD Total number of AORs scheduled (incl. cal & engineering): ( h) Statistics from M. Kidger, R. HSC.

7 What+Priority+1+leJ+to+release+ AORs Hours HIFI PPhoto PSpec SPhoto SSpec SPParallel Total The total number of ODs of un-released AORs 2. Almost all are in time constrained/too programmes.

8 What+Priority+2+leJ+to+release+ AORs Hours HIFI PPhoto PSpec SPhoto SSpec SPParallel Total The total number of ODs of un-released AORs 4.

9 Some+Observing+Stats:++ Instrument+Mode+Distribu<on+in+the+HSA+ [Includes science, calibration+engineering] This+gives+a+ qualita<ve+feel+for+ the+amount+of+<me+ spent+per+mode,+ through+aug Since+August,+several+ long+hifi+campaigns+ of+mainly+ot1+p1+ have+increased+ observing+to+20%+of+ total+(appearance+of+ Orion,+Sgr/GC,+ Taurus,+etc).+ +

10 Herschel+topical+Mee<ngs+of+Interest+ 28L31+January:++Herschel+mapLmaking+workshop+at+ESAC,+ Madrid+herschel.esac.esa.int/2013Mapmaking.shtml+ Late+February:++next+HIFI+DP+webinar,+likely+focus+on+baselines+ and+standing+wave+treatment.++to+be+announced+in+nhsc s+el newsleger.+ 25L27+March:++Herschel+Calibra<on+Workshop+(open)+at+ESAC,+ Madrid.++herschel.esac.esa.int/Calibra<onWorkshop5.shtml+ 17L21+March:++Infrared+and+Submillimeter+Probes+of+Gas+in+ Galaxies:+From+the+Milky+Way+to+the+Distant+Universe,+ Pasadena,+CA+conference.ipac.caltech.edu/gasconf/+ 15L18+October:++The+Universe+Explored+by+Herschel+at+ESA/ ESTEC,+Noordwijk,+The+Netherlands+ herschel.esac.esa.int/the_fir_universe.shtml+ +

11 HSA+and+Product+Deliveries+ Currently+there+are+ UPDPs +from+9+programs+linked+at+the+hsc s+web+pages.+

12 UPDPs+ All+the+files+have+been+checked+(and+modified)+to+be+ readable+by+any+standard+fits+viewer+and+hipe+ The+first+HIFI+delivery+(HIFISTARS)+has+been+posted:+ two+simple+spectrum+(usb+++lsb)+++the+original+class+ file+per+observa<on+(366)+ Many+more+are+under+checking+of+format+(HEXOS,+ HOBY, )+ Next+version+of+the+HSA+(around+March+2013)+will+ implement+the+system+to+distribute+updps+from+the+ interface.++

strong+enough+to+meet+required+frequency+ calibra<on+accuracy.

13 HIFI+Instrument+News+ The+instrument+is+opera<ng+normally.+ WBSLV+comb+signal+has+been+fading+but+s<ll+ strong+enough+to+meet+required+frequency+ calibra<on+accuracy.++hrs+xlcal+is+available.+ No+loss+of+observing+<me+due+to+a+Single+Event+ Upset+(CR+hit+on+sensi<ve+electronics+or+areas+of+ memory)+has+occurred+in+the+last+few+months.+

14 New+HIFI+Flier+ Visit+nhscsci.ipac.caltech.edu+ + For the latest info on HIFI DP topics, calibrations and documentation, visit herschel.esac.esa.int/twiki/bin/view/public/hificalibrationweb

15 Level 2.5 Products

spectral scans, it contains a default deconvolved product For both cases, all defects unaccounted for at the

16 Level 2.5 Products Introduction of a level 2.5 for maps and spectral scans For maps, it contains the cubes that were so far present in the level2 For spectral scans, it contains a default deconvolved product For both cases, all defects unaccounted for at the level2 are propagated into these products (e.g. baseline distortion, unflagged spurs) These are used to generate the BROWSE PRODUCTS Seen in both the archive and the obscontext within HIPE

17 Browse Products For spectral scans these are the deconvolved level 2.5 products For maps, a combination of collapsed spectra over the map and integrated cubes over the whole spectral coverage In HIPE 9, this is done over the stitched 4 GHz WBS range, in HIPE 10 this is separated per WBS sub-band (see next slide) Browse product for spectral scan Browse product for map in HIPE 9

18 Map Browse Product in 10

19 Configurable Pipeline In general, level 2.5 is meant to be science grade As an example, a spectral scan is usually meant to be run through the dodeconvolution task to produce a final spectrum. The level 2.5 pipeline can be adapted to fit other needs. In this example, I have added the fitbaseline task in lights out mode to clean up the data before it is deconvolved.

20 Configurable Pipeline In general, level 2.5 is meant to be science grade As an example, a spectral scan is usually meant to be run through the dodeconvolution task to produce a final spectrum. The level 2.5 pipeline can be adapted to fit other needs. In this example, I have added the fitbaseline task in lights out mode to clean up the data before it is deconvolved.

21 Improved pointing products for data collected between OD 320 and 761

22 Pointing improvements Summary of affected data between OD 320 and 761: 3887 HIFI observations affected, with offsets up to 7.6 (Taurus area) Shift can be discontinuous with a mapping observation Model of the problem created, tested, and incorporated into HIPE 9.2 herschel.esac.esa.int/twiki/bin/view/public/howtouseimprovedpointingproducts! The above web page contains a lookup table of obsid and offset. Also included are detailed instructions on how to correct your observations

23 Distribution of pointing issues Distribution of astrometry shifts Discrepancy with modelled predictions Shift for worse case in Taurus!

24 Shifts during map observations Flux in map will likely be redistributed after pointing correction!!

25 Improved pointing products (4) DISCONTINUOUS SHIFTS OVER A MAP: AFFECT CALIBRATION MONITORING TOO!! L2 PUP CO 8-7 FOR FPG3 TESTS NGC7023H2 C+ IN CYCLE 41!!

History of pointing performance SEE http://herschel.esac

26 History of pointing performance SEE Five main periods over the mission OD# RANGE 0L L L866 Description warm +STR+(star+tracker),+presence+of+sporadic+soLcalled+ speed+ bumps STR+temperature+lowered+to+solve+the+above,+resul<ng+in+an+ increased+scale+plate+error+and+degraded+poin<ng+performance 1LD+correc<on+(i.e.+same+linear+factors+along+Y+and+Z)+uploaded+to+ solve+str+focal+length+at+low+temperature 866L1011 Improved+2LD+correc<on+permanently+uploaded APE# (arcsec)# 1.9L L L full+str+fp+distor<on+correc<on+uploaded,+and+on+odl1032,+73+ dubious+stars+removed+from+the+str+catalogue 0.8L0.9

27 NHSC HIFI Webinar December 17, 2012 HIFI Webinar: HEB bands 6 and 7 Standing Wave Removal Adwin Boogert based on slides provided by David Teyssier and Max Avruch

HEB Standing Waves Strongest standing waves in level 2 spectra HEB bands 6 and 7 have electronic, not optical origin. Optical waves also present in bands 6 and 7, but are weaker.

28 HEB Standing Waves Strongest standing waves in level 2 spectra HEB bands 6 and 7 have electronic, not optical origin. Optical waves also present in bands 6 and 7, but are weaker. HEB waves are not sine waves. Exact shape and amplitude depends on power on detectors. FitHifiFringe can approximate HEB waves with 2-3 sine waves. More reliable removal of HEB waves requires special treatment: match on-source spectra with off-source spectra from large database

HEB Matching Technique As of HIPE 9, a script is available that subtracts best-matching off spectra, i.e., it modifies the standard HIFI pipeline: scripts/hifi/scripts/users/engineering/hebstwvcatalogcorrection.

29 HEB Matching Technique As of HIPE 9, a script is available that subtracts best-matching off spectra, i.e., it modifies the standard HIFI pipeline: scripts/hifi/scripts/users/engineering/hebstwvcatalogcorrection.py Not a standard task. Only available in expert branch of HIPE. This is advanced users script! Currently covers only DBS mode Continuum is conserved Version 0.52 available in HIPE 9.2 and 10.0 Possible to use ad hoc public obsids to feed a larger set of OFF spectra use data from the same mixer band! (6/7). Band 6A: obsid was made public.

HEB Matching Technique Example on obsid 1342196583 WBS-V old WBS-H old Use OFFs from same

30 HEB Matching Technique Example on obsid WBS-V old WBS-H old Use OFFs from same observation WBS-V old WBS-V new WBS-H old Use OFFs from other (much longer) observation. WBS-V new

31 HEB Matching Technique For every on-source spectrum, these plots are produced (if option plot=true): Heavily smoothed on and off spectra (smoothing speeds up processing) Before and after spectra On minus best Off On Best Off (smoothed)

32 HEB Matching Technique Tips on how to run script: This script is for advanced users! Contact NHSC if you need help! Copy script to local directory: scripts/hifi/scripts/users/engineering/hebstwvcatalogcorrection.py There is no manual, but script contains point-by-point instructions, including which parameters to modify: Which observation to process Which observation to use for reference OFF spectra Create or use calibration database? Process WBS-H or WBS-V (not both!) Plot intermediate results? Script runs pipeline, which requires local calibration database hifi-cal Run script line by line using HIPE's single green arrow

33 Future developments: HEB Matching Technique Make script work for non-dbs modes Data mining for more standing wave templates Speed up processing and reduce memory usage by storing spline points in table instead of spectra. Make more user-friendly task

34 HIPE 10 Preview

35 HSPOT parameters Pre-HIPE 10, many useful values determined (or entered) with HSPOT were available only within the AOR.

36 HSPOT parameters Introduction of HifiUplinkProduct! Propagates more information on the parameters of the as-performed observation (e.g. achieved noise, exact map size) from HSpot In principle transparent to the user use in pipeline to define optimal cube size in maps Useful also to recall the requested smoothing width

mkdbsreftask Introduction of reference spectra automatically computed for DBS observations Consists of the difference between the OFF phases in the two nods Stored in the

37 mkdbsreftask Introduction of reference spectra automatically computed for DBS observations Consists of the difference between the OFF phases in the two nods Stored in the calibration>pipeline-out>referencespectra product Expect degraded baseline since this is a simple difference that do not optimally correct standing waves from the unmatched optical paths

38 mkdbsreftask Important to inspect this in your data Positive signal means flux is present in the first nod position. Flag signal means either the two reference beams are emission free or have the same emission (not likely)

39 Row Flag summary Introduction of a row flag summary table at level1 (quality product) Limited to on-target science frames of level1 products Useful for getting a sense of where data needs user attention Some functionality will be folded into the flagtool

40 flagtool HIPE10 has an overhaul of the flagtool! More user friendly, and easier to find and flag anomolous data. Hooks into fitfringe and fitbaseline make it a one stop shop for HIFI data reduction. Left Complete list of all spectra LO value Indicator for each subband Right Spectrum viewer. Flags color coded and listed Top Flag selector Bottom Hooks to other tools Undo operator

41 flagtool

42 BRIGHT_LINE/decon New option to ignore data flagged as a BRIGHT_LINE! Useful for ensuring decon does not propagate flux incorrectly Should only affect lines >>10K Decon also allows ghostfinding natively.

43 The end QUESTIONS

esac Herschel/HIFI Spectral Map Highly Processed Data Products Release Notes

esac Herschel/HIFI Spectral Map Highly Processed Data Products Release Notes esac European Space Astronomy Centre (ESAC) P.O. Box 78 28691 Villanueva de la Cañada, Madrid Spain NHSC NASA Herschel Science Center IPAC, Caltech 1200 E. California Blvd, Pasadena, CA, USA Herschel/HIFI