The INES Archive in the era of Virtual Observatories E. Solano Spanish Virtual Observatory LAEX CAB / INTA-CSIC PV2009, ESAC, Dec 2009
Outline Introduction: Archives in in Astronomy The IUE Archive Curation and preservation Added-value products Catalogues Virtual Observatory Conclusions
Astronomy: a communal organisation Community-oriented instruments Public data (after short proprietary times) Community-oriented S/W Common format (FITS) Archives have become a fundamental tool for for modern astrophysics
Archives in Astronomy Efficiency: One set of of observations can serve many different scientific purposes, including some not considered when the observations were made.
Archives in Astronomy Exploitation of ofthe thetime domain: Periodic phenomena: Variable stars Transient phenomena: SN, GRB
The IUE mission Jan1978 Sep1996 > 11000 objects and > 110000 spectra. Data archiving was considered a fundamental activity from the very beginning of the IUE project.
The IUE archive: a thirthy years history 1978: First astronomical archive with fully calibrated data. Science-ready data.
The IUE archive: a thirthy years history 1986: ULDA: First astronomical archive in the Internet (data access and data distribution via e-mail). More than 235 000 spectra since 1987 to to1995.
The IUE archive: Data curation The IUE Final Archive (1990 s): The problem: Along the years of the mission, different reduction systems and calibrations were used making unreliable the comparison among data observed at different epochs. Goal I: Produce a uniformly processed, fully-intercomparable archive. Improve photometric accuracy and S/N ratio by applying new image processing algorithms. Quality info. at pixel and spectrum level. Quality control on metadata. Cross-match with previous catalogues and hand-written observing logs. Goal II: Deliver IUE data to the community in a simple and efficient way.
Accessing IUE data: The INES system
The connectivity problem in the nineties Goal II: II: Deliver IUE data to tothe thecommunity in in a a simple and efficient way. Nowadays
INES: The IUE Distribution System in the nineties but butin in the thenineties No No Youtube, Google, Wikipedia or orfacebook Only a a few newspapers and magazines have begun to to put put their articles online. Your computer at at home takes about 20 20 seconds to to load each page using aa modem. Costa Rica First link link to tothe theinternet in in 1993. Only 12 12 nodes (research institutes). Speed: 64 64 Kbps
INES: The IUE Distribution System in the nineties A Principal Centre (LAEX) and a a Mirror Site (CADC) both containing the the Master Archive A number of of National Hosts containing low low resolution spectra.
Data preservation in the INES system Master Archive (all): 429 GB (DVD s 3.9 GB) Fire/water proof safe. On-line archive: 129 GB. (disk) + backup (tapes)
Adding value to the archive: The INES guides The INES Guides: Even though access to to INES data is is extremely easy the quantity of of data in in the archive is is so so large that the task of of addressing the data efficiently in in the context of of an an astrophysical problem is is not simple. Goal of ofthe guides: Facilitate the use of of INES data for for scientists with a specific astrophysical problem in in mind.
Adding value to the archive: Accessing other archives Many astrophysical projects require the analysis of of data covering different wavelengths of ofthe electromagnetic spectrum.
Interoperability The ability of of diverse systems providing diverse information (spectra, images, catalogues, bibliographic info., ) to to work together.
The Interoperability problem in Astronomy Data Discovery: How can astronomers find the the relevant data to to their scientific needs? Data access & transfer: Astronomers need to to learn about different user interfaces, access and download procedures. archive 1 service 1 archive 2 archive 3 survey 1 survey 2 service2 service3 survey 3
The Interoperability problem in Astronomy Data characterisation: lack of of Data Models to to describe similar observations in in the same way.
The Interoperability problem in Astronomy
The solution: The Virtual Observatory INES was one of of the first spectroscopic All archives the world s in in the world to to become VO-compliant. astronomical data should feel like it sits on the astronomer s desktop International agreement on standards. 2002 -- Standard semantic: UCDs Standard access protocols Science Standard output format Standard data models Tools Automated discovery tools (registries) Uptake by the Data Centres Data Grid. Stds
What s the result of all these efforts? A&A 90 80 70 60 50 40 30 20 10 0 2003 2004 2005 2006 2007 2008 Year Queries 90000 80000 70000 60000 50000 40000 30000 20000 10000 0 2007 2008 2009 SSAP Web Year
Summary Astronomical archives represent a fundamental tool in in modern Astrophysics. IUE, a pioneer project in in this field. The Virtual Observatory is is a consolidated international initiative that constitutes the framework where to to develop archive-related activities. INES, the IUE Archive System, is is being intensively used more than 30 30 years after launch and more than ten years after the end of of operations. In In particular, VO accesses are growing every year. INES is is an an excellent example of of how archives can extend the projects effective lifetime reducing very considerably the euro per observation cost.