Historical lessons, inter-disciplinary comparison, and their application to the future evolution of the ESO Archive Facility and Archive Services Paul Eglitis and Dieter Suchar ESO, Karl-Schwarzschild Str. 2, 85748 Garching near Munich, Germany Chajnantor (5000m) Paranal (2600 m) La Silla (2400 m) EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 1
Introduction Inter-disciplinary thoughts The ESO Science Archive Facility Physical experiences: storage and infrastructure Future growth requirements Lessons Learnt Future evolution EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 2
Archive Requirements: Inter-disciplinary Variation and Similarity Space Mission Earth Observation Ground Based Astronomy Particle Physics Customer Science Service, Science Science Science Configuration Stable (Mission) Stable (Long Term) Variable Variable Data Timeliness On Opportunity Periodic Variable In house++ Data Volume Always more! Interoperability IPDA, SPASE EUROPLANET GEOSS, INSPIRE, GIGAS,HMA FITS, VO, IVOA E GRID EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 3
CHILE OBSERVATORIES On-Line Data Transfer DATA HANDLING STAGING Traveling Disk Replicated Database Infrastructure Replication HQ SECONDARY ARCHIVE PRIMARY ARCHIVE ESO SCIENCE ARCHIVE USER INTERFACES Replicated Database Infrastructure Local Databases EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 4
Physical Archive History GB Hardware Changes * * CD DVD * DISK * * Inexpensive Upgrades * cited.be Racks 2 0.5 0.25 Year EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 5
Multi-mission Concept in Archival VLT Very Large Telescope CRIRES CRyogenic high resolution InfraRed Echelle Spectrograph FLAMES Fibre Large Array Multi Element Spectrograph FORS1 FOcal Reducer/low dispersion Spectrograph 1 FORS2 FOcal Reducer/low dispersion Spectrograph 2 HAWK I High Acuity Wide field K band Imager ISAAC Infrared Spectrometer And Array Camera NACO Nasmyth Adaptive Optics System (NAOS) Near Infrared Imager and Spectrograph (CONICA) SINFONI Spectrograph for INtegral Field Observations in the Near Infrared Paranal UVES VIMOS VISIR X SHOOTER VLTI VLTI MIDI VLTI AMBER VISTA MASCOT Leading observatories and stateof-the-art instrumentation Ultraviolet and Visual Echelle Spectrograph VIsible MultiObject Spectrograph VLT Imager and Spectrometer for mid Infrared a multi wavelength (300 2500nm) medium resolution spectrograph Very Large Telescope Interferometer MID infrared Interferometric instrument Astronomical Multi BEam combiner Visible and Infrared Survey Telescope for Astronomy Mini All Sky Cloud Observation Tool NTT ESO 3.6m MPG/ESO 2.2m WFI Wide Field Imager FEROS The Fiber fed Extended Range Optical Spectrograph EFOSC2 ESO Faint Object Spectrograph and Camera HARPS High Accuracy Radial velocity Planet Searcher SOFI Son of ISAAC La Silla ALMA APEX VST E ELT The Atacama Large Millimeter/submillimeter Array Atacama Pathfinder Experiment telescope Chajnanator VLT Survey Telescope Future European Extremely Large Telescope Images Spectra Calibration and Quality Information Data products managed in the ESO Data Centre and Archive Facilities
ESO Archive System 2009 EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 7
Design and construction of the ESO Data Centre Requirements: Construction at different physical location On an area of 7 offices in an existing building Modular approach: Dismantle, move and rebuild the Data Centre at ESO premises.
Archive Holdings Past and Projected EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 9
Paranal and the ESO Very Large Telescope La Silla EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 10
VISTA-survey-telescope EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 11 Credit: Steven Beard/UKATC/ESO
ALMA ~ 1 rack/yr Credit: ALMA EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 12
E-ELT 17 PB/yr? EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 13
Lessons Learnt Preconditioning: physical constraints of data flow a.o. domain practices bias archive solutions. Physical installation: in-house, out-source (all or part)? Industry support: risks, costs, maintenance continuity. Engineering changes: return on investment, cost of change, operational costs. Art-of-balance: especially when a facility serves multiple purposes, e.g. Operations, Preservation, Exploitation. Interoperability: many parallel efforts are ongoing. Long-term view and inter-disciplinary initiatives like the PV- Series critical. EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 14
ESO Archive Future Evolution History of effective response to rapid changes has led to system heterogeneity Homogenisation of metadata and interfaces Development of new HMIs Standards compliance Converge on next storage technology: Analyse current usage of disk-based system in detail Assessment of growth and access requirements End-User, Operations, Maintenance and Development requirements The ESO Science Archive is a complex part of the complete operational Data Flow System at ESO. EGLITIS and SUCHAR PV2009 ESAC 03.12.2009 Slide 15
Thank you Acknowledgement: observatory images courtesy of the ESO Education and Public Outreach Department. See http://www.eso.org/gallery