Project Engineer SKA Science Data Processor Consortium
|
|
|
- Bryan Briggs
- 6 years ago
- Views:
Transcription
1 SKA Science Data Processing Storage Requirements BigStorage Initial Workshop 3 rd March 2016 Bojan Nikolic Astrophysics Group, Cavendish Lab University of Cambridge b.nikolic@mrao.cam.ac.uk Project Engineer SKA Science Data Processor Consortium
2 SKA Science Data Processing Storage Requirements BigStorage Initial Workshop 3 rd March 2016 Bojan Nikolic Astrophysics Group, Cavendish Lab University of Cambridge b.nikolic@mrao.cam.ac.uk We are hiring! PostDocs + PhD students with computing background Project Engineer SKA Science Data Processor Consortium
3 Outline Introduction to the Square Kilometre Array SKA Science Data Processor Storage Requirements SDP Architecture (SKA Science) Q&A
4 THE SQUARE KILOMETRE ARRAY TELESCOPE
5 What is the Square Kilometre Array (SKA)?
6 Mid frequency dishes and dense aperture array concept
7 Low frequency array and the survey telescope concept
8 Phased Aperture array for MHz
9 Phased Aperture array: 3 million antennas
10 Scientific Context a partner to ALMA, EELT, JWST Credit:A. Marinkovic/XCam/ALMA(ESO/NAOJ/NRAO) Credit:ESO/L. Calçada (artists impression) Credit: Northrop Grumman (artists impression) Credit: SKA Organisation (artists impression)
11 Scientific Context a partner to ALMA, EELT, JWST ALMA: 66 high precision sub-mm antennas Completed in 2013 Budget ~1.5 bn USD European ELT ~40m optical telescope Completion ~2025 Budget ~1.1 bn EUR Credit:A. Marinkovic/XCam/ALMA(ESO/NAOJ/NRAO) JWST: 6.5m space near-infrared telescope Launch 2018 Budget ~8 bn USD Credit:ESO/L. Calçada (artists impression) Square Kilometre Array Two next generation low-frequency arrays Completion ~2022 for Phase 1 Budget 0.65 bn EUR for Phase 1 Construction Credit: Northrop Grumman (artists impression) Credit: SKA Organisation (artists impression)
12 What will the Square Kilometre Array (SKA) be? Radio Telescope Makes Images of the Sky at radio (5m-3cm) wavelengths ~100x faster than current telescopes Complements ALMA, JWST (successor to Hubble), and E-ELT Currently in Design Construction begins 2018 Full operations expected at end 2022 Good fraction of funds already committed by participating countries Major Engineering Project Major ICT Project Two remote desert sites >100k receiving elements Subject of this talk!
13 SCIENCE DATA PROCESSOR CONTEXT
14 The SKA Observatory Phase 1 : SKA 1 People, buildings, roads, ground works, communications, electrical power, maintenance, transportation, catering at desert sites. Data Processing: general computing Digital Signal Processing: FPGA + (maybe) ASICS & GPUs Receptors: Aperture Arrays and Dishes
15 SKA Context Diagram Monitor and Control SKA1 Low: Low Frequency Aperture Array LFAA Correlator/ Beam Former Science Data Processor Implementation (Australia) Pulsar Search Processor (Australia) SKA1 Mid: Dish Antennas with Single-Pixel feeds SKA1 Mid Correlator/ Beam Former Pulsar Search Processor (South Africa) Science Data Processor Implementation (South Africa) Astronomers These are offsite! (In Perth & Cape Town)
16 Role of computing/processing in SKA: COMPUTING IS THE MAJOR PART OF SKA TELESCOPES BY DESIGN
17 Large D vs Large N GBT 100-m diameter telescope SKA LFAA prototype array No 1 aim: collect as many photons as possible No 2 aim: maximum separation of collectors -> achieve high angular resolution
18 SKA SCIENCE DATA PROCESSOR OVERVIEW
19 SDP Top-level Components & Key Performance Requirements -- SKA Phase 1 Science Data Processor Telescope Manager SDP Local Monitor & Control C S P 1 Tera Byte/s Data Processor High Performance ~100 PetaFLOPS Data Intensive ~100 PetaBytes/observation (job) Partially real-time ~10s response time Partially iterative ~10 iterations/job (~3 hour) Data Preservation High Volume & High Growth Rate ~100 PetaByte/year Infrequent Access ~few times/year max Delivery System Data Distribution ~100 PetaByte/year from Cape Town & Perth to rest of World Data Discovery Visualisation of 100k by 100k by 100k voxel cubes Regional Centres & Astronomers
20 SDP Top-level Components & Key Performance Requirements -- SKA Phase 1 Science Data Processor Telescope Manager SDP Local Monitor & Control C S P 1 Tera Byte/s Data Processor High Performance ~100 PetaFLOPS Data Intensive ~100 PetaBytes/observation (job) Partially real-time ~10s response time Partially iterative ~10 iterations/job (~3 hour) Data Preservation High Volume & High Growth Rate Goal ~100 PetaByte/year is to extract information from data and Infrequent then discard Access the data ~few times/year max Delivery System Data Distribution ~100 PetaByte/year from Cape Town & Perth to rest of World Data Discovery Visualisation of 100k by 100k by 100k voxel cubes Regional Centres & Astronomers
21 Key Characteristics of SKA Data Processing Noisy Data Sparse and Incomplete Sampling Corrupted Measurements Multiple dimensions of data parallelism Very large data volumes, all data are processed in each observation Corrected for by deconvolution using iterative algorithms (~10 iterations) Corrected by jointly solving for the sky brightness distribution and for the slowly changing corruption effects using iterative algorithms Loosely coupled tasks, large degree of parallelism is inherently available
22 Radio Telescopes make noisy measurements Sums of rows Illustration only not real data Sum of all Can clearly see the fringes!
23 Radio Interferometers sample sparsely & irregularly Sampling - Each pair of telescopes results in a measured visibility - Loosely equivalent to a sample in the Fourier domain - Inevitably areas without any samples: - At high radius -> limit of attainable resolution - At small radius -> limit of largest detectable structure - Regular distribution of telescopes leads to a regular distribution of samples in uv plane Illustrative example for the 27 antenna JVLA!
24 Radio Interferometers sample sparsely & irregularly Sampling - Earth rotation effective moves the sampling point of each pair of telescopes - This limits the possible integration time duration - Fills in the gaps between the previous samples (but not the central gap and outside the limit) Illustrative example for the 27 antenna JVLA!
25 Radio Interferometers sample sparsely & irregularly Sampling - More rotation Illustrative example for the 27 antenna JVLA!
26 Radio Interferometers sample sparsely & irregularly & redundantly! Redundant + Sparse Sampling With SKA1 there will be very many individual visibilities in each observation: Accumulation of visibilities improves the signal/noise Fills in most of the uv plane In general however to retain best signal/noise significant unevenness in sampling of the uv plane must be accepted. -> Deconvolution Illustrative example for the 27 antenna JVLA!
27 Measurements are imperfect corrupted by slowly changing mechanical, electrical & atmospheric effects Uncalibrated Offset Calibration Rick Perley & Oleg Smirnov: High Dynamic Range Imaging,
28 Iterative & joint solving for the image of the Sky & Calibration Self-Calibration closure error calibration Rick Perley & Oleg Smirnov: High Dynamic Range Imaging,
29 Data-parallelism schemes Data-parallelism schemes Time & baseline Frequency Visibility data Exploit frequency independence o o o Data parallelism: Dominated by frequency Provides dominant scaling Nothing more needed if each processing node can manage a frequency channel complete processing Grid and degrid Buffered UV data FFT Processing nodes
30 Data-parallelism schemes Time & baseline Frequency FFT Visibility data Exploit frequency independence Sort and distribute visibility data and target Grid and degrid Gather target grids o o o o Further data parallelism in locality in UVW-space Use to balance memory bandwidth per node Some overlap regions on target grids needed UV data buffered either on a locally shared object store of locally on each node
31 Data-parallelism schemes Time & baseline Frequency Visibility data FFT o o Exploit frequency independence Distribute visibility data, duplicate target Grid and degrid Gather and accumulate target grids To manage total I/O from buffer/bus distribute Visibility data across nodes for same target grid which is duplicated Duplication of target provides fall-over protection
32 The challenges? Power efficiency Funding agencies more tolerant of cap-ex then power op-ex Cost of hardware and complexity of software Scalability of software Hardware roadmaps indicate h/w will reach requirements Demonstrated software scaling is only 1/1000 th of requirement Project risks Inevitable significant interaction between software engineers and idiosyncratic domain specific knowledge Software project Extensibility, system scalability, maintainability SKA1 is the first milestone expecting significant expansion in the 2020s 50yr observatory lifetime
33 SKA SDP STORAGE REQUIREMENTS
34 SDP Storage Functions/Use cases Temporary Storage of Raw Data Temporary Storage of Intermediate Data Products Temporary persistence of key data / program state Archiving of science data Large volume Write once, ~read 10 times Predictable read patterns and locality with respect to processing Rewritten every ~12 hours Objective is to reduce working memory size At least one reads per write (perhaps a few on average) Objective is failure recovery Write many times read once Write once, read few times Most data will be permanently stored Long term reliability important
35 Radio Astronomy Parallelisation Current Best Practice Routinely achieve ~4000 way parallelism: ~100 separate observations analysed in parallel Each on a ~40 core shared memory system Often with frequent human interaction in each ( The human pipeline ) Current limited by number of people with expertise in astronomical radio interferometry For SKA we think we will be limited by cost of storing the raw data for long enough
36 Power & Cost of Storage Capacity determine the amount of embarrassing parallelism for SKA
37 Telescope State from TM Observation B Visibilities From Correlator Telescope State from TM Observation A Visibilities From Correlator Current baseline: double buffered operation Double buffering processing scheme for batch processing Buffer 1 Ingest Processing Near-realtime Processing Batch Processing of Observation A Iterate on the buffer data in batch processing Archive Buffer 2 Ingest Processing Near-realtime Processing Batch Processing of Observation B Iterate on the buffer data in batch processing Archive Flow of time
38 Storage H/W level requirements Average aggregate write of ~1 TeraByte/s and average aggregate read of ~10 TeraByte/s Large block sizes (> 1MB) are fine Capacity minimum 100 PetaByte Compatible, power, cost balance with computing doing ~5000 FLOPS/byte of I/O High endurance: ~1000 write cycles/year
39 Storage S/W layer requirements We need: High read/write throughput efficiency for large blocks Sophisticated management of read latency Explicit pre-fetch probably necessary Both local and remote read of data Local write probably sufficient Modest resilience for subset of data We don t think we need: Posix filesystem, locking, concurrent access Remote read between arbitrary pair of end-points island level remote read probably sufficient Cross-node striping Any security, permissions, ownership mechanisms
40 In contrast: HPC Simulation check pointing almost diametrically opposed: Write many read once Resilience/recovery after crash main use case Weak interaction with processing Traditional big-data : Small objects, high information contents Diverse algorithms re-run on the same dataset Multi-tenant datacentres, data protection, security
41 Design Ideas Mix storage and processing throughout the system, perhaps in each node Enable data-locality by distributing the raw data so that the processing load is roughly balanced Distributed object storage system Dataflow programming model, allowing scheduling of both computation and data movement
42 Questions Cost & power models for disk and solid state Are the benefits of data locality worth the software complexity? How to integrate with the dataflow programming model? Integration with interconnect fabrics What is the filesystem/object layer? Can we replace (large)message passing with put/get? Will the traditional driver/kernel/libc/application stack limit performance?
43 SKA Science Data Processor SDP ARCHITECTURE HIGHLIGHTS
44 Programming model Hybrid programming model: Dataflow at coarse-grained level: About 1 million tasks/s max over the whole processor (-> ~10s 100s milli second tasks), consuming ~100 MegaByte each Static scheduling at coarsest-level (down to data-island ) Static partitioning of the large-volume input data Dynamic scheduling within data island: Failure recovery, dynamic load-balancing Data driven (all data will be used) Shared memory model at fine-grained level e.g.: threads/openmp/simt-like ~100s active threads per shared memory space Allows manageable working memory size, computational efficiency
45 Why? Shared memory model essential at fine-grain to control working memory requirements Dataflow (but all of these are still to be proven in our application): Maintainability Load-balancing Minimisation of data movement Handling failure Adaptability to different system architectures
46 Fault Tolerance Non-Precious Data Classify arcs in the dataflow graph as precious or non-precious Precious data are treated in usual way failover, restart, RAID, etc. Non-precious data can be dropped: If they are input to a map-type operation then no output If they are input to a reduction then result is computed without them Stragglers outputting non-precious data can be terminated after a relatively short time-out
47 The non-precious data concept - illustration FULL DATASET FULL DATASET SPLIT BY FREQ SPLIT BY FREQ FREQ 1 FREQ 2 FREQ 3 FREQ NF FREQ 1 FREQ 2 FREQ 3 FREQ NF GRID GRID GRID GRID GRID GRID GRID GRID FFT FFT FFT FFT FFT FFT FFT FFT Single Frequency Image Single Frequency Image Single Frequency Image Single Frequency Image Single Frequency Image Single Frequency Image Single Frequency Image Single Frequency Image Reduce by pixelvise addition 2 Freq Image 2 Freq Image 2 Freq Image 1 Freq Image Reduce by pixelvise addition Reduce by pixelvise addition Combined Image Combined Image Normalise by 4 Normalise by 3 Combined Image Combined Image
48 LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT LNK FDX DIAG RPS PWR /ACT /HDX COMBO PORTS FAN TEMP RESET LNK/ACT LNK/ACT LNK/ACT LNK/ACT Data Ingress and Interconnect Concept CSP Node CSP Node CSP Node CSP Node CSP Node CSP Node CSP Node CSP Node CSP Node INGEST LAYER 1st stage BDN 1st stage LLN 1st stage BDN 1st stage LLN 1st stage BDN 2nd stage BDN 2nd stage LLN 2nd stage BDN 2nd stage LLN 2nd stage BDN 2nd stage LLN 2nd stage BDN 2nd stage LLN 2nd stage BDN 3rd stage switch Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Compute Island Science Archive switch Science Archive switch High Performance Buffer Medium Performance Buffer Storage Pod Storage Pod Archive Network Low-Latency Network Long Term Storage DELIV Bulk Data Network
49 The known-unknowns HIGHEST-PRIORITY SKA1 SCIENCE OBJECTIVES
50 Epoch of Re-Ionisation
51 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
52 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
53 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
54 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
55 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
56 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
57 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
58 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
59 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
60 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
61 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
62 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
63 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
64 Epoch of Re-ionization Furlanetto et al. (2003) z= Mpc comoving Dn=0.1 MHz
65 Pulsar Surveys: Testing gravity The SKA will detect around 30,000 pulsars in our own galaxy, 2000 msec pulsars accurate clocks Relativistic binaries give unprecedented strongfield test of gravity expect ~100 Timing net of ms pulsars to detect gravitational waves via timing residuals Expect timing accuracy to improve by ~100
66 Pulsar timing array Nano-Hertz range of frequencies MBH-MBH binaries: resolved objects and stochastic background Cosmic strings and other exotic phenomenon Timing residual ~10s ns need ms pulsars
67 Finding the unexpected Hubble Deep Field (HDF) Very Large Array observation of HDF ~ 3000 galaxies ~ 3000 galaxies ~15 radio sources
68 Finding the unexpected (2) Hubble Deep Field (HDF) Simulation of SKA Observation ~ 3000 galaxies
69
SKA Science Data Processing or MAKING SENSE OF SENSOR BIG DATA
SKA Science Data Processing or MAKING SENSE OF SENSOR BIG DATA Bojan Nikolic & SDP Consortium Astrophysics Group, Cavendish Laboratory Project Engineer/Architecture Lead SKA Science Data Processor What
The Square Kilometre Array & High speed data recording
The Square Kilometre Array & High speed Astrophysics Group, Cavendish Laboratory, University of Cambridge http://www.mrao.cam.ac.uk/ bn204/ CRISP Annual Meeting FORTH, Crete, Greece March 2012 Outline
New Zealand Impacts on CSP and SDP Designs
New Zealand Impacts on CSP and SDP Designs Dr Andrew Ensor Director HPC Research Laboratory/ Director NZ SKA Alliance Mid-Year Mini SKA Colloquium 2018 12 July 2018 Past, Present, Future Mega-Science Projects
Overview of the Square Kilometre Array. Richard Schilizzi COST Workshop, Rome, 30 March 2010
Overview of the Square Kilometre Array Richard Schilizzi COST Workshop, Rome, 30 March 2010 The Square Kilometre Array A global program Time line 2000-07 Initial Concept Stage 2008-12 System Design Stage
The Square Kilometre Array Radio Telescope Project : An Overview
Science with the SKA IISER Mohali 19th March The Square Kilometre Array Radio Telescope Project : An Overview Yashwant Gupta NCRA-TIFR Background : what is the SKA? The SKA is the most ambitious Radio
Square Kilometre Array. Steve Torchinsky
Square Kilometre Array Steve Torchinsky steve.torchinsky@obspm.fr A square kilometre of collecting area for each of three frequency bands SKA Low frequency 50MHz to 450MHz to be built in Western Australia
Science Operations with the Square Kilometre Array
Science Operations with the Square Kilometre Array Dr Antonio Chrysostomou Head of Science Operations Planning a.chrysostomou@skatelescope.org Outline Introduction to the SKA Science Programme Operational
High-performance computing and the Square Kilometre Array (SKA) Chris Broekema (ASTRON) Compute platform lead SKA Science Data Processor
High-performance computing and the Square Kilometre Array (SKA) Chris Broekema (ASTRON) Compute platform lead SKA Science Data Processor Radio Astronomy and Computer Science Both very young sciences (1950s)
SKA Industry Update. Matthew Johnson Head of the UK SKA Project Office STFC
SKA Industry Update Matthew Johnson Head of the UK SKA Project Office STFC Square Kilometre Array Rebaselining announced in March 2015 UK selected as HQ host in April 2015 Business case approved by Jo
The Square Kilometre Array. Richard Schilizzi SKA Program Development Office
The Square Kilometre Array Richard Schilizzi SKA Program Development Office EVN Symposium, Manchester September 2010 1995-00 2000-07 Science 2008-12 SKA Science & Engineering 2013-23 Committee 2020-50+
Square Kilometre Array Nigel Rix
Square Kilometre Array Nigel Rix Technology Business Manager, ESP KTN nigel.rix@ukskaic.org Meet the Buyer Event Heathrow, 27 th September 2011 What When Who How ORIGINS Neutral hydrogen in the universe
SKA Precursors and Pathfinders. Steve Torchinsky
SKA Precursors and Pathfinders Steve Torchinsky steve.torchinsky@obspm.fr A square kilometre of collecting area for each of three frequency bands SKA Low frequency 50MHz to 450MHz to be built in Western
The international SKA project
The international SKA project - Towards a giant global radio telescope - Wim van Driel Paris Observatory - GEPI 3 rd MCCT SKADS School, Paris, 24/08/2009 SKA - Square Kilometre Array Necessary for a breakthrough
Large Field of View Radio Astronomy; relevant for many KSP s
Large Field of View Radio Astronomy; relevant for many KSP s SKA Key Science Drivers ORIGINS Probing the Dark Ages When & how were the first stars formed? Cosmology and Galaxy Evolution Galaxies, Dark
SKA Science Data Processing
SKA Science Data Proceing B. Nikolic (SDP Project Engineer) & SDP Conortium Team Atrophyic Group, Cavendih Laboratory, Univerity of Cambridge http://www.mrao.cam.ac.uk/ bn204/ 1 April 2014 AASTCS 2: Exacale
Software, Computing and Data Storage for the
Software, Computing and Data Storage for the Square Kilometre Array Duncan Hall XLDB 2009 2009 August 28 Outline SKA: What are the drivers? How does radio astronomy work? What are the SKA s prime characteristics?
The Future of Radio Astronomy. Karen O Neil
The Future of Radio Astronomy Karen O Neil The Present Arecibo Telescope 305m diameter dish 0.4-10 GHz 7-pixel FPA (1.2-1.5 GHz) -01⁰ < δ < 38⁰ Resolution : 15 (0.3 GHz) 0.4 (10 GHz) 60% of astronomy time
The Square Kilometer Array. Presented by Simon Worster
The Square Kilometer Array Presented by Simon Worster The Square Kilometer Array SKA for short. Will contain over a square kilometer of collecting area. Will be the largest radio telescope ever built.
An Introduction to Radio Astronomy
An Introduction to Radio Astronomy Second edition Bernard F. Burke and Francis Graham-Smith CAMBRIDGE UNIVERSITY PRESS Contents Preface to the second edition page x 1 Introduction 1 1.1 The role of radio
International Project Update
International Project Update - Mike Garrett (ASTRON & Leiden) Chair SKA Science & Engineering Committee This talk - SKA: a global vision for Radio Astronomy - Science drivers - Pathfinders, Precursors,
Big-Data as a Challenge for Astrophysics
Big-Data as a Challenge for Astrophysics ESA / Euclid Consortium Hubble deep field, NASA/STScI HESS collaboration, F. Acero and H. Gast ESA/ATG medialab; ESO/S. Brunier Volker Beckmann Institut National
The Square Kilometre Array
The Square Kilometre Array An example of international cooperation a.chrysostomou@skatelescope.org Antonio Chrysostomou Head of Science Operations Planning SKA Key Science Drivers: The history of the Universe
An Introduction to ASKAP Bringing Radio Interferometers Into the Multi-pixel Era
An Introduction to ASKAP Bringing Radio Interferometers Into the Multi-pixel Era Aidan Hotan and Lisa Harvey-Smith 3 rd October 2014 CSIRO ASTRONOMY AND SPACE SCIENCE Introducing ASKAP The Australian SKA
Data analysis of massive data sets a Planck example
Data analysis of massive data sets a Planck example Radek Stompor (APC) LOFAR workshop, Meudon, 29/03/06 Outline 1. Planck mission; 2. Planck data set; 3. Planck data analysis plan and challenges; 4. Planck
Lecture Outlines. Chapter 5. Astronomy Today 8th Edition Chaisson/McMillan Pearson Education, Inc.
Lecture Outlines Chapter 5 Astronomy Today 8th Edition Chaisson/McMillan Chapter 5 Telescopes Units of Chapter 5 5.1 Optical Telescopes 5.2 Telescope Size 5.3 Images and Detectors 5.4 High-Resolution Astronomy
Max-Planck-Institut für Radioastronomie, Bonn Istituto di Radioastronomia - INAF, Bologna
RadioNet3 & SKA Franco Mantovani Project Scientist RadioNet3 Max-Planck-Institut für Radioastronomie, Bonn Istituto di Radioastronomia - INAF, Bologna RadioNet3 Advanced Radio Astronomy in Europe www.radionet-eu.org
Future Radio Interferometers
Future Radio Interferometers Jim Ulvestad National Radio Astronomy Observatory Radio Interferometer Status in 2012 ALMA Covers much of 80 GHz-1 THz band, with collecting area of about 50% of VLA, for a
The Square Kilometre Array and Data Intensive Radio Astronomy 1
The Square Kilometre Array and Data Intensive Radio Astronomy 1 Erik Rosolowsky (U. Alberta) Bryan Gaensler (U. Toronto, Canadian SKA Science Director) Stefi Baum (U. Manitoba) Kristine Spekkens (RMC/Queen
Radio Telescopes of the Future
Radio Telescopes of the Future Cristina García Miró Madrid Deep Space Communications Complex NASA/INTA AVN Training School HartRAO, March 2017 Radio Telescopes of the Future Characteristics FAST SKA (EHT)
From LOFAR to SKA, challenges in distributed computing. Soobash Daiboo Paris Observatory -LESIA
From LOFAR to SKA, challenges in distributed computing Soobash Daiboo Paris Observatory -LESIA Overview LOFAR telescope Data processing with LOFAR NenuFar SKA Summary LOFAR science drivers Key science
Radio Interferometry and Aperture Synthesis
Radio Interferometry and Aperture Synthesis Phil gave a detailed picture of homodyne interferometry Have to combine the light beams physically for interference Imposes many stringent conditions on the
Credit: ESO/B. Tafreshi (twanight.org) Operating the Very Large Telescope
Credit: ESO/B. Tafreshi (twanight.org) Operating the Very Large Telescope Introduction: From scientific idea to data legacy Credit: ESO/B. Tafreshi (twanight.org) The Very Large Telescope (VLT) was built
Gravitational Wave Data (Centre?)
ARC Centre of Excellence for Gravitational Wave Discovery Gravitational Wave Data (Centre?) Matthew Bailes (Director) Swinburne University of Technology ARC Laureate Fellow OzGrav Director Gravitational
Welcome and SKA Overview Michiel van Haarlem. Interim Director General SKA Organisation 15 February 2012
Welcome and SKA Overview Michiel van Haarlem Interim Director General SKA Organisation 15 February 2012 SKA Concept A large radio telescope for transformational science: up to 1 million m 2 antenna collecting
Square Kilometer Array:
Square Kilometer Array: The worlds most powerful radio telescope! Paul Bourke ivec@uwa With support from ICRAR (International Centre for Radio Astronomy) Outline History SKA (Square Kilometer Array) ASKAP
A Pulsar Timing Array for Gravitational Wave Detection. Paul Demorest, NRAO
A Pulsar Timing Array for Gravitational Wave Detection Paul Demorest, NRAO About 10% of known radio pulsars are recycled millisecond pulsars (MSPs). These are spun up by accreting matter from a companion
evlbi Networks Bringing Radio Telescopes Together
evlbi Networks Bringing Radio Telescopes Together Dr Chris Phillips evlbi Project Scientist 11 November 2009 With Thanks Introduction o to Radio Astronomy o Radio Astronomy encompasses long wavelength
Radio Astronomy module
Radio Astronomy module Contact tony@ska.ac.za Notes: NRAO Essential radio astronomy course: http://www.cv.nrao.edu/course/astr534/era.shtml See also http://www.haystack.mit.edu/ edu/undergrad/materials/ra_tutorial.html
Atmospheric phase correction for ALMA with water-vapour radiometers
Atmospheric phase correction for ALMA with water-vapour radiometers B. Nikolic Cavendish Laboratory, University of Cambridge January 29 NA URSI, Boulder, CO B. Nikolic (University of Cambridge) WVR phase
Principles of Interferometry. Hans-Rainer Klöckner IMPRS Black Board Lectures 2014
Principles of Interferometry Hans-Rainer Klöckner IMPRS Black Board Lectures 2014 acknowledgement Mike Garrett lectures James Di Francesco crash course lectures NAASC Lecture 5 calibration image reconstruction
Determining the Specification of an Aperture Array for Cosmological Surveys
Determining the Specification of an Aperture Array for Cosmological Surveys P. Alexander University of Cambridge, Department of Physics, Cavendish Laboratory, Cambridge, UK. A. J. aulkner Jodrell Bank
An introduction to closure phases
An introduction to closure phases Michelson Summer Workshop Frontiers of Interferometry: Stars, disks, terrestrial planets Pasadena, USA, July 24 th -28 th 2006 C.A.Haniff Astrophysics Group, Department
Science advances by a combination of normal science and discovery of anomalies.
Science advances by a combination of normal science and discovery of anomalies. Many revolutions come from long periods of normal science reinforced by exceptional science. example: accelerating universe
Astronomy. Optics and Telescopes
Astronomy A. Dayle Hancock adhancock@wm.edu Small 239 Office hours: MTWR 10-11am Optics and Telescopes - Refraction, lenses and refracting telescopes - Mirrors and reflecting telescopes - Diffraction limit,
Square Kilometre Array Science Data Challenge 1
Square Kilometre Array Science Data Challenge 1 arxiv:1811.10454v1 [astro-ph.im] 26 Nov 2018 Anna Bonaldi & Robert Braun, for the SKAO Science Team SKA Organization, Jodrell Bank, Lower Withington, Macclesfield,
FIVE FUNDED* RESEARCH POSITIONS
OBSERVATION Sub-GROUP: 1. Masters (MSc, 1 year): Exploring extreme star-forming galaxies for SALT in the Sloan Digital Sky Survey 2. Masters (MSc,1 year): HI masses of extreme star-forming galaxies in
ALMA Development Program
ALMA Development Program Jeff Kern CASA Team Lead Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Opportunities for Software
The well-composed image was recorded over a period of nearly 2 hours as a series of 30 second long, consecutive exposures on the night of October 5.
Happy Thursday! The well-composed image was recorded over a period of nearly 2 hours as a series of 30 second long, consecutive exposures on the night of October 5. The exposures were made with a digital
Light and Telescope 10/24/2018. PHYS 1403 Introduction to Astronomy. Reminder/Announcement. Chapter Outline. Chapter Outline (continued)
PHYS 1403 Introduction to Astronomy Light and Telescope Chapter 6 Reminder/Announcement 1. Extension for Term Project 1: Now Due on Monday November 12 th 2. You will be required to bring your cross staff
An Introduction to Radio Astronomy
An Introduction to Radio Astronomy Bernard F. Burke Massachusetts Institute of Technology and Francis Graham-Smith Jodrell Bank, University of Manchester CAMBRIDGE UNIVERSITY PRESS Contents Preface Acknowledgements
Chapter 3 Telescopes 2/19/2014. Lecture Outline. 3.1 Optical Telescopes. 3.1 Optical Telescopes. Units of Chapter Optical Telescopes
Lecture Outline Chapter 3 Telescopes Images can be formed through reflection or refraction. Reflecting mirror Chapter 3 Telescopes Refracting lens Units of Chapter 3 Optical Telescopes Telescope Size High-Resolution
Future radio galaxy surveys
Future radio galaxy surveys Phil Bull JPL/Caltech Quick overview Radio telescopes are now becoming sensitive enough to perform surveys of 107 109 galaxies out to high z 2 main types of survey from the
Astronomy of the Next Decade: From Photons to Petabytes. R. Chris Smith AURA Observatory in Chile CTIO/Gemini/SOAR/LSST
Astronomy of the Next Decade: From Photons to Petabytes R. Chris Smith AURA Observatory in Chile CTIO/Gemini/SOAR/LSST Classical Astronomy still dominates new facilities Even new large facilities (VLT,
Square Kilometre Array: status. Philip Diamond SKA Director-General 16 th September 2013
Square Kilometre Array: status Philip Diamond SKA Director-General 16 th September 2013 Great Observatories for the coming decades E- ELT E- ELT oop(cal/ p(cal/ir IR Construc(on Construc(on approved approved
e-vlbi Radio interferometers How does VLBI work? What science do we do with VLBI? How has the technology changed? Advantages of e-vlbi e-astronomy
e-vlbi Radio interferometers How does VLBI work? What science do we do with VLBI? How has the technology changed? Advantages of e-vlbi e-astronomy Simple interferometer In a simple interferometer, two
Todays Topics 3/19/2018. Light and Telescope. PHYS 1403 Introduction to Astronomy. CCD Camera Makes Digital Images. Astronomical Detectors
PHYS 1403 Introduction to Astronomy Light and Telescope Chapter 6 Todays Topics Astronomical Detectors Radio Telescopes Why we need space telescopes? Hubble Space Telescopes Future Space Telescopes Astronomy
Cherenkov Telescope Arrays
Cherenkov Telescope Arrays Michael Daniel University of Durham michael.daniel@durham.ac.uk Workshop on Stellar Intensity Interferometry 1 CONTENTS Introduction to Cherenkov telescopes Characteristics of
416 Distributed Systems
416 Distributed Systems RAID, Feb 26 2018 Thanks to Greg Ganger and Remzi Arapaci-Dusseau for slides Outline Using multiple disks Why have multiple disks? problem and approaches RAID levels and performance
A potential large-scale scientific collaborative project Africa/Europe to study the infant Universe. Ricardo Génova Santos (IAC)
A potential large-scale scientific collaborative project Africa/Europe to study the infant Universe Ricardo Génova Santos (IAC) AERAP investing in People, Prosperity and Peace Round table seminar, 2 April
New physics is learnt from extreme or fundamental things
New physics is learnt from extreme or fundamental things New physics is learnt from extreme or fundamental things The Universe is full of extremes and is about as fundamental as it gets! New physics is
The Square Kilometre Array: An international project to realize the world s
The Square Kilometre Array: An international project to realize the world s The The Square Square The Kilometre Square Kilometre Kilometre Array: Array: An Array international An largest radio telescope
Radio Interferometry Fundamentals. John Conway Onsala Space Obs and Nordic ALMA ARC-node
Radio Interferometry Fundamentals John Conway Onsala Space Obs and Nordic ALMA ARC-node So far discussed only single dish radio/mm obs Resolution λ/d, for D=20m, is 30 at mm-wavelengths and 30 (diameter
Square Kilometre Array: World s Largest Radio Telescope Design and Science drivers
Square Kilometre Array: World s Largest Radio Telescope Design and Science drivers Miroslava Dessauges Geneva Observatory, University of Geneva With my thanks to Philip Diamond (SKA director-general),
Classical Interferometric Arrays. Andreas Quirrenbach Landessternwarte Heidelberg
Classical Interferometric Arrays Andreas Quirrenbach Landessternwarte Heidelberg The VLT Interferometer Tucson 11/14/2006 Andreas Quirrenbach 2 Optical / Infrared Interferometry Today Access to milliarcsecond-scale
TECHNICAL REPORT NO. 86 fewer points to average out the noise. The Keck interferometry uses a single snapshot" mode of operation. This presents a furt
CHARA Technical Report No. 86 1 August 2000 Imaging and Fourier Coverage: Mapping with Depleted Arrays P.G. Tuthill and J.D. Monnier 1. INTRODUCTION In the consideration of the design of a sparse-pupil
From the VLT to ALMA and to the E-ELT
From the VLT to ALMA and to the E-ELT Mission Develop and operate world-class observing facilities for astronomical research Organize collaborations in astronomy Intergovernmental treaty-level organization
Square Kilometer Array
Square Kilometer Array YERVANT TERZIAN CORNELL UNIVERSITY The Big Bang Space Time A rich history of discovery Many discoveries over the past 50yr Pulsars
Introduction to Interferometry
Introduction to Interferometry Ciro Pappalardo RadioNet has received funding from the European Union s Horizon 2020 research and innovation programme under grant agreement No 730562 Radioastronomy H.Hertz
SKA-NL Roadmap: SaDT and VLBI. Arpad Szomoru, JIVE
SKA-NL Roadmap: SaDT and VLBI Arpad Szomoru, JIVE JIVE - Joint Institute for VLBI ERIC Promote and advance the use of VLBI for astronomy Central correlation for European VLBI Network Operational feedback
International Facilities
International Facilities A report to the National Committee for Astronomy for the Australian Astronomy Decadal Plan 2006-2015 By Working Group 3.1 September 2005 Executive Summary Over the past decade
2.6 Complexity Theory for Map-Reduce. Star Joins 2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51
2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51 Star Joins A common structure for data mining of commercial data is the star join. For example, a chain store like Walmart keeps a fact table whose tuples each
Galaxy Classification and the Hubble Deep Field
Galaxy Classification and the Hubble Deep Field A. The Hubble Galaxy Classification Scheme Adapted from the UW Astronomy Dept., 1999 Introduction A galaxy is an assembly of between a billion (10 9 ) and
Chapter 6 Light and Telescopes
Chapter 6 Light and Telescopes Guidepost In the early chapters of this book, you looked at the sky the way ancient astronomers did, with the unaided eye. In chapter 4, you got a glimpse through Galileo
The conceptual view. by Gerrit Muller University of Southeast Norway-NISE
by Gerrit Muller University of Southeast Norway-NISE e-mail: gaudisite@gmail.com www.gaudisite.nl Abstract The purpose of the conceptual view is described. A number of methods or models is given to use
The Square Kilometre Array Epoch of Reionisation and Cosmic Dawn Experiment
Peering towards Cosmic Dawn Proceedings IAU Symposium No. 333, 2018 Vibor Jelić & Thijs van der Hulst, eds. c 2018 International Astronomical Union DOI: 00.0000/X000000000000000X The Square Kilometre Array
Polarimetry with Phased-Array Feeds
Polarimetry with Phased-Array Feeds Bruce Veidt Dominion Radio Astrophysical Observatory Herzberg Institute of Astrophysics National Research Council of Canada Penticton, British Columbia, Canada Provo,
RLW paper titles:
RLW paper titles: http://www.wordle.net Astronomical Surveys and Data Archives Richard L. White Space Telescope Science Institute HiPACC Summer School, July 2012 Overview Surveys & catalogs: Fundamental
The South African SKA/meerKAT Project. Kobus Cloete
The South African SKA/meerKAT Project Kobus Cloete Team Project Director Project Scientists Infrastructure meerkat Lead Project Manager HCD HR and Finances KAT Project Manager Support Staff Subsystems
APPLICATIONS FOR PHYSICAL SCIENCE
APPLICATIONS FOR PHYSICAL SCIENCE A. Bulgarelli (INAF) NATIONAL INSTITUTE FOR ASTROPHYSICS (INAF) The National Institute for Astrophysics (INAF) is the main Italian Research Institute for the study of
Short-Spacings Correction From the Single-Dish Perspective
Short-Spacings Correction From the Single-Dish Perspective Snezana Stanimirovic & Tam Helfer (UC Berkeley) Breath and depth of combining interferometer and single-dish data A recipe for observing extended
Radio astronomy in Africa: Opportunities for cooperation with Europe within the context of the African-European Radio Astronomy Platform (AERAP)
Radio astronomy in Africa: Opportunities for cooperation with Europe within the context of the African-European Radio Astronomy Platform (AERAP) Square Kilometre Array (SKA) Global mega-science project
Reliability at Scale
Reliability at Scale Intelligent Storage Workshop 5 James Nunez Los Alamos National lab LA-UR-07-0828 & LA-UR-06-0397 May 15, 2007 A Word about scale Petaflop class machines LLNL Blue Gene 350 Tflops 128k
Foundations of Astronomy 13e Seeds. Chapter 6. Light and Telescopes
Foundations of Astronomy 13e Seeds Chapter 6 Light and Telescopes Guidepost In this chapter, you will consider the techniques astronomers use to study the Universe What is light? How do telescopes work?
FPGA Implementation of a Predictive Controller
FPGA Implementation of a Predictive Controller SIAM Conference on Optimization 2011, Darmstadt, Germany Minisymposium on embedded optimization Juan L. Jerez, George A. Constantinides and Eric C. Kerrigan
The Cherenkov Telescope Array
The Cherenkov Telescope Array Gamma-ray particle astrophysics Dark Matter Space time Cosmic rays...? Gamma-ray particle astrophysics Dark Matter Space time Cosmic rays...? Particle Dark Matter Direct Detection
Exoplanet Detection and Characterization with Mid-Infrared Interferometry
Exoplanet Detection and Characterization with Mid-Infrared Interferometry Rachel Akeson NASA Exoplanet Science Institute With thanks to Peter Lawson for providing material Sagan Workshop July 21, 2009
Astro Tourism OPPORTUNITIES FOR ASTRO TOURISM IN THE NORTHERN AND WESTERN CAPE
Astro Tourism OPPORTUNITIES FOR ASTRO TOURISM IN THE NORTHERN AND WESTERN CAPE NSTF DISCUSSION FORUM SCIENCE TOURISM PRESENTER: Lorenzo Raynard South African Astronomical Observatory (SAAO) National facility
Chapter 5 Telescopes
Chapter 5 Telescopes Units of Chapter 5 Telescope Design Images and Detectors The Hubble Space Telescope Telescope Size High-Resolution Astronomy Radio Astronomy Interferometry Space-Based Astronomy Full-Spectrum
Real Astronomy from Virtual Observatories
THE US NATIONAL VIRTUAL OBSERVATORY Real Astronomy from Virtual Observatories Robert Hanisch Space Telescope Science Institute US National Virtual Observatory About this presentation What is a Virtual
The Square Kilometre Array
Cosmology, Galaxy Formation and Astroparticle Physics on the pathway to the SKA Klöckner, H.-R., Rawlings, S., Jarvis, M. & Taylor, A. (eds.) April 10th-12th 2006, Oxford, United Kingdom The Square Kilometre
Mid-Term Review: Workpackage 5
Mid-Term Review: Workpackage 5 Advanced Radiometric Phase Correction (ARPC) John Richer (Project Leader for Advanced WVR Techniques) Bojan Nikolic (Project Scientist for Advanced WVR Techniques) Cambridge
Transient Cosmic Phenomena and their Influence on the Design of the SKA Radio Telescope
Transient Cosmic Phenomena and their Influence on the Design of the SKA Radio Telescope Research Review Curtin Institute of Radio Astronomy 5 May 2009 ToC Parameter space Discovering phenomena ASKAP &
Pulsar Key Science with the SKA
Pulsar Key Science with the SKA Strong-field tests of gravity using Pulsars and Black Holes Michael Kramer Krabi, April 2007 Pulsar Key Science with the SKA Strong-field tests of gravity using Pulsars
Radio Transient Surveys with The Allen Telescope Array & the SKA. Geoffrey C Bower (UC Berkeley)
Radio Transient Surveys with The Allen Telescope Array & the SKA Geoffrey C Bower (UC Berkeley) Transient Science is Exploding New Phenomena An Obscured Radio Supernova in M82 Discovered serendipitously
The Square Kilometre Array and the radio/gamma-ray connection toward the SKA era
The Square Kilometre Array and the radio/gamma-ray connection toward the SKA era Marcello Giroletti INAF Istituto di Radioastronomia 12th AGILE Workshop Roma, 8/5/2014 What s in this talk: SKA basics and
ASKAP Array Configurations: Options and Recommendations. Ilana Feain, Simon Johnston & Neeraj Gupta Australia Telescope National Facility, CSIRO
ASKAP Array Configurations: Options and Recommendations Ilana Feain, Simon Johnston & Neeraj Gupta Australia Telescope National Facility, CSIRO August 6, 2008 Executive Summary In this discussion document
ASKAP. and phased array feeds in astronomy. David McConnell CASS: ASKAP Commissioning and Early Science 16 November 2017
ASKAP and phased array feeds in astronomy David McConnell CASS: ASKAP Commissioning and Early Science 16 November 2017 Image credit: Alex Cherney / terrastro.com 1 Credits ASKAP Commissioning & Early Science
Unveiling the Radio Sky with the JVLA and VLASS. Julie Hlavacek-Larrondo Canada Research Chair, Université de Montréal
Unveiling the Radio Sky with the JVLA and VLASS Julie Hlavacek-Larrondo Canada Research Chair, Université de Montréal Very Large Array What is it? Located in Southern New Mexico at 2100 m altitude Composed
The high angular resolution component of the SKA
The high angular resolution component of the SKA Tingay, S. 1, Bignall, H. 1, Colegate, T. 1, Aben, G. 2, Nicolls, J. 2 & Weston, S. 3 Prof. Steven Tingay SKA2010 Deputy Director, ICRAR March 22 25, 2010
Progress in multigigabit Point to Point Connectivity between South Africa and GEANT.
Progress in multigigabit Point to Point Connectivity between South Africa and GEANT. Applications to Radio Astronomy M J Gaylard 1, J F H Quick 1, B. Becker 2 1 HartRAO, P. O. Box 443, Krugersdorp 1740,