New Opportunities in Petascale Astronomy University of Illinois
Overview The Challenge New Opportunity: Probabilistic Cosmology New Opportunity: Model-Based Mining New Opportunity: Accelerating Analyses Always keep in mind the Opportunity Costs LCDM Collaborators: Nick Ball, Adam Myers, Brian Wilhite, and Britt Lundgren, Ashley Ross
The (Obvious)Challenge LSST: Large Synoptic Survey Telescope Credit: LSST Project Credit: LSST Project
LSST Data Challenges Why LSST: Prototypical large astronomy survey I am involved in three working groups Some standard opportunities: How to handle the data volume? How to sweep the data? How to mine the data?
LSST Data Challenges Credit: LSST Project
The (Not So Obvious) Challenge Petascale Astronomy eduction does not (typically) cover: 1) How to use databases 2) How to find rare events 3) How to find similar events 4) How to statistically characterize events 5) How to do it all quickly! Credit: Penguin Publishers Illinois Pilot Project: Astro Informatics in the classroom
So we ask for help! Collaborations are not easy and take real effort to make them work. Credit: Morgan Kaufmann
So we ask for help! Credit: W. W. Norton & Company
So we ask for help!?
Building Bridges Now to find funding!
A Brief History Lesson Spectro Photo Credit: SDSS Project Credit: SDSS Project
A Brief History Lesson Spectroscopic Data Photometric Data FOV Full Moon One hour, 600 spectra to r ~ 17.7 One minute, multiband photometry to r ~ 22.6
The Rest of the Story Photo Limit Spectral Limit Credit: Brunner, Connolly, & Szalay 1999 Factor of 100 increase!
Spectroscopic Cosmology Credit: Zehavi et al. 2004
Photometric Cosmology Credit: Ross, Percival, & Brunner 2010
The Future Spectroscopic Data LSST Photometric Data LAMOST r~20 AAΩ r~21 KAOS on Suburu r~22.5 r~24 One Hour FOV Full Moon
Going Forward LSST Final LSST KAOS LAMOST Credit: Brunner, Connolly, & Szalay 1999
A New Opportunity: Probabilistic Cosmology Enormous increase in source count Multi-band data to estimate classification and redshift (distance) Not the death of spectroscopy, but entirely new (sub-)field or approach. Note that vast majority of sources (> 99.9%?) will never be followed up spectroscopically!
Photometric Redshifts Credit: SDSS Project Credit: ESO
Photometric Redshifts Given best-fit spectral model and a corresponding redshift provides a photometric redshift estimation. Note: Spectral redshifts can currently only be determined reliably to I < 24 Credit: Benitez, N 2000, ApJ, 536, 571
Probabilistic Redshifts Main Galaxy L.R. Galaxy Credit: Ball, Brunner, Myers et al. 2008 Quasar NN PDF constructed by sampling from measurement errors
Photometric Cosmology Credit: Myers et al. 2009 6 z peak = 0.42 1.8 < z spec < 2.2 dn/dz 4 2 6 z peak = 2.17 dn/dz 4 2 6 z peak = 1.79 dn/dz 4 2 0.5 1.0 1.5 2.0 z Typical analysis only uses peak photometric redshift
Probabilistic Cosmology 10 3 f( ) (hmpc -1 ) 2.0 1.5 1.0 0.5 3.0 2.0 1.0 2.0 1.5 1.0 0.5 5.0 4.0 3.0 2.0 1.0 f 1 = 0.59 1.8 < z spec < 2.2 <f( *)> = 1.26 f 2 = 1.05 f 3 = 1.64 Credit: Myers et al. 2009 3200 3400 3600 3800 4000 (h -1 Mpc) More sophisticated analysis weights by distance PDF.
Probabilistic Cosmology Credit: Lundgren et al. 2009 Cross-Correlation of Spectroscopic + Photometric samples
Another New Opportunity
Time Domain Astronomy Credit: LSST Project Good reference site: DotAstro.org
Time Domain Astronomy Spectral Credit: Wilhite et al. 2005 Credit: Wilhite et al. 2006 Variability 5 4 3 2 1 SDSS J115031.03-004403.1 High S/N Epoch 0-1 -2.0 10 4-1.5 10 4-1.0 10 4-5.0 10 3 0 5.0 10 3 1.0 10 4 5 4 3 2 1 SDSS J115031.03-004403.1 Low S/N Epoch 0-1 -2.0 10 4-1.5 10 4-1.0 10 4-5.0 10 3 0 5.0 10 3 1.0 10 4 1.0 0.5 0.0-0.5-1.0-1.5-2.0-2.0 10 4-1.5 10 4-1.0 10 4-5.0 10 3 0 5.0 10 3 1.0 10 4 QSO rest frame velocity (km/s) 1.0 0.5 0.0-0.5-1.0-1.5-2.0-2.0 10 4-1.5 10 4-1.0 10 4-5.0 10 3 0 5.0 10 3 1.0 10 4 QSO rest frame velocity (km/s) Credit: Lundgren et al. 2007
A New Opportunity: Model Based Mining Synoptic Astronomy is rapidly growing field Rare or transient events are extremely interesting But the ability to constrain physical models by using synoptic data is an entirely new (sub-)field. Mine Petascale data based on analytic models! Borrow from other fields, including financial markets
Model Based Mining Continuous time first order auto-regressive process: CAR(1) dx(t) = 1 τ X(t)dt + σ dt(t)+bdt τ, σ,t>0 MBH R t lc =1.1 10 8 days M 100R S 3/2 MBH R t orb = 104 10 8 days M 100R S α 1 3/2 MBH R t th =4.6 0.01 10 8 yr M 100R S Credit: Kelly et al. 2009
Another New Opportunity
A New Opportunity: Accelerating Analysis Leverage Commodity Graphics Processing Units to Accelerate Cosmology Codes Cluster manager and Login server Lustre file system servers Credit: Kindratenko & Brunner 2009 24-port Topspin 120 Server InfiniBand switch Netgear Prosafe 24- port Gigabit Ethernet switch HP xw9400 workstation with NVIDIA Quadro Plex Model IV modules 16 cluster compute nodes
Opportunity Costs Challenge: Leaving the comfort zone. Challenge: Building Bridges Challenge: Implementing Solutions Computations Faster algorithms Faster hardware Opportunities: New discoveries!
Good Luck! Interdisciplinary collaborations are not easy and take real effort to make them work.