Astronomical Time Series Analysis for the Synoptic Survey Era. Joseph W. Richards. UC Berkeley Department of Astronomy Department of Statistics

Transcription

1 Astronomical Time Series Analysis for the Synoptic Survey Era Joseph W. Richards UC Berkeley Department of Astronomy Department of Statistics Workshop on Algorithms for Modern Massive Data Sets July 11, 2012

2 Center for Time-Domain Informatics UC Berkeley (UCB): Faculty/Staff Josh Bloom, Dan Starr (Astro), John Rice, Noureddine El Karoui (Stats), Martin Wainwright, Masoud Nikravesh (CS) Post-Docs Dovi Poznanski, Brad Cenko, Nat Butler, Berian James, JWR Grad Students Adam Miller, Adam Morgan, Chris Klein, James Long, Tamara Broderick, Isaac Shivvers, Henrik Brink, Sharmo Bhattacharyya Undergrads Maxime Rischard, Justin Higgins, Rachel Kennedy, Arien Crellin-Quick, Pierre Christian, Tatyana Gavrilchenko, Stuart Gegenheimer, Anthony Paredes, Benjamin Gerard, Pedro Rodriguez, Pauline Arriaga Lawrence Berkeley National Laboratory (LBNL): Peter Nugent, David Schlegel, Nic Ross, Horst Simon Visit our website: J. Richards Astronomical Time Series 2

3 Motivation: Photometric Light Curves Transients Type Ia supernova, DES simulation Variable Stars Pulsating star, Hipparcos Survey Eclipsing binary system, OGLE J. Richards Astronomical Time Series 3

4 Data Deluge from Photometric Surveys Massive Upcoming Surveys Large Synoptic Survey Telescope (LSST) Light curves for 800M sources every 3 nights 10 6 SNe/yr, 10 7 eclipsing binaries 3.2 gigapixel camera, 20 TB/night Gaia space astrometry mission billion stars observed 70 times over 5 years Will observe 20K supernovae Many other photometric surveys have already produced data: SDSS, Palomar Transient Factory (PTF), Pan-STARRS, Hipparcos, OGLE, ASAS, Kepler, LINEAR, DES (soon) J. Richards Astronomical Time Series 4

5 Large Synoptic Survey Telescope (circa 2020) 400,000 every 3 nights J. Richards Astronomical Time Series 5

6 Motivation: Automated Learning on Light Curves We need machine-learned classification of light curves for: 1 detection and discovery of events in real time, condensing a data deluge into a trickle of astrophysical goodness 2 optimal allocation of (expensive) follow-up resources, deciding, in real time, which objects require detailed study 3 construction of pure & complete samples of, e.g., Type Ia Supernovae (expansion history of Universe), RR Lyrae Variable Stars (structure of Milky Way), Eclipsing star systems (stellar mass, radius, age, distance) 4 outlier detection to find objects from new or rare classes J. Richards Astronomical Time Series 6

7 Motivation: Automated Learning on Light Curves We need machine-learned classification of light curves for: 1 detection and discovery of events in real time, condensing a data deluge into a trickle of astrophysical goodness 2 optimal allocation of (expensive) follow-up resources, deciding, in real time, which objects require detailed study 3 construction of pure & complete samples of, e.g., Type Ia Supernovae (expansion history of Universe), RR Lyrae Variable Stars (structure of Milky Way), Eclipsing star systems (stellar mass, radius, age, distance) 4 outlier detection to find objects from new or rare classes Discovery on massive data streams is not assured J. Richards Astronomical Time Series 6

8 Real-time Source Discovery & Classification with Josh Bloom, Adam Morgan, James Long, Dan Starr (UCB), Nat Butler (ASU), Dovi Poznanski (TAU) Bloom & Richards (2011) arxiv: Overview of Machine-Learned Classification for Photometric Surveys Bloom, et al. (2011) arxiv: Classification for PTF Morgan, et al. (2012) arxiv: Classification for GRB Follow-up J. Richards Astronomical Time Series 7

9 Real-time Discovery & Classification Goal - Quickly sift through 10 6 candidates per night to find the 10s of objects that are of astrophysical interest Challenges 1 Classification statements are required in near real-time classification features must be cheap to compute context (e.g., position in sky, historical data) is crucial missing values are common, and must be dealt with 2 Training sets are limited in size and scope depend on spectroscopic analysis and/or visual analysis 3 Classification problem can be cast as one of resource allocation can put dollar amount on follow-up telescope resources J. Richards Astronomical Time Series 8

10 Classification for Palomar Transient Factory (PTF) Law et al. (2009, PASP, 121, 1395) J. Richards Astronomical Time Series 9

11 From Detection to Discovery Is this a real time-varying astrophysical source or a bogus detection? New Reference Subtraction Brink, et al. (2012), in prep. PTF obtains 1.5M detections/night Only 0.1% are real astrophysical sources Real-Bogus Classifier Make automated real/bogus decision about each detected source based on pixelated image and contextual features J. Richards Astronomical Time Series 10

12 Real-Bogus Classifier Performance Performance Improvement J. Richards Astronomical Time Series 11

13 Discovery of Most Nearby Supernova in 25 years Real-bogus classifier allowed detailed follow-up. 11 hours after explosion Nugent et al. (2011), Nature Li et al. (2011), Nature Bloom et al. (2011) Nature J. Richards Astronomical Time Series 12

14 Sensitivity of RB2 to Label Noise Sensitivity of RB2 to Training (gray) and Testing (red) label noise Figure of merit Training set contamination Testing set contamination % contamination Astro training sets are difficult to compile For RB2, results are insensitive to noise in the training set until 7% error Implications The training set does not have to be perfectly labeled. J. Richards Astronomical Time Series 13

15 Supernova Zoo Classification Galaxy Zoo Supernovae project: a citizen-science approach to supernova identification Q: Is this candidate star-like and approximately circular? Our approach: Use the first year of user data from Galaxy Zoo Supernovae to train a Random Forest classifier to predict the Zoo consensus for each newly detected object J. Richards Astronomical Time Series 14

16 Supernova Zoo Classification In predicting which objects are supernovae, our Random Forest classifier dominates the SN Zoo Current research - simultaneously model user aptitude and SN viability Richards et al. 2012, in Missed Detection Rate (MDR) Classification of known SNe Random Forest Supernova Zoo preparation False Positive Rate (FPR) J. Richards Astronomical Time Series 15

17 Supernova Zoo: Nightly PTF Operation J. Richards Astronomical Time Series 16

18 Retrospective Light Curve Classification with Josh Bloom, Dan Starr, Adam Miller, Henrik Brink, James Long (UCB), Nat Butler (ASU) Richards et al. (2011) arxiv: VarStar Classification Richards et al. (2011) arxiv: Active Learning for Classification Richards et al. (2011) arxiv: Supernova Typing Richards et al. (2012) arxiv: ML ASAS Classification Catalog J. Richards Astronomical Time Series 17

19 Retrospective Light-Curve Classification Goal - Use long-baseline time series to classify millions of objects to perform astrophysical and cosmological inference Challenges 1 Time-domain surveys presently observe > 10 7 objects, and soon will retrieve data for 10 9 sources need fast estimation of periodic features fast cross-referencing with other catalogs is imperative 2 Training sets are limited and biased require methods that can mitigate sample-selection bias 3 Must characterize all biases for downstream inference calibrate class probabilities and understand FPR / MDR outlier detection to identify and follow up the weirdos J. Richards Astronomical Time Series 18

20 DES Supernova Classification Challenge Question: What type of supernova is this? Great interest in separating Type Ia, II, Ib/c supernovae DES SN Photometric Classification Challenge to test supernova light curve classification methods Our Approach Detect sparse structure in the supernova light curve data to use as features in a classifier J. Richards Astronomical Time Series 19

21 Photometric Supernova Classification Semi-supervised Approach: 1 Construct local dissimilarity measure between light curves 2 Use all data to find a low-dimensional embedding for all SNe via diffusion map 3 Use labeled SNe to train random forest classifier on diffusion map coordinates 4 Predict the class of each unlabeled SNe J. Richards Astronomical Time Series 20

22 Diffusion Map Diffusion map non-linear method to uncover low-dimensional structure in data Coifman & Lafon 2006, Lafon & Lee 2006, Richards et al. 2009; ApJ 691, 32 Idea estimate the true discrepancy between data via a fictive diffusion process (i.e., Markov random walk). Related spectral methods: LLE, Laplacian eigenmaps, Hessian eigenmaps, Isomap, etc. Other uses of diffusion map in Astrophysics: 1 Spectral basis for galaxy population synthesis modeling Richards et al. 2009; MNRAS 399, Adaptive regression for photometric redshift estimation Freeman et al. 2009; MNRAS 398, 2012 J. Richards Astronomical Time Series 21

23 Diffusion Map: Intuition p 1 (x i, x j ) RW transition prob. Diffusion distance at t: D 2 t (x i, x j ) = k (p t(x i,x k ) p t(x j,x k )) 2 φ 0 (x k ) J. Richards Astronomical Time Series 22

24 Diffusion Map: Procedure How to Construct a Diffusion Map 1 Construct weighted graph on data set {x 1,..., x n }, with weights w(x i, x j ) = exp ( l(x i, x j ) 2 ) ɛ l is a local discrepancy measure ɛ is a tuning parameter 2 p 1 (x i, x j ) = w(x i, x j )/ k w(x i, x k ) is transition probability in a fictive Markov random walk on the data 3 Find svd of P: p 1 (x i, x j ) = l 0 λ lψ l (x i )φ l (x j ) 4 The m-dimensional diffusion map is: Ψ : x i [ λ t 1ψ 1 (x i ), λ t 2ψ 2 (x i ),, λ t mψ m (x i ) ] Result: D t (x i, x j ) Ψ(x i ) Ψ(x j ) 2 J. Richards Astronomical Time Series 23

25 SN Classification: Diffusion Map Coordinates IIn+IIP+IIL We can obtain physical intuition by visualizing Ib+Ic+Ib/c how SNe vary across the diffusion map coordinate space Diffusion Co Ia Diffusion Coordinate 3 Normalized Flux + offset B1 B2 B3 B Time since r band max J. Richards Astronomical Time Series 24

26 Sample Selection Bias in Light Curve Classification with Dan Starr, Adam Miller, Nat Butler, James Long, John Rice, Josh Bloom (UC Berkeley), Henrik Brink & Berian James (DARK) Richards et al. (2011), ApJ, 744, 192. arxiv: J. Richards Astronomical Time Series 25

27 Sample Selection Bias In astronomical problems, the training (labeled) and testing (unlabeled) sets are often drawn from different distributions. Left: Training set Right: Testing set This problem is referred to as Sample Selection Bias or Covariate Shift. SN Challenge Data Kessler et al. (2010) arxiv: J. Richards Astronomical Time Series 26

28 Sample Selection Bias: SN Typing Classification Result For SN typing, it is better to use deeper spectroscopic training samples, even though they observe fewer SNe S m,25-25th mag-limited spectroscopic survey was optimal. 23.5th mag was used in SN Challenge (7 times more SNe) Figure: Type Ia SN Purity and Efficiency of diffusion map Random Forest classifier on SN Challenge testing data J. Richards Astronomical Time Series 27

29 Sample Selection Bias in Astro Classification Training sets in astronomy are biased: 1 Populations of well-studied objects are inherently biased toward brighter/nearby sources with better quality data 2 Available training data are typically from older, lower quality detectors 3 Each survey has different characteristics, aims, cadences... 4 Training data are often generated from idealized models This can cause significant problems for off-the-shelf supervised methods: 1 Poor model selection risk minimization (e.g., by cross-validation) is performed with respect to P Train (x, y) 2 Regions of feature space ignored by the training data catastrophically bad extrapolation J. Richards Astronomical Time Series 28

30 Methods: Active Learning (AL) Idea: Identify and manually label the testing set data that would most help future iterations of the classifier Key: In astronomy, we often have the ability to selectively follow up on sources: Spectroscopic study Query other databases; cross-match Look at the data; Citizen Science projects Pool-based, batch-mode Active Learning: On each AL iteration, select a batch of objects from the entire testing set for manual labeling via a query function J. Richards Astronomical Time Series 29

31 Methods: Active Learning (AL) - P RF (y x) is the estimated RF probability - ρ(x, x) is the RF proximity measure Proposed RF AL query functions; Richards et al. (2011) AL1. Select testing data point (x U) that is most under-sampled by the training data (L): S 1 (x ) = x U ρ(x, x)/n Test (1) z L ρ(x, z)/n Train AL2. Select testing data point that maximizes the total change in the RF probabilities over the testing data: S 2 (x ) = x U ρ(x, x)(1 max y PRF (y x)) z L ρ(x, z) + 1 (2) J. Richards Astronomical Time Series 30

32 Experiment: Variable Star Classification Error Rate IW ST / CT Active Learning "Random" AL RF Def. IW ST CT CT.p AL1.d AL1.t AL2.d AL2.t AL.rand Method Note: AL routines evaluated only on non-selected testing data J. Richards Astronomical Time Series 31

33 Results: All-Sky Automated Survey Classifications Performance on 28-class variable star classification problem Percent Agreement with ACVS RF w/ Active Learning Error Rate = 20.5% Off-the-shelf RF Error Rate = 34.5% Percent of Confident ASAS RF Labels fold increase in classifier confidence AL Iteration AL Iteration Machine-learned ASAS Classification Catalog: J. Richards Astronomical Time Series 32

34 Machine-learned ASAS Classification Catalog J. Richards Astronomical Time Series 33

35 Concluding Remarks Machine learning has been imperative for PTF and is critical for future time-domain surveys Methods & algorithms that handle high data rates Statistical guarantees on performance Reproducible and transparent Both astrophysical insight and machine learning expertise are essential elements in this endeavor J. Richards Astronomical Time Series 34

36 Concluding Remarks Machine learning has been imperative for PTF and is critical for future time-domain surveys Methods & algorithms that handle high data rates Statistical guarantees on performance Reproducible and transparent Both astrophysical insight and machine learning expertise are essential elements in this endeavor Some of our ongoing research for time-domain analysis 1 Template-free photometric supernova typing methods 2 Period estimation methods for multi-band photometry 3 Novel feature estimation: diffusion map, DTW, wavelets 4 Structured Classification to exploit class taxonomy 5 Active Learning to overcome sample selection bias 6 Modeling user aptitude in crowdsourced data sets 7 Semi-supervised anomaly detection J. Richards Astronomical Time Series 34

37 Center for Time-Domain Informatics Publications Butler, Nathaniel R. & Bloom, Joshua S. Optimal Time-Series Selection of Quasars (2011, AJ, 147, 93) Richards, Joseph W., et al. On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data (2011, ApJ, 733, 1) Bloom, Joshua S. & Richards, Joseph W. Data Mining and Machine-Learning in Time-Domain Discovery & Classification (2011, Chapter in the forthcoming book Advances in Machine Learning and Data Mining for Astronomy ) Klein, Christopher R., Richards, Joseph W., Butler, Nathaniel R. & Bloom, Joshua S. Mid-infrared Period-luminosity Relations of RR Lyrae Stars Derived from the WISE Preliminary Data Release (2011, ApJ, 732, 2) Richards, Joseph W., Homrighausen, Darren, Freeman, Peter E., Schafer, Chad M. & Poznanski, Dovi Semi-supervised Learning for Photometric Supernova Classification (2012, MNRAS, 419, 1121) Richards, Joseph W. et al. Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification (2012, ApJ, 744, 192) Bloom, Joshua S. et al. Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era (2011, arxiv: ) J. Richards Astronomical Time Series 35