Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University
|
|
- Juniper McGee
- 5 years ago
- Views:
Transcription
1 Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University
2 Outline What is Surprise Detection? Example Application: The LSST Project New Algorithm for Surprise Detection: KNN-DD
3 Outline What is Surprise Detection? Example Application: The LSST Project New Algorithm for Surprise Detection: KNN-DD
4 Outlier Detection has many names Semi-supervised Learning Outlier Detection Novelty Detection Anomaly Detection Deviation Detection Surprise Detection
5 Outlier Detection has many names Semi-supervised Learning Outlier Detection Novelty Detection Anomaly Detection Deviation Detection Surprise Detection
6 Outlier Detection as Surprise Detection Benefits of very large datasets: best statistical analysis of typical events automated search for rare events Surprise! Graphic from S. G. Djorgovski
7 Basic Knowledge Discovery Problem Outlier detection: (unknown unknowns) Finding the objects and events that are outside the bounds of our expectations (outside known clusters) These may be real scientific discoveries or garbage Outlier detection is therefore useful for: Novelty Discovery is my Nobel prize waiting? Anomaly Detection is the detector system working? Science Data Quality Assurance is the data pipeline working? One person s garbage is another person s treasure. One scientist s noise is another scientist s signal.
8 Outlier detection: (unknown unknowns) Simple techniques exist for uni- and multi-variate data: Outlyingness: O(x) = x μ(x n ) / σ(x n ) Mahalanobis distance: Normalized Euclidean distance: Numerous (countless?) outlier detection algorithms have been developed. For example, see these reviews: Novelty Detection: A Review Part 1: Statistical Approaches, by Markos & Singh, Signal Processing, 83, (2003). A Survey of Outlier Detection Methodologies, by Hodge & Austin, Artificial Intelligence Review, 22, (2004 ). Capabilities of Outlier Detection Schemes in Large Datasets, by Tang, Chen, Fu, & Cheung, Knowledge and Information Systems, 11 (1), (2006). Outlier Detection, A Survey, by Chandola, Banerjee, & Kumar, Technical Report (2007). How does one optimally find outliers in D parameter space? or in interesting subspaces (lower dimensions)? How do we measure their interestingness?
9 Outline What is Surprise Detection? Example Application: The LSST Project New Algorithm for Surprise Detection: KNN-DD
10 LSST = Large Synoptic Survey Telescope meter diameter primary mirror = 10 square degrees! Hello!
11 Observing Strategy: One pair of images every 40 seconds for each spot on the sky, then continue across the sky continuously every night for 10 years (~ ), with time domain sampling in log(time) intervals (to capture dynamic range of transients). LSST (Large Synoptic Survey Telescope): Ten-year time series imaging of the night sky mapping the Universe! ~1,000,000 events each night anything that goes bump in the night! Cosmic Cinematography! The New Education and Public Outreach have been an integral and key feature of the project since the beginning the EPO program includes formal Ed, informal Ed, Citizen Science projects, and Science Centers / Planetaria.
12 LSST Key Science Drivers: Mapping the Dynamic Universe Solar System Inventory (moving objects, NEOs, asteroids: census & tracking) Nature of Dark Energy (distant supernovae, weak lensing, cosmology) Optical transients (of all kinds, with alert notifications within 60 seconds) Digital Milky Way (proper motions, parallaxes, star streams, dark matter) LSST in time and space: When? ~ Where? Cerro Pachon, Chile Architect s design of LSST Observatory
13 3-Gigapixel camera LSST Summary One 6-Gigabyte image every 20 seconds 30 Terabytes every night for 10 years 100-Petabyte final image data archive anticipated all data are public!!! 20-Petabyte final database catalog anticipated Real-Time Event Mining: 1-10 million events per night, every night, for 10 yrs Follow-up observations required to classify these Repeat images of the entire night sky every 3 nights: Celestial Cinematography
14 The LSST will represent a 10K-100K times increase in the VOEvent network traffic. This poses significant real-time classification demands on the event stream: from data to knowledge! from sensors to sense!
15 The LSST Data Mining Raison d etre More data is not just more data more is different! Discover the unknown unknowns. Massive Data-to-Knowledge challenge.
16 The LSST Data Mining Challenges 1. Massive data stream: ~2 Terabytes of image data per hour that must be mined in real time (for 10 years). 2. Massive 20-Petabyte database: more than 50 billion objects need to be classified, and most will be monitored for important variations in real time. 3. Massive event stream: knowledge extraction in real time for 1,000,000 events each night. Challenge #1 includes both the static data mining aspects of #2 and the dynamic data mining aspects of #3. Look at these in more detail...
17 LSST challenges # 1, 2 Each night for 10 years LSST will obtain the equivalent amount of data that was obtained by the entire Sloan Digital Sky Survey My grad students will be asked to mine these data (~30 TB each night 60,000 CDs filled with data):
18 LSST challenges # 1, 2 Each night for 10 years LSST will obtain the equivalent amount of data that was obtained by the entire Sloan Digital Sky Survey My grad students will be asked to mine these data (~30 TB each night 60,000 CDs filled with data): a sea of CDs
19 LSST challenges # 1, 2 Each night for 10 years LSST will obtain the equivalent amount of data that was obtained by the entire Sloan Digital Sky Survey My grad students will be asked to mine these data (~30 TB each night 60,000 CDs filled with data): a sea of CDs Image: The CD Sea in Kilmington, England (600,000 CDs)
20 LSST challenges # 1, 2 Each night for 10 years LSST will obtain the equivalent amount of data that was obtained by the entire Sloan Digital Sky Survey My grad students will be asked to mine these data (~30 TB each night 60,000 CDs filled with data): A sea of CDs each and every day for 10 yrs Cumulatively, a football stadium full of 200 million CDs after 10 yrs The challenge is to find the new, the novel, the interesting, and the surprises (the unknown unknowns) within all of these data. Yes, more is most definitely different!
21 LSST data mining challenge # 3 Approximately 1,000,000 times each night for 10 years LSST will obtain the following data on a new sky event, and we will be challenged with classifying these data:
22 flux LSST data mining challenge # 3 Approximately 1,000,000 times each night for 10 years LSST will obtain the following data on a new sky event, and we will be challenged with classifying these data: time
23 flux LSST data mining challenge # 3 Approximately 1,000,000 times each night for 10 years LSST will obtain the following data on a new sky event, and we will be challenged with classifying these data: more data points help! time
24 flux LSST data mining challenge # 3 Approximately 1,000,000 times each night for 10 years LSST will obtain the following data on a new sky event, and we will be challenged with classifying these data: more data points help! Characterize first! Classify later. time
25 Characterization includes Feature detection and extraction: Identify and describe features in the data Extract feature descriptors from the data Curate these features for scientific search & re-use Find other parameters and features from other archives, other databases, other sky surveys and use those to help characterize (ultimately classify) each new event. hence, cope with a highly multivariate parameter space Outlier / Anomaly / Novelty / Surprise detection Clustering: unsupervised learning ; class discovery Correlation discovery
26 Outline What is Surprise Detection? Example Application: The LSST Project New Algorithm for Surprise Detection: KNN-DD (work done in collaboration Arun Vedachalam)
27 Challenge: which data points are the outliers?
28 Inlier or Outlier? Is it in the eye of the beholder?
29 3 Experiments
30 Experiment #1-A (L-TN) Simple linear data stream Test A Is the red point an inlier or and outlier?
31 Experiment #1-B (L-SO) Simple linear data stream Test B Is the red point an inlier or and outlier?
32 Experiment #1-C (L-HO) Simple linear data stream Test C Is the red point an inlier or and outlier?
33 Experiment #2-A (V-TN) Inverted V-shaped data stream Test A Is the red point an inlier or and outlier?
34 Experiment #2-B (V-SO) Inverted V-shaped data stream Test B Is the red point an inlier or and outlier?
35 Experiment #2-C (V-HO) Inverted V-shaped data stream Test C Is the red point an inlier or and outlier?
36 Experiment #3-A (C-TN) Circular data topology Test A Is the red point an inlier or and outlier?
37 Experiment #3-B (C-SO) Circular data topology Test B Is the red point an inlier or and outlier?
38 Experiment #3-C (C-HO) Circular data topology Test C Is the red point an inlier or and outlier?
39 KNN-DD = K-Nearest Neighbors Data Distributions f K (d[x i,x j ])
40 KNN-DD = K-Nearest Neighbors Data Distributions f O (d[x i,o])
41 KNN-DD = K-Nearest Neighbors Data Distributions f O (d[x i,o]) f K (d[x i,x j ])
42 The Test: K-S test Tests the Null Hypothesis: the two data distributions are drawn from the same parent population. If the Null Hypothesis is rejected, then it is probable that the two data distributions are different. This is our definition of an outlier: The Null Hypothesis is rejected. Therefore the data point s location in parameter space deviates in an improbable way from the rest of the data distribution.
43 Advantages and Benefits of KNN-DD Based on the non-parametric K-S test It makes no assumption about the shape of the data distribution or about normal behavior It compares the cumulative distributions of the data values (i.e., the sets of inter-point distances) without regard to the nature of those distributions (to be continued)
44 Cumulative Data Distribution (K-S test) for Experiment 1A (L-TN)
45 Cumulative Data Distribution (K-S test) for Experiment 2B (V-SO)
46 Cumulative Data Distribution (K-S test) for Experiment 3C (C-HO)
47 Results of KNN-DD experiments Experiment ID Short Description of Experiment KS Test p-value Outlier Index = 1-p = Outlyingness Likelihood Outlier Flag (p<0.05?) L-TN L-SO L-HO V-TN V-SO V-HO C-TN C-SO C-HO Linear data stream, True Normal test Linear data stream, Soft Outlier test Linear data stream, Hard Outlier test V-shaped stream, True Normal test V-shaped stream, Soft Outlier test V-shaped stream, Hard Outlier test Circular stream, True Normal test Circular stream, Soft Outlier test Circular stream, Hard Outlier test % False % Potential Outlier % TRUE % False % Potential Outlier % TRUE % False % TRUE % TRUE The K-S test p value is essentially the likelihood of the Null Hypothesis.
48 Results of KNN-DD experiments Experiment ID Short Description of Experiment KS Test p-value Outlier Index = 1-p = Outlyingness Likelihood Outlier Flag (p<0.05?) L-TN L-SO L-HO V-TN V-SO V-HO C-TN C-SO C-HO Linear data stream, True Normal test Linear data stream, Soft Outlier test Linear data stream, Hard Outlier test V-shaped stream, True Normal test V-shaped stream, Soft Outlier test V-shaped stream, Hard Outlier test Circular stream, True Normal test Circular stream, Soft Outlier test Circular stream, Hard Outlier test % False % Potential Outlier % TRUE % False % Potential Outlier % TRUE % False % TRUE % TRUE The K-S test p value is essentially the likelihood of the Null Hypothesis.
49 Astronomy data experiment #1: Star-Galaxy separation Approximately 100 stars and 100 galaxies were selected from the SDSS & 2MASS catalogs 8 parameters (ugriz and JHK magnitudes) were extracted 7 colors were computed: u-g, g-r, r-i, i-z, z-j, J-H, H-K these are used to locate each object in feature space hence, we have a 7-dimensional parameter feature space The galaxies are treated as the outliers relative to the stars i.e., can KNN-DD separate them from the stars? Results: (for p=0.05 and K=20) 78% of the galaxies were correctly classified as outliers (TP) 1% of the stars were incorrectly classified as outliers (FP)
50 Astronomy data experiment #2: Star-Quasar separation 1000 stars selected from the SDSS & 2MASS catalogs 100 quasars selected from Penn St. astrostats website 8 parameters (ugriz and JHK magnitudes) were extracted 7 colors were computed: u-g, g-r, r-i, i-z, z-j, J-H, H-K these are used to locate each object in feature space hence, we have a 7-dimensional parameter feature space The quasars are treated as the outliers relative to the stars i.e., can KNN-DD separate them from the stars? Results: (for p=0.05 and K=20) 100% of the quasars were correctly classified as outliers (TP) 29% of the stars were incorrectly classified as outliers (FP)
51 Advantages and Benefits of KNN-DD continued KNN-DD: operates on multivariate data (thus solving the curse of dimensionality) is algorithmically univariate (by estimating a function that is based only on the distance between data points, which themselves occupy high-dimensional parameter space) is simply extensible to higher dimensions is computed only on small-k local subsamples of the full dataset of N data points (K << N) is easily (embarrassingly) parallelized when testing multiple data points for outlyingness
52 Future Work for KNN-DD (i.e., deficiencies that need attention) Validate our choices of p and K, which are not well determined or justified Measure the KNN-DD algorithm s learning times Determine the algorithm s complexity Compare the algorithm against several other outlier detection algorithms We started doing that, but there are a very large number Evaluate the KNN-DD algorithm s effectiveness on much larger datasets (e.g., 77,000 SDSS quasars) We started doing that, but with mixed results, which we are still analyzing Test its usability on event streams (streaming data)
53 Future Work in Surprise Detection Test and validate many more of the existing outlier detection algorithms on astronomical data: score them according to their effectiveness and efficiency Derive an interestingness index, which will probably be based upon a mixture of outlyingness metrics: test and validate this on single data points, data sequences (e.g., trending data), and data series (e.g., time series) Apply resulting algorithms and indices on very large datasets (e.g., SDSS+2MASS+GALEX+WISE catalogs, Kepler time series, etc.) test on LSST simulated data and catalogs, in preparation for the real thing at the end of the decade Investigate applicability of algorithms to SDQA (Science Data Quality Assessment) tasks for large sky surveys
Surprise Detection in Science Data Streams Kirk Borne Dept of Computational & Data Sciences George Mason University
Surprise Detection in Science Data Streams Kirk Borne Dept of Computational & Data Sciences George Mason University kborne@gmu.edu, http://classweb.gmu.edu/kborne/ Outline Astroinformatics Example Application:
More informationAstroinformatics: massive data research in Astronomy Kirk Borne Dept of Computational & Data Sciences George Mason University
Astroinformatics: massive data research in Astronomy Kirk Borne Dept of Computational & Data Sciences George Mason University kborne@gmu.edu, http://classweb.gmu.edu/kborne/ Ever since humans first gazed
More informationScientific Data Flood. Large Science Project. Pipeline
The Scientific Data Flood Scientific Data Flood Large Science Project Pipeline 1 1 The Dark Energy Survey Study Dark Energy using four complementary* techniques: I. Cluster Counts II. Weak Lensing III.
More informationAstroinformatics in the data-driven Astronomy
Astroinformatics in the data-driven Astronomy Massimo Brescia 2017 ICT Workshop Astronomy vs Astroinformatics Most of the initial time has been spent to find a common language among communities How astronomers
More informationAstronomy of the Next Decade: From Photons to Petabytes. R. Chris Smith AURA Observatory in Chile CTIO/Gemini/SOAR/LSST
Astronomy of the Next Decade: From Photons to Petabytes R. Chris Smith AURA Observatory in Chile CTIO/Gemini/SOAR/LSST Classical Astronomy still dominates new facilities Even new large facilities (VLT,
More informationReal Astronomy from Virtual Observatories
THE US NATIONAL VIRTUAL OBSERVATORY Real Astronomy from Virtual Observatories Robert Hanisch Space Telescope Science Institute US National Virtual Observatory About this presentation What is a Virtual
More informationTransient Alerts in LSST. 1 Introduction. 2 LSST Transient Science. Jeffrey Kantor Large Synoptic Survey Telescope
Transient Alerts in LSST Jeffrey Kantor Large Synoptic Survey Telescope 1 Introduction During LSST observing, transient events will be detected and alerts generated at the LSST Archive Center at NCSA in
More informationThe Large Synoptic Survey Telescope
The Large Synoptic Survey Telescope Philip A. Pinto Steward Observatory University of Arizona for the LSST Collaboration 17 May, 2006 NRAO, Socorro Large Synoptic Survey Telescope The need for a facility
More informationDoing astronomy with SDSS from your armchair
Doing astronomy with SDSS from your armchair Željko Ivezić, University of Washington & University of Zagreb Partners in Learning webinar, Zagreb, 15. XII 2010 Supported by: Microsoft Croatia and the Croatian
More informationAn end-to-end simulation framework for the Large Synoptic Survey Telescope Andrew Connolly University of Washington
An end-to-end simulation framework for the Large Synoptic Survey Telescope Andrew Connolly University of Washington LSST in a nutshell The LSST will be a large, wide-field, ground-based optical/near-ir
More informationLarge Synoptic Survey Telescope
Large Synoptic Survey Telescope Željko Ivezić University of Washington Santa Barbara, March 14, 2006 1 Outline 1. LSST baseline design Monolithic 8.4 m aperture, 10 deg 2 FOV, 3.2 Gpix camera 2. LSST science
More informationAstronomical Notes. Astronomische Nachrichten. A machine learning classification broker for the LSST transient database
Astronomical Notes Astronomische Nachrichten A machine learning classification broker for the LSST transient database K.D. Borne George Mason University, Department of Computational and Data Sciences, MS
More informationSDSS Data Management and Photometric Quality Assessment
SDSS Data Management and Photometric Quality Assessment Željko Ivezić Princeton University / University of Washington (and SDSS Collaboration) Thinkshop Robotic Astronomy, Potsdam, July 12-15, 2004 1 Outline
More informationTaking the census of the Milky Way Galaxy. Gerry Gilmore Professor of Experimental Philosophy Institute of Astronomy Cambridge
Taking the census of the Milky Way Galaxy Gerry Gilmore Professor of Experimental Philosophy Institute of Astronomy Cambridge astrophysics cannot experiment merely observe and deduce: so how do we analyse
More informationParametrization and Classification of 20 Billion LSST Objects: Lessons from SDSS
SLAC-PUB-14716 Parametrization and Classification of 20 Billion LSST Objects: Lessons from SDSS Ž. Ivezić, T. Axelrod, A.C. Becker, J. Becla, K. Borne, D.L. Burke, C.F. Claver, K.H. Cook, A. Connolly,
More informationMachine Learning Applications in Astronomy
Machine Learning Applications in Astronomy Umaa Rebbapragada, Ph.D. Machine Learning and Instrument Autonomy Group Big Data Task Force November 1, 2017 Research described in this presentation was carried
More informationAnomaly Detection. Jing Gao. SUNY Buffalo
Anomaly Detection Jing Gao SUNY Buffalo 1 Anomaly Detection Anomalies the set of objects are considerably dissimilar from the remainder of the data occur relatively infrequently when they do occur, their
More informationData Exploration vis Local Two-Sample Testing
Data Exploration vis Local Two-Sample Testing 0 20 40 60 80 100 40 20 0 20 40 Freeman, Kim, and Lee (2017) Astrostatistics at Carnegie Mellon CMU Astrostatistics Network Graph 2017 (not including collaborations
More informationWhat shall we learn about the Milky Way using Gaia and LSST?
What shall we learn about the Milky Way using Gaia and LSST? Astr 511: Galactic Astronomy! Winter Quarter 2015! University of Washington, Željko Ivezić!! The era of surveys... Standard: What data do I
More informationFrom the Big Bang to Big Data. Ofer Lahav (UCL)
From the Big Bang to Big Data Ofer Lahav (UCL) 1 Outline What is Big Data? What does it mean to computer scientists vs physicists? The Alan Turing Institute Machine learning examples from Astronomy The
More informationPhysics Lab #10: Citizen Science - The Galaxy Zoo
Physics 10263 Lab #10: Citizen Science - The Galaxy Zoo Introduction Astronomy over the last two decades has been dominated by large sky survey projects. The Sloan Digital Sky Survey was one of the first
More informationGalaxies. Introduction. Different Types of Galaxy. Teacher s Notes. Shape. 1. Download these notes at
1. Introduction A galaxy is a collection of stars, the remains of stars, gas and dust, and the mysterious dark matter. There are many different types and sizes of galaxies, ranging from dwarf galaxies
More informationNew Astronomy With a Virtual Observatory
New Astronomy With a Virtual Observatory S. G. Djorgovski (Caltech) With special thanks to R. Brunner, A. Szalay, A. Mahabal, et al. Introduction: Astronomy in the Era of Information Abundance The Virtual
More informationReal-time Variability Studies with the NOAO Mosaic Imagers
Mem. S.A.It. Vol. 74, 989 c SAIt 2003 Memorie della Real-time Variability Studies with the NOAO Mosaic Imagers R. Chris Smith 1 and Armin Rest 1,2 1 NOAO/Cerro Tololo Inter-American Observatory, Colina
More informationData Intensive Computing meets High Performance Computing
Data Intensive Computing meets High Performance Computing Kathy Yelick Associate Laboratory Director for Computing Sciences, Lawrence Berkeley National Laboratory Professor of Electrical Engineering and
More informationSome ML and AI challenges in current and future optical and near infra imaging datasets
Some ML and AI challenges in current and future optical and near infra imaging datasets Richard McMahon (Institute of Astronomy, University of Cambridge) and Cameron Lemon, Estelle Pons, Fernanda Ostrovski,
More informationLecture 25: Cosmology: The end of the Universe, Dark Matter, and Dark Energy. Astronomy 111 Wednesday November 29, 2017
Lecture 25: Cosmology: The end of the Universe, Dark Matter, and Dark Energy Astronomy 111 Wednesday November 29, 2017 Reminders Online homework #11 due Monday at 3pm One more lecture after today Monday
More informationTime Domain Astronomy in the 2020s:
Time Domain Astronomy in the 2020s: Developing a Follow-up Network R. Street Las Cumbres Observatory Workshop Movies of the Sky Vary in depth, sky region, wavelengths, cadence Many will produce alerts
More informationAstronomy 1. 10/17/17 - NASA JPL field trip 10/17/17 - LA Griffith Observatory field trip
Astronomy 1 10/17/17 - NASA JPL field trip 10/17/17 - LA Griffith Observatory field trip CH 1 Here and NOW Where do we fit in the Universe? How-small-we-really-are-in-this-universe Start here: The figure
More informationScience Alerts from GAIA. Simon Hodgkin Institute of Astronomy, Cambridge
Science Alerts from GAIA Simon Hodgkin Institute of Astronomy, Cambridge Simon Hodgkin, IoA, Cambridge, UK 1 Discover the Cosmos, CERN, Sept 1-2 2011 A word on nomenclature Definition of a science alert:
More informationHow Did the Universe Begin?
How Did the Universe Begin? As we will discuss in this lecture, it looks like the Universe started about 14 billion years ago and has been expanding (space stretching) ever since. The model of what happened
More informationIgor Soszyński. Warsaw University Astronomical Observatory
Igor Soszyński Warsaw University Astronomical Observatory SATELLITE vs. GROUND-BASED ASTEROSEISMOLOGY SATELLITE: Outstanding precision! High duty cycle (no aliases) HST MOST Kepler CoRoT Gaia IAU GA 2015,
More informationLearning algorithms at the service of WISE survey
Katarzyna Ma lek 1,2,3, T. Krakowski 1, M. Bilicki 4,3, A. Pollo 1,5,3, A. Solarz 2,3, M. Krupa 5,3, A. Kurcz 5,3, W. Hellwing 6,3, J. Peacock 7, T. Jarrett 4 1 National Centre for Nuclear Research, ul.
More informationHubble s Law and the Cosmic Distance Scale
Lab 7 Hubble s Law and the Cosmic Distance Scale 7.1 Overview Exercise seven is our first extragalactic exercise, highlighting the immense scale of the Universe. It addresses the challenge of determining
More informationLSST Science. Željko Ivezić, LSST Project Scientist University of Washington
LSST Science Željko Ivezić, LSST Project Scientist University of Washington LSST@Europe, Cambridge, UK, Sep 9-12, 2013 OUTLINE Brief overview of LSST science drivers LSST science-driven design Examples
More informationOverview of Modern Astronomy. Prof. D. L. DePoy
Astronomy 111: Overview of Modern Astronomy Prof. D. L. DePoy Fall 2013 Course Description This course will cover the roots of modern astronomy, the scientific method, fundamental physical ysca laws, the
More informationDirected Reading A. Section: The Life Cycle of Stars TYPES OF STARS THE LIFE CYCLE OF SUNLIKE STARS A TOOL FOR STUDYING STARS.
Skills Worksheet Directed Reading A Section: The Life Cycle of Stars TYPES OF STARS (pp. 444 449) 1. Besides by mass, size, brightness, color, temperature, and composition, how are stars classified? a.
More informationThe Sloan Digital Sky Survey
The Sloan Digital Sky Survey Robert Lupton Xiaohui Fan Jim Gunn Željko Ivezić Jill Knapp Michael Strauss University of Chicago, Fermilab, Institute for Advanced Study, Japanese Participation Group, Johns
More informationA Look Back: Galaxies at Cosmic Dawn Revealed in the First Year of the Hubble Frontier Fields Initiative
A Look Back: Galaxies at Cosmic Dawn Revealed in the First Year of the Hubble Frontier Fields Initiative Dr. Gabriel Brammer (ESA/AURA, STScI) Hubble Science Briefing / November 6, 2014 1 The Early Universe
More informationFrom DES to LSST. Transient Processing Goes from Hours to Seconds. Eric Morganson, NCSA LSST Time Domain Meeting Tucson, AZ May 22, 2017
From DES to LSST Transient Processing Goes from Hours to Seconds Eric Morganson, NCSA LSST Time Domain Meeting Tucson, AZ May 22, 2017 Hi, I m Eric Dr. Eric Morganson, Research Scientist, Nation Center
More informationLecture 22: The expanding Universe. Astronomy 111 Wednesday November 15, 2017
Lecture 22: The expanding Universe Astronomy 111 Wednesday November 15, 2017 Reminders Online homework #10 due Monday at 3pm Then one week off from homeworks Homework #11 is the last one The nature of
More informationLecture Outlines. Chapter 25. Astronomy Today 7th Edition Chaisson/McMillan Pearson Education, Inc.
Lecture Outlines Chapter 25 Astronomy Today 7th Edition Chaisson/McMillan Chapter 25 Galaxies and Dark Matter Units of Chapter 25 25.1 Dark Matter in the Universe 25.2 Galaxy Collisions 25.3 Galaxy Formation
More informationarxiv:astro-ph/ v1 3 Aug 2004
Exploring the Time Domain with the Palomar-QUEST Sky Survey arxiv:astro-ph/0408035 v1 3 Aug 2004 A. Mahabal a, S. G. Djorgovski a, M. Graham a, R. Williams a, B. Granett a, M. Bogosavljevic a, C. Baltay
More informationHow do telescopes "see" on Earth and in space?
How do telescopes "see" on Earth and in space? By NASA, adapted by Newsela staff on 03.28.17 Word Count 933 Level 970L TOP IMAGE: The Hubble Space Telescope orbiting in space over Earth. SECOND IMAGE:
More informationD4.2. First release of on-line science-oriented tutorials
EuroVO-AIDA Euro-VO Astronomical Infrastructure for Data Access D4.2 First release of on-line science-oriented tutorials Final version Grant agreement no: 212104 Combination of Collaborative Projects &
More informationAstronomical "color"
Astronomical "color" What color is the star Betelgeuse? It's the bright star at upper left in this picture of Orion taken by a student at the RIT Observatory. Orange? Red? Yellow? These are all reasonable
More informationClassifying Galaxy Morphology using Machine Learning
Julian Kates-Harbeck, Introduction: Classifying Galaxy Morphology using Machine Learning The goal of this project is to classify galaxy morphologies. Generally, galaxy morphologies fall into one of two
More informationScience at the Kavli Institute
Science at the Institute SLUO Presentation July 11 2003 Institute Collaboration Joint initiative by SLAC/campus faculties Science at the interface of physics and astronomy Technology from physics and astronomy
More informationVirtual Observatory: Observational and Theoretical data
Astrophysical Technology Group - OAT Virtual Observatory: Observational and Theoretical data Patrizia Manzato INAF-Trieste Astronomical Observatory 06 Dec 2007 Winter School, P.sso Tonale - P.Manzato 1
More informationChandra: Revolution through Resolution. Martin Elvis, Chandra X-ray Center
Chandra: Revolution through Resolution Martin Elvis, Chandra X-ray Center The Chandra X-ray Observatory Launched 5 years ago revolutionized X-ray astronomy, and all of astronomy. What is X-ray Astronomy?
More informationThe New LSST Informatics and Statistical Sciences Collaboration Team. Kirk Borne Dept of Computational & Data Sciences GMU
The New LSST Informatics and Statistical Sciences Collaboration Team Kirk Borne Dept of Computational & Data Sciences GMU We are facing a huge problem! The Tsunami We are facing a huge problem! The Data
More informationANTARES: The Arizona-NOAO Temporal Analysis and Response to Events System
ANTARES: The Arizona-NOAO Temporal Analysis and Response to Events System Thomas Matheson and Abhijit Saha National Optical Astronomy Observatory 950 North Cherry Avenue Tucson, AZ 85719, USA and Richard
More informationLecture Outlines. Chapter 26. Astronomy Today 8th Edition Chaisson/McMillan Pearson Education, Inc.
Lecture Outlines Chapter 26 Astronomy Today 8th Edition Chaisson/McMillan Chapter 26 Cosmology Units of Chapter 26 26.1 The Universe on the Largest Scales 26.2 The Expanding Universe 26.3 The Fate of the
More informationTest ABCDE. 1. What is the oldest era on the geological timescale? A. Precambrian B. Paleozoic C. Mesozoic D. Cenozoic
Test - 8.8 ABCDE 1. What is the oldest era on the geological timescale? A. Precambrian B. Paleozoic C. Mesozoic D. Cenozoic 2. A light-year is defined as- F. the distance from Earth to the Sun. G. the
More informationOutline Challenges of Massive Data Combining approaches Application: Event Detection for Astronomical Data Conclusion. Abstract
Abstract The analysis of extremely large, complex datasets is becoming an increasingly important task in the analysis of scientific data. This trend is especially prevalent in astronomy, as large-scale
More informationPractice Test: ES-5 Galaxies
Class: Date: Practice Test: ES-5 Galaxies Multiple Choice Identify the letter of the choice that best completes the statement or answers the question. Light Years * The distance between stars and galaxies
More informationRick Ebert & Joseph Mazzarella For the NED Team. Big Data Task Force NASA, Ames Research Center 2016 September 28-30
NED Mission: Provide a comprehensive, reliable and easy-to-use synthesis of multi-wavelength data from NASA missions, published catalogs, and the refereed literature, to enhance and enable astrophysical
More informationBasics of Multivariate Modelling and Data Analysis
Basics of Multivariate Modelling and Data Analysis Kurt-Erik Häggblom 2. Overview of multivariate techniques 2.1 Different approaches to multivariate data analysis 2.2 Classification of multivariate techniques
More informationLSST. Pierre Antilogus LPNHE-IN2P3, Paris. ESO in the 2020s January 19-22, LSST ESO in the 2020 s 1
LSST Pierre Antilogus LPNHE-IN2P3, Paris ESO in the 2020s January 19-22, 2015 LSST ESO in the 2020 s 1 LSST in a Nutshell The LSST is an integrated survey system designed to conduct a decadelong, deep,
More informationIntroduction: Don t get bogged down in classical (grade six) topics, or number facts, or other excessive low-level knowledge.
Introduction: Comments: SES4U: Earth and Space Science, Grade 12 John R. Percy Department of Astronomy and Astrophysics University of Toronto Email: john.percy@utoronto.ca; August 2012 Emphasize astronomical
More informationLarge Synoptic Survey Telescope, Computational-science Requirements of. George Beckett, 2 nd September 2016
Large Synoptic Survey Telescope, Computational-science Requirements of George Beckett, 2 nd September 2016 Overview Large Synoptic Survey Telescope (LSST) New optical telescope being constructed in northern
More informationEEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1
EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle
More informationAstronomy is remote sensing
Astronomy is remote sensing We cannot repeat (or change) the Universe in a controlled environment. We cannot make planets, stars, or galaxies. We cannot make the vacuum of space, nor the shape of spacetime
More informationMassive Event Detection. Abstract
Abstract The detection and analysis of events within massive collections of time-series has become an extremely important task for timedomain astronomy. In particular, many scientific investigations (e.g.
More informationSearching for Needles in the Sloan Digital Haystack
Searching for Needles in the Sloan Digital Haystack Robert Lupton Željko Ivezić Jim Gunn Jill Knapp Michael Strauss University of Chicago, Fermilab, Institute for Advanced Study, Japanese Participation
More informationCollecting Light. In a dark-adapted eye, the iris is fully open and the pupil has a diameter of about 7 mm. pupil
Telescopes Collecting Light The simplest means of observing the Universe is the eye. The human eye is sensitive to light with a wavelength of about 400 and 700 nanometers. In a dark-adapted eye, the iris
More informationExascale IT Requirements in Astronomy Joel Primack, University of California, Santa Cruz
Exascale IT Requirements in Astronomy Joel Primack, University of California, Santa Cruz Director, University of California High-Performance AstroComputing Center Exascale IT Requirements in Astronomy
More informationCOSMOLOGY PHYS 30392 OBSERVING THE UNIVERSE Part I Giampaolo Pisano - Jodrell Bank Centre for Astrophysics The University of Manchester - January 2013 http://www.jb.man.ac.uk/~gp/ giampaolo.pisano@manchester.ac.uk
More informationASTRONOMY (ASTRON) ASTRON 113 HANDS ON THE UNIVERSE 1 credit.
Astronomy (ASTRON) 1 ASTRONOMY (ASTRON) ASTRON 100 SURVEY OF ASTRONOMY 4 credits. Modern exploration of the solar system; our galaxy of stars, gas and dust; how stars are born, age and die; unusual objects
More informationAstronomy 1 Fall 2016
Astronomy 1 Fall 2016 One person s perspective: Three great events stand at the threshold of the modern age and determine its character: 1) the discovery of America; 2) the Reformation; 3) the invention
More informationThe Galaxy Zoo Project
Astronomy 201: Cosmology Fall 2009 Prof. Bechtold NAME: The Galaxy Zoo Project 200 points Due: Nov. 23, 2010, in class Professional astronomers often have to search through enormous quantities of data
More informationCosmology with the Sloan Digital Sky Survey Supernova Search. David Cinabro
Cosmology with the Sloan Digital Sky Survey Supernova Search David Cinabro Cosmology Background The study of the origin and evolution of the Universe. First real effort by Einstein in 1916 Gravity is all
More informationANSWER KEY. Stars, Galaxies, and the Universe. Telescopes Guided Reading and Study. Characteristics of Stars Guided Reading and Study
Stars, Galaxies, a the Universe Stars, Galaxies, and the Universe Telescopes Use Target Reading Skills Check student definitions for accuracy. 1. Electromagneticradiationisenergythatcan travel through
More informationLocal Volume, Milky Way, Stars, Planets, Solar System: L3 Requirements
Local Volume, Milky Way, Stars, Planets, Solar System: L3 Requirements Anthony Brown Sterrewacht Leiden brown@strw.leidenuniv.nl Sterrewacht Leiden LSST@Europe2 2016.06.21-1/8 LSST data product levels
More informationOptical Synoptic Telescopes: New Science Frontiers *
Optical Synoptic Telescopes: New Science Frontiers * J. Anthony Tyson Physics Dept., University of California, One Shields Ave., Davis, CA USA 95616 ABSTRACT Over the past decade, sky surveys such as the
More informationIntroduction to SDSS -instruments, survey strategy, etc
Introduction to SDSS -instruments, survey strategy, etc (materials from http://www.sdss.org/) Shan Huang 17 February 2010 Survey type Status Imaging and Spectroscopy Basic Facts SDSS-II completed, SDSS-III
More informationJINA Observations, Now and in the Near Future
JINA Observations, Now and in the Near Future Timothy C. Beers Department of Physics & Astronomy Michigan State University & JINA: Joint Institute for Nuclear Astrophysics Examples SDSS-I, II, and III
More informationAPS Science Curriculum Unit Planner
APS Science Curriculum Unit Planner Grade Level/Subject The Universe Stage 1: Desired Results Enduring Understanding Galaxies are some of the largest collections of matter in the Universe, and we reside
More informationPDF hosted at the Radboud Repository of the Radboud University Nijmegen
PDF hosted at the Radboud Repository of the Radboud University Nijmegen The following full text is a publisher's version. For additional information about this publication click this link. http://hdl.handle.net/2066/84153
More informationCS570 Data Mining. Anomaly Detection. Li Xiong. Slide credits: Tan, Steinbach, Kumar Jiawei Han and Micheline Kamber.
CS570 Data Mining Anomaly Detection Li Xiong Slide credits: Tan, Steinbach, Kumar Jiawei Han and Micheline Kamber April 3, 2011 1 Anomaly Detection Anomaly is a pattern in the data that does not conform
More informationStudying the universe
Studying the universe What is astronomy? A branch of science that deals with study of stars, planets, the universe as a whole The idea is that we live in a clockwork universe and is governed by laws that
More informationAnomaly (outlier) detection. Huiping Cao, Anomaly 1
Anomaly (outlier) detection Huiping Cao, Anomaly 1 Outline General concepts What are outliers Types of outliers Causes of anomalies Challenges of outlier detection Outlier detection approaches Huiping
More informationA Convex Hull Peeling Depth Approach to Nonparametric Massive Multivariate Data Analysis with Applications
A Convex Hull Peeling Depth Approach to Nonparametric Massive Multivariate Data Analysis with Applications Hyunsook Lee. hlee@stat.psu.edu Department of Statistics The Pennsylvania State University Hyunsook
More informationGalaxies and Cosmology
4/28/17 The Discovery of Galaxies Up to the 1920 s, astronomers were not sure exactly how far away galaxies were, and thus didn t know how big they are! Spiral Nebulae could be assumed to be inside our
More informationChapter 26: Cosmology
Chapter 26: Cosmology Cosmology means the study of the structure and evolution of the entire universe as a whole. First of all, we need to know whether the universe has changed with time, or if it has
More informationCosmology with the Sloan Digital Sky Survey Supernova Search. David Cinabro
Cosmology with the Sloan Digital Sky Survey Supernova Search David Cinabro Cosmology Background The study of the origin and evolution of the Universe. First real effort by Einstein in 1916 Gravity is all
More informationFast Hierarchical Clustering from the Baire Distance
Fast Hierarchical Clustering from the Baire Distance Pedro Contreras 1 and Fionn Murtagh 1,2 1 Department of Computer Science. Royal Holloway, University of London. 57 Egham Hill. Egham TW20 OEX, England.
More informationGLAST. Exploring the Extreme Universe. Kennedy Space Center. The Gamma-ray Large Area Space Telescope
GLAST Gamma-ray Large Area Space Telescope The Gamma-ray Large Area Space Telescope Exploring the Extreme Universe Kennedy Space Center Dave Thompson GLAST Deputy Project Scientist David.J.Thompson@nasa.gov
More informationComputational Intelligence Challenges and. Applications on Large-Scale Astronomical Time Series Databases
Computational Intelligence Challenges and 1 Applications on Large-Scale Astronomical Time Series Databases arxiv:1509.07823v1 [astro-ph.im] 25 Sep 2015 Pablo Huijse, Millennium Institute of Astrophysics,
More informationAnswer Key for Exam C
Answer Key for Exam C 1 point each Choose the answer that best completes the question. Read each problem carefully and read through all the answers. Take your time. If a question is unclear, ask for clarification
More informationAnswer Key for Exam B
Answer Key for Exam B 1 point each Choose the answer that best completes the question. Read each problem carefully and read through all the answers. Take your time. If a question is unclear, ask for clarification
More informationASTRONOMY 202 Spring 2007: Solar System Exploration
ASTRONOMY 202 Spring 2007: Solar System Exploration Instructor: Dr. David Alexander Web-site: www.ruf.rice.edu/~dalex/astr202_s07 Class 3: Our Place in the Universe [1/19/07] Announcements Scale in the
More informationDark Energy and Dark Matter
Dark Energy and Dark Matter Attendance Quiz Are you here today? (a) yes (b) no Here! (c) The Dark Lord wants his ring back! Final Exam The final exam is Thursday, 6/8, from 11:30am to 1:30pm (2 hours),
More informationPRACTICAL ANALYTICS 7/19/2012. Tamás Budavári / The Johns Hopkins University
PRACTICAL ANALYTICS / The Johns Hopkins University Statistics Of numbers Of vectors Of functions Of trees Statistics Description, modeling, inference, machine learning Bayesian / Frequentist / Pragmatist?
More informationV. Astronomy Section
EAS 100 Planet Earth Lecture Topics Brief Outlines V. Astronomy Section 1. Introduction, Astronomical Distances, Solar System Learning objectives: Develop an understanding of Earth s position in the solar
More informationLecture 16 The Measuring the Stars 3/26/2018
Lecture 16 The Measuring the Stars 3/26/2018 Test 2 Results D C B A Questions that I thought were unfair: 13, 18, 25, 76, 77, 80 Curved from 85 to 79 Measuring stars How far away are they? How bright are
More informationo Terms to know o Big Bang Theory o Doppler Effect o Redshift o Universe
Standard 1: Students will understand the scientific evidence that supports theories that explain how the universe and the solar system developed. They will compare Earth to other objects in the solar system.
More informationHomework on Properties of Galaxies in the Hubble Deep Field Name: Due: Friday, April 8 30 points Prof. Rieke & TA Melissa Halford
Homework on Properties of Galaxies in the Hubble Deep Field Name: Due: Friday, April 8 30 points Prof. Rieke & TA Melissa Halford You are going to work with some famous astronomical data in this homework.
More informationRFI Detectives Activity for Large Public Venues
RFI Detectives Activity for Large Public Venues Adapted from the NRAO s Be An Interference Detective activity Background Most people living near big cities are familiar with light pollution and how it
More informationCosmology. Stellar Parallax seen. The modern view of the universe
Cosmology The modern view of the universe SC/NATS 1730, XXVIIICosmology 1 Stellar Parallax Copernicus said stellar parallax couldn t be seen because the stars were so far away. A strictly ad hoc explanation
More information