The SDSS - data & how to get it Jarle Brinchmann Leiden,

Transcription

1 The SDSS - data & how to get it Jarle Brinchmann Leiden,

2 An outline Galaxy bimodality The AGN population Environment - does it matter? The M-Z relation. Large-scale structure stuff The Sloan Digital Sky Survey (SDSS) The general characteristics of the survey The data obtained and the quantities measured Gotcha s Navigation of the SDSS online data Databases - a slight detour SQL - clever searching Extracting data from the SDSS and making use of it all Going further - web services, matches to other surveys etc.

3 What might it be useful for during your research? You might be able to use the SDSS to create a comparison sample for your analysis. E.g. find similar stars, galaxy clusters and extract these to put your results in context. You might learn a bit of SQL, doesn t hurt. Quick way to create finding charts. You may simply have need for the data!... Hopefully you ll have ideas of your own after this presentation!

4 The SDSS The most ambitious survey of the sky ever undertaken. Imaging survey of 8600 square degrees. Redshifts of more than 1,000,000 galaxies & QSOs. Robotic 2.5m telescope - imaging & Spectrscopy

5 Parts of the RC3 NGC 1068 NGC 3718 NGC 2782 NGC 3310 Plans exist to create a Gunn Atlas of Galaxies NGC 1068

6 The SDSS DR7 - (Autumn 2008) Legacy 5 band imaging over 8423 deg 2 down to r~21.5 ~230 million objects Spectra covering 3800Å-9200Å with R~ ,567 galaxy spectra, 109,862 QSO spectra with z<2.3 and 8,802 high-z QSOs Median seeing SEGUE 5 band imaging over 3240 deg 2 ~127 million objects. 229,466 spectra of stars of type K and earlier, and 7,922 M stars and later. log g, Teff, [Fe/H] and And: Supernova survey, M 31 and other special scans & low galactic latitude runs

7 DR5

8 DR5

9 DR5 Sloan Great Wall

10 The Workings Images are taken in a drift-scan using a mosaic camera with five filters. Each scan is called a strip, but to cover gaps a second scan is made and the combination is called a stripe. The exposure time is ~54s with ~73s between each filter. The images are then analysed using a code called Photo and based on the measurements objects are selected for spectroscopy. Spectroscopy is carried out the following season using a fibre spectrograph with 3 fibres covering 3800Å-9000Å. Spectroscopic data are analysed using two pipelines and imaging & spectroscopy is released to the public.

11 Advantages/ Disadvantages Very uniform and well-characterised photometry & spectroscopy and automatic pipeline. Very large dataset & advanced data base interface allow (relatively) simple access. Decent spectral resolution & S/N. r-band selection. Images have mediocre seeing & relatively shallow. Limited amount of matching data (radio, X-ray, submm, HST etc.) Automatic data reduction not ideal for all types of objects.

12 Did the SDSS find anything new? The low-z universe has been well studied up through the years - was there anything new to be found? Tully-Fisher Faber-Jackson Luminosity functions Mg 2 -σ Colour-Magnitude etc.

13 Did the SDSS find anything new? The low-z universe has been well studied up through the years - was there anything new to be found? Main progress areas: Yes! Tully-Fisher Faber-Jackson Luminosity functions Mg 2 -σ Colour-Magnitude etc. Well-understood selection function allows the construction of distribution functions. The vastness of the sample provides large samples of extreme objects. Well-known trends can be studied with very high precision.

14 Luminosity function Is it a Schechter function?

15 Luminosity function Is it a Schechter function?

16 Luminosity function Not quite. But quite close... Is it a Schechter function?

17 Luminosity function Not quite. But quite close... Is it a Schechter function? And remember: z= is 2.5 Gyr Evolution is important

18 Mass Function Panter et al (2004)

19 Mass Function Panter et al (2004) Note excess at large mass/luminosity - likely either to be cd galaxies or photometric problems.

20 Bi-modality A key result from the SDSS was to show how ubiquitous bi-modality is in the local universe

21 Bi-modality A key result from the SDSS was to show how ubiquitous bi-modality is in the local universe In colour. (Baldry et al 2004; Blanton et al 2003; Hogg et al ) Baldry et al (2004) >50% of all galaxies M r < -22 are on the red sequence u-r

22 Bi-modality A key result from the SDSS was to show how ubiquitous bi-modality is in the local universe In D4000 & HδA (SFH) (Kauffmann et al 2003ab)

23 Bi-modality A key result from the SDSS was to show how ubiquitous bi-modality is in the local universe In D4000 & HδA (SFH) (Kauffmann et al 2003ab) Characteristic mass: ~3x10 10 M Also characteristic µ * =3x10 8 M

24 Bi-modality A key result from the SDSS was to show how ubiquitous bi-modality is in the local universe And in SFR/M* (Brinchmann et al 2004) Again a transition around M* ~ 3x10 10 M Fraction Log SFR/M * [yr -1 ]

25 Galaxy-Galaxy lensing The SDSS is not ideal for lensing because the image quality is rather mediocre, but there is plenty of galaxies! This has been used by Mandelbaum et al in a series of papers to study shape correlations, halo mass vs stellar mass etc. Here is the halo mass vs stellar mass comparison:

26 Galaxy-Quasar lensing and dust in galaxy halos QSOs Lensing Observer

27 Galaxy-Quasar lensing and dust in galaxy halos QSOs Lensing + reddening Observer

28 Galaxy-Quasar lensing and dust in galaxy halos QSOs Lensing + reddening Observer Gravitational lensing is achromatic, so if you correlate QSOs with galaxies and look at magnification and the dependence on wavelength you might find signs of dust in galaxy halos.

29 Galaxy-Quasar lensing and dust in galaxy halos Ménard et al (2009)

30 Galaxy-Quasar lensing and dust in galaxy halos Ménard et al (2009) E(B-V) ~ 1.4x10-3 (θ/1 ) They argue that the main contribution of dust is from L ~ 0.5L * galaxies and this gives a typical dust mass in the halo of ~10 7 M The mean opacity of the Universe is A V ~ 0.05 mag to z~1

31 So, how do you do these studies using the imaging data?

32 Magnitude System(s) Normal magnitudes: m=-2.5 Log 10 f + zp asinh magnitudes (luptitudes): m=-2.5/ln(10) [asinh((f/f0)/(2b)) + ln(b)] The difference is small (irrelevant) for bright objects but is very important at faint flux levels! nano-maggies: A linear flux measurement such that 1 nmgy corresponds to a conventional magnitude of Widely used in NYU VAGC & Princeton data.

33 Magnitude System(s)

34 Magnitude System(s) Relationship to other photometric calibrations: The SDSS is a near AB magnitude system. Thus conversion to Janskys is fairly easy: AB=2.5 ( 23 - log 10 F [Jy] ) But there are some small offsets, such that: u AB = u SDSS & z AB = z SDSS To convert to Johnson etc systems: An example (for stars, from Robert Lupton): B = g *(g - r) V = g *(g - r) R = r *(r - i) I = i *(i - z)

35 Magnitude System(s) Relationship to other photometric calibrations: The SDSS is a near AB magnitude system. Thus conversion to Janskys is fairly easy: AB=2.5 Be aware ( 23 that - log 10 there F [Jy] now ) - is 48.6 Ubercalibration: Reduced systematic effects across the entire survey (~1%). But there are some small offsets, such that: u AB = u SDSS & z AB = z SDSS To convert to Johnson etc systems: (Padmanabhan et al 2007) An example (for stars, from Robert Lupton): B = g *(g - r) V = g *(g - r) R = r *(r - i) I = i *(i - z)

36 The offerings: Photometry Photometric data comes from Photo:

37 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF]

38 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF] Sizes [Petrosian radii, Image moments]

39 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF] Sizes [Petrosian radii, Image moments] Positions [better than 0.1, often multiple observations]

40 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF] Sizes [Petrosian radii, Image moments] Positions [better than 0.1, often multiple observations] Shapes, orientations [image moments]

41 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF] Sizes [Petrosian radii, Image moments] Positions [better than 0.1, often multiple observations] Shapes, orientations [image moments] Images [full field, JPGs, objects only - atlas images ]

42 The offerings: Photometry Photometric data comes from Photo: Magnitudes [ugriz, Petrosian, Model, Apertures, PSF] Sizes [Petrosian radii, Image moments] Positions [better than 0.1, often multiple observations] Shapes, orientations [image moments] Images [full field, JPGs, objects only - atlas images ] Image quality [position dependent PSF as PCAs]

43 The offerings: Photometry - Advice Reliability:

44 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality.

45 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5).

46 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5). r<14.5 requires care and sky estimates for very large galaxies are often poor. See e.g. Blanton et al (2005) for details.

47 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5). r<14.5 requires care and sky estimates for very large galaxies are often poor. See e.g. Blanton et al (2005) for details. Large, actively star forming galaxies are sometimes shredded with multiple spectroscopic targets.

48 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5). r<14.5 requires care and sky estimates for very large galaxies are often poor. See e.g. Blanton et al (2005) for details. Large, actively star forming galaxies are sometimes shredded with multiple spectroscopic targets. Take care to check imaging flags.

49 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5). r<14.5 requires care and sky estimates for very large galaxies are often poor. See e.g. Blanton et al (2005) for details. Large, actively star forming galaxies are sometimes shredded with multiple spectroscopic targets. Take care to check imaging flags. What should I use? For colours: Model magnitudes or aperture magnitudes. For total flux: Petrosian or optimal model magnitudes. Fiber magnitudes are useful to check spectrophotometry.

50 The offerings: Photometry - Advice Reliability: The r-band is in general the most reliable - u & z might be poor quality. r<21.5 should in general be fine, but be aware of singleband detections (typical detection limit rab ~ 22.5). r<14.5 requires care and sky estimates for very large galaxies are often poor. See e.g. Blanton et al (2005) for details. Large, actively star forming galaxies are sometimes shredded with multiple spectroscopic targets. Take care to check imaging flags. What should I use? For colours: Model magnitudes or aperture magnitudes. For total flux: Petrosian or optimal model magnitudes. Fiber magnitudes are useful to check spectrophotometry.

51 Imaging Flags... These are set as individual bits in a long integer to indicate whether all was well with the photometric analysis. It is crucial to check these when doing accurate work! Some examples: SATURATED: Tells you whether the image of an object was saturated. EDGE: Is the object on the edge (commonly the case for large galaxies). CHILD: Is this object part of a larger object that was split? MOVED: Did the object move? etc. etc. See catalogs/flags.html for details.

52 Databases & SQL

53 The need for Databases Very large data sets require advanced techniques for analysis. Most researchers lack the skills to do e.g. correlation functions for >10 8 objects. Normal files (e.g. text files, FITS files) are inconvenient when >1000 properties are measured for each object. The indexing properties of databases make searches in general very fast and flexible and allow for powerful combinations of tables. It is even useful outside of astronomy... :)

54 Interplay Web server HTML pages (typically) Browser User XML Data base server Often SQL User interface Efficient data transfer Data store (local or anywhere on the net)

55 A simple version of the SDSS model Object ID Image Position u,g,r,i,z Profiles Object ID Object ID SpecObjID Redshift Spectra Via SpecLine Emission line flux SpecObj

56 A simple version of the SDSS model Object ID Image SpecObjID Spectra Position u,g,r,i,z Profiles Redshift Via SpecLine Emission line flux SpecObj Object ID Object ID Note that a given position can pay host to different objects! Note also that the photometric objects really live in a separate data base table from the spectroscopic objects - the keys connect the two. Note also that there is a lot of information outside this structure e.g.:

57 A simple version of the SDSS model Object ID Image SpecObjID Spectra Position u,g,r,i,z Profiles Redshift Via SpecLine Emission line flux SpecObj Object ID Object ID Note that a given position can pay host to different objects! Note also that the photometric objects really live in a separate data base table from the spectroscopic objects - the keys connect the two. Note also that there is a lot of information outside this structure e.g.: X-ray data, common names ++

58 The SDSS Database Tables PhotoObjAll, SpecObjAll, Photoz, RC3, etc... These contain the different sets of data and form the backbone Views PhotoObj, SpecObj, Galaxy, Star, Sky etc... These are provide convenient interfaces to Tables. This is often what you want to use! Functions fdistanceeq, ffield, fgetnearbyobjeq etc... Functions that might be very useful in queries and can also save you quite a bit of work. They are particularly powerful when combined with iteration To find out exactly what is available: Look at the Schema

59 Tables The SDSS Database PhotoObjAll, SpecObjAll, Photoz, RC3, etc... These contain the different sets of data and form the backbone Important: Views Best vs. Target PhotoObj, SpecObj, Galaxy, Star, Sky etc... These are provide convenient interfaces to Tables. This is often what you want to use! The photometry that was used when deciding spectroscopic targets is stored in Target - this is often different from the Functions fdistanceeq, ffield, fgetnearbyobjeq etc... Functions that might be very useful in queries and can also save you quite a bit of work. They are particularly powerful when combined currently with iteration Best reductions. You have to choose one of these two contexts. To find Normally out exactly you what want is available: to use Best. Look at the Schema

60 Tables The SDSS Database Important: Primary & SciencePrimary PhotoObjAll, SpecObjAll, Photoz, RC3, etc... These contain the different sets of data and form the backbone Important: Views Best vs. Target PhotoObj, SpecObj, Galaxy, Star, Sky etc... Each real object on the sky should have one Primary object These are associated provide convenient with it. interfaces They might to Tables. also This have is often Secondary what you want to use! objects associated but you will often only need primary The photometry that was used when deciding spectroscopic targets is stored in Target - this is often different from the Functions objects. fdistanceeq, ffield, fgetnearbyobjeq etc... Functions that might be very useful in queries and can also save you quite a bit of work. They are particularly powerful when combined currently with iteration Best reductions. You have to For spectra there is a similar sounding quantity called SciencePrimary - this indicates whether the spectrum is choose one of these two contexts. considered good enough quality AND that this is not a reobservation. To find Normally out exactly you what want is available: to use Best. Look at the Schema For SEGUE this distinction is not very useful

61 Look for keys Access to various parts of the database The view, table or function

62 Structured Query Language - SQL A computer language designed for efficient query of databases. It is used both to create tables/databases and to search these. In our work we are often only concerned with the search aspect: SELECT specifies what is to be returned FROM specifies what table to use WHERE is an optional clause that specifies a subset of the full data. But be aware that for advanced searches it is often necessary/convenient to be able to CREATE temporary tables. Combinations of tables can be done using JOINs. These require that there are entries in each table that are in common.

63 Structured Query Language - SQL SELECT objid, u, g, r, FROM PhotoPrimary WHERE u - g < 0.4 and g - r < 0.7

64 Structured Query Language - SQL SELECT objid, u, g, r, Select something FROM PhotoPrimary WHERE u - g < 0.4 and g - r < 0.7

65 Structured Query Language - SQL SELECT objid, u, g, r, FROM PhotoPrimary WHERE u - g < 0.4 and g - r < 0.7 Select something From a table/view

66 Structured Query Language - SQL SELECT objid, u, g, r, FROM PhotoPrimary WHERE u - g < 0.4 and g - r < 0.7 Select something From a table/view According to a criterion

67 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2

68 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 SELECT COUNT(*) AS "Low redshift galaxies"

69 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 SELECT COUNT(*) AS "Low redshift galaxies" SQL Statement: we want the following returned

70 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 SELECT COUNT(*) AS "Low redshift galaxies" SQL Statement: we want the following returned Count all matches to the WHERE statement below.

71 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 SELECT COUNT(*) AS "Low redshift galaxies" SQL Statement: we want the following returned Count all matches to the WHERE statement below. Alias for the result - in some systems the return data would be named according to this.

72 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2

73 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 FROM SpecObj

74 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 FROM SpecObj SQL Statement: use the following table or view.

75 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 FROM SpecObj SQL Statement: use the following table or view. The table to carry out the search within.

76 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2

77 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 WHERE z BETWEEN AND 0.2

78 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 WHERE z BETWEEN AND 0.2 SQL Statement: Only include the objects satisfying the following constraints

79 Count galaxies: Some examples: SELECT COUNT(*) AS "Low redshift galaxies" FROM SpecObj WHERE z BETWEEN AND 0.2 WHERE z BETWEEN AND 0.2 SQL Statement: Only include the objects satisfying the following constraints Only objects with redshifts between and 0.2

80 Bimodality search select top 5000 modelmag_u-extinction_u as u, modelmag_r-extinction_r as r, z, dbo.fcosmoabsmag(petromag_r-extinction_r, z) as AbsMag_R FROM SpecPhoto Where z between 0.05 and 0.07

81 Classifying emission line galaxies: The BPT diagram Metal content Hardness of ionization Baldwin, Phillips & Terlevich (1981)

82 SFR Inventory (I) Size Type Brinchmann et al (2004) Kauffmann et al 2003; Blanton et al 2003

83 SFR Inventory (I) So the SFR density in the local universe is dominated by disk galaxies similar to the Milky Way in size Size Type Brinchmann et al (2004) Kauffmann et al 2003; Blanton et al 2003

84 SFR Inventory (II) SFH Dust Metals Brinchmann et al (2004)

85 SFR Inventory (II) Most new stars in the low-z universe have ~solar abundance and while most galaxies show little attenuation, most of the stars are extincted by ~1 magnitude at H-alpha. SFH Dust Metals Brinchmann et al (2004)

86 The Mass- Metallicity Relation Tremonti et al (2004): Clear turn-over in the relation between stellar mass and gas metallicity. 12+Log O H= Log M * Log M * 2 Turn-over around Log M=10.5. Again that mass Log M=10.5! Note that the absolute metal abundance is uncertain and quite likely too high by ~0.3 dex.

87 Star Formation Histories Galaxy spectra are records of the past history of star formation in galaxies - so one can aim to invert the galaxy spectra to get their star formation history (e.g. using MOPED, STECMAP etc.) Heavens et al (2004) Downsizing very clear - without looking back in time! The method loses resolution as one looks back in time. See also Mathis et al (2006) Heavens et al 2004; Panter et al 2004; Mathis et al 2006

88 Working with spectra

89 The offerings: Spectroscopy Spectroscopic data comes from spectro1d:

90 The offerings: Spectroscopy Spectroscopic data comes from spectro1d: Absorption line indices [Lick + a few more]

91 The offerings: Spectroscopy Spectroscopic data comes from spectro1d: Absorption line indices [Lick + a few more] Emission lines [Fluxes, EWs, widths]

92 The offerings: Spectroscopy Spectroscopic data comes from spectro1d: Absorption line indices [Lick + a few more] Emission lines [Fluxes, EWs, widths] Velocity dispersions [Two pipelines, Elodie PCA]

93 The offerings: Spectroscopy Spectroscopic data comes from spectro1d: Absorption line indices [Lick + a few more] Emission lines [Fluxes, EWs, widths] Velocity dispersions [Two pipelines, Elodie PCA] Probably the area where using Value-Added Catalogues is a useful alternative! In particular the MPA-JHU database with more careful continuum subtraction and the NYU VAGC which has more matching and is an excellent base sample for further work.

94 The offerings: Spectroscopy - Advice Reliability:

95 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies.

96 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped.

97 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped. Bright sky and/or CCD defects might cause some lines not to be measured.

98 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped. Bright sky and/or CCD defects might cause some lines not to be measured. Sky subtraction is not very good in the red [c.f. Wild et al 2005]

99 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped. Bright sky and/or CCD defects might cause some lines not to be measured. Sky subtraction is not very good in the red [c.f. Wild et al 2005] The spectrophotometric calibration for DR6 is tied to PSF magnitudes - watch out for galaxies!

100 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped. Bright sky and/or CCD defects might cause some lines not to be measured. Sky subtraction is not very good in the red [c.f. Wild et al 2005] The spectrophotometric calibration for DR6 is tied to PSF magnitudes - watch out for galaxies! The slope of the spectrum in the blue is only good to the ~few % level.

101 The offerings: Spectroscopy - Advice Reliability: [O II]3727Å is not measured for low-z galaxies. Very strong emission lines are sometimes clipped. Bright sky and/or CCD defects might cause some lines not to be measured. Sky subtraction is not very good in the red [c.f. Wild et al 2005] The spectrophotometric calibration for DR6 is tied to PSF magnitudes - watch out for galaxies! The slope of the spectrum in the blue is only good to the ~few % level.

102 The offerings: SEGUE Stellar analysis from SEGUE

103 The offerings: SEGUE Stellar analysis from SEGUE Absorption line indices [Lick + a few more]

104 The offerings: SEGUE Stellar analysis from SEGUE Absorption line indices [Lick + a few more] Stellar parameters [Teff, log g, [Fe/H], R V ]

105 The offerings: SEGUE Stellar analysis from SEGUE Absorption line indices [Lick + a few more] Stellar parameters [Teff, log g, [Fe/H], R V ] The stellar parameters are derived from a number of different pipelines. Since the spectra in the SDSS are not optimal to derive stellar parameters it is necessary to compare these estimates to assess the quality of the results! And also to get external data with better resolution. ([Fe/H] ~ 0 show noticeable systematic offset). SEGUE in general targets lower Galactic latitudes and crowded fields and go outside the normal SDSS footprint. Thus much of the data is not in the normal CAS context!

106 The offerings: Some Others The SDSS Supernova search - repeat observations of Stripe 82. These are sometimes taken in poor conditions and are therefore provided in uncalibrated form.

107 The offerings: Some Others The SDSS Supernova search - repeat observations of Stripe 82. These are sometimes taken in poor conditions and are therefore provided in uncalibrated form. Moving objects [from repeat observations]

108 The offerings: Some Others The SDSS Supernova search - repeat observations of Stripe 82. These are sometimes taken in poor conditions and are therefore provided in uncalibrated form. Moving objects [from repeat observations] Variability [from repeat observations]

109 The offerings: Some Others The SDSS Supernova search - repeat observations of Stripe 82. These are sometimes taken in poor conditions and are therefore provided in uncalibrated form. Moving objects [from repeat observations] Variability [from repeat observations] Observing conditions [extinction, seeing etc.]

110 The offerings: Some Others The SDSS Supernova search - repeat observations of Stripe 82. These are sometimes taken in poor conditions and are therefore provided in uncalibrated form. Moving objects [from repeat observations] Variability [from repeat observations] Observing conditions [extinction, seeing etc.] Raw data [raw counts etc.]

111 Value-Added Data Very important! When people have done some careful job they might offer this as a value-added catalogue. Using these, rather than the official SDSS data might save you a lot of time and effort! Finding out about these: Read papers!! Check the SDSS VAC page: Listen carefully - as I will say this only once

112 Value-Added Data The NYU VAGC Large-Scale Structure samples with well characterised selection functions. Cross-matches of SDSS to other surveys. MPA-JHU VAGC Improved spectroscopic data reduction, line fluxes and indices. Derived quantities such as SFR, O/H and stellar masses etc. QSO catalogues (Schneider et al) & BAL-QSOs. White Dwarf catalogues (Eisenstein et al) Variable star catalogues from Stripe 82 & CV catalogues Moving objects catalogues Galaxy cluster catalogues (MaxBCG, cut & enhance) & more!

113 See: Blanton et al (2005) The NYU VAGC Large-scale structure sample. Careful characterisation of the survey geometry. Low-z galaxy sample (this is non-trivial to construct). SDSS data sweeps - files with a useful subset of all the data with manageable size (~100 Gb). Sersic fits & a number of other quantities. Various pieces of software - in particular kcorrect which is the most widely used software to calculate k- corrections for SDSS data. Anyone wanting a complete sample of galaxies with photometry & spectroscopy should consider this sample!

114 The MPA-JHU VAGC See e.g: Tremonti et al (2004); Brinchmann et al (2004); Improved continuum subtraction. Pipeline optimised to measure emission lines and absorption line indices on galaxy spectra. Emission lines, absorption line indices. Star formation rates, oxygen abundances, stellar masses, emission line classification of galaxies and stellar metallicities. All the photometric information available in tsobj files for spectroscopic targets. Data organised in a set of FITS files and presently only complete for DR4 but most quantities are present for DR7 as well.

115 Navigating it all

116 Navigating it all

117 The Web Interfaces The web site: Interactive browsing of the sky, finding charts, generic entry point etc. The DAS (Data Archive Server) Access to individual files - flatfiles. For large sets of files and for automatic download The CAS (Catalog Archive Server) Access to the database interfaces Google Earth & Sky

118 Navigate Point and click interface to browsing the sky. Intuitive and with useful overlays of spectroscopic/photometric targets etc. Inspection of individual objects provides much more information and direct access to flat files and NED/Simbad etc. Extremely useful for checking individual objects, problems & spectroscopic targeting.

119 Image Lists Allows the submission of a list of objects to get access to their images and provides links to navigate further. Very useful when matching to old catalogues with moderately good astrometry for instance.

120 Image Lists Allows the submission of a list of objects to get access to their images and provides links to navigate further. Very useful when matching to old catalogues with moderately good astrometry for instance.

121 A small worked example Scientific Question: How does the metal content of stars vary as a function of Galactic latitude? Step 1: Decide what data you require. We need positions & metallicity. Ra, Dec & [Fe/H] (+ quality, flags etc) Step 2: Determine what tables you can get this from. Do you need VACs? sppparams contains what we want (and more) Step 3: Write an appropriate SQL statement & download data Step 4: Read in data & carry out post-processing to visualize the results.

122 A small example Question: How does the metal content of stars vary as a function of Galactic latitude? Step 3: Write an appropriate SQL statement & download data SELECT fehw, ra, dec FROM sppparams WHERE fehwn > 3

123 A small example Question: How does the metal content of stars vary as a function of Galactic latitude? Step 3: Write an appropriate SQL statement & download data SELECT fehw, ra, dec FROM sppparams WHERE fehwn > 3 This is a weighted average of [Fe/H] estimates.

124 A small example Question: How does the metal content of stars vary as a function of Galactic latitude? Step 3: Write an appropriate SQL statement & download data SELECT fehw, ra, dec FROM sppparams WHERE fehwn > 3 This is a weighted average of [Fe/H] estimates. This ensures that only stars with at least 4 different estimates of [Fe/H] are chosen. (In practice I did this within CASJobs and then published the resulting catalogue and downloaded it as a FITS file)

125 A small example Question: How does the metal content of stars vary as a function of Galactic latitude? Step 4: Read in data & carry out post-processing to visualize the results. Convert Ra, Dec to l, b Plot [Fe/H] as a function of l, b and b alone.

126 A small example Question: How does the metal content of stars vary as a function of Galactic latitude? Step 4: Read in data & carry out post-processing to visualize the results. Convert Ra, Dec to l, b Plot [Fe/H] as a function of l, b and b alone.

127 Joins - Combining Tables Table A (e.g. PhotoAll) ID1 u g... Table B (e.g. SpecObjAll) ID1 ID2 Hα z Normal language: Get z from Table B and g from Table A for the same object. SQL: Need a quantity that is common between two tables. Here ID1. Then create a join - where A

128 Joins - Combining Tables Table A (e.g. PhotoAll) ID1 u Table B (e.g. SpecObjAll) ID1 ID2 SELECT A.u, A.g, B.z FROM PhotoAll as A, SpecObjAll as B WHERE A.ID1=B.ID1 g... Hα z Notice that we here use the as construct to give tables a nickname. Note also that this will only get objects that are in both A & B! If you need to combine tables including all elements in A and B you need to do an OUTER JOIN.

129 Joins - Combining Tables Table A (e.g. PhotoAll) ID1 u g... Table B (e.g. SpecObjAll) ID1 ID2 Hα z SELECT A.u, A.g, B.z FROM PhotoAll as A LEFT OUTER JOIN SpecObjAll as B ON A.ID1=B.ID1 The most common left outer join is between photometry and spectroscopy - for this reason there is a view called SpecPhoto which contains this join pre-computed. Use this when you want both spectroscopic info and photometry.

130 Looking at an SDSS Join Find galaxies with EW(Ha) > 40Å SELECT G.ObjID -- we want the photometric ObjID FROM Galaxy as G, SpecObj as S, SpecLine as L WHERE G.ObjID = S.bestObjID -- the spectroscopic object should -- be (photometrically) a galaxy and S.SpecObjID = L.SpecObjID -- and spectral line L is detected in spectrum and L.LineId = 6565 and L.ew > and line L is the H alpha line -- and H alpha is at least angstroms wide

131 Looking at an SDSS Outer Join Find stars with & without spectra in a range: SELECT TOP 20 S.ra, S.dec, S.objID, S.specObjID FROM Star as S LEFT OUTER JOIN SpecObj as Sp ON S.ObjID = Sp.bestObjID WHERE S.ra > 180 AND S.ra < 181 AND abs(s.dec) < 1

132 And a more complex one select fld.run, fld.avg_sky_mujy, fld.runarea as area, isnull(fp.nfirstmatch,0) from ( select run, sum(stripearea) as runarea, 3631e6*avg(power(cast(10. as float),-0.4*sky_r)) as avg_sky_mujy from field group by run ) as fld left outer join ( select p.run, sum(fm.match) as nfirstmatch from First as fm inner join photoprimary as p on p.objid=fm.objid group by p.run ) as fp on fld.run=fp.run order by fld.run

133 Other ways of getting data

134 Flat-Files The SDSS data are kept in a set of FITS files stored at Fermilab - in the DAS. This the reference set of data and is a superset of the CAS. Advantages: Can be analysed off-line using your own code All data available You will be able to measure quantities that have not been measured by the SDSS pipelines. Provided in a manner that matches the observing strategy Disadvantages: Contains often a lot more than what you want Requires some understanding of the survey observing strategy Requires a lot of disk space to store in full - searching the data subsequently requires significant effort.

135 When do you need them? Flat-Files You want to analyse everything You need/prefer to work off-line SQL makes the hair on your back stand up You need data that is not in the database (e.g. images, spectra) You have a specialized analysis pipeline you prefer to use. e.g: Spectroscopic analysis pipeline with better continuum subtraction (Tremonti et al 2004; Brinchmann et al 2004) Improved/specialised image analysis - improved treatment of large galaxies for instance.

136 Flat-Files How do you get them? The Data Archive Server (DAS) You can either browse the file system or create a file containing the necessary information to identify your required files and then upload this to the DAS. Some time later you will be told where to get your data.

137 Flat-Files How do you get them? The Data Archive Server (DAS) What can you get? fpc - calibrated image frames tsobj - One file per fpc with all detected objects spplate - 2D spectra for one plate with 640 fibres spspec - 1D spectra for a plate with measurements psfield - Extinction, PSFs etc. +++ Many more +++

138 Going Further

139 VO access Many key players in SDSS are involved in the Virtual Observatory so this can be used to access the data. Direct access to images in Aladin. Sky search services like OpenSkyQuery ( Spectrum services (Hungary, Baltimore) A number of web services are currently available. The JHU developer website is a good start: skyservice.pha.jhu.edu/develop/vo/

140 Some Hints CASJobs allows the creation of your own tables. This allows complex queries to be made. You need to know how to create tables. You might also need to use Procedures. You can then create a table and loop over this to put final results in a CASJobs table. Google Sky allows you to create KML files to overlay your own information on the sky. Can be useful for matching. And you can write your own scripts to interact with the SDSS - personally I often run these via the casjobs.jar Java library provided by JHU.