C.A.L. Bailer-Jones. Mon. Not. R. Astron. Soc. 000, (0000) Printed 22 December 2009 (MN LATEX style file v2.2)

Transcription

1 Mn. Nt. R. Astrn. Sc. 000, (0000) Printed 22 December 2009 (MN LATEX style file v2.2) The ILIUM frward mdelling algrithm fr multivariate parameter estimatin and its applicatin t derive stellar parameters frm Gaia spectrphtmetry C.A.L. Bailer-Jnes Max-Planck-Institut für Astrnmie, Königstuhl 17, Heidelberg, Germany Accepted 2009 Nvember 26. Received 2009 Nvember 26; in riginal frm 2009 Octber 19 ABSTRACT I intrduce an algrithm fr estimating parameters frm multidimensinal data based n frward mdelling. It perfrms an iterative lcal search t effectively achieve a nnlinear interplatin f a template grid. In cntrast t many machine learning appraches it avids fitting an inverse mdel and the prblems assciated with this. The algrithm makes explicit use f the sensitivities f the data t the parameters, with the gal f better treating parameters which nly have a weak impact n the data. The frward mdelling apprach prvides uncertainty (full cvariance) estimates in the predicted parameters as well as a gdness-f-fit fr bservatins, thus prviding a simple means f identifying utliers. I demnstrate the algrithm, ILIUM, with the estimatin f stellar astrphysical parameters (APs) frm simulatins f the lw reslutin spectrphtmetry t be btained by Gaia. The AP accuracy is cmpetitive with that btained by a supprt vectr machine. Fr zer extinctin stars cvering a wide range f metallicity, surface gravity and temperature, ILIUM can estimate T eff t an accuracy f 0.3% at G=15 and t 4% fr (lwer signal-t-nise rati) spectra at G=20, the Gaia limiting magnitude (mean abslute errrs are quted). [Fe/H] and lg g can be estimated t accuracies f dex fr stars with G 18.5, depending n the magnitude and what prirs we can place n the APs. If extinctin varies a priri ver a wide range (0 10 mag) which will be the case with Gaia because it is an all sky survey then lg g and [Fe/H] can still be estimated t 0.3 and 0.5 dex respectively at G=15, but much prer at G=18.5. T eff and A V can be estimated quite accurately (3 4% and mag respectively at G=15), but there is a strng and ubiquitus degeneracy in these parameters which limits ur ability t estimate either accurately at faint magnitudes. Using the frward mdel we can map these degeneracies (in advance), and thus prvide a cmplete prbability distributin ver slutins. Additinal infrmatin frm the Gaia parallaxes, ther surveys r suitable prirs shuld help reduce these degeneracies. Key wrds: surveys methds: data analysis, statistical techniques: spectrscpic stars: fundamental parameters ISM: extinctin 1 INTRODUCTION Inferring parameters frm multidimensinal data is a cmmn task in astrnmy, whether this be inference f csmlgical parameters frm CMB experiments, phtmetric-redshifts f galaxies r physical prperites frm stellar spectra. Imprtant questins abut the structure and evlutin f stars and stellar ppulatins require knwledge f abundances and ages, which must be btained spectrscpically via the stellar atmspheric parameters effective temperature (T eff ), surface gravity (lg g) and irn-peak metallicity ([Fe/H]). calj@mpia.de Numerus publicatins have presented methds fr estimating such astrphysical parameters (APs) frm spectra. The methds can be divided int tw brad categries. The first, based n high reslutin and high signal-t-nise rati (SNR) spectra, makes use f specific line indices selected t be sensitive predminantly t the phenmena f interest. Examples include the detectin f BHB stars via Calcium and Balmer lines (e.g. Brwn 2003) and spectral type classificatin f M, L and T dwarfs via mlecular band indices (e.g. Hawley et al. 2002). In each case these methds use a specific, identified phenmenn and sare generally limited t a narrw part f the AP space where the values f the ther APs are reasnably well knwn. While simple and useful, these methds d nt use all available infrmatin nr can they nrmally be used with lw resc 0000 RAS Cntent is c C.A.L. Bailer-Jnes

2 2 C.A.L. Bailer-Jnes lutin data, because the effects f individual APs cannt then be separated. The secnd categry f methds is glbal pattern recgnitin, which try t use the full set f available data. These are generally used when we want t estimate ne r mre APs ver a wide range f parameter space. In such cases there is rarely a simple relatin between a single feature and the physical quantity f interest. (An example is determinatin f surface gravity, where the relevant lines t use change with temperature.) We must therefre infer which features are relevant t which APs in which parts f the AP space. As this is generally a nnlinear, multidimensinal prblem, the standard apprach is t use a machine learning algrithm t learn the mapping frm the data space t the AP space based n labelled template spectra (spectra with knwn APs). Varius mdels have been used in astrnmy fr a variety f prblems, including (t name just a few): neural netwrks fr stellar parameter estimatin (e.g. Re Firentin et al. 2007) r phtmetric redshift estimatin (e.g. Firth et al. 2003); supprt vectr machines fr quasar classificatin (e.g. Ga et al. 2003, Bailer-Jnes et al. 2008) r galaxy mrphlgy classificatin (e.g. Huertas- Cmpany et al. 2008); classificatin trees fr identifying csmic rays in images (e.g. Salzberg et al. 1995); linear basis functin prjectin methds (e.g. the methd f Reci-Blanc et al fr spectral parameter estimatin, which cnstructs the basis functins frm mdel spectra). Mre examples f machine learning methds and their use in astrnmy can be fund in the vlume edited by Bailer-Jnes (2008). Nte that the first categry f methds, line indices, is really just a special case f the secnd in which drastic feature selectin has taken place t enable use f lw (ne r tw) dimensinal mdels. In bth cases we must learn sme relatinship AP = g (Data) frm a mdel r based n sme labelled templates. But this is an inverse relatin: mre than ne set f APs may fit a given set f data (e.g. a lw extinctin cl star r high extinctin ht star culd prduce the same clurs). Despite this nn-uniqueness we nnetheless try and fit a unique mdel. This causes fitting prblems which becme mre severe the lwer the quality f the data (the lwer the number f independent measures) and the larger the number f APs we want t estimate frm it, and culd lead t pr AP estimates r biases. In cntrast, the frward mapping (r generative mdel), Data = g(ap) is unique, because this is a causal, physical mdel (e.g. a stellar atmsphere and radiative transfer mdel f a spectrum). A further issue is that the mdel must learn the sensitivity f each input t each AP (and hw nise affects this). Yet this infrmatin we in principle have already frm the gradients f the generative mdel. A pattern recgnitin methd which tries t vercme sme f these prblems is k-nearest neighburs, which has als been applied t many prblems in astrnmy (e.g. Katz et al. 1998, Ball et al. 2008; plus extensins theref such as kernel density estimatin, e.g. Richards et al. 2004, r the methd f Shkedy et al. 2007, which cnverts the distances int likelihds and uses prirs t create a full prbabilistic slutin.) In many ways this is the mst natural way t slve the prblem: we create a grid f labelled templates and find which are clsest t ur bservatin (perhaps smthing ver several neighburs). On the assumptin that the APs vary smthly with the data between the grid pints, this may prvide a gd estimate. But fr this t be accurate (and nt t biased), the grid must be sufficiently dense that multiple grid pints lie within the errr ellipse f the bservatin (the cvariance f these neighburs then prvides a measure f the uncertainty in the estimated APs). If the SNR is high, the grid must therefre be very dense. Mrever, as the number f APs increases, the required grid density grws expnentially with it: Fr a stellar parametrizatin prblem with 5 APs we might need an average f 100 samples per AP, resulting in = templates. There is als the issue f what distance metric t use. The cvariance-weighted Mahalanbis distance is ften used, but it ignres the sensitivities f the inputs. (If sme inputs are smetimes dminated by irrelevant csmic scatter, this will add unmdelled nise t the distance estimate.) This is a prblem when we have a mix f APs, sme f which have a large and thers a small impact n the variance ( strng and weak APs). If we just use the Mahalanbis distance we lse sensitivity t the weak APs. We culd vercme these prblems if we did n-the-fly interplatin f the template grid t generate new templates as we need them. Running stellar mdels is far t time cnsuming fr this, but als unnecessary because the generative mdel is smth: We can instead fit a frward mdel t a lw density grid f templates as an apprximatin t the generative mdel. As we shall see, pssessin f a frward mdel pens up pprtunities nt available t the inverse methds, such as direct uncertainty estimates and gdness-f-fit assessment f the slutin. In this article I intrduce an algrithm fr AP estimatin based n this frward mdelling idea and iterative interplatin. I will demnstrate it using simulatins f lw reslutin spectra t be btained frm the Gaia missin (e.g. Lindegren et al. 2008). 1 Gaia will bserve mre than 10 9 stars dwn t 20 th magnitude ver the whle Galaxy, stars which a priri span a very wide range in several APs. This includes the line-f-sight extinctin parameter, A V, which must be estimated accurately if we want t derive intrinsic stellar luminsities frm the Gaia parallaxes. AP estimatin (Bailer-Jnes 2005) is therefre an integral part f the verall Gaia data prcessing and cmprises ne f the Crdinatin Units in the Gaia Data Prcessing and Analysis Cnsrtium (DPAC) (Mignard & Drimmel 2007, O Mullane et al. 2007). I will nw describe the basic algrithm (sectin 2). In sectin 3 I then intrduce the simulated Gaia spectrscpy t which ILIUM is applied, with the results and discussin theref presented in sectins 4, 5 and 6. The latter sectin als reprts n a strng and ubiquitus degeneracy between T eff and A V. I summarize and cnclude in sectin 8. Additinal plts, results and discussins can be fund in a series f fur Gaia technical ntes (Bailer-Jnes 2009a,b,c,d) available frm 2 THE ILIUM ALGORITHM I utline the algrithm using the terminlgy f spectra and stellar astrphysical parameters, althugh it is quite general and applies t any multivariate data. Table 1 summarizes the ntatin. Band refers t a flux measurement in the spectrum. In general it culd be a phtmetric band, a single pixel in the spectrum, r a functin f many pixels. 2.1 Frward mdelling I will call the true relatinship between the APs and the flux in a band i the generative mdel, g i(φ). This prvides the bserved 1 c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

3 Parameter estimatin with ILIUM 3 Table 1. Ntatin flux, p I i p i J j φ j s ij S f i (φ) n p1 p0 p2 number f bands (pixels in spectrum) cunter ver band, i = 1... I phtn cunts in band i (p is a spectrum) number f APs (astrphysical parameters) cunter ver AP, j = 1... J AP j (φ is a set f APs) sensitivity f band i t AP j, p i φ j sensitivity matrix, I J matrix with elements s ij frward mdel fr band i iteratin; e.g. φ(n) is the AP at iteratin n δ p φ2 δφ true φ φ1 frward mdel measurement generative functin = templates AP, φ Figure 1. Sketch f the search methd fr ne band (I=1) and ne AP (J=1). The dashed blue curve is the (unknwn) generative mdel, and the slid blue curve is the frward mdel (ur apprximatin t the generative mdel) frmed by fitting a functin t the templates. (The difference between the tw is exaggerated.) The straight red line shws the lcal linear apprximatin f the frward mdel (tangent at φ 1 ) used t calculate the first AP step. (In the case shw the frward mdel culd be inverted. But this is nt generally the case, nt even in ne dimensin if it has a turning pint.) spectrum fr a given set f APs and thus encapsulates the underlying stellar mdel, radiative transfer, interstellar extinctin, instrument mdel etc. (There is a separate functin fr each band, but fr simplicity I will refer t it in the singular.) Generally we dn t have an explicit functin fr this mdel, s it remains unknwn. All we have fr ding AP estimatin is a discrete template grid f example spectra with knwn APs generated by the generative mdel. We apprximate the generative mdel using a frward mdel, f i(φ), which is a (nnlinear) parametrized fit t this grid and prvides flux estimates at arbitrary APs, i.e. ff the grid. (Frward mdels can be fit independently fr each band.) Demanding the frward mdels t be cntinuus functins ensures we can als use them t calculate the sensitivities, which by definitin are the gradients f the flux with respect t each AP. The frward mdel fitting is dne just nce fr a given grid and is kept fixed when predicting APs. In ther wrds it is a training prcedure. 2.2 Cre algrithm The basic idea f ILIUM is t use the Newtn-Raphsn methd t find that frward mdel-predicted spectrum (and assciated APs) which best fits the berved spectrum. In detail, the algrithm is as fllws (Fig. 1). Cnsider first a single AP and single band. The measurement is p(0) and we want t estimate its AP. The frward mdel, ˆp = f(φ), has been fit and remains fixed. The prcedure is as fllws (n is the iteratin number) (i) Initialize: find nearest grid neighbur t p(0), i.e. the ne which minimizes the sum-f-squares residual δp T δp. Call this [p(1), φ(1)]. φ(1) is the initial AP estimate. (ii) Use the frward mdel t calculate the lcal sensitivities, p φ, at the current AP estimate. (iii) Calculate the discrepancy (residual) between the predicted flux and the measured flux, δp(n) = p(n) p(0). (iv) Estimate the AP ffset as δφ(n) = δp(n), φ p φ(n) i.e. a Taylr expansin truncated t the linear term. (Nte that this partial derivative is the reciprcal sensitivity.) (v) Make a step in AP space, φ(n1) = φ(n) δφ(n), tward the better estimate. This is the new AP predictin. (vi) Use the frward mdel t predict the crrespnding (ffgrid) flux, p(n 1) (vii) Iterate steps ii vi until cnvergence is achieved r a stp is impsed. The algrithm is basically minimizing δp. At each iteratin we btain an estimate f the APs (step v) and the crrespnding spectrum (step vi). Cnvergence culd be defined in several ways, e.g. when changes in the spectrum r the APs (r their rate f change) drp belw sme threshld. Alternatively we culd simply stp after sme fixed number f iteratins. There is n guarantee f cnvergence. Fr example, if the AP steps were sufficiently large t mve t a part f the functin with a sensitivity f the ppsite sign, then the mdel culd diverge r get stuck in a limit cycle. Likewise, if initialized t far frm the true slutin the algrithm culd becme stuck in a lcal minimum far frm the true slutin. Fr this, and ther reasns, the algrithm in practice has sme additinal features (discussed in sectin 2.5). Als, 2.3 Generalizatin t multiple APs and bands In general we have several bands and several APs. The flux perturbatin due t small changes in the APs is then δp = S δφ (1) where S is the I J sensitivity matrix with elements s ij = p i/ φ j. Nte that I > J. Multiplying this equatin n the left by (S T S) 1 S T gives δφ = (S T S) 1 S T δp (2) s the AP update equatin (step v in the algrithm) becmes φ(n 1) = φ(n) (S T S) 1 S T δp(n) (3) The I frward mdels are nw functins f J variables, and this turns ut t be a critical matter. 2.4 The frward mdel The cre algrithm just described can make use f any frm fr the frward mdel, n the cnditin that it prvides values f the functin and its first derivatives fr arbitrary values f the APs. The mst bvius frward mdel wuld be a multidimensinal, nnlinear regressin f the frm ˆp = f(φ), which in principle wrks fr any number f APs. Hwever, I fund that it was difficult t get a mdel which simultaneusly fits bth T eff, a strng AP, and lg g, a weak AP t sufficient accuracy. Strng here means c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

4 4 C.A.L. Bailer-Jnes flux strng AP, k weak AP, l Figure 2. Schematic diagram f a tw-cmpnent frward mdel. Nte the strnger variatin in the flux in the directin f the strng AP (the cntrast is typically much larger fr the prblems cnsidered in this article). strng AP, k weak AP, l Figure 3. Schematic diagram f the grid f AP values, k = , l = The slid (red) pints dente thse used t fit the weak cmpnent f the frward mdel fr ne value f the strng AP. that it explains much f the variance in the flux data, i.e. is a strng predictr f the flux. Weak is a relative term, indicating that the AP explains much less f the variance. The reasn fr pr fits ver the weak APs is that the mdel is fit by minimizing a single bjective functin, namely the errr in reprducing the flux. As the weak AP has very little impact n the flux, its influence has little impact n the errr, s the mdel ptimizatin des little t prduce a gd fit ver this AP. T vercme this prblem I use a tw-cmpnent frward mdel t separately fit the strng and weak APs. Cnsider the case f a single strng AP and a single weak AP (this is generalized later). The strng cmpnent is a 1D nnlinear functin f the strng AP which is fit by marginalizing ver the weak AP. This fits mst f the flux variatin. Then, at each discrete value f the strng AP in the grid, we fit the residual flux as a functin f the weak AP; these are the secnd cmpnents (als 1D). They prvide a flux increment dependent n the weak AP, which is added t the flux predicted given the strng AP. A schematic illustratin f such a tw-cmpnent frward mdel is illustrated in Fig flux flux residual abut strng cmpnent strng AP, k weak AP, l Figure 4. Schematic illustratin f the tw cmpnents f the frward mdel fit ver all the strng and weak pints in the grid in Fig. 3 Tp: the fit ver the strng APs. Bttm: ne f the fits ver the weak APs (the slid/red pints in the tp panel at k=14). T be mre precise the mdel is fit as fllws. Let φ S dente a strng AP, φ W a weak AP and f i(φ S, φ W ) the cmplete (2D) frward mdel fr band i. Let subscripts k and l dente specific values in the grid f the strng and weak APs respectively (see Fig. 3). We mdel the flux at an arbitrary AP pint as f i(φ S, φ W ) = f S i (φ S ) f W i,k(φ W ; φ S = φ S k ) (4) where bth fi S and fi,k W are 1D functins. fi S (φ S ) is the single strng cmpnent (fr band i). It is a fit t the average value f φ W at each φ S (the curve in the tp panel f Fig. 4). fi,k W is the k th weak cmpnent, which is a fit with respect t the weak AP with the strng AP fixed at φ S k (e.g. the slid/red pints in Fig. 3), i.e. it is fit t the residuals { p(φ S = φ S k ) p(φ S = φ S k ) } as illustrated in in the bttm panel f Fig. 4. This fitting apprach clearly requires us t have a semiregular grid: ne which has a range f values f the weak AP fr each value f the strng AP. (This requirement is easily fulfilled when using synthetic grids.) The number f weak cmpnents in the mdel is equal t the number f unique values f φ S, 20 in this schematic case. If we have mre than ne strng r weak cmpnent then we raise the dimensinality f the strng r weak cmpnent (see sectin 6). Applying the frward mdel is easy. Given (φ S, φ W ) we (i) evaluate f S i, the strng cmpnent; (ii) find the nearest value, φ S k, in the grid t φ S, i.e. identify the clsest weak cmpnent; (iii) evaluate f W i,k, the increment frm the weak cmpnent; c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

5 Parameter estimatin with ILIUM 5 (iv) sum the tw cmpnents, fi S mdel predictin. f W i,k, t give the frward Althugh the weak cmpnent we use changes discntinuusly as φ S varies, the weak cmpnent is nly specifying an increment t the strng cmpnent fit. As bth cmpnents are smth in their respective APs the cmbined functin is als smth alng any directin parallel t the AP axes. It is nt smth alng arbitrary directins, but this is unimprtant because explicit calculatins with the frward mdel (e.g. f the sensitivities) are nly carried ut parallel t the AP axes. 2.5 Practical algrithm The practical realities f wrking with real (nisy) data mean that the basic algrithm shuld be extended in rder t make it mre rbust. These I nw describe tgether with ther implementatin aspects used fr the experiments described later. Fr the purpses f this paper the algrithm has been implemented in R 2. A Java implementatin is in prgress, which will be necessary fr larger scale applicatins. data with a similar nise level as the target data (e.g. Snider et al. 2001).This precipitates the need fr multiple mdels when used n survey data with a range f SNRs, smething which is nt an issue fr ILIUM Sensitivity estimatin If the frward mdel is a simple analytic functin then it may have analytical first derivatives which can be used t calculate the sensitivities. But in the general case we can use the methd f first differences p φ j φ f(φ δφj) f(φ δφj) 2 δφ j (5) I select δφ j t be slightly smaller than the maximum precisin a priri pssible in an AP. As the frward mdel must be smth at this reslutin, the first difference apprximatin is sufficiently accurate. Fr the examples shwn later, I chse δφ j t be 0.05 dex fr lg g and [Fe/H], fr lg(t eff ) (0.1% fr T eff ) and 0.03 mag fr A V Standardized variables The spectral variables are bserved phtn cunts (t within an irrelevant cnstant factr). The APs are all n lgarithmic scales: A V in magnitudes, lg g and [Fe/H] in dex, and lg(t eff ). In rder t bring each variable t the same level I standardize the flux in each band and each AP (linearly scale each t have zer mean and unit variance). If there were crrelatins in the spectra these culd be remved by sphereing ( prewhitening ), a cvariate generalizatin f standardizatin which gives the data unit diagnal cvariance (e.g. Bishp 2006) Lwer limit n sensitivities (singularity avidance) Given that the AP updates depend upn the inverse f the sensitivity matrix (equatin 2), it is prudent t prevent the sensitivities being t small in rder t avid S T S being singular. Fr this reasn, a lwer limit is placed n the abslute value f each sensitivity, s ij, f (with p and φ in standardized units). Fr the examples shwn later, this limit rarely had t be applied in practice and had negligible impact n the results. T avid singularity S T S must als have a rank f at least J, i.e. t estimate J APs we need at least J independent measures in the spectrum Frward mdel functins The strng and weak cmpnents f the frward mdel are fit using smthing splines (e.g. Hastie et al. 2001). Cnventinal cubic splines have the drawback that ne must cntrl their cmplexity (smthness) using the number and psitin f the knts. Smthing splines circumvent this prblem by setting a knt at every pint (which wuld verfit the data) and then applying a smthing penalty which is cntrlled by specifying the effective degrees-f-freedm (df). I set this by trial and errr via inspectin f the resulting fits. Fr the 2D prblems TG (T eff and lg g) and TM (T eff and [Fe/H]) described in sectins 4 and 5, bth the strng and the weak mdel splines are 1D. The strng mdel uses df = n Teff /2 = 16.5 where n Teff is the number f unique T eff pints. As the maximum number f lg g pints is 10 (fr the training data), and because the variatin with lg g is quite smth, I set the df fr these fits t be 4. Hwever, many f the T eff values in the training grid have fewer lg g r [Fe/H] pints: T avid verfitting, if n lgg (T eff ) 4 then a linear fit is used. If n lgg (T eff ) = 1, then n fit is perfrmed and this weak cmpnent f the frward mdel is zer. (Practical aspects f higher dimensinal frward mdels are described in sectin 6.) The frward mdel is always fit t nise-free data. It is well knwn (and the authr s experience) that inverse mdelling methds such as ANNs and SVMs perfrm best when trained n AP update cntributin clipping Equatin 2 can be written φ(n 1) = φ(n) M δp(n) (6) where M = (S T S) 1 S T (7) is a J I matrix. Equatin 6 gives J update equatins, ne fr each AP. The update fr AP j can be written as the dt prduct f tw vectrs, the j th rw f M, m j with p(n), i.e. φ j(n 1) = φ j(n) m j δp(n) = φ j(n) X i = φ j(n) X i m ijδp i(n) u ij(n) (8) which defines u ij. Thus we see that the update t AP j is a sum ver I individual updates, which we can view as an update spectrum. If we inspect these updates (see Bailer-Jnes2009a) we see that n ccasin sme are much larger than the thers. This dminance f the update by just ne f a few elements is undesirable, because they may be affected by nise (δp(n) is a nisy measurement). Fr this reasn, I clip utliers in this spectrum. (It is valid t cmpare the updates fr different bands, because we wrk with standardized fluxes.) T be rbust, I set an upper (lwer) limit which is a multiple c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

6 6 C.A.L. Bailer-Jnes c f the median f thse pints abve (belw) the median. Using the ntatin θ() t dente median, the limits are u upper = θ(u i) c[θ(u i > θ(u i)) θ(u i)] u lwer = θ(u i) c[θ(u i < θ(u i)) θ(u i)] (9) I smewhat arbitrarily set c = 10 s as t be relatively cnservative in clipping Upper limit n AP update step size The AP steps at each iteratin (equatin 2) culd be very large. This is undesirable, because the updates are based n a lcal linear apprximatin t the generative mdel. The cde therefre impses upper limits n the AP updates, crrespnding t steps n larger than abut 2.0 dex in lg g and [Fe/H], 0.04 in lg(t eff ) (10% in T eff ) and 0.3 mag in A V. A larger step size is permitted fr the weaker APs because the initial nearest neighbur ffset can be quite incrrect. These limits are impsed mre ften fr nisy data, but still relatively rarely Limit AP extraplatin We d nt expect the frward mdel t make gd predictins beynd the AP extremes f the grid, s I set upper and lwer limits n the AP estimates which ILIUM can prvide. These are set as e times the range f each AP, i.e. upper limit = max φ j e(max φ j min φ j) lwer limit = min φ j e(max φ j min φ j) (10) I set e = Stpping criterin The algrithm is simply run fr a fixed number f iteratins (20). We ften bserve gd natural cnvergence, s a mre sphisticated stpping criterin is nt applied at this time, althugh it may be imprtant n high variance data sets (lw SNR r mre APs). as a functin f the sensitivity (calculated at the estimated APs) and the cvariance in the measured phtmetry, C p. (This equatin assumes that ILIUM prvides unbiased AP estimates and that the sensitivities have zer cvariance. It can als be written C φ = MC pm T where M is the update matrix intrduced in equatin 7.) C p can be estimated frm a phtmetric errr mdel, and will be diagnal if the phtmetric errrs in the bands are independent. Even in this case C φ is generally nn-diagnal: the AP estimates are crrelated n accunt f the sensitivities. Because we have a frward mdel we can calculate a gdness-f-fit (GF) fr any estimate f the APs. Here I simply use the reduced-χ 2 t measure the difference between the bserved spectrum and predicted spectrum 3 GF = 1 Xi=I «2 pi ˆp i (12) I 1 i=1 σ pi where ˆp i = f i(φ j ) is the frward mdel predictin and {σ 2 p i } = diag(c p) is the expected phtmetric nise. (Despite the name, a larger value refers t a prer fit) As the GF can be measured withut knwing the true APs, it can be used fr detectin f pr slutins r utliers. Cnventinal methds f AP estimatin via direct inverse mdelling (e.g. with SVMs r ANNs) d nt naturally prvide uncertainty estimates and must usually resrt t time-intensive sampling methds, such as resampling the measured spectrum accrding t its estimated cvariance. They cannt prvide a GF at all because they lack a frward mdel. 2.8 Signal-t-nise weighted AP updates The update equatin (2) nly takes int accunt the sensitivity f the bands, nt their SNR. Hwever, even if a band is very sensitive t an AP in principle, if its measurement is very nisy then it is less useful. We culd accmmdate this by including a factr prprtinal t C 1 p int equatin 2 which wuld dwn-weight nisier measurements. Preliminary results using this n the TG prblem (see sectin 3.2) shw it actually degrades perfrmance at G=15, but gives sme imprvement at G=18.5 (Bailer-Jnes 2009c). 2.6 Perfrmance statistics The mdel perfrmance is assessed via the AP residuals (estimated minus true, δφ) n an evaluatin data set. I reprt three statistics : (1) the rt-mean-square (RMS) errr, which I abbreviate with ɛ rms; (2) the mean abslute errr, δφ, abbreviated as ɛ mae; (3) the mean residual, δφ, a measure f the systematic errr, abbreviated as ɛ sys. (Nte that as (1) and (2) are statistics with respect t the true values they als include any systematic errrs.) I mstly use ɛ mae rather than RMS because the frmer is mre rbust. If the residuals had a Gaussian distributin then the RMS wuld equal the Gaussian 1σ which is p π/2 = 1.25 times larger than ɛ mae. But usually there are utliers which increase the RMS significantly beynd this. 2.7 Uncertainty estimates If vectrs y and x are related by a transfrmatin y = Ax then a standard result f matrix algebra is that the cvariance f y is C y = AC xa T where C x is the cvariance f x. Applying this t equatin 2 gives us an expressin fr the cvariance in the APs C φ = (S T S) 1 S T C ps(s T S) 1 (11) 3 ASSESSING THE ALGORITHM 3.1 Gaia simulatins T illustrate ILIUM I apply it t estimate stellar APs frm simulated Gaia stellar spectra and thereby als make preliminary predictins f the missin perfrmance. Gaia will bserve all f its targets with tw lw-dispersin slitless prism spectrgraphs, tgether cvering the wavelength range frm nm. (These are creatively called BP fr blue phtmeter and RP fr red phtmeter.) The dispersin varies frm 3 nm/pixel at the blue end t 30 nm/pixel at the red end (Brwn 2006). The blue and the red spectra are each sampled with 60 pixels, but as the line-spreadfunctin f the spectrgraph is much brader, these samples are nt independent. After remving lw SNR regins f the mdelled spectra, I retain 34 pixels in BP cvering nm and 34 pixels in RP cvering nm. This is a slightly narrwer range (and 18 pixels fewer) than the ne adpted by Bailer-Jnes et al. (2008) fr quasar classificatin with similar spectra. 3 I use I 1 degrees f freedm rather than I because all the spectra have a cmmn G magnitude, s the bands are nt all strictly independent. c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

7 Parameter estimatin with ILIUM 7 lgg / dex Teff / K Figure 5. The T eff lg g grid f the data used in the experiments. Extensive libraries f BP/RP spectra have been simulated by the Gaia DPAC using the GOG (Gaia Object Generatr; Luri et al. (2005), Isasi 2009) instrument mdel and libraries f input spectra. 4 Here I use the Basel (Lejeune et al. 1997) and Marcs (e.g. Gustafssn et al. 2008) stellar libraries. The frmer (as used here) includes 17 T eff values frm K with nn-unifrm spacing and the latter 17 T eff values frm K in unifrm steps f 250 K. (Tgether there are 33 unique T eff values because 8000 K is in bth.) Tgether the grids span lg g values frm 0.5 t 5.0 dex in steps f 0.5 dex, althugh the grid is incmplete fr astrphysical reasns (Fig. 5). [Fe/H] ranges frm 4.0 dex t 1.0 dex with 13 discrete values fr the cler stars (with T eff 8000 K): The spacing is 0.25 dex frm 1 t 1, fllwed by pints at 1.5, 2.0, 3.0 and 4.0. Nt all [Fe/H] are present at all T eff lg g cmbinatins. Each star has been simulated at ne f ten values f the interstellar extinctin, A V {0, 0.1, 0.5, 1, 2, 3, 4, 5, 8, 10} with R V = 3.1. (Nte that A V is the extinctin parameter defined by Cardelli et al It is nt the extinctin in the V band.) The Marcs library additinally shws variatin in the alpha element abundances, [α/fe], representing five values frm dex in 0.1 dex steps. Hence this cmbined library shws variance in five APs, T eff, A V, lg g [Fe/H], [α/fe], the first tw f which are strng and the latter three weak. The ttal number f spectra is ILIUM will be used t estimate the first fur APs; [α/fe] will be ignred and cntributes csmic scatter. GOG simulates the number f phtelectrns ( cunts ) in the spectral bands. (While these will be calibrated in physical flux units befre being published t the cmmunity, the classificatin wrk by DPAC is currently dne in phtelectrn space.) The variance in the spectra due t the fur APs f interest is demnstrated in the example spectra pltted in Figs. 6, 7, 8 and 9. The first plt shws the variance due t T eff nly. The break between the BP and RP instruments arund 660 nm is clear, as is the highly variable dispersin. The lwer cunts in RP immediately t the right f the break cmpared t BP is primarily due t the higher dispersin (fewer phtns per band). In the ther plts tw APs are varied while the ther tw are held cnstant. Fig. 7 demnstrates the strng impact f bth T eff and A V variatins. (As I will discuss in sectin 6.4, 4 Fr Gaia pundits: I use the CU8 cycle 3 simulatins f the nminal (discrete) libraries (Srd & Vallenari 2008) cunts wavelength / nm Figure 6. Nise-free Gaia spectra fr slar metallicity dwarfs at zer extinctin with T eff = {4000, 5000, 6000, 7000, 8500, 10000, 15000} K, increasing mntnically frm bttm (red) t tp (vilet) at lng wavelengths. They are cmpsed f tw spectra (BP and RP) t the left and right f abut 660 nm. The rdinate is in units f phtelectrns (t within sme cnstant multiple), nt energy flux. cunts Teff=15000K Teff=10000K Teff=8500K Teff=7000K Teff=6000K Teff=5000K Teff=4000K wavelength / nm Figure 7. Nise-free Gaia spectra at a range f T eff (as in Fig. 6) and A V (0.0, 0.1, 0.5, 1, 2, 3, 4, 5, 8, 10) ranging frm 0.0 mag (lwest line at lng wavelengths; in red) t 10.0 mag (highest line at lng wavelengths; in vilet). Each temperature blck has been ffset by 1200 cunts fr clarity (the zer levels are shwn by the dashed lines). lg g = 4.0 dex, [Fe/H]= 0.0 dex and [α/fe] = 0.0 dex in all cases. c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

8 8 C.A.L. Bailer-Jnes cunts Teff=15000K Teff=12500K Teff=9000K Teff=7000K Teff=5000K Teff=4000K cunts Teff=8500K Teff=7000K Teff=6000K Teff=5000K Teff=4500K Teff=4000K wavelength / nm wavelength / nm Figure 8. Nise-free Gaia spectra at a range f T eff and lg g ( 0.5, 0.5, 2, 3, 4, 5), with the lwest gravity (red) frming the lwest curve at the red end f the spectrum and the highest gravity (vilet) the highest. Nte that nt all gravities are present at all T eff due t the limitatins f reality. Each temperature blck has been ffset by 600 cunts fr clarity (the zer levels are shwn by the dashed lines) A V = 0.0 mag, [Fe/H]= 0.0 dex and [α/fe] = 0.0 dex in all cases. Figure 9. Nise-free Gaia spectra at a range f T eff and [Fe/H] ( 3, 2, 1, 0, 0.5), with the lwest metallicity (red) frming the lwest curve at the red end f the spectrum and the highest metallicity (vilet) the highest. Nte that nt all metallicities are present at all T eff due t limitatins f the simulated libraries. Each temperature blck has been ffset by 600 cunts fr clarity (the zer levels are shwn by the dashed lines) A V = 0.0 mag, lg g = 4.0 dex and [α/fe] = 0.0 dex in all cases. these tw APs are highly degenerate in these data.) The third and furth figures clearly demnstrate why lg g and [Fe/H] are weak parameters: they have little impact n the spectra cmpared t the variance due t T eff r A V. In particular, at high temperatures the spectra shw essentially n sensitivity t [Fe/H]. The AP estimatin accuracy is, f curse, a strng functin f the SNR in the spectrum. As Gaia has a fixed sky scanning law, the SNR depends n the surce magnitude and the number f bservatins (because the individual bservatins are cmbined int a single end-f-missin spectrum). I adpt here 72 bservatins fr all spectra (the mean number f bservatins per surce fr a 5-year missin), and reprt results as a functin f just the magnitude. The simulatr nise mdel takes int accunt the surce, backgrund and backgrund-subtractin phtn (Pissn) nise as well as the CCD readut nise. At present errrs due t the cmbinatin f spectra, charge-transfer inefficiency and CCD radiatin damage are nt explicitly included. There is instead a factr t accunt fr general prcessing and calibratin errrs. All f these nise terms are cmbined int a zer mean Gaussian mdel fr each pixel, the standard deviatin f which is a functin f the G-band apparent magnitude. (The G-band is the filterless band defined by the mirrr and CCD respnse in which the Gaia astrmetry is btained, cvering a range similar t BP/RP.) This defines a sigma spectrum fr each star frm which I generate randm numbers in rder t simulate nisy spectra at G=15, 18.5 and 20. The resulting SNR is a strng functin f wavelength and the specific surce, and is summarized in Fig Data sets and frward mdel fitting In the fllwing sectins I will apply Gaia t fur distinct prblems accrding t the APs we are trying t determine. In each case the frward mdel is fit t a grid varying in at least thse APs, and in sme cases the grid als shws variance in anther AP (which therefre acts t prvide additinal csmic scatter ). Fr each case the ILIUM frward mdel is fit (trained) using the full, nise-free data set ver the cmplete range f the represented APs. The crrespnding nisy data set (at G=15, 18.5 r 20) is split randmly int tw equal halves: ne half is used fr initializatin (selecting the nearest neighbur) and ILIUM is applied t the ther half (r a randm subset f it where indicated belw) n which the perfrc 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

9 Parameter estimatin with ILIUM 9 SNR per band wavelength / nm Figure 10. The median signal-t-nise rati (SNR) (slid line) and 0.1 and 0.9 quartiles (dashed lines) acrss the set f zer extinctin dwarf stars fr G=18.5 (red/thicker lines) and G=20.0 (blue/thinner lines). Cmpared t the G=18.5 curve, the SNR at G=15 is 8 6 times larger between 400 and 660 nm and 7 12 times larger between 660 and 1000 nm. mance is evaluated. 5 (The legitimacy f this prcedure fr evaluating perfrmance is discussed in appendix A.) In additin t reprting glbal results (ver the full AP ranges present in the training set) I als measure perfrmance n subsets f the evaluatin data set, in particular the [Fe/H] just fr cl stars (the ILIUM frward mdel is nt refit). The prblems and their grids are as fllws (the name indicates the APs being determined: Temperature, Gravity, Metallicity and/r Extinctin) TG: Estimatin f T eff lg g (1 strng and 1 weak AP), fr stars with A V = 0 and [Fe/H] = 0, sme 274 stars. I als build a secnd TG mdel (TG-allmet) which is trained and evaluated with the full [Fe/H] range in the grid ( 4 t 1 dex). This cntains 4361 stars, f which a quarter are used in the evaluatin set. TM: Estimatin f T eff [Fe/H] (1 strng and 1 weak AP), fr stars with A V = 0 and either lg g {4.0, 4.5, 5.0} (TM-dwarfs; 1716 stars), r lg g {1.0, 1.5, 2.0, 2.5, 3.0}, (TM-giants; 1882 stars). I als build a third mdel (TM-allgrav) which is trained and evaluated with the full lg g range in the grid ( 0.5 t 5 dex). This cntains 4361 stars (it s the same grid as used fr TG-allmet f curse), f which a quarter are used in the evaluatin set. TAG: Estimatin f (T eff, A V)lg g (2 strng and 1 weak AP), fr stars with [Fe/H] = 0. This has 2740 stars, f which a randm selectin f 1000 is used fr evaluatin. TAM: Estimatin f (T eff, A V)[Fe/H] (2 strng and 1 weak AP), fr dwarfs with lg g {4.0, 4.5, 5.0}. This has stars f which a randm selectin f 1000 is used fr evaluatin. TGM: Estimatin f T eff (lg g, [Fe/H]) (1 strng and 2 5 These target spectra must first be nrmalized t have the same cunts level as thse n which ILIUM was trained. Fr this I just use the G magnitude t scale the cunts. Hwever, as the G-band is nt identical t the BP/RP band adpted, this gives rise t a nrmalizatin ffset between spectra even fr a cmmn G magnitude. Fr example, ver the TG grid the integrated BP/RP cunts varies by up t 10%, dependent primarily n T eff. A better nrmalizatin might be area nrmalizatin, i.e. dividing the cunts in each band by the sum ver all bands fr that spectrum. My G-band nrmalizatin is in principle cnservative, as it mimics a small calibratin bias. weak APs, fr stars with A V = 0. This has 4361 stars f which a randm selectin f 1000 is used fr evaluatin. In appendix B the ILIUM results are cmpared with results frm an SVM n sme f these prblems. 4 APPLICATION TO THE T eff lg g PROBLEM (TG) First the frward mdel is fit fr each band t the T eff lg g grid shwn in Fig. 5. As a reminder, each frward mdel cmprises the 1D functin ver T eff (the strng cmpnent) and 33 1D functins in lg g (the weak cmpnents), as described in sectin 2.4. All f these are smthing splines (sectin 2.5.2). Figs. 11 and 12 shw the frward mdel fit fr 12 bands at cuts f cnstant lg g and T eff respectively. The fits are gd: they shw the degree f smthness we wuld expect fr these data plus a rbust extraplatin. If we cmpare the flux scales between Figs. 11 and 12 (these are standardized variables) we see hw small the flux variatin is as lg g varies ver its full range cmpared t T eff : this is what it means t be a weak AP. Nte als the small discntinuity in sme f the bands in Fig. 11 at 8000 K (lg(t eff ) = 3.903) where the Marcs and Basel libraries jin. Having trained ILIUM I apply it t the evaluatin data set at G=15. Fig. 13 shws five examples f hw the AP estimates evlve. The first iteratin is the nearest neighbur initializatin; the final is the adpted AP estimate. The red (hrizntal) lines shw the true APs. Lking at many examples we see a range f cnvergence behaviurs. Smetimes cnvergence is rapid, fr ther stars it takes lnger. Smetimes it is smth, ther times nt. It can be quite different fr the tw APs fr a given star and depends als n the specific spectrum (which is nisy). Smetimes the nearest neighbur estimate is the crrect ne, and ILIUM may actually iterate away frm this and cnverge n a different value. Cnvergence (n smething) is almst always achieved n this prblem, even thugh there is nthing adaptive in the algrithm. This is an encuraging prperty. Limit cycles are als seen in a handful f cases, but with negligible amplitudes n this prblem (high SNR). Fr this specific prblem, the extraplatin limits n the AP estimatins (sectin 2.5.7) and the limits n the AP updates at each step (sectin 2.5.6) never had t be enfrced by the algrithm. Fig. 14 plts the ILIUM residuals (estimated minus true APs) n the 137 stars in the evaluatin set; the statistics are summarized in line 1 f Table 2. We see that the APs can be estimated very accurately (n significant systematic errr, ɛ sys) and very precisely (lw scatter, ɛ mae r ɛ rms). Applying the same mdel t nisier spectra bviusly increases the errrs (lines 2 and 3 in the table), but at G=18.5 the mean abslute errrs are still an acceptable 1% in T eff and 0.35 dex in lgg. At G=20 lg g cannt be estimated accurately enugh t reliably distinguish dwarfs and giants, althugh T eff is still kay with an expected errr f 260 K at 6000 K, fr example. This evaluatin set is relatively small s the errr statistics are subject t variatin. Applying ILIUM t an ensemble f randmly selected evaluatin sets (at G=18.5), we see that the errr statistics vary by 5 10% (inter-quartile range) f their mean. The reprted values are therefre reasnably representative. The perfrmance f an SVM n these TG prblems is given in Table B1. Fr bth APs and all three magnitudes ILIUM is similar t r significantly better than SVM. (As there is sme variance in the results frm bth methds I nly cnsider the perfrmance significantly different if the better ne has a ɛ mae at least 25% smaller.) S in this limited variance prblem, at least, the frward mdelling apprach imprves perfrmance. c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

10 10 C.A.L. Bailer-Jnes Table 2. Summary statistics (defined in sectin 2.6) f the ILIUM perfrmance n varius experiments. The first clumn indicates the mdel and grid used (defined in sectin 3.2). The secnd clumn defines the magnitude and cmpsitin f the evaluatin data set: F means the full AP ranges (as present in the training set); L means nly stars with T eff 7000 K. < in the ɛ sys clumn indicates that the systematic errr has a magnitude less than twice the standard errr in the mean ( ɛ sys ɛ mae/ N), i.e. is statistically insignificant. (Quite a few nt s marked are nly marginally significant.) The units f the errr statistics are lgarithmic fr all variables (dex fr all except A V which is in magnitudes) and are n the prper variables (nt the standardized variables). Multiplying by 2.3 cnverts t fractinal errrs fr lg(t eff ). mdel evaluatin A V lg(t eff ) lg g [Fe/H] sample ɛ sys ɛ mae ɛ rms ɛ sys ɛ mae ɛ rms ɛ sys ɛ mae ɛ rms ɛ sys ɛ mae ɛ rms TG F G=15 < < TG F G=18.5 < < TG F G= < TG-allmet F G=15 7.4e < TG-allmet F G= e < TG-allmet F G= TM-dwarfs L G=15 < < TM-dwarfs L G=18.5 < TM-dwarfs L G=20 < < TM-giants L G=15 < < TM-giants L G= e < TM-giants L G= TM-allgrav L G=18.5 < < TM-allgrav F G= TAG F G= TAG F G= TAM L G= < TAM L G= TGM L G=15 8.3e TGM L G= e TGM F G= e Figure 11. Predictins f the full frward mdel fr the TG prblem as a functin f lg(t eff ) with lg g fixed at 4.0 dex fr 12 different bands (the central wavelength f which in indicated at the tp f each panel in nm). The black crsses are the (nise-free) grid pints, the small red stars are the frward mdel predictins (at randmly selected AP values) and the blue circles the nisy (G=15) grid pints. The phtn cunts pltted n the rdinate are in standardized units. c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes

11 Parameter estimatin with ILIUM Figure 12. As Fig. 11, but nw shwing predictins f the full frward mdel as a functin f lg g at cnstant T eff =5000 K. It is hard t fairly cmpare the perfrmance with nearest neighburs (NN), because n the ne hand NN is limited by the density f its template grid, but n the ther hand it can reprt the exact AP values. If we used a full, nise-free template grid and nisy evaluatin bjects, then prvided the nise is lw enugh, 1-NN will give exact results. If we instead split the template and evaluatin sets t have n cmmn bjects, then the precisin f at least ne f the APs is limited by the grid spacing. We culd instead average ver the k nearest neighburs, but then NN des badly because it averages ver a wide range f the weak AP (a shrtcming which party mtivates ILIUM). T give sme cmparisn, hwever, I estimate the parameters f all 274 stars in the TG grid f nisefree spectra via leave-ne-ut crss validatin. The errrs in bth APs are six times larger than the ILIUM result at G=15. This at least cnfirms that ILIUM vercmes the grid reslutin limitatin f NN. The abve results are fr stars with [Fe/H] = 0 dex. I retrained and evaluated ILIUM n a grid with the full range f metallicities ( 4 t 1.0 dex; TG-allmet). The errrs averaged ver this full data set at G=15, reprted in line 4 f Table 2, are 6 8 times larger than the slar metallicity case. This is entirely due t the new metallicity variance which is nt accunted fr (mdelled) by ILIUM and s is a cnfusing factr. (A T eff accuracy f 1% at G=15 and 2% at G=18.5 is nnethelss gd fr stars which a priri shw variance ver the full range f T eff, lg g and [Fe/H]). Curiusly, if we apply the mdel t G=20 data, then we see that the perfrmance is n wrse than when we limited the prblem t slar metallicity stars (cmpare lines 3 and 6 f Table 2). This is because the phtmetric nise dminates ver the variance intrduced by the (unmdelled) metallicity range. If the AP residuals had a Gaussian distributin, then ɛ rms = 1.25ɛ mae in Table 2. The fact that ɛ rms is always larger (smetimes much larger), indicates that there are utliers, which justifies the use f ɛ mae as a mre rbust errr statistic. In terms f verall errr, an SVM des better than ILIUM n the TG-allmet prblem at all three magnitudes (Table B1). It s nt clear why this is s, given that ILIUM was better n the prblem limited t slar metallicity (see sectin B). 5 APPLICATION TO THE T eff [Fe/H] PROBLEM (TM) I nw use ILIUM t estimate T eff and [Fe/H] n the tw grids TMdwarfs and TM-giants defined in sectin 3.2. In each case we effectively assume we already have a rugh lg g estimate. The remaining spread in lg g in each case acts as csmic scatter. The mdels are then applied t the crrespnding evaluatin sets with nise levels at three magnitudes. The summary perfrmance statistics are shw in six lines in Table 2. As there is essentially n sensitivity t metallicity at T eff abve 7000 K (Fig. 9), I nly reprt results n cler stars, even thugh the ILIUM mdels were fit t the full T eff range. Lking first at the T eff perfrmance, we see that the precisin is twice as gd as TG at all magnitudes (e.g dex cmpared t dex). This just indicates that we can estimate T eff mre precisely fr cl stars. Within the T eff range K there is n strng dependence f the T eff precisin with T eff r [Fe/H]: ILIUM can estimate T eff equally well at all metallicities. Turning nw t metallicity, we see gd perfrmance at G=15 and G=18.5 fr dwarfs and giants: randm errrs f 0.3 dex r less and negligible systematics. At G=20 the perfrmance is quite a lt wrse ( dex). There is little dependence f [Fe/H] precisin r accuracy with [Fe/H], as can be seen in the lwer panel f Fig. 15: Even fr the mst metal pr stars in the sample at [Fe/H] = 4.0 dex the precisin is still 0.5 dex. This plt als shws that the AP estimates hardly ever exceed the limits f the training c 0000 RAS, MNRAS 000, Cntent is c C.A.L. Bailer-Jnes