Revew: Ft a lne to data ponts Correlated parameters: L y = a x + b Orthogonal parameters: J y = a (x ˆ x + b For ntercept b, set a=0 and fnd b by optmal average: ˆ b = y, Var[ b ˆ ] = For slope a, set b=0 and fnd a by optmal scalng: ˆ a = y ( x x ˆ ( x x ˆ o need to terate. (Why?, Var[ a ˆ ] = Pvot pont: ˆx x ( x x ˆ ˆ x ˆ y
Ft a lne => ft patterns => ft M patterns Model : y = b + a x ˆ x ( = α 0 P 0 (x + α P (x ( Patterns : P 0 (x = P (x = x ˆ x Optmal Scalng: ˆ α 0 = ˆ α = ( y α ˆ P P 0 P 0 ( y α ˆ 0 P 0 P P, Var[ α ˆ 0 ] =, Var[ α ˆ ] = P 0 P ˆx x ˆ Pvot pont: x ˆ y Iterate (f patterns not orthogonal. LIEAR REGRESSIO: Generalse model to M patterns: M k= y = α k P k (x Iterated Optmal Scalng: smple algorthm, easy to code, often adequate.
Example: Sne Curve + Background Data : X ± at t = t Model : X(t = A + S sn(ω t + Ccos(ω t 3 Patterns :, s = sn(ω t, c = cos(ω t Iterated Optmal Scalng: ˆ A = ˆ S = ˆ C = ( X S ˆ s C ˆ c ( X A ˆ C ˆ c s s ( X A ˆ S ˆ s c c, Var[ A ˆ ] =, Var[ S ˆ ] =, Var[ C ˆ ] = Iterate ( f patterns not orthogonal. s c Varance formulas assume orthogonal parameters, otherwse gve error bars too small. Use nverse of Hessan matrx (see later.
χ = 0 = χ a = y (a x + b x ( y a x b σ 0 = χ b = ( y a x b σ The ormal Equatons : a x σ + b x σ = x y σ a x σ + b σ = y σ Matrx form : Σ x /σ Σ x /σ Σ x /σ Σ /σ a b = Σ x y /σ Σ y /σ H α = c(y Soluton : α = H c(y χ analyss of the straght lne ft ( H = Hessan matrx y = a x + b
ormal Equatons : H α = c(y Σ x /σ Σ x /σ Σ x /σ Σ/σ a b = Σ x y /σ Σ y /σ Soluton : a = b Δ α = H c(y Σ/σ Σ x /σ Σ x y /σ Σ x /σ Σ x /σ Σ y /σ Hessan Determnant : Δ = ( Σ/σ ( Σ x /σ ( Σ x /σ Orthogonal bass : x (x ˆ x Σ (x ˆ x /σ = 0, Δ = Σ/σ a = b Δ a ˆ = Σ (x x ˆ y /σ Σ (x x ˆ /σ x ˆ ( Σ x /σ ( Σ/σ ( ( Σ (x x ˆ /σ Σ/σ 0 Σ (x x ˆ y /σ 0 Σ (x x ˆ /σ Σ y /σ b ˆ = Σ y /σ Σ/σ χ analyss of the straght lne ft ( same as Optmal Scalng y = a x + b ( Dagonal Hessan Matrx
H jk Example: y = a x + b. χ a = x The Hessan Matrx χ a j a k, / χ a b = x / x χ /σ x /σ b = /σ, so H = x / /σ χ χ = a = y (a x + b x ( y a x b σ χ b = ( y a x b σ For lnear models, Hessan matrx s ndependent of the parameters, and χ surface s parabolc.
Parameter Uncertantes Hessan matrx descrbes the curvature of the χ surface : χ (α = χ ( ˆ α + j,k ( α j α ˆ j H j k α k ˆ ( +... α k H jk χ a j a k, For lnear models, Hessan matrx s ndependent of the parameters, and χ surface s parabolc. For a one-parameter ft: f ˆ α mnmzes χ, then Var( ˆ α = χ / α. For a mult-parameter ft the covarance of any par of parameters s an element of the nverse-hessan matrx: Cov(a k,a j = χ α k α j = [ H ] k j
Prncpal Axes of the χ Ellpsod b b Egenvectors of H defne the prncpal axes of the χ ellpsod. Equvalent to rotatng the coordnate system n parameter space. y = a x + b = a x cosθ snθ ( + b xsnθ + cosθ ( θ a a Can also dagonalse H by : a x + b a (x x ˆ + b Ths shears the parameter space, gvng H = x ˆ / 0 0 / σ Dagonalsng the Hessan matrx orthogonalses the parameters. b a
Scale M Patterns: General Lnear Regresson Example: Lnear Model: y(x = a P (x + a P (x +... = M k Polynomal : y(x = a 0 + a x + a x +...+ a M x M a k P k (x χ = y y ormal Equatons: = = 0 = χ # M & = y a j P j a % ( k $ j ' M y a j P j σ j P k k =...M M # P j P k & y % ( ( a j = P k P k P k $ ' j M P H j k a j = c k (y H j k = j P k c k (y = j y P k
Prncpal Axes for general Lnear Models In the general lnear case we ft M functons P k (x wth scale factors a k : M y(x = a k P k (x k= The (M x M Hessan matrx has elements: H jk χ a j a k = ormal equatons (M equatons for M unknowns: H jk a k = c j where c j = = P j P k y P j Ths gves M-dmensonal ellpsodal surfaces of constant χ whose prncpal axes are the M egenvectors of the Hessan matrx H. Use standard matrx methods to fnd lnear combnatons of P that dagonalse H. ( More detals later =
Lnear vs on-lnear Models Lnear Model: y(x = H j k χ = α j α k on - Lnear Models : power - law : y = A x B = M blackbody : f ν = Ω B ν (λ,t k α k P k (x P j P k H j k χ depends on parameters α j α k χ (α = χ ( α ˆ + α j α ˆ j ( α k +... j,k ( H j k α k ˆ M scale parameters α k Ellptcal χ contours, unque soluton by lnear regresson (matrx nverson. Banana-shaped contours, multple local mnma, requre teratve methods.
Method : Lnearse the on-lnear Model Lnearsaton: use local lnear approxmaton to the model, gvng a quadratc approxmaton to χ surface. Solve by lnear regresson, then terate. Example : gaussan peak + background : µ = A g + B g e η / Δµ ΔA µ A + ΔB µ B + Δx 0 µ A = g µ = A g η /σ x 0 µ B = µ σ = A g η /σ η x x 0 σ µ + Δσ µ x 0 σ µ A = g µ x 0 µ σ Guess x 0 and σ, ft lnear parameters A and B, evaluate dervatves, adjust x 0 and σ usng lnear approxmaton, terate. (Levenberg-Marquadt method: add constant to Hessan dagonal to prevent over-steppng. See e.g. umercal Recpes. A and B are scale parameters. µ B = x0 and σ are non-lnear parameters.
Method : Amoeba (Downhll Smplex Amoeba (downhll smplex Smplex = cluster of M+ ponts n the M-dmensonal parameter space. 3. Evaluate χ at each node.. Pck node wth hghest χ, move t on a lne thru the centrod of the other M nodes, usng smple rules to fnd new place wth lower χ. 3. Repeat untl converged. 4 5 6 8 7 Amoeba requres no dervatves : Amoeba crawls downhll, adjustng shape to match the χ landscape, then shrnks down onto a local mnmum. See umercal Recpes for full descrpton.
Method 3: Markov Chan Monte Carlo (MCMC. Start wth M+ ponts n the M-dmensonal parameter space.. Evaluate for each parameter (and covarance matrx from last n ponts. 3. Take a random step, e.g. usng a Gaussan random number wth same (and covarances as recent ponts. Δα ~ G(0, 4. Evaluate Δχ = χ new - χ old and keep the step wth probablty 5. Iterate steps -4 untl convergence. P = mn, exp Δχ / ( MCMC requres no dervatves : MCMC generates a chan of ponts tendng to move downhll, then settlng nto a pattern matchng the posteror dstrbuton of the parameters. Can escape from local mnma. Can also nclude pror dstrbutons on the parameters.
Example: MCMC ft of exoplanet model to transt lghtcurves and radal velocty curve data. 0.5 Relatve radal velocty (km s - 0-0.5 - -0.08 0.6 0.8..4 Transt phase -0.06-0.04 Dfferental magntude -0.0 0 0.0 0.04 0.06 0.08 0. 0. 0.4 0.95.05 Transt phase