Stanford University School of Earth Sciences Course on Inverse Problems Albert Tarantola Third Lesson: Probability (Elementary Notions)
Let u and v be two Cartesian parameters (then, volumetric probabilities and probability densities are identical). Given a probability density f (u, v), one defines the two marginal probability densities f u (u) = f v (v) = dv f (u, v) du f (u, v) and the two conditional probability densities f u v (u v = v ) = f v u (v u = u ) = f (u, v ) du f (u, v ) f (u, v) dv f (v, u )
marginal conditional.25.12.1.8.6.4.2.2.15.1.5-1 -5 5 1 1 conditional.25.2 joint.15.1.5-1 -5 5 1 5-5 -1-1 -5 5 1 marginal.15-1 -5 5 1.125.1.75.5.25-1 -5 5 1 conditional.14.12.1.8.6.4.2-1 -5 5 1.14.12.1.8.6.4.2 conditional
Let u and v be two Cartesian parameters (then, volumetric probabilities and probability densities are identical). Given a probability density f (u, v), one defines the two marginal probability densities f u (u) = f v (v) = dv f (u, v) du f (u, v) and the two conditional probability densities f u v (u v = v ) = f v u (v u = u ) = f (u, v ) du f (u, v ) f (u, v) dv f (v, u )
Let u and v be two Cartesian parameters (then, volumetric probabilities and probability densities are identical). Given a probability density f (u, v), one defines the two marginal probability densities f u (u) = f v (v) = dv f (u, v) du f (u, v) and the two conditional probability densities f u v (u v ) = f v u (v u ) = f (u, v ) du f (u, v ) f (u, v) dv f (v, u )
Let u and v be two Cartesian parameters (then, volumetric probabilities and probability densities are identical). Given a probability density f (u, v), one defines the two marginal probability densities f u (u) = f v (v) = dv f (u, v) du f (u, v) and the two conditional probability densities f u v (u v) = f v u (v u) = f (u, v) du f (u, v) f (u, v) dv f (v, u)
One has f u v (u v) = f v u (v u) = f (u, v) f v (v) f (u, v) f u (u) from where (a joint distribution can be expressed by a conditional distribution times a marginal distribution) f (u, v) = f u v (u v) f v (v) = f v u (v u) f u (u) from where (Bayes theorem) f u v (u v) = f v u(v u) f u (u) f v (v)
Recall: f (u, v) = f u v (u v) f v (v) = f v u (v u) f u (u). The two quantities u and v are said to have independent uncertainties if, in fact, f (u, v) = f u (u) f v (v) (the joint distribution equals the product of the two marginal distributions). This implies (and is implied by) f u v (u v) = f u (u) ; f v u (v u) = f v (v).
two quantities with independent uncertainties (the joint distribution is the product of the two marginal distributions) -5 5 1 15 2 2 15 1 5-5 -1-5 5 1-1 -5 5 1
Let u and v be two Cartesian parameters (then, volumetric probabilities and probability densities are identical). Let f (u, v), be a probability density that is not qualitatively different from a two-dimensional Gaussian. The mean values are the variances are u = v = c uu = σ 2 u = c vv = σ 2 v = and the covariance is c uv = du du du du du dv u f (u, v) dv v f (u, v) dv (u u) 2 f (u, v) dv (v v) 2 f (u, v) dv (u u)(v v) f (u, v)
The covariance matrix is ( ) cuu c uv ( ) σ 2 u c uv C = c vu c vv = c vu σ 2 v. It is symmetric and positive definite (or, at least, non-negative). Note: the correlation, defined as ρ uv = c uv σ u σ v = has the property 1 ρ uv +1. c uv cuu cvv,
The general form of a covariance matrix is c 11 c 12 c 13... σ 2 1 c 12 c 13... c 21 c 22 c 23... c 21 σ 2 2 c 23... C = = c 31 c 32 c 33... c 31 c 32 σ 2 3............... The quantities with immediate interpretation are the standard deviations {σ 1, σ 2, σ 3,... } and the correlation matrix 1 ρ 12 ρ 13... ρ 21 1 ρ 23... R = ρ 31 ρ 32 1..........
The multidimensional Gaussian distribution is defined as f (x 1, x 2,..., x n ) f (x) = k exp ( 1 2 (x x ) t C -1 (x x ) ) Its mean is x and its covariance is C (not obvious!).