Least squares: introduction to the network adjustment

Transcription

1 Least squares: introduction to the network adjustment Experimental evidence and consequences Observations of the same quantity that have been performed at the highest possible accuracy provide different values therefore high precision observations are not deterministic phenomena, but are characterized by several, unpredictable, errors. High precision observations can be modeled by random variables A proper model of random variable to describe high precision observations is the gaussian or normal distribution, completely defined by the mean ( µ) and the covariance ( C ) xx

2 f ( x)= 1 ( 2π ) n/2 ( detc ) xx n/2 e 1 2 x µ x ( ) T C 1 xx ( x µ x ) Geodetic observations are always high precision observations and can be described as extractions from random variables (normally distributed). Geodetic observations are acquired to estimate positions of points in a given reference system (the frame of the observations).

3 Given X0Y (and measured angles and distances), the coordinates of the points should be determined. From a geometrical point of view, the necessary and sufficient number of observations to estimate positions can always be univocally defined.

4 Two problems arise positions estimates depend on the observation errors errors cannot be checked Solution to the problems the positions of the points are modeled as random variables (normally distributed): accuracies must be computed beside positions redundant (more than necessary) observations are required to perform cross-checks redundant observations allow the evaluation of the errors

5 The effect of redundant observations Given the height H of point 1, the 1 heights H, 2 H of 2 and 3 have to be 3 determined. From a geometric point of view, two observations are necessary and sufficient, for example 12 ΔH : 0 H 2=H 1+ΔH 120 H 3=H 1+ΔH 12 + ΔH ΔH, 23 0 The subscript o denotes the numerical value of the observation, that contain the unknown observation error.

6 Each error ε in the observations 12 0 into the estimates H and 2 H : 3 Ĥ 2=H 1+ΔH 12+ε 12 Ĥ =H +ΔH + ε + ΔH + ε ΔH e 23 0 ΔH directly propagates The redundant observation 31 ΔH is introduced

7 Geometric condition ΔH 12 + ΔH 23 + ΔH 31 = 0 The effect of observation errors: first problem ΔH 12 + ΔH ΔH 0 31 = ε 0 0 ε is the closure error and provides an overall measure of the observations error ε= ε12 + ε23 + ε 31 Note that ε can be computed, while individual errors remain unknown. Second problem

8 The estimates of height (differences) depend on the path * ΔH 12 = ΔH 12 0, ** Δ H12 = ΔH31 Δ H , ΔH ΔH * ** A possible solution The closure error ε is distributed among all the observations in such a way that their a posteriori estimates satisfy the geometric condition ΔH ˆ + ΔH ˆ + ΔH ˆ = 0 the estimate of the heights (differences) is univocal

9 Algebraic position of the example 2 unknowns: H, 2 H in 3 observations 3 3 observation equations H H = ΔH s = H H = ΔH H H = ΔH ΔH 12 0, ΔH 23 0, ΔH 31 0 that are completely equivalent to 1 condition equation (the sum of the above three) ΔH 12 + ΔH 23 + ΔH 31 = 0 that is not satisfied by the 3 equations

10 The system s 0 of 3 equations in 2 unknown is algebraically impossible H H = ΔH s = H H = ΔH H H = ΔH because each equation linearly depends on the other two at the left side of the system, but not at right, due to the observation errors

11 Therefore 3 unknown, ε 12, ε 23, ε 31, are considered, that represent the corrections to apply to the observations, in order to obtain their (corrected) calculated values ΔH ˆ ˆ ˆ 12, ΔH23, Δ H that satisfy 31 s The resulting system ŝ is algebraically underdetermined ŝ = H 2 H 1 + ε 12 = ΔH 12 0 H 3 H 2 + ε 23 = ΔH 230 H 1 H 3 + ε 31 = ΔH unknowns H, 2 H, 3 ε 12, ε 23, ε in only 3 equations 31 Therefore

12 2 more equations are needed to build a system S with an algebraic univocal solution the 2 equations are obtained by imposing an (arbitrary) condition on the unknowns the condition provides a criterion to distribute the closure error and poses the estimation principle for the unknowns.

13 One example of estimation principle Equal distribution of the closure error ε =ε =ε = ε 3 computed observations: observables estimates Δ Ĥ =ΔH ε Δ Ĥ =ΔH ε Δ Ĥ =ΔH ε Heights: parameters estimates H ˆ =H +ΔH ˆ

14 H ˆ =H +ΔH ˆ Verify: estimates do not any more depend on the path! Remarks Estimation of the parameters (positions) starting from redundant observations by applying one estimation principle Moreover estimation of the parameters accuracies by applying the same principle Least Squares is the estimation principle usually adopted to adjust networks. Least Squares principles

15 General properties not dependent on the probability distribution of the observations minimal variance estimates within the class of unbiased and linear ones parameters and their accuracy can be computed in a closed form by simulations final accuracies of the parameters can be predicted from the a priori precision of the observations however not robust estimates

16 Least Squares: data y : observations vector (m elements), extracted from the 0 random variable y(observables), defined in the y must belong to a linear subspace (manifold) dimensions in m R n y V (linear functional model) m R space n V with n the covariance of y is known apart a proportionality factor m 2 σ C yy 2 = σ Q (stochastic model)

17 Least Squares: goals One estimate ŷ of the mean y such that One estimate Cyy = σˆ 2 Q C yy ˆˆ of Estimation principle y V 2 C yy, that means one estimate of σ, given Q ( y y) Q ( y y ) = min T n

18 Geometric interpretation: the leveling triangle y ΔH = ΔH Δ H Functional model (linear) ΔH 12 + ΔH 23 + ΔH 31 = 0 y V (equation of a plane in the space m=2) Stochastic model 2 Q= I Cyy = σ I n

19 Least Squares estimator T 1 ( y0 y) Q ( y0 y) = min n y V ŷ is the point n V at minimum distance from y 0

20 Geometric interpretation of the condition and observation equations: the leveling triangle Condition equation: ΔH 12 + ΔH 23 + ΔH 31 = 0 Observation equation: ΔH 12 = H2 H1 ΔH 23 = H3 H1 ΔH 31 = H1 H3 Both of them represent equation of a plane in R3

21 Application of Least Squares to the network adjustment Condition or Observation equations? Typically, not the observables but other parameters are of "interest", that are typically unknowns but are related to the observables by observation equations In leveling observables: height differences parameters: heights observation equations: H j H i= ΔHij

22 generally, condition equations cannot be easily implemented. On the contrary, observation equations can be easily implemented: one equation for each kind of observation In the following, the observation equations model will be adopted.

23 The observation equation model Given m observations y o y1 o y 2o =... ym o y y ε, [ ] 0 = + E y = y, 0 C C Q 2 yy = εε = σ y is the vector of the unknown observables,

24 ε is a sample drawn from an m-variate random variable modeling the observing error, unknown too, 2 σ is the a-priori variance of the observations error, Q is the cofactor matrix of the m-variate random variable, both known. Given the n-dimensional vector x, containing the unknown parameters: x x 1 2 =... x n x, n m

25 We introduce the following deterministic model, describing the functional relationship between x and y y= Ax+ b A is the m n design matrix, b is an m-dimensional known vector.

26 Least squares principle and estimators We look for two vectors ˆx and ŷ compatibles with each other, where ŷ is at a minimum distance from y ; namely o ˆx and ŷ such that yˆ = Axˆ+ b ˆ T 1 ( yo y) Q ( yo y ) = min ˆ

27 From the previous conditions it follows the normal system: T 1 ˆ = ( o ) T 1 Nx A Q y b, N= A Q A N is called normal matrix; two cases are possible: A is full rank (its columns are linearly independent one from the others) Ax = 0 x = 0 in this case there is no rank deficiency and the normal matrix N is invertible.

28 A is not a full rank matrix, (some of its columns are linear combinations of some others) Ax = 0 for some x 0 in this case the problem has a rank deficiency, N is not invertible.

29 Rank deficiency in a system of observations equations: the leveling triangle The 3 redundant observations ΔH, 12 ΔH, 23 ΔH are not sufficient 31 to estimate the 3 heights H, 1 H, 2 H 3

30 Indeed the 3 redundant observations are not modified by adding a constant H to the 3 heights ( ) ( ) ( ) ( ) ( ) ( ) ΔH 12 = H2 H 1 = H 2 + H H 1 + H s = ΔH 23 = H3 H 2 = H 3 + H H 2 + H ΔH 31 = H1 H 3 = H 1 + H H 3 + H Heights present one degree of freedom with respect to the observables, therefore the system s presents a rank deficiency equal to 1 and the redundant observations allow the estimation of 2 heights when the third has been a priori fixed.

31 Important Redundancy and rank deficiency are completely separate characteristics in a system of observations equations: redundancy is related to the cross check of the observations rank deficiency is related to the estimability of a given set of parameters wrt a given set of observables

32 Rank deficiency: intuitive definitions Rank deficiency can be eliminated by constraining the degrees of freedom of the parameters wrt the observables When the rank deficiency is eliminated by fixing only the degrees of freedom of the parameters, the adjustment is called minimal constraints adjustment Example: by fixing the height of point 1, a reference frame in which A has height H is defined 1

33 Rank deficiency: formalization We define the kernel K of A as follows: { } K ( A) = x Ax = 0 ; 0 0 if ˆx is the solution of yˆ = Axˆ+ b, then also xˆ + x is a solution: 0 Ax ( ˆ+ x) + b= Axˆ+ Ax + b= yˆ+ 0= y ˆ 0 0

34 The observations don t contain enough information to estimate all the unknown parameters; this happens even if the problem is redundant and it is due to the problem design. There is an infinite number of possible solutions for the unknown parameters, which can satisfy the optimal estimation principle.

35 The levelling triangle y O yy ΔH = ΔH Δ H 12O 23O 31O H H 2 = H C = σ I 3 = Ax + ν 2 0 v A is not full rank; in particular

36 1 T K ( A) = 1 H, H, N= A A, det( N) = 0 1 Notes A: R R, K( A) R ; A : R R, S( A ) R n m n T m n T n T K( A) S ( A )

37 Exercise: the kernel identification Let the rank of A be equal to r, with r< n: r columns are linearly independent, the remaining n r= d columns are a combination of the previous ones; by rearranging the columns, it is therefore possible to write A = a1 a2... an = Ar Ad = Ar Ar D = Ar Ir D m n m 1 m 1 m 1 m r m d m r m r r d m r r r r d where D is a coefficient matrix. It is easy to show that

38 D r d K ( A) = xd x Id d 1 d d In fact, one has d D D A x = A [ I D] x = A ( D+ D) x = 0 x I I d r r d r d d d d In order to prove the viceversa, let Ax = 0; consider the decomposition

39 x n 1 xr r 1 = xd d 1 xr Ar[ Ir D] x= 0 [ Ir D] = 0 because Ar is full rank; therefore, x d xr + Dxd = 0 xr = Dx, namely d Dxd D x such as Ax= 0 x= = xd x xd Id d

40 The constrained solution To eliminate the rank deficiency, stochastic constraints Hx = 0 can be introduced on the parameters Hx = 0, ( dim(h) = n k,k d ) to build a new system y 0 = Ax + b, C yy = σ 2 Q 0 = Hx, C hh = σ h 2 I such that A H has full rank. The estimate is obtained by the solution of

41 h 0 = y 0 0 = A H C hh = σ 2 Q hh, Q hh = x + b 0 Q 0, 0 1/ λi, λ = σ 2 2 / σ h and is given by ˆx H = R 1 A T Q 1 (y O b), R = N + λh T H C x x H H = σ R NR 2 1 1

42 The minimum constraints solution Constraints are minimal when T K ( A) + S( H ) = R T K ( A) S( H ) = 0 which implies v n = d. Practical and comprehension exercise: compute all possible solutions for the leveling triangle!

43 Two definitions and a note We define as intrinsic the rank deficiency of a network, made by a given type of observations, which cannot be reduced by modifying the network design; that is, by adding new observables of the same kind to the design itself. In the geodetic network adjustment, solutions which eliminate the intrinsic rank deficiency must be adopted and define the reference system and frame.

44 The not linear problem There s no LS formulation for the not linear problem y O = y + ε = f(x) + ε where f(x) = f 1 (x 1,x 2,...,x n ) f 2 (x 1,x 2,...,x n )... f m (x 1,x 2,...,x n ) In this case at first we have to linearize the problem.

45 To this aim, approximate values of the unknown parameters have to be known!x " x The linearization is obtained by using a Taylor s development around!x, truncated at the first order y = f(!x) + J f (!x)(x!x) The original problem becomes η O = η + ε = Aξ + ε η = y f(!x),ξ = x!x, A ij = J ij = f i x j (!x)

46 Final estimates provided by LS Not rank deficient problems! Unknown parameters estimate ˆx = N 1 A T Q 1 (y o b); Observables and residuals estimates: ŷ = Aˆx + b = AN 1 A T Q 1 (y o b) + b ˆε = ŷ o ŷ = (I AN 1 A T Q 1 )(y 0 b);

47 Redundancy or degrees of freedom: R= m n 2 A posteriori variance, σ, estimate: σˆ 2 = ˆε T Q 1 ˆε m n ; Covariance matrix of estimated parameters: C ˆxˆx = σˆ 2 N 1

48 Covariance matrix of estimated observables: C ŷŷ = σˆ 2 AN 1 A T Covariance matrix of estimated residuals C ˆε ˆε = ˆ σ 2 (Q AN 1 A T ) Remark LS produce unbiased and minimum variance estimates of the unknown parameters; the estimates are independent from apriori variance.

49 Outliers identification and removal Least Squares are not robust: the estimates can be distorted if the functional and/or the stochastic models do not properly describe the observations: outliers however the stochastic and functional models that describe the observations of classic geodetic networks are very simple and well known the stochastic and functional models relevant to the GPS observations are complicated and only partly known in any case Normality hypothesis on the observations

50 Classic geodetic networks Model errors typically due to outliers: external observations wrt the stochastic model N µ N e µ σ µ+σ µ+σ e

51 The observations N[µ,σ 2 ] are well described by the normal distribution The observations are outliers wrt the normal distribution N[µ,σ 2 ], but are consistent with the other normal distribution N e [µ,σ e 2 ]

52 GNSS observations Model errors caused by both outliers and an approximate knowledge of the stochastic model N a N c

53 N a : approximated stochastic model N c : correct stochastic model outlier with respect to both the models Generally, the accuracy of the observations is over estimated therefore at first outliers should be identified, then the stochastic model has to be corrected

54 Algorithms exist in order to a posteriori Verify the global unbiasedness of the adopted models Identify possible errors affecting single observations Assess the stochastic model Assess the reliability of the adjustment results.

55 Hypothesis testing Determine if, with a fixed probability of error (confidence level), a hypothesis H0 can be accepted A statistics is built, whose distribution is known if H0 holds true and that, when H0 is wrong, assumes values very high, the sample value of this statistics is compared to an acceptance interval, the hypothesis is accepted if the sample value belongs to this interval. The test significance level it is the probability to make an error by refusing an hypothesis which is true, the usually adopted values are: 0.01, 0.05, 0.10.

56 Global test on the model (functional and stochastic) Fundamental hypothesis 2 2 H ˆ 0 : σ = σ σ Test statistic: ˆ 2 σ (m n) = χ 2 2 exp 2 2 χ exp ~ χ if o ( m n) H is true 2 χ ( m n) chi square distribution with (m-n) degrees of freedom

57

58 α significance level of the test 2 χ lim 2 = χ m n (α ) such that P(0 χ χ ) = 1 α 2 2 m n lim 2 2( α) if χ χ( m n), H o is accepted 2 2( α) if ( m n) χ > χ, Ho is rejected (probable presence of model errors)

59 Steps to perform the test on the global model Least squares estimate of ˆx, ŷ, ˆε and ˆ 2 σ ; given the significance level α, the value 2 2 χ exp is computed and compared with χ lim. 2 χ lim is determined (tables); Note The test on the global model has been originally designed to identify errors in the deterministic model, but it can fail just because of errors in the stochastic model.

60 Local test on single observations (independent observations) i Fundamental hypothesis: = τexp ~ τ( m n) ˆ ε σ ε i where σ e i = ˆ σ q εε i i τ( n m) Thomson distribution τ at (n-m) degrees of freedom (similar to a normal distribution when (n-m) is big)

61 α /2 The theoretical τlim = τ( m n) is fixed such that P(0 τ τ ) = 1 α, P( τ > τlim) = α τ if exp ( m n) lim τ H0 is accepted, otherwise, H0 is rejected the i-th lim observation is a suspected outlier

62 Remark 1 The acceptance interval leaves two tails of a given probability α: either negative residuals or positive residuals whose absolute value are too high wrt the limits are to be rejected. Remark 2 The not robustness of the estimates complicates the outliers identification: indeed one outlier modifies also the residuals of the other observations Therefore an iterative process is needed to identify outliers (Data Snooping)

63 Data Snooping at each iteration, we eliminate the K-th observation, for which τ > τ exp k lim τ exp =max( τsp ) k Iterations stop when no more suspected observations remain Final residuals of the rejected observations are used to decide whether eliminate them definitively of reintroduce them

64 The test on the stochastic model In some cases, the test on the global model fails but no large outliers (normalized residuals) can be isolated by data snooping. In this case, typically, the hypotheses on the covariance matrix structure were wrong: the accuracy of some groups of observations were over estimated: the corresponding diagonal blocks of ("small") C were under estimated yyii C yy

65 A posteriori estimate of the stochastic model. Let y i1 2 i =..., ii = σ i ii y C Q, y ip 2 y 1 σ1q y =..., C= , 2 y q σ 0 0 qqqq 2 where even the different σ are to be considered as unknown. i 2 Fixed an arbitrary σ we solve the least squares problem by using K σ = σ = ii = C Q; Q ; K Q i2 σ 0 0 K qq ii

66 where, as a first approximation, we can put 2 σ i 1 2 σ ; 2 The rigorous model for the joined estimate of σ and i x is so numerically complex that has no practical use. A reasonable compromise between rigorousness and simplicity is given by ˆ σ i 2 = ˆε i T K ii 1 ˆε i m i tr A i N 1 A i T K ii 1 { } that can be also written as follows

67 σˆ 2 i = ˆε T 1 i K ii ˆε i r jj j i The process is iterative; at each step: LS estimate of the parameters, the residuals and new covariances the process is stopped when the results converge to stable values.

68 Accuracy of the parameters Hypothesis: global test and Data Snooping have been successfully performed C C overall accuracy: covariance matrix 2 1 xx = σˆ N accuracy of the point i: submatrix = ˆ σ EN E 2 1 T xx i i i i E i : matrix to extract the estimates ˆi parameters xˆ i = Ex ˆ i x of the point i, from the vector of

69 Confidence interval of each point I: Region of the p-dimensional space (p is the number of coordinates that define the position of I in the network) to which i belongs with a given probability, centered on the estimates xˆ, C ( x i ˆx ) T i C 1( xi xi ˆx ) ( α ) x i i F p, m n ( ) i x x i i Error ellipse (2D) and ellipsoid (3D) Error ellipse of a point i: 2-D confidence interval (p=2)

70 ( x i ˆx ) T i C 1( xi xi ˆx ) ( α ) x i i F 2, m n α F 2, n m ( ) ( ) ( ) : Fisher distribution with (2,(n-m)) degrees of freedom α : generally the values 0.01, 0.05, 0.10 are adopted Standard error ellipse of a point i: ˆ 1 ˆ 1 T ( x x ) C ( x x ) i i x x i i ( ) ( ) α F 2, m n i i =1 α! 0.61 for ( n m) >10:

71 Error ellipsoid of a point i: 3-D confidence interval (p=3) ( x i ˆx ) T i C 1( xi xi ˆx ) ( α ) x i i F 3, m n ( ) ( ) ( ) : Fisher distribution with (3,(n-m)) degrees of freedom α F 3, n m α : generally the values 0.01, 0.05, 0.10 are adopted Standard error ellipsoid of a point i: ˆ 1 ˆ 1 T ( x x ) C ( x x ) i i x x i i ( ) ( ) α F 3, n m i i =1 α! 0.81 for ( n m) >10

72 Geometric parameters of the error ellipse Given the 2D covariance matrix of point i: C ˆ σ ˆ σ = 2 X XY xx ˆˆ 2 ˆ σ ˆ XY σy σ max : major semiaxis, σ min : minor semiaxis θ: orientation angle of major semiaxis wrt x 1 axis σ are the square root of the eigenvalues of C ˆˆ, max,min xx

73 2 ˆ σ ˆ X λ σ XY det( Cxx ˆˆ λi ) = det 0 2 = ˆ σ ˆ XY σy λ Therefore 2 2 ˆ ˆ X + σy 1 σ λ ˆ ˆ ˆ max,min = ± ( σ σ ) + 4σ X Y XY 2 2 ˆ σ ˆ X + σy ˆ ˆ ˆ STD max, STDmin = ± ( X Y ) + 4 XY σ σ σ σ σ 2 2 Let's use θ to indicate the counter-clockwise angle between X axis and error ellipse major axis. The unitary vector

74 e max cosθ = sinθ provides the direction of the maximum eigenvector e max λ max, C e = λ Ie ; ( C λ I) e = 0 xx ˆˆ max max max xx ˆˆ max max ( ˆ σ λ ) e + ˆ σ e = 0, ˆ σ e + ( ˆ σ λ ) e = X max 1 XY 2 XY 1 Y max 2 tanθ e e 2 = = 1 max max max max λ ˆ σ ˆ σ max XY 2 X

75 from the trigonometric equality 2 tan 2θ 2tan θ / (1 tan θ) =, it follows tan 2θ = ˆ σ ˆ σ 2 ˆ σ 2 2 X Y XY therefore θ = 1 ˆ ˆ arctg σ σ 2 2ˆ σ 2 2 X Y XY The parameters of other ellipses, with F ( ) 1 2,( ) α, are given by the m n σmax = F2,( m n) ( α) σstdmax, σmin = F2,( m n) ( α) σstdmin

76 Internal reliability of the observable Represent the capability of the observables to reciprocally monitor, for example: the height difference Δ H is BD not controlled from any other observation, so it has no reliability Δ H AB, H BC from zero Δ, Δ HCA control each other, so their reliability is different The local r i redundancy of each observable is an internal reliability index.

77 Maximum outlier not identified in each observation (according to Baarda) Outlier identification is based on the mutual control of all the observations therefore the absolute value of the not identified outlier in each observation depends on its reliability Depending on the local redundancy r i and accuracy σ i of each observation, fixed a no-centrality a-priori value δ, according to the probability: α: accept the hypothesis H a (presence of the outlier) even if the outlier is not present (the hypotesis H 0 is true) 1 β ; accept the hypothesis H 0 (outlier not present) even if one outlier δ is present (the hypothesis H a is true)

78 Test reliability Under the alternative hypothesis Ha (one outlier δ i in observation y i) the normalized residual follows a τ ( m n) distribution with not centrality parameter Q εε ˆ σ ii δi Q εε δ ii i : τ ( m n ) ( ) ˆ σ

79 τ lim 1 β = τ (m n) ( Q εε ii δ i β is the power of the test with respect to δ. σˆ ) Assigned α, τ lim can be determined; assigned β, δ i can be computed such that lim ( Qεε δ ii i τ ) 1 ( m n) = β. ˆ σ τ

80 The maximum embedded error in observation i is the maximum error δ i which is not possible to reveal with a test power β. δ i = k α,β It can be proved that σ σ! k (Q 1 α,β P A ) ii P A ii where P = I AN A Q 1 T 1 A

81 Local redundancy of the observation i P A = ii r i Two extreme cases r 0: δ ; r 1: δ min i i i i Internal reliability of the observation i Maximum embedded error δ ; the worst internal reliability: δ = maxδ i i

82 External reliability The influence of an embedded error δ in the unknown parameter i estimation: δxˆ( δ ) = N A Q e δ 1 T 1 i i i Two extreme cases: r = 0: the whole error propagates into the unknown estimates. i r 1: the whole error goes into residuals and does not affect at all i the unknown estimates.

83 External reliability of the parameter x j: δxˆ = max( δxˆ ( δ )) j j i i The worst external reliability: δxˆ = max( δxˆ ). j j

84 Parameters estimability and conditioning of the normal system Even after the removal of the rank deficiency, some parameters could be affected by a bad estimability problem this can be checked by singular value decomposition of the normal matrix T ENE E: matrix of the orthonormal Eigenvectors of A Λ: diagonal matrix containing the Eigenvalues of A The parameters whose Eigenvalues are almost 0 are badly estimable and must be calculated = Λ

85 the conditioning number of the normal system is one index of the estimability of the solution: it is computed accordingly to v λ max = v should be almost equal to 1 λ min if λ min ; 0, v, the solution is unstable or ill conditioned This is neither a rank deficiency problem, nor a accuracy one: however, an ill posed system can degenerate into a rank deficient one, in any case, final accuracies of estimated parameters are mediocre