Geometric Constructions Relating Various Lines of Regression

Transcription

1 Applied Mathematical Sciences, Vol. 0, 06, no. 3, - 30 HIKARI Ltd, Geometric Constructions Relating Various Lines of Regression Brian J. M c Cartin Applied Mathematics, Kettering University 700 University Avenue, Flint, MI USA Copyright c 06 Brian J. M c Cartin. This article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract As a direct consequence of the Galton-Pearson-McCartin Theorem [0, Theorem ], the concentration ellipse provides a unifying thread to the Euclidean construction of various lines of regression. These include lines of coordinate regression [7], orthogonal regression [3], λ-regression [8] and (λ, µ)-regression [9] whose geometric constructions are afforded a unified treatment in the present paper. Mathematics Subject Classification: 5M5, 5N0, 6J05 Keywords: geometric construction, linear regression, concentration ellipse, Galton-Pearson-McCartin Theorem Introduction In fitting a linear relationship [6] to a set of (x, y) data (Figure ), it is many times assumed that one ( independent ) variable, say x, is known exactly while the other ( dependent ) variable, say y, is subject to error. The line L (Figure ) is then chosen to minimize the total square vertical deviation Σd y. This is known as the line of regression of y-on-x. Reversing the roles of x and y minimizes instead the total square horizontal deviation Σd x, thereby yielding the line of regression of x-on-y. Either method may be termed coordinate regression. Deceased January 9, 06

2 Brian J. McCartin This procedure may be generalized to the situation where both variables are subject to independent errors with zero means, albeit with equal variances. In this case, L is chosen to minimize the total square orthogonal deviation Σd (Figure ). This is referred to as orthogonal regression. 0.5 y x Figure : Linear Fitting of Discrete Data This approach may be further extended to embrace the case where the error variances are unequal. Defining σ u = variance of x-error, σ v = variance of y- error and λ = σ v/σ u, a weighted least squares procedure [, 6] is employed whereby +m λ+m Σd is minimized in order to determine L. This is referred to as λ-regression. Finally, if the errors are correlated with p uv = covariance of errors and µ = p uv /σ u then +m λ µm+m Σd is minimized in order to determine L. This is referred to as (λ, µ)-regression. In the next section, the concept of oblique linear least squares approximation is introduced (Figure 3). In this mode of approximation, one first selects a direction specified by the slope s. One then obliquely projects each data point in this direction onto the line L. Finally, L is then chosen so as to minimize the total square oblique deviation Σd i. It is known that this permits a unified treatment of all of the previously described flavors of linear regression [0]. Galton [7] first provided a geometric interpretation of coordinate regression in terms of the best-fitting concentration ellipse which has the same first moments and second moments about the centroid as the experimental data [3]. Pearson [3] then extended this geometric characterization to orthogonal regression. McCartin [8] further extended this geometric characterization to

3 Geometric construction of regression lines 3 Figure : Coordinate and Orthogonal Regression λ-regression. Finally, McCartin [9] succeeded in extending this geometric characterization to the general case of (λ, µ)-regression. In a similar vein, a geometric characterization of oblique linear least squares approximation [0] further generalizes these Galton-Pearson-McCartin geometric characterizations of linear regression. This result is known as the Galton-Pearson-McCartin Theorem and is reviewed below (Theorem ). The primary purpose of the present paper is to utilize the Galton-Pearson- McCartin Theorem to provide a unified approach to the Euclidean construction of these various lines of regression. As such, the concentration ellipse will serve as the adhesive which binds together these varied geometric constructions. However, before this can be accomplished, the basics of oblique least squares, linear regression and the concentration ellipse must be reviewed. Oblique Least Squares Consider the experimentally observed data {(x i, y i )} n i= where x i = X i +u i and y i = Y i + v i. Here (X i, Y i ) denote theoretically exact values with corresponding random errors (u i, v i ). It is assumed that E(u i ) = 0 = E(v i ), that successive observations are independent, and that V ar(u i ) = σ u, V ar(v i ) = σ v, Cov(u i, v i ) = p uv irrespective of i. Next, define the statistics of the sample data corresponding to the above population statistics. The mean values are given by x = n x i ; ȳ = n i= n n y i, () i=

4 4 Brian J. McCartin the sample variances are given by Figure 3: Oblique Projection σx = n i x) n i=(x ; σy = n the sample covariance is given by n (y i ȳ), () i= p xy = n (x i x) (y i ȳ), (3) n i= and (assuming that σ x σ y 0) the sample correlation coefficient is given by r xy = p xy σ x σ y. (4) Note that if σx = 0 then the data lie along the vertical line x = x while if σy = 0 then they lie along the horizontal line y = ȳ. Hence, we assume without loss of generality that σx σy 0 so that r xy is always well defined. Furthermore, by the Cauchy-Buniakovsky-Schwarz inequality, p xy σ x σ y (i.e. r xy ) (5) with equality if and only if (y i ȳ) (x i x), in which case the data lie on the line y ȳ = p xy (x x) = σ y (x x), (6) σx p xy since p xy 0 in this instance. Thus, we may also restrict < r xy <. Turning our attention to Figure 3, the oblique linear least squares approximation, L, corresponding to the specified slope s may be expressed as L : y ȳ = m(x x), (7)

5 Geometric construction of regression lines 5 since it is straightforward to show that it passes through the centroid of the data, ( x, ȳ) [0]. L is to be selected so as to minimize the total square oblique deviation n n S := d i = [(x i x ) + (y i y ) ]. (8) i= i= Furthermore, this optimization problem may be reduced [0] to choosing m to minimize S(m) = + s (m s) thereby producing the extremizing slope [0] n [m(x i x) (y i ȳ)], (9) i= m = σ y s p xy. (0) p xy s σx Equations (7) and (0) together comprise a complete solution to the problem of oblique linear least squares approximation. 3 Linear Regression Some of the implications of oblique linear least squares approximation for linear regression are next considered, beginning with the case of coordinate regression. Setting s = 0 in Equation (0) produces regression of x on y: m + x := σ y p xy, m x := 0. () Likewise, letting s in Equation (0) produces regression of y on x: m + y := p xy, m σx y :=. () Unsurprisingly, we see that the coordinate regression lines [7] are special cases of oblique regression. The more substantial case of orthogonal regression is now considered. Setting s = in Equation (0) produces m p xy m (σ y σ x) m p xy = 0, (3) whose roots provide both the best orthogonal regression line m + = (σ y σ x) + (σ y σ x) + 4p xy p xy, (4)

6 6 Brian J. McCartin as well as the worst orthogonal regression line m = (σ y σ x) (σ y σ x) + 4p xy p xy. (5) Thus, orthogonal regression [3] is also a special case of oblique regression. Turning now to the case of λ-regression, set s = λ in Equation (0): m p xy m (σ y λσ x) m λp xy = 0, (6) whose roots provide both the best λ-regression line m + λ = (σ y λσ x) + (σ y λσ x) + 4λp xy p xy, (7) as well as the worst λ-regression line m λ = (σ y λσ x) (σ y λσ x) + 4λp xy p xy. (8) Thus, λ-regression [8] is also a special case of oblique regression. Proceeding to the case of (λ, µ)-regression, set s = µm λ in Equation (0): m µ (p xy µσ x) m (σ y λσ x) m (λp xy µσ y) = 0, (9) whose roots provide both the best (λ, µ)-regression line m + λ,µ = (σ y λσx) + (σy λσx) + 4(p xy µσx)(λp xy µσy), (0) (p xy µσx) as well as the worst (λ, µ)-regression line m λ,µ = (σ y λσx) (σy λσx) + 4(p xy µσx)(λp xy µσy). () (p xy µσx) Thus, (λ, µ)-regression [9] is also a special case of oblique regression. 4 Concentration Ellipse As established in the previous section, all of the garden varieties of linear regression are subsumed under the umbrella of oblique linear least squares approximation. The generalization of the Galton-Pearson-McCartin geometric characterizations of linear regression to oblique linear least squares approximation is now considered.

7 Geometric construction of regression lines 7 Figure 4: Concentration Ellipse Define the concentration ellipse, E, (Figure 4) via (x x) σ x p xy (x x) (y ȳ) (y ȳ) + σ x σ y σ x σ y σy = 4( p xy ), () σxσ y which has the same centroid and second moments about that centroid as does the data [3, pp ] (see [0] for proof). In this sense, it is the ellipse which is most representative of the data points without any a priori statistical assumptions concerning their origin. The concentration ellipse is homothetic to the inertia ellipse with ratio of similitude 4n(σ xσ y p xy) [8]. In order to achieve our characterization, the concept of conjugate diameters of an ellipse is required [4, p. 46]. A diameter of an ellipse is a line segment passing through the center connecting two antipodal points on its periphery. The conjugate diameter to a given diameter is that diameter which is parallel to the tangent to the ellipse at either peripheral point of the given diameter (Figure 5). If the ellipse is described by ax + hxy + by + gx + fy + c = 0 (h < ab), (3) then, according to [0, p. 46], the slope of the conjugate diameter is s = a + h m h + b m, (4)

8 8 Brian J. McCartin A A Figure 5: Conjugate Semidiameters where m is the slope of the given diameter. Consequently, m = a + h s h + b s, (5) thus establishing that conjugacy is a symmetric relation. As a result, these conjugacy conditions may be rewritten symmetrically as b m s + h (m + s) + a = 0. (6) For the concentration ellipse, Equation (), take so that a = σ y, h = p xy, b = σ x, (7) s = σ y p xy m p xy + σ x m ; m = σ y p xy s p xy + σ x s, (8) which implies that s and m satisfy the symmetric conjugacy condition σ x (m s) p xy (m + s) + σ y = 0. (9) Direct comparison of Equations (8) and (0) demonstrates that the best and worst slopes of oblique regression satisfy Equation (9) and are thus conjugate to one another. Hence, the following pivotal result has been established [0]:

9 Geometric construction of regression lines 9 Theorem (Galton-Pearson-McCartin) The best and worst lines of oblique regression are conjugate diameters of the concentration ellipse whose slopes satisfy the conjugacy relation Equation (9). Note that s = µm λ m µ may be rearranged to reproduce the geometric characterization for (λ, µ)-regression of McCartin [9]: (m µ) (s µ) = µ λ. Furthermore, subsequently setting µ = 0 reproduces the geometric characterization for λ-regression of McCartin [8]: m s = λ. Now, setting λ = reproduces Pearson s geometric characterization for orthogonal regression [3]: it is the major axis of the concentration ellipse. Finally, setting λ = 0, reproduces Galton s geometric characterization for coordinate regression [7]: they are the lines displayed in Figure 6 connecting the points of horizontal/vertical tangencies, respectively, to the centroid. 5 Geometric Constructions Previous work on constructing an ellipse from its moments [] will now be exploited in order to construct various lines of regression. This is possible because, as has been demonstrated above, all of theses regression lines are associated with the concentration ellipse. Once the moments x, ȳ, σ x, p xy, σ y have been computed from the discrete data set {(x i, y i )} n i=, the equation of the concentration ellipse, E, is given by the positive definite quadratic form [ x x y ȳ ] [ σ x p xy p xy σ y and the elliptical area may be expressed as ] [ x x y ȳ ] = 4, (30) A E = 4π σ xσ y p xy. (3) The slopes of the principal axes of the concentration ellipse are the roots of the quadratic so that since m + has the same sign as p xy [8]. m + σ x σ y p xy m = 0, (3) m + = (σ y σ x) + (σ y σ x) + 4p xy p xy, (33) m = (σ y σ x) (σ y σ x) + 4p xy p xy, (34)

10 0 Brian J. McCartin The corresponding squared magnitudes of the principal semiaxes [4, p. 58] are the roots of the quadratic Σ 4(σ x + σ y) Σ + 6(σ xσ y p xy) = 0, (35) so that (Σ + ) = [ (σ x + σ y) + (σ x σ y) + 4p xy], (36) (Σ ) = [ (σ x + σ y) (σ x σ y) + 4p xy]. (37) Equation () may be expanded thereby yielding σ y (x x) p xy (x x)(y ȳ) + σ x (y ȳ) = 4(σ xσ y p xy) (38) as the equation of the concentration ellipse. Setting y = ȳ yields the corresponding horizontal points of intersection ( x ± σ x p xy/σ y, ȳ ), (39) while setting x = x yields the corresponding vertical points of intersection ( x, ȳ ± σ y p xy/σ x ). (40) The points of vertical and horizontal tangency are [0], respectively, ( x ± σ x, ȳ ± p ) xy ; σ x ( x ± p ) xy, ȳ ± σ y. (4) σ y With reference to Figure 5, recall that two semidiameters are conjugate to one another if they are parallel to each others tangents []. For this to occur, it is necessary and sufficient that their slopes m and s satisfy the symmetric conjugacy condition, Equation (9). Thus, the semidiameter to a point of vertical/horizontal tangency is conjugate to the semidiameter to a point of vertical/horizontal intersection, respectively. (See Figure 6.) In the succeeding subsections, straightedge-compass constructions of the various lines of regression, given the moments, x, ȳ, σx, p xy, σy, will be developed. In this endeavor, certain primitive arithmetic/algebraic Euclidean constructions [, pp. -], as next described, will be required. Proposition (Primitive Constructions) Given two line segments of nonzero lengths a and b, one may construct line segments of length a + b, a b, a b, a/b and a using only straightedge and compass.

11 Geometric construction of regression lines m y m x + m y + m x Figure 6: Coordinate Regression: y-on-x (Left); x-on-y (Right) 5. Coordinate Regression As previously noted, the semidiameter of a point of vertical intersection is conjugate to the semidiameter to a point of vertical tangency. Thus, by Equations (40) and (4), the vectors 0, σ y p xy/σ x ; σ x, p xy σ x (4) define a pair of conjugate semidiameters emanating from the centroid, ( x, ȳ). (See Figure 6, Left.) Construction (Regression of y-on-x) Given the first and second moments x, ȳ, σ x, p xy, σ y, the best and worst lines of regression of y-on-x are given, respectively, by the pair of conjugate semidiameters x + y, y + y := σ x, p xy σ x ; x y, y y := 0, σ y p xy/σ x which may be constructed using Proposition. The corresponding best and worst mean-squared vertical deviations are given, respectively, by [5, pp. 0-] n ( ) S y + 4(σ = xσy p xy) = σ xσy p xy, (x + y ) σx n ( ) Sy 4(σ = xσy p xy) =. (x y ) (43)

12 Brian J. McCartin Likewise, the semidiameter of a point of horizontal intersection is conjugate to the semidiameter to a point of horizontal tangency. Thus, by Equations (39) and (4), the vectors σ y p xy/σ x, 0 ; p xy σ y, σ y (44) define a pair of conjugate semidiameters emanating from the centroid, ( x, ȳ). (See Figure 6, Right.) Construction (Regression of x-on-y) Given the first and second moments x, ȳ, σ x, p xy, σ y, the best and worst lines of regression of x-on-y are given, respectively, by the pair of conjugate semidiameters x + x, y + x := p xy σ y, σ y ; x x, y x := σ y p xy/σ x, 0 which may be constructed using Proposition. The corresponding best and worst mean-squared vertical deviations are given, respectively, by [5, pp. 0-] n ( ) S x + 4(σ = xσy p xy) = σ xσy p xy, (y x + ) σy n ( ) Sx 4(σ = xσy p xy) =. (yx ) (45) 5. Orthogonal Regression The following beautiful construction due to Carlyle [5, p. 99] and Lill [4, pp ] will be invoked in order to construct the remaining lines of regression. Proposition (Graphical Solution of a Quadratic Equation) Draw a circle having as diameter the line segment joining (0, ) and (a, b). The abscissae of the points of intersection of this circle with the x-axis are the roots of the quadratic x ax + b = 0. A double application of Proposition, in concert with Equations (3) and (35), produces the following construction for the best and worst orthogonal regression lines. Construction 3 (Orthogonal Regression) Given the first and second moments x, ȳ, σ x, p xy, σ y, the best and worst lines of orthogonal regression are given by the pair of principal axes determined as follows:

13 Geometric construction of regression lines 3 (4(σ x +σ y ),6(σ x σ pxy y )) (0,) m (σ y σ )/pxy x + m ((σ y σ x )/pxy, ) (0,) Σ Σ Figure 7: Carlyle-Lill Construction: Directions (Left); Magnitudes (Right). According to Equation (3), apply the Carlyle-Lill construction, Proposition, with (a, b) = ( σ y σx p xy, ) to construct the directions of the principal axes. (See Figure 7, Left.). According to Equation (35), apply the Carlyle-Lill construction, Proposition, with (a, b) = (4(σ x + σ y), 6(σ xσ y p xy)) to construct the squared magnitudes of the principal semiaxes. (See Figure 7, Right.) 3. Apply the -operator, Proposition, to these squared magnitudes in order to construct the magnitudes, {Σ +, Σ }, of the principal semiaxes emanating from the centroid, ( x, ȳ). (See Figure 8.) 4. Denote by x ±, y± the tips of these principal semiaxes lying on the concentration ellipse, E. By Equation (36), the corresponding best mean-squared orthogonal deviation is given by n ( ) S + 4(σ = xσy p xy) (x + ) + (y + = (σxσ y p [ xy) ) (σ x + σy) + (46) (σx σy) + 4pxy], while, by Equation (37), the corresponding worst mean-squared orthogonal deviation is given by n ( ) S 4(σ = xσy p xy) (x ) + (y = (σxσ y p [ xy) ) (σ x + σy) (47) (σx σy) + 4pxy].

14 4 Brian J. McCartin Σ + Σ Figure 8: Orthogonal Regression Equations (46) and (47) may be combined and further simplified to read 6 λ-regression n ( ) S ± = [ (σx + σy) (σx σy) + 4pxy]. (48) Turning to the case of λ-regression corresponding to the slope s = λ m in Figure 3, it is shown in [8] that the x-expansion(λ > )/compression(λ < ) ξ := λ x σ ξ = λ σ x, p ξy = λ p xy, ˆm = m λ (49) transforms λ-regression into orthogonal regression so that the slopes of the best and worst λ-regression lines are, by Equation (3), roots of the quadratic m + λσ x σ y p xy m λ = 0. (50) Specifically, by Equation (33), the slope of the best λ-regression line is m + λ = (σ y λσ x) + (σ y λσ x) + 4λp xy p xy, (5) while, by Equation (34), the slope of the worst λ-regression line is m λ = (σ y λσ x) (σ y λσ x) + 4λp xy p xy. (5)

15 Geometric construction of regression lines 5 Σ λ Σ λ + Figure 9: λ-regression By Equation (35), the squared magnitudes of the corresponding conjugate semidiameters are the roots of the quadratic Σ λ 4(λσ x + σ y) Σ λ + 6λ(σ xσ y p xy) = 0, (53) so that (Σ + λ ) = [ (λσ x + σ y) + (λσ x σ y) + 4λp xy], (54) (Σ λ ) = [ (λσ x + σ y) (λσ x σ y) + 4λp xy]. (55) Thus, Construction 3 may be modified to accommodate λ-regression. Construction 4 (λ-regression) Given the first and second moments x, ȳ, σ x, p xy, σ y, the best and worst lines of λ-regression are given by the pair of conjugate semidiameters determined as follows:. According to Equation (50), apply the Carlyle-Lill construction, Proposition, with (a, b) = ( σ y λσx p xy, λ) to construct the directions of the conjugate semidiameters.. According to Equation (53), apply the Carlyle-Lill construction, Proposition, with (a, b) = (4(λσ x + σ y), 6λ(σ xσ y p xy)) to construct the squared magnitudes of the conjugate semidiameters. 3. Apply the -operator, Proposition, to these squared magnitudes in order to construct the magnitudes, {Σ + λ, Σ λ }, of the conjugate semidiameters emanating from the centroid, ( x, ȳ). (See Figure 9.)

16 6 Brian J. McCartin 4. Denote by x ± λ, y± λ the tips of these conjugate semidiameters lying on the concentration ellipse, E. By Equation (46), the corresponding best mean-squared λ-deviation is given by n ( ) S λ + = max (, λ ) 4λ(σ xσy p xy) λ(x + λ ) + (y λ + ) max (, λ ) λ(σ = xσy p [ xy) (λσ x + σy) + (λσx σy) + 4λpxy] = max (, λ ) [ (λσ x + σy) ] (λσx σy) + 4λp xy,(56) while, by Equation (47), the corresponding worst mean-squared λ-deviation is given by n ( ) Sλ = max (, λ ) 4λ(σ xσy p xy) λ(x λ ) + (yλ ) max (, λ ) λ(σ = xσy p [ xy) (λσ x + σy) (λσx σy) + 4λpxy] = max (, λ ) [ (λσ x + σy) + ] (λσx σy) + 4λp xy.(57) 7 (λ, µ)-regression Turning to the case of (λ, µ)-regression corresponding to the slope s = µm λ m µ in Figure 3, it is shown in [9] that the linear mapping ξ := λ µ x, η := µ x + y ˆm = m µ λ µ, (58) σ ξ = (λ µ ) σ x, σ η = µ σ x µ p xy +σ y, p ξη = λ µ (p xy µσ x) (59) transforms (λ, µ)-regression into orthogonal regression so that the slopes of the best and worst (λ, µ)-regression lines are, by Equation (3), roots of the quadratic m + λσ x σy m λp xy µσy p xy µσx p xy µσx = 0. (60) Specifically, by Equation (33), the slope of the best (λ, µ)-regression line is m + λ,µ = (σ y λσx) + (σy λσx) + 4(p xy µσx)(λp xy µσy), (6) (p xy µσx)

17 Geometric construction of regression lines 7 Σ λ,µ + Σ λ,µ Figure 0: (λ, µ)-regression while, by Equation (34), the slope of the worst (λ, µ)-regression line is m λ,µ = (σ y λσx) (σy λσx) + 4(p xy µσx)(λp xy µσy). (6) (p xy µσx) By Equation (35), the squared magnitudes of the corresponding conjugate semidiameters are the roots of the quadratic Σ λ,µ 4 [ (λ µ )σ x + (µ σ x µp xy + σ y) ] Σ λ,µ +6(λ µ ) [ σ x(µ σ x µp xy + σ y) (p xy µσ x) ] = 0, (63) so that (Σ + λ,µ ) = [ (λ µ )σ x + (µ σ x µp xy + σ y) ] + [(λ µ )σ x (µ σ x µp xy + σ y) ] + 4(λ µ )(p xy µσ x), (64) (Σ λ,µ ) = [ (λ µ )σ x + (µ σ x µp xy + σ y) ] [(λ µ )σ x (µ σ x µp xy + σ y) ] + 4(λ µ )(p xy µσ x). (65) Thus, Construction 3 may be modified to accommodate (λ, µ)-regression.

18 8 Brian J. McCartin Construction 5 ((λ, µ)-regression) Given the first and second moments x, ȳ, σ x, p xy, σ y, the best and worst lines of (λ, µ)-regression are given by the pair of conjugate semidiameters determined as follows:. According to Equation (60), apply the Carlyle-Lill construction, Proposition, with (a, b) = ( σ y λσx µσy λ p xy µσ, x p xy µσ ) to construct the directions of the x conjugate semidiameters.. According to Equation (63), apply the Carlyle-Lill construction, Proposition, with (a, b) = (4 [ (λ µ )σ x + (µ σ x µp xy + σ y) ], 6(λ µ ) [ σ x(µ σ x µp xy + σ y) (p xy µσ x) ] ) to construct the squared magnitudes of the conjugate semidiameters. 3. Apply the -operator, Proposition, to these squared magnitudes in order to construct the magnitudes, {Σ + λ,µ, Σ λ,µ }, of the conjugate semidiameters emanating from the centroid, ( x, ȳ). (See Figure 0.) 4. Denote by x ± λ,µ, y± λ,µ the tips of these conjugate semidiameters lying on the concentration ellipse, E. By Equation (46), the corresponding best mean-squared (λ, µ)-deviation is given by ( ) n S + λ,µ = max (, λ ) 4(λ µ ) [ σx(µ σx µp xy + σy) (p xy µσx) ] λ(x + λ,µ ) µx + λ,µ y+ λ,µ + (y+ λ,µ ) = max (, λ ) (λ µ ) [ σ x(µ σ x µp xy + σ y) (p xy µσ x) ] (λ µ )σ x + (µ σ x µp xy + σ y) + [(λ µ )σ x (µ σ x µp xy + σ y) ] + 4(λ µ )(p xy µσ x) = max (, λ ) [(λ µ )σ x + (µσ x µpxy + σy) [ (λ µ )σ x (µσ x µpxy + σy) ] + 4(λ µ )(pxy µσ x) ] while, by Equation (47), the corresponding worst mean-squared (λ, µ)-deviation is given by ( ) n S λ,µ = max (, λ ) 4(λ µ ) [ σx(µ σx µp xy + σy) (p xy µσx) ] λ(x λ,µ ) µx λ,µ y λ,µ + (y λ,µ ), = max (, λ ) (λ µ ) [ σ x (µ σ x µpxy + σ y ) (pxy µσ x )] (λ µ )σ x + (µ σ x µpxy + σ y ) [(λ µ )σ x (µ σ x µpxy + σ y )] + 4(λ µ )(p xy µσ x ) [(λ µ )σ x + (µσ x µpxy + σy) + [ (λ µ )σ x (µσ x µpxy + σy) ] + 4(λ µ )(pxy µσ x) ] = max (, λ ).

19 Geometric construction of regression lines 9 8 Conclusion In the foregoing, Euclidean constructions were supplied for various lines of regression. These included lines of coordinate regression [7], orthogonal regression [3], λ-regression [8] and (λ, µ)-regression [9]. The adhesive which bound together these geometric constructions was provided by the concentration ellipse [3]. This unified approach was, in turn, a direct consequence of the Galton-Pearson-McCartin Theorem [0, Theorem ] which establishes that, as special cases of oblique regression [0], all of the associated best and worst regression lines form pairs of conjugate diameters of the concentration ellipse. Acknowledgements. I would like to thank my devoted wife Mrs. Barbara A. McCartin for her indispensable aid in constructing the figures. References [] F. S. Acton, Analysis of Straight-Line Data, Dover, New York, 966. [] R. Courant and H. Robbins, Geometrical Constructions, Chapter III in What Is Mathematics?, Oxford University Press, New York, 94, [3] H. Cramér, Mathematical Methods of Statistics, Princeton University Press, Princeton, New Jersey, 946. [4] L. E. Dickson, Constructions with Ruler and Compasses; Regular Polygons, Chapter in Monographs on Topics of Modern Mathematics Relevant to the Elementary Field (J. W. A. Young, Editor), Dover, New York, 955, [5] H. Eves, An Introduction to the History of Mathematics, Sixth Edition, HBJ-Saunders, New York, 990. [6] R. W. Farebrother, Fitting Linear Relationships: A History of the Calculus of Observations , Springer, New York, [7] F. Galton, Family Likeness in Stature, with Appendix by J. D. H. Dickson, Proc. Roy. Soc. London, 40 (886), [8] B. J. McCartin, A Geometric Characterization of Linear Regression, Statistics, 37 (003), no.,

20 30 Brian J. McCartin [9] B. J. McCartin, The Geometry of Linear Regression with Correlated Errors, Statistics, 39 (005), no., -. [0] B. J. McCartin, Oblique Linear Least Squares Approximation, Applied Mathematical Sciences, 4 (00), no. 58, [] B. J. McCartin, A Matrix Analytic Approach to the Conjugate Diameters of an Ellipse, Applied Mathematical Sciences, 7 (03), no. 36, [] B. J. McCartin, Geometric Construction of an Ellipse from its Moments, Applied Mathematical Sciences, 0 (06), no. 3, [3] K. Pearson, On Lines and Planes of Closest Fit to Systems of Points in Space, Philosophical Magazine Series 6, (90), [4] G. Salmon, A Treatise on Conic Sections, Sixth Edition, Chelsea, New York, 954. [5] W. M. Smart, Combination of Observations, Cambridge University Press, Cambridge, 958. [6] P. Sprent, Models in Regression, Methuen, London, 969. Received: January, 06; Published: March 30, 06