A METHOD FOR DETECTING OUTLIERS IN FUZZY REGRESSION

Transcription

1 OPERATIONS RESEARCH AND DECISIONS No Barbara GŁADYSZ* A METHOD FOR DETECTING OUTLIERS IN FUZZY REGRESSION In ths artcle we propose a method for dentfyng outlers n fuzzy regresson. Outlers n a sample may have an mportant nfluence on the form of the regresson equaton. For ths reason there s great scentfc nterest n ths ssue. The method presented s analogous to the method of fndng outlers based on the studentzed dstrbuton of resduals. In order to dentfy outlers, regresson models are constructed wth an addtonal explanatory varable for each observaton. Next, the sgnfcance of a fuzzy regresson coeffcent s analysed consderng ths addtonal explanatory varable. Illustratve examples are presented. Keywords: fuzzy regresson, outlers, possblty theory 1. Introducton In 1965 Zadeh proposed hs possblty theory, [12]. We wll present the basc notons of ths theory. Frst, we wll present the concept of a fuzzy varable. Let X be a sngle valued varable whose value s not precsely nown. The possblty dstrbuton for X s a normal, quas concave and upper sem contnuous functon μ X : R [,1], see X [1], [13]. The value μ X (x) for x R denotes the possblty of the event that the fuzzy varable X taes the value of x. We denote t as follows: μ ( x ) = Pos( X = x). (1) An L L fuzzy varable s one whose possblty dstrbuton can be expressed n the followng form: * Insttute of Organzaton and Management, Department of Management, Wrocław Unversty of Technology, ul. Smoluchowsego 25, Wrocław, e-mal: barbara.gladysz@pwr.wroc.pl

2 26 B. GŁADYSZ x mx μ (x) = L (2) lx where: m x s a constant and L s a contnuous, symmetrc functon whch attans the value 1 for argument and s ncreasng n the doman of non-negatve numbers (m x s called a centre of fuzzy varable X, l x a spread of fuzzy varable X). For a gven fuzzy varable X and a gven λ, the λ-level s defned as the closed nterval [X] λ = {x : μ(x) λ}. Examples of such L(x) functons are L(x) = max{, 1 x p }, L(x) = (1 + x p ) 1, L(x) = exp( x p ). An L L fuzzy varable wll be denoted as X = (m X, l X ). Consder two fuzzy varables X, Y wth possblty dstrbutons μ X (x), μ Y ( y), respectvely. The possblty dstrbutons of the fuzzy varables Z = X + Y and V = XY are defned by means of Zadeh s extenson prncple [12] as follows: μ Z z= x+ y ( z) = sup (mn( μ ( x), μ ( y))), (3) μ V v= xy X Y ( v) = sup (mn( μ ( x), μ ( y))). (4) We are nterested n comparng X to Y,.e. we want to characterze the possblty of the event that the value taen by X wll be greater (not smaller) than the value taen by Y. To descrbe the possblty of these events Dubos and Prade proposed the followng ndces [1]: X Y Pos( X Y ) = sup (mn μ ( x), μ ( y))), (5) x y X Pos( X > Y ) = sup sup (mn μ ( x),1 μ ( y))). (6) x y We want now to characterse the possblty of the event that the realsaton of the varable X wll be equal to the varable Y. Ths possblty s defned as follows [1]: x X Pos( X = Y ) = mn( Pos( X Y ), Pos( Y X )). (7) The measure of necessty of the event that the varable X s dfferent to the varable Y s Both ndces tae a value from the nterval [, 1]. Y Nec ( X Y ) = 1 Pos( X = Y ). (8) Y

3 A method for detectng outlers n fuzzy regresson Detecton of outlers n fuzzy regresson A fuzzy regresson s a lnear dependence Y ˆ = A + A 1 X A X, (9) n whch the dependent varable Y, the ndependent varables X 1,..., X and the regresson coeffcents A, A 1,..., A are fuzzy varables. In specal cases the regresson coeffcents or the explanatory varables can be real numbers. There are many methods of estmatng a fuzzy regresson equaton. A revew of ths feld can be found n [4]. Let us consder the fuzzy regresson model whch was ntroduced by TANAKA et al. [11]. Let us assume that we have observatons (y, x ), y, x R, where y s an observaton of a trangular fuzzy varable Y, = 1,..., n. The model for estmatng the coeffcents of fuzzy regresson (9) s formulated as: wth the constrants: A = l x mn (1) y a x + la x L ( λ ), for =,...,, (11) = = y a x la x L ( λ ), for =,...,, (12) = l = A for =,...,, (13) L( x) = max{,1 x }. (14) Y Y The form of the optmal soluton to the fuzzy regresson problem defned by (1) (14) depends on the assumed λ-level. If we now the optmal soluton for a gven λ-level, we can fnd the optmal soluton for other λ-levels. Namely, f the ~ coeffcents A = ( a, la ) gve the optmal soluton for the λ-level, then the optmal ~ L ( λ) soluton for the λ -level s A, = a l, a ( ) L λ =,...,. Let us assume that we have fuzzy observatons [Y ] λ = [ L ( λ), R ( λ)] of the dependent varable for a gven λ-level, = 1,..., n. Let us defne Y ˆ = [ ( λ), [ ] λ L Y ˆ

4 28 B. GŁADYSZ ˆ R ( λ)] to be estmators of [Y ] λ, = 1,..., n. The upper fuzzy regresson model s Y gven by the followng: wth the constrants n = 1 [ R Yˆ Yˆ ( λ ) L ( λ)] mn (15) ˆ L Y Y L ( λ) ( λ) for = 1,..., n, (16) Y ˆ R ( λ) R ( λ) for = 1,..., n. (17) Y Many papers consder the ssue of dentfyng outlers n the models of fuzzy regresson gven by (1) (14) and (15) (17). Here, we present some chosen methods n whch outlers are dentfed by usng soft boundares n the regresson model: see [2], [7], [8]. To dentfy outlers, PETERS [8] uses soft lmtatons on the obectve functon (1) and constrants (11) (12), thus constructng the followng model wth the constrants: n 1 Λ = Λ max n 1 = (18) n la x = 1 = ( 1 Λ) p d, (19) A = = ( 1 Λ ) p + a x + l x y for = 1,..., n, (2) A = = ( 1 Λ ) p a x + l x y for = 1,..., n, (21) A for =,...,, (22) l Λ 1 for = 1,..., n, (23) L ( λ) = 1. (24) where p s the wdth of the tolerance nterval for the obectve functon and p s the tolerance nterval for the observaton y.

5 A method for detectng outlers n fuzzy regresson 29 The parameters p and p, = 1,..., n, must be gven all together. The parameter d descrbes the tolerance for the value of the obectve functon. The suggested value s d =, whch means that the preferences are for a model of crsp type n la x =. In the model defned by (14) (24) the varable Λ descrbes = 1 = a compromse between the obectve functon (mnmzaton of the wdth of regresson coeffcents) and lmtng the number of wrong observatons. Such a formulaton of the problem mples that a trade-off between the number of rght and wrong observatons wll be acheved. Each observaton has an nfluence on the obectve functon of weght 1/n. We may also consder the weghted average Λ = n w = 1 n Λ, where Defnton 1 [8]. An outler s an observaton whch s charactersed by a low value of Λ. ÖZELKAN and DUCKSTEIN [7] propose a method for dentfyng outlers usng a multobectve fuzzy regresson model n whch, besde the crteron of mnmsng the sum of the wdths of the estmators of the dependent varable, the crteron of mnmsng the sum of the devatons of the observatons from the regresson equaton. Let us assume that we have fuzzy observatons of the dependent varable [Y ] λ = [ L Y ( λ), R ( λ)] Y for a gven λ-level, = 1,..., n. Let ˆ ] [ Y λ = [ L ˆ ( λ), R ( )] Y ˆ λ. The Y two-crteron model for detectng outlers wth soft boundares proposed n [7] s n = 1 = 1 1 ˆ Y ˆ Y = 1 w = 1. [ R ( λ) L ( λ)] mn, (25) wth the constrants: n = 1 p L p R [ ε + ε ] mn (26) 1 L L n Y ˆ ( λ ) Y ( λ) ε L for = 1,...,, (27) 1 1 RY ( λ ) R ˆ ( λ)] ε R for = 1,..., n, (28) 1 Y ε L, ε for = 1,..., n (29) R where p s an nteger. In the model gven by (25) (29) both of the constrants (27) (28) are actve, although not all observatons are used n the determnaton of the values of the regresson coeffcents. Non-domnated Pareto-optmal solutons are consdered as a soluton of ths model.

6 3 B. GŁADYSZ Defnton 2 [7]. An outler s an observaton whch s charactersed by a sgnfcantly large value of ε L ( ε ). R Özelan and Ducsten proved the followng property. Lemma 1 [7] The models of Tanaa et al., Peters and the classcal regresson model are partcular cases of the model gven by (25) (29). Another multcrtera model for the dentfcaton of outlers was proposed by n p p GŁADYSZ and KUCHTA [2], n whch the crteron [ ε L + ε R ] mn s replaced by p L = 1 Me [ ε ε ] mn. (3) + p R In ths artcle we wll propose a method for detectng outlers usng n regresson models each wth an addtonal explanatory varable correspondng to a gven observaton of the dependent varable. An observaton s consdered to be an outler f the regresson component correspondng to the approprate addtonal varable has a sgnfcant nfluence on the predcted value of the dependent varable. 3. Method of dentfyng outlers When we estmate the regresson parameters ˆ 1 y = a + a1x a x (31) usng the classcal least squares method, t may happen that one or more observatons have an sgnfcant nfluence on the values of the regresson coeffcents. Such observatons are called outlers. The basc queston of data analyss where outlers occur s the followng queston: whch observatons are outlers? There are many methods of dentfyng outlers proposed n the lterature, see e.g. [3], [9]. We wll descrbe one of them. It s a method that analyses the changes n the predctons of the dependent varable caused by the excluson of a partcular observaton from a data set. The analyss s carred out based on resduals from predcton and studentzed resduals. Both nds of resduals can be defned usng a bnary varable and the fact that a studentzed resdual s the resdual from predcton dvded by ts standard error [5]. To determne a studentzed resdual for the -th observaton, we construct a regresson model based on the set of all observatons, addng to the set of explanatory varables a varable d of the form:

7 A method for detectng outlers n fuzzy regresson 31 d 1 = for for -th other observaton. observatons The regresson coeffcent correspondng to the varable d s called a predctve resdual and the t-student statstc s called a studentzed resdual. Ths statstc has a t-student dstrbuton wth (n 1) degrees of freedom. If the studentzed resdual (t-student statstc) belongs to the crtcal set, the -th observaton s nferred to be an outler. In ths artcle we propose a method of detectng outlers n fuzzy regresson analogous to the method descrbed above for classcal regresson. Let us consder the fuzzy regresson (9) Y ˆ = A + A 1 X A X. To verfy whether the -th observaton s an outler, we buld a fuzzy regresson model based on the set of all observatons, addng to the set of explanatory varables a varable d descrbed by formula (32). Hence, we determne the coeffcents of fuzzy regresson for the model Y ˆ = A... + (32) + A1 X1 + + A X ADd. (33) Let us characterse the regresson model (33) as a sum of two components: Y ˆ = Yˆ + A D d, (34) where Y ˆ = A + A 1 X A X. Next, we analyse whether the second component A D d of the model (34) has a sgnfcant nfluence on the predcton of the value of the dependent varable for the -th observaton. If ts nfluence s sgnfcant, the -th observaton s classfed as an outler. Let us defne an outler n the followng way. Defnton 3. The -th observaton s classfed as an outler when the followng occurs for the model (34) Pos ˆ. (35) ( Y = Yˆ ) λ The parameter λ s specfed subectvely by a decson-maer. Inequalty (35) s equvalent to the nequalty Nec ( Yˆ Yˆ λ, (36) ) 1 whch means that the necessty that the predcton Yˆ s not equal to ˆ Y s not lower than λ. Let us formulate a lnear model of coeffcents estmaton n fuzzy regresson (33) for the case of real data. Its form wll be the followng

8 32 B. GŁADYSZ wth the constrants: A = l x + l mn (37) A D y a x + = l A = x L ( λ) for = 1,..., n,, (38) y a x = l A = x L ( λ) for = 1,..., n,, (39) y a x + la x L ( λ) + ad + l L ( λ) for =, (4) = = A D y a x la x L ( λ) + ad l L ( λ) for =, (41) = = A D la, l for =,...,, (42) A D L( x) = max{,1 x }. (43) In the model gven by (37) (43), the constrants (4) (41) for observaton are soft boundary constrants. So the method of dentfyng outlers proposed n ths artcle nvolves relaxaton of the constrants. A relaxng varable s a fuzzy varable. An observaton s treated as an outler f the correspondng relaxng varable has a sgnfcant nfluence on predcton. A lnear model for estmatng regresson coeffcents can be constructed n a smlar way to the regresson model gven by (25) (29) for fuzzy data. 4. Examples To llustrate the method of detectng outlers proposed n ths artcle we wll present ts mplementaton for a set of real and fuzzy data Example 1 The data are presented n Table 1. All the observatons are real numbers. The ffth observaton s an outler.

9 A method for detectng outlers n fuzzy regresson 33 Table 1. Data for Example x y Source: [8]. Let us consder the followng fuzzy regresson ˆ = A + A x. (44) Y 1 For the data from Table 1, Tanaa s regresson model (44) taes the form Y ˆ = 2.25, 2.4) + (.95, )x. We wll determne ten regresson models (44) each wth one addtonal varable d for observaton = 1,..., 1. Y ˆ = A + A 1 x + A D d. (45) The results of such estmaton are presented n Table 2. For = 1,..., 8, 5 the same regresson models were obtaned n the form of Yˆ = (2.25, 2.4) + (.95, )x + (, )d. In these models the coeffcent of the varable A D = (, ). So the element A D d n the model (45) for observatons = 1,..., 8, 5 s nsgnfcant. Table 2. Regresson coeffcents for the model (45) and the sgnfcance of the component A D d A A 1 A D Pos ( Yˆ = Y ) ˆ Nec Yˆ ( ) Yˆ 1 (2.25, 2.4) (.95, ) (, ) 1 2 (2.25, 2.4) (.95, ) (, ) 1 3 (2.25, 2.4) (.95, ) (, ) 1 4 (2.25, 2.4) (.95, ) (, ) 1 5 (.175,.325) (,975,.325) (4.35, ) 1 6 (2.25, 2.4) (.95, ) (, ) 1 7 (2.25, 2.4) (.95, ) (, ) 1 8 (2.25, 2.4) (.95, ) (, ) 1 9 (2.25, 2.4) (.95, ) (, ) 1 1 (2.25, 2.4) (.95, ) (, ) 1 Source: Author s own wor. For the ffth observaton we obtan the followng regresson equaton Y ˆ = (.175,.325) + (.975,.325) x + (4.35, ) d 5 (46)

10 34 B. GŁADYSZ where the measure of sgnfcance of the element A D d 5 s Nec ( Yˆ Y ˆ ) = 1. So the element (4.35, )d 5 n the model (46) sgnfcantly nfluences the predcted value of the dependent varable. The ffth observaton has been detected as an outler. The results of estmaton are presented n Fgures 1 and 2. Tanaa s regresson model calculated for the complete data set s presented n Fgure 1. It should be noted that the outler (ffth observaton) mples a bg wdth usng Tanaa s model. Fgure 2 shows the model estmated after excludng the ffth observaton from the data set. Fg. 1. Observatons and fuzzy regresson (44) for the complete data set (λ =, λ = 1) Fg. 2. Observatons and fuzzy regresson (44) for the data wthout the ffth observaton (λ =, λ = 1) If we construct regresson models usng: the Peters model, the Özelan Ducsten model and the Gładysz Kuchta model, for the data from Table 1 we get the followng results:

11 A method for detectng outlers n fuzzy regresson 35 the Peters model ˆ~ Y = (.229,.271) + (.974,.25)x, Özelan and Ducsten s multcrteron model ˆ~ Y = (.258,.48) + (.967,.17)x, Gładysz and Kuchta s multcrteron model ˆ~ Y = (.258,.48) + (.967,.17)x. The data and ther weghts Λ (the Peters model) and the values of the relaxng varables ε L, ε R are presented n Table 3. The results from the Gładysz Kuchta model are dentcal to those obtaned from the Özelan Ducsten model. For the ffth observaton (the outler) the weght Λ 5 s relatvely small compared to the other weghts and the value of the relaxaton varable ε R5 s much bgger than the value of the relaxaton varable for the other observatons. Table 3. Indcators of outlers for Example 1 x y Λ ε L ε R Source: [7]. All these three methods dentfy the ffth observaton as an outler, as does the method proposed n ths artcle Example 2 Let us consder the data from [1, ], presented n Table 4. Both the observatons of the dependent varable and the ndependent varable are symmetrc trangular fuzzy numbers (L(x) = max{, 1 x }). The ffth observaton s an outler.

12 36 B. GŁADYSZ Let us consder the followng fuzzy regresson model ˆ = A + a X, (47) Y 1 where the coeffcent A s a fuzzy coeffcent and the coeffcent a 1 of the fuzzy explanatory varable s a real coeffcent. Table 4. Data for Example 2 x l x y l y Source: [1]. To detect outlers, let us ntroduce addtonal varables d accordng to formula (32) and construct regresson models: Y ˆ = A + a 1 X + A D d. (48) Table 5. Regresson coeffcents for model (48) and sgnfcance of the component A D d A A 1 A D Pos ( Yˆ = Y ) ˆ Nec Yˆ ( ) Yˆ 1 (1.5, 6.85).4 (.3, ) (9.7, 6.86).43 (.3, ) (9.7, 6.86).44 (.3, ) (9.7, 6.86).44 (.3, ) (3.75, 1.5).5 (1.5,.25) (8.45, 6.65).6 (.5, ) (9.7, 6.86).44 (2.56, ) (9.7, 6.86).44 (.91, ).94.6 Source: Author s own wor. The coeffcents of regresson n (48) for = 1,..., n are presented n Table 5. Table 5 also shows the measures for the event that the component A D d n model (48) s not sgnfcant. Let us assume that λ =.6. It can be observed that the component A D d

13 A method for detectng outlers n fuzzy regresson 37 has an mportant nfluence on predctng the dependent varable only for the ffth observaton. For the other observatons Nec ( Yˆ Y ˆ ) 1 λ. So the ffth observaton has been detected as an outler. The results of estmaton are shown n Fgures 3 and 4. Fgure 3 shows the fuzzy regresson model for the complete set of data and Fgure 4 llustrates the estmated model after excludng the ffth observaton from the data set. Fg. 3. Observatons and fuzzy regresson (47) for the complete data set (λ = and λ = 1) Fg. 4. Observatons and fuzzy regresson for model (47) for the complete data set (λ = and λ = 1)

14 38 B. GŁADYSZ When we construct the Özelan Ducsten model and the Gładysz Kuchta model for the data from Table 4, we obtan the followng models: Özelan Ducsten multcrteron model ˆ~ ~ Y = (3.944,.278) +.444X, Gładysz Kuchta multcrteron model ˆ~ ~ Y = (3.19, 1.4) +.6X. The relaxaton varables ε L, ε R are presented n Table 6. Table 6. Indcators of outlers for Example 2 Özelan Ducsten Gładysz Kuchta ε L ε R ε L ε R Source: Author s own wor. For the ffth observaton the value of the relaxaton varable s much bgger than the values of the relaxng varables for other observatons, both for the Özelan Ducsten model and the Gładysz Kuchta model. Both methods dentfed the ffth observaton as an outler, as does the method presented n ths artcle. 5. Conclusons In ths artcle we have proposed a method of detectng outlers n fuzzy regresson. In order to dentfy outlers, regresson models are constructed each wth an addtonal explanatory varable correspondng to a gven observaton. Next, the sgnfcance of the fuzzy regresson coeffcent correspondng to the addtonal explanatory varable s analysed. If ts nfluence s sgnfcant, ths observaton s treated as an outler. Two examples were presented. In the frst example the observatons are real numbers (realsatons of fuzzy numbers). In the second example the data are fuzzy numbers.

15 A method for detectng outlers n fuzzy regresson 39 References [1] DUBOIS D., PRADE H., Possblty Theory: An Approach to Computerzed Processng of Uncertanty, Plenum Press New Yor, [2] GŁADYSZ B., KUCHTA D., Mult crtera programmng n robust estmaton for nterval data, Foundaton of Computng and Decson Scences, 28, 33 (2), [3] HUBER P.J., Robust statstcs, John Wley & Sons, Hoboen, New Jersey, 24. [4] KACPRZYK J., FEDRIZZI M., Fuzzy Regresson Analyss, Omntech Press Warsaw and Physca-Verlag Helderberg, Warsaw, [5] MADDALA G.S., Eonometra, PWN, Warszawa, 26. [6] NASRABADI M.M., NASRABADI E., NASRABADI A.R., Fuzzy lnear regresson analyss: A multobectve programmng approach, Appled Mathematcs and Computaton, 25, 163, [7] ÖZELKAN E.C., DUCKSTEIN L., Mult-obectve fuzzy regresson: a general framewor, Computers and Operaton Research, 22, 27, [8] PETERS G., Fuzzy lnear regresson wth fuzzy ntervals, Fuzzy Sets and Systems, 1994, 63, [9] ROUSSEEUW P., LEROY A.M., Robust regresson and outler detecton, John Wley & Sons, [1] SAKAWA M., YANO H., Multobectve fuzzy lnear regresson analyss for fuzzy nput-output data, Fuzzy Sets and Systems, 1992, 47, [11] TANAKA H., UEJIMA S., ASAI K., Lnear regresson analyss wth fuzzy model, IEEE Transacton on Systems Man and Cybernetcs, 1982, 12, [12] ZADEH L.A., Fuzzy Sets, Informaton and Control, 1965, (8), [13] ZADEH L.A., Fuzzy sets as a bass of theory of possblty, Fuzzy Sets and Systems, 1978, (1), 3 28.