COMPUTING CONFIDENCE INTERVALS FOR OUTPUT ORIENTED DEA MODELS: AN APPLICATION TO AGRICULTURAL RESEARCH IN BRAZIL

Transcription

1 COMPUTING CONFIDENCE INTERVALS FOR OUTPUT ORIENTED DEA MODELS: AN APPLICATION TO AGRICULTURAL RESEARCH IN BRAZIL Gerald da Silva e Suza Miria Oliveira de Suza Eliae Gçalves Gmes Brazilia Agricultural Research Crprati (Embrapa) SGE Parque Estaçã Bilógica, Av. W3 Nrte fial, Asa Nrte, , Brasília, DF, Brazil {gerald.suza, miria.suza, eliae.gmes}@embrapa.br ABSTRACT We defie ad mdel the research prducti at Embrapa, the mar Brazilia istituti respsible fr applied agricultural research. The mai theretical framewrk is Data Evelpmet Aalysis DEA. We explre the ecmic iterpretati ad the statistical prperties f these mdels t cmpute cfidece itervals fr utput rieted efficiecy measuremets, based a parametric flexible mdel, defied by the trucated rmal distributi. These results prvide a better isight the efficiecy classificati ad allw cmparis amg the DMUs ivlved i the evaluati prcess takig it accut iefficiecy radm variati. Key wrds: Cfidece itervals; DEA; Agricultural research. Mai area: DEA RESUMO Neste artig é defiid e mdelad sistema de prduçã de pesquisa da Embrapa, a mair istituiçã brasileira de pesquisa agrpecuária. A ferrameta teórica pricipal é Aálise Evltória de Dads DEA. Explram-se a iterpretaçã ecômica e as prpriedades estatísticas desses mdels, para calcular itervals de cfiaça para as medidas de eficiêcia rietadas a prdut. Aqueles sã baseads em um mdel paramétric flexível defiid pela distribuiçã rmal trucada. Esses resultads geram melhres etedimets sbre a classificaçã das medidas de eficiêcia e permitem cmparações etre as DMUs evlvidas prcess de avaliaçã, csiderad a variaçã aleatória da ieficiêcia. Palavras chave: Itervals de cfiaça; DEA; Pesquisa agrpecuária. Área de classificaçã pricipal: DEA XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 57

2 . Itrducti It is f imprtace t admiistratrs f research istitutis t have at their dispsal measures ad prcedures makig feasible a evaluati f the quatum f prducti, as well as the techical efficiecy f the prducti prcess f their istitutis. I times f cmpetiti ad budget cstraits a research istituti eeds t kw by hw much it may icrease its prducti, with quality, withut absrbig additial resurces. The quatitative mitrig f the prducti prcess allws fr a effective admiistrati f the resurces available ad the bservati f predefied research patters ad gals. I this ctext, the Brazilia Agricultural Research Crprati (Embrapa) develped a prducti mdel based the iput-utput data f its research uits. The theretical framewrk fr this mdel is the aalysis f prducti frtiers, kw as Data Evelpmet Aalysis (DEA). Several uses are made f the efficiecy measuremets by Embrapa s admiistrati. Thse iclude mitrig f prducti targets, resurce allcati ad rewardig. Admiistrative actis regardig a give rakig f uits will have mre impact if they take it accut the stchastic variati imbedded i the measuremets f prducti variables. This leads t the csiderati f statistical prducti mdels, frm which e may ifer statistical prperties fr efficiecy estimates. Fr the stchastic frtier aalysis, with prper parametric specificatis f the prducti r cst fuctis, this is a atural prcess, as ca be see i Kumbhakar ad Lvell (2000) ad Celli et al. (2005). Fr the parametric frtier appraches iduced by classical DEA (Celli et al., 2005) r the Free Dispsal Hull (FDH) f Depris et al. (984), sme techical issues arise ad a prper apprach has t be put frward t guaratee the derivati f sud statistical results. This is the lie f wrk carried ut by Baker (993), Baker ad Nataraa (2004, 2008), Simar ad Wils (2004, 2007), Darai ad Simar (2007), Suza ad Staub (2007) ad Suza et al. (2009a). I this article we cmbie the results f Baker (993), Baker ad Nataraa (2008), Simar ad Wils (2007) ad Suza ad Staub (2007) t cme up with cfidece itervals fr DEA efficiecy measuremets, rbust relative t prducti fucti chices ad efficiecy distributis withi reas. These itervals are mre appealig tha thse geerated by the btstrap f Simar ad Wils (2004, 2007) that may prduce uexpected results, like e uit beig regarded as iefficiet after beig bserved as a bechmark r geeratig cfidece limits that d t iclude bserved efficiecy measuremets. Our discussi prceeds as fllws. I Secti 2 we review the ccepts leadig t the mdels fr which e may view DEA estimates as parametric maximum likelihd, ad fr which statistical prperties may be derived fr efficiecy estimates. I Secti 3 we review Embrapa s prducti mdel. Secti 4 deals with the statistical results f ur applicati ad, fially, i Secti 5 we summarize ur fidigs. 2. Data Evelpmet Aalysis Prducti Mdels Csider a prducti prcess cmpsed f decisi makig uits (DMUs). Each DMU uses varyig quatities f m differet iputs t prduce varyig quatities f s differet utputs. Dete by Y = ( y, y2,..., y ) the s prducti matrix f the DMUs. The rth clum f Y is the utput vectr f DMU r. Dete by X = x, x,..., x ) the m iput matrix. The rth ( 2 ( i clum f X is the iput vectr f DMU r. The matrices Y = y ) ad X = x ) must satisfy: p i 0, i pi > 0 ad p i > 0, where p is x r y. The measure f techical efficiecy f prducti (uder cstat returs t scale) fr DMU {, 2,..., }, deted E CR ( ), is the sluti f the liear prgrammig prblem (). ( i XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 572

3 E CR y u ( ) = max u, v x v subect t () ' i) x v =, ' ' ii) y u x v 0, =,2,..., ad iii) u 0, v 0 If we lk at the cefficiets u ad v as iput ad utput prices, we see that the measure f techical efficiecy f prducti is very clse t the ti f prductivity (utput icme/iput expediture). Techical efficiecy, i this ctext, basically, is lkig fr the price system ( u, v) fr which DMU achieves the best relative prductivity rati. The dual prblem f the liear prgrammig prblem () has a imprtat ecmic Y y iterpretati, which we will explre. This is equivaletly, frmulati (2). mi θ,λ θ, subect t x 0 0 X I θ λ 0 0 r, mi, θ λ θ subect t i) Yλ y, ii) Xλ θx ad iii) λ 0, θ free (2) The matrix prducts Y λ ad X λ, with λ 0, represet liear cmbiatis f the clums f Y ad X, respectively, i.e., a srt f weighted averages f utput ad iput vectrs. I this way, fr each λ, we ca geerate a ew prducti relati, a ew pseud prducer. Trivially, the set f DMUs, 2,..., are icluded amg thse ew prducers. Makig allwace fr these ewly defied prducti relatiships, the questi that the dual iteds t aswer is: What prprtial reducti f iputs θ x it is pssible t achieve fr DMU ad still prduce at least utput vectr y? The sluti θ ( x, y ) is the smallest θ with this prperty. We ca defie the ccept f techical efficiecy f prducti i a ctext f fixed iputs istead f fixed utputs, i.e., i a prgram f utput augmetati. I this evirmet the measure f techical efficiecy f prducti f DMU, uder cstat returs t scale, is the e defied i (3). φ ( x, y ) = max, φ subect t i) Yλ φy φ λ, ii) Xλ x ad iii) λ 0, φ free (3) I the utput augmetati prgram the questi we ask is: what prprtial rate φ ca be uifrmly applied t augmet the utput vectr y, withut icreasig the iput vectr x? The sluti φ is the largest φ with this prperty. This is the apprach we will explre here. Questis f scale ca be dealt prperly impsig prper restrictis i the liear prgrammig prblem. Oe btais the variable returs DEA impsig the additial cditi λ = the weight vectr λ. We w tur ur atteti t prducti statistical mdels. We fllw alg the lies f Baker (993). Suppse m = (a sigle utput) ad assume the existece f a ctiuus frtier prducti fucti g : K R defied the cvex ad cmpact subset K f the psitive rthat f s R. Fr each DMU, the iput bservatis x are pits i K. Let (4). XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 573

4 K = x K; x λ x, λ 0, λ = (4) The DEA frtier prducti fucti is defied fr x K by (5), ad it ca be shw that fr DMU, g ( x ) = φ y, where we are assumig variable returs t scale. = g ( x) sup λ y ; x λ x, λ 0, λ = λ = (5) Suppse that bservatis ( x, y ) are iterir pits t K ad that they are geerated i accrdace with the statistical mdel (6), y = g( ) ε (6) x where: a) The iefficiecies ε are iid with a cmm desity f (ε ). b) The cmm distributi fucti F(x) f the iefficiecies is strictly psitive i ( 0, + ). c) The iputs x represet a radm sample frm a desity h(x) strictly psitive i the iterir f K. d) The iputs x ad the iefficiecies are idepedet. The:. g ( x ) is the parametric maximum likelihd estimate f g ( x ) if f (ε ) is mtically decreasig i ( 0, + ). 2. g ( x ) is weakly csistet fr g ( x ). 3. Let M be ay fixed subset f DMUs. If is large, the it distributi f the estimated iefficiecies ˆ ε = y g ( x ), M, is, apprximately, the it distributi f the true iefficiecies ε, M. These results ca be used t test hypthesis abut the ature f the prducti prcess. A example is the verificati f weather the techlgy shws cstat returs t scale r variable returs t scale. We may perfrm this test cmparig the empirical distributi fuctis f the εˆ (estimated iefficiecy errrs) uder the assumptis f cstat ad variable returs t scale cmputig Klmgrv-Smirv test statistic (Cver, 998). The statistical prperties f uivariate DEA estimates were exteded by Baker ad Nataraa (2004, 2008) t ecmpass stchastic frmulatis f the prducti mdel, ad by Suza ad Staub (2007) t allw fr iid iefficiet cmpets. Checkig weather r t the prducti mdel fits the data is a matter f verifyig if the pstulated iefficiecy distributi fits the efficiecy bservatis. As i stchastic frtier aalysis, the three cmmly used family f distributis used t mdel iefficiecy errrs are the expetial, the half-rmal ad the trucated rmal, the latter havig flexibility prperties (Celli et al., 2005). Agai, Klmgrv-Smirv test ca be used t assess prper fits f these families. Fr the expetial ad half-rmal cases, Baker (993) ad Suza ad Staub (2007) have derived simple statistics, which allw the study f the quality f the fit. Uder the expetial mdel, the statistic (7) will have qui-square distributi with 2 degrees f freedm, where d is the stadard errr f the iefficiecy errrs. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 574

5 w = 2 d ˆ ε i (7) Uder the assumpti f a half-rmal mdel, bth w 2 ad w 3, i (8), are apprximately quisquare with degrees f freedm. w 2 = π ( ˆ ε i ) ( ˆ ε ) 2 i ( ε ) w 2 = π 3 2 d (8) If e is ccered with 00 ( α ) % cfidece itervals fr the efficiecy measuremet φ thse ca be cmputed as (9), where i ˆq is the crrespdig quatile f the estimated iefficiecy errr distributi. This fllws frm a similar result reprted i Suza ad Staub (2007). ˆ ; ˆ φ φ + i i qˆ yi (9) The quatile ca be cmputed by parametric ad parametric methds. If the prducti mdels are subect als t radm errrs, assumig that the radm errr distributi has supprt i a buded clsed iterval, the iterval becmes (0), where the cstat a shuld be estimated as i Baker ad Nataraa (2008) r i Suza et al. (2009b). These cectures i the ctext f iterval estimati are ew t the best f ur kwledge. The extesi f these results t the multivariate utput is t clear i the literature. ˆ a ˆ a qˆ φ ; φ + i i y y y i i i (0) 3. Embrapa s Prducti System Embrapa s research system cmprises 37 uits (DMUs) f research ceters. Iput ad utput variables are defied frm a set f perfrmace idicatrs kw t the cmpay sice 99. The set cmprises 28 utputs ad 3 iputs. We begi ur discussi with the utput. The utput variables are classified it fur categries: Scietific prducti; Prducti f techical publicatis; Develpmet f techlgies, prducts, ad prcesses; Diffusi f techlgies ad image. By scietific prducti we mea the publicati f articles ad bk chapters aimed maily t the academic wrld. We require each item t be specified with cmplete bibligraphical referece. The categry f techical publicatis grups publicatis prduced by research ceters aimig, primarily, agricultural busiesses ad agricultural prducti. The categry f develpmet f techlgies, prducts, ad prcesses grups idicatrs related t the effrt made by a research uit t make its prducti available t sciety i the frm f a fial prduct. Oly ew techlgies, prducts ad prcesses are csidered. Thse must be already tested at the cliet s level i the frm f prttypes, r thrugh demstrati uits, r be already pateted. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 575

6 Fially, the categry f diffusi f techlgies ad image ecmpasses prducti variables related t Embrapa s effrt t make its prducts kw t the public ad t market its image. The iput side f Embrapa s prducti prcess is cmpsed f three variables. Persel expediture, Operatial Csts (csumpti materials, travel ad services less icme frm prducti prects), ad Capital (measured by depreciati). All utput variables are measured as cuts ad rmalized by the cmpay s mea f the crrespdig variable fr the year uder aalysis. Likewise, the iputs are rmalized by the mea. As a fial utput we take a weighted average f all variables i all categries f prducti. The weights are user defied ad reflect the admiistrati percepti f the relative imprtace f each variable t each research ceter r DMU. Defiig weights is a hard ad questiable task. I ur applicati i Embrapa we fllwed a apprach based the Law f Categrical Judgmets f Thurste (Trgers, 958). Mre details this issue may be see i Gmes ad Suza (2008). The mdel is cmpetitive with the AHP methd f Saaty (994) ad is well suited whe several udges are ivlved i the evaluati prcess. Basically, we set ut abut 500 questiaires t researchers ad admiistratrs ad asked them t rak i imprtace scale frm t 5, each prducti categry ad each prducti variable withi the crrespdig prducti categry. A set f weights was determied uder the assumpti that the psychlgical ctiuum f the respses prects t a rmal distributi. DEA mdels implicitly assume that the DMUs are cmparable. This is t strictly the case i Embrapa. T make them cmparable it is ecessary a effrt t defie a utput measure adusted fr differeces i perati ad perceptis. At the level f the partial prducti categries we iduced this measure allwig a distict set f weights fr each DMU. I priciple e culd g ahead ad use DEA with multiple utputs. This wuld miimize the effrt f defiig weights leavig t DEA the task f fidig these cefficiets. The prblem with such apprach is that there is a kid f dimesiality curse i DEA mdels. As the umber f factrs (iputs ad utputs) icreases, the ability t discrimiate betwee DMUs decreases. Thus we fud cveiet t exted the weight system t prduce a sigle measure f utput y. A persel scre was created fr each uit dividig its umber f emplyees by the cmpay s mea. Outputs ad iputs were further rmalized by this variable. This further established a cmm basis t cmpare research uits ad avided the icidece f spurius efficiet uits ad zer utput (shadw) prices, ather cmm ccurrece i multiple utput mdels, ad als a disturbig fact fr maagemet iterpretati. A sigle utput als allws the use f the statistical tests described i the previus secti. DEA mdels are kw t be sesitive t utliers. I ur applicati ctrl f utliers is particularly imprtat fr utput variables. I this ctext we use bx plt feces t adust the Q3 +.5 Q3 Q are reduced t this value fr values f utlyig bservatis. Values abve ( ) ay variable. Here Q ad Q3 dete the first ad third quartiles, respectively. 4. Statistical Results I Table we shw Embrapa s prducti data fr We begi ur aalysis f the data i Table checkig the scale f perati f Embrapa s research uits. Efficiecy estimates uder cstat ad variable returs ad the crrespdig iefficiecy errrs are shw i Table 2. This test is particularly imprtat sice we further rmalize the data by the idex f quatum f persel. The Klmgrv-Smirv statistics fr this hypthesis is D=0.324 with a p-value f 0.04, sigificat. The data used i this aalysis is shw i Table 2 ad are iefficiecy errrs ( φ ) y cmputed frm the efficiecy measuremets. O the basis f this test statistic we assume a variable returs t scale techlgy. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 576

7 Table. Embrapa s mea rmalized prducti data. Y is the utput, X is persel expeditures, X2 is ther expeses ad X3 is capital. Research Ceter Y X X2 X XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 577

8 Table 2. Prduct rieted measures f techical efficiecy ad iefficiecy errrs cmputed uder cstat returs t scale (CRS) ad variable returs t scale (VRS). Research Ceter Efficiecy Errr CRS VRS CRS VRS Table 3 shws gdess f fit statistics related t the fit f the expetial ad half-rmal distributis. We see that statistic w reects the expetial mdel. The statistics w 2 ad w 3 d t shw evidece agaist the half-rmal distributi. Table 3. Gdess f fit tests fr the expetial ad half-rmal distributis. Distributi df Qui-square p-value Expetial ( w ) < 0.00 Half-rmal ( w 2 ) Half-rmal ( w ) XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 578

9 The prbability distributi fucti fr the expetial distributi with parameter γ > 0 is + give by () ad fr the trucated rmal N ( µ, σ ) is (2), where Φ (.) is the distributi fucti f the stadard rmal. The half-rmal btais makig µ = 0. F( ε ) = exp{ γε} () ε µ µ Φ Φ σ σ F( ε ) = µ Φ σ (2) Estimatig the parameters ivlved by maximum likelihd methds e seeks t maximize (3) fr the expetial distributi ad (4) fr the trucated rmal. Here φ (.) detes the prbability desity fucti f the stadard rmal distributi. L( γ ) = l( λ) γ errr > 0 (3) k k = φ k = errr ( k µ σ ) L( µ, σ ) = l (4) σ Φ( µ σ ) Table 4 shws maximum likelihd estimates f γ, µ ad σ. T cmpute maximum likelihd estimates we fllw Simar ad Wils (2007) ad elimiate errrs assciated with efficiet uits. This practice des t destry the asympttic apprximatis. Table 4. Maximum likelihd estimates uder the assumptis f expetial, half-rmal ad trucated rmal distributi. Parameter Estimate Stadard errr p-value σ (Half-rmal) <0.000 µ (Trucated rmal) <0.000 σ (Trucated rmal) <0.000 γ (Expetial) <0.000 The Klmgrv-Smirv test statistics fr the hyptheses trucated-rmal, half-rmal ad expetial are 0.078, 0.29 ad 0.36, respectively. The ly sigificat result is that related t the trucated rmal distributi, sice the 95% asympttic quatile fr the Klmgrv-Smirv test statistic uder the ull hypthesis is These results are t i agreemet with the statistical tests reprted i Table 3, which d t reect the half-rmal hypthesis. I this ctext, sice lcati is sigificatly differet frm zer i the trucated rmal specificati, we iterpret Table 3 results as lack f eugh pwer t reect the halfrmal ull hyptheses ad fllw the trucated rmal errr assumpti. The quatile plt shw i Figure prvides visual supprt fr ur cclusis regardig gdess f fit. This is further evidece i favr f the trucated rmal. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 579

10 ,0 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0, 0,0 0,0 0, 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9,0 Half-Nrmal Expetial Trucated Nrmal Figure. Quatile plts fr Half-Nrmal, Expetial ad Trucated Nrmal distributis. Fially, Table 5 shws the efficiecy estimates uder the assumpti f variable returs t scale ad 95% idividual cfidece itervals, cmputed uder a parametric assumpti fr DEA residuals. The 95% estimated quatile fr the errr distributi is.93. Fr the estimated trucated rmal distributi the equivalet quatity is.30. The frmer leads t smaller cfidece limits i the preset case. The mai advatage f the techique used here relative t ther cfidece itervals suggested i the literature, as the prpsal f Simar ad Wils (2004, 2007), is that the actual efficiecy estimates are lwer buds ad they defie real pssibilities fr the crrespdig ppulati values. Hwever, the impsiti f stchastic errrs may destry this prperty. The parametric fittig is particularly cveiet whe cvariates are preset, sice residuals will t be iid ad quatiles will chage with the DMU level. 5. Summary ad Cclusis Uder the assumpti f a statistical mdel ctaiig ly iefficiecy errrs we cmpute cfidece itervals fr the efficiecy measuremets cmputed usig Data Evelpmet Aalysis. The prducti variables are rmalized, which ptetially leads e t csider variable returs t scale techlgies. Nparametric gdess f fit test idicates sigificat differeces, relative t the hypthesis f cstat returs t scale. Residuals frm the estimated prducti mdel fllw a trucated rmal distributi, which is used t cmpute the 95% upper limits f the idividual efficiet measuremets. Oly efficiet uits are used fr parameter estimati via maximum likelihd. The expetial ad half-rmal distributis fail t prvide a gd fit. The itervals prvided iclude actual efficiecies as pssible values. This prperty hwever may be destryed if e icludes buded radm errrs i the mdel specificati. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 580

11 Table 5. 95% idividual cfidece itervals fr prduct rieted efficiecy measures usig empirical quatile. Itervals scaled t be i [0,]. Research Ceter Lwer bud Upper Bud Ackwledgmet T the Natial Cucil fr Scietific ad Techlgical Develpmet (CNPq), fr the fiacial supprt. 7. Refereces Baker, R.D. (993), Maximum likelihd, csistecy ad DEA: a statistical fudati, Maagemet Sciece, 39(0), Baker, R.D. ad Nataraa, R., Statistical tests based DEA efficiecy scres, i Cper, W.W., Seifrd, L.M. ad Zhu, J. (Eds.), Hadbk Data Evelpmet Aalysis, Kluwer Iteratial Series, Bst, , XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 58

12 Baker, R.D. ad Nataraa, R. (2008), Evaluatig ctextual variables affectig prductivity usig data evelpmet aalysis. Operatis Research, 56, Celli, T.J., Prasada Ra, D.S., O'Dell, C.J. ad Battese, G.E., A Itrducti t Efficiecy ad Prductivity Aalysis, 2d Editi, Spriger, New Yrk, Cver, W.J., Practical Nparametric Statistics, Wiley, New Yrk, 998. Darai, C. ad Simar, L. Advaced Rbust ad Nparametric Methds i Efficiecy Aalysis, Spriger, New Yrk, Depris, D., Simar, L. ad Tulkes, H. Measurig labr iefficiecy i pst ffices, i Marchad, M., Pestieau, P. ad Tulkes, H. (Eds.), The Perfrmace f Public Eterprises: Ccepts ad Measuremets, Nrth-Hllad, Amsterdam, , 984. Gmes, E.G. e Suza, G.S. (2008), Percepções psicmétricas sbre a imprtâcia de atividades de pesquisa, Aais d XL Simpósi Brasileir de Pesquisa Operacial. Kumbhakar, S. ad Lvell, C.A.K. Stchastic Frtier Aalysis, Cambrigde Uiversity Press, New Yrk, Saaty, T.L. The Fudametals f Decisi Makig ad Pririty Thery with the Aalytic Hierarchy Prcess, RWS Publicati, Pittsburgh, 994. Simar, L. ad Wils, P.W. (2007), Estimati ad iferece i tw-stage, semi-parametric mdels f prducti prcesses, Jural f Ecmetrics, 36 (), Simar, L. ad Wils, P.W. Perfrmace f the btstrap fr DEA estimatrs ad iteratig the priciple, i Cper, W.W., Seifrd, L.M. ad Zhu, J. (Eds.), Hadbk Data Evelpmet Aalysis, Kluwer Iteratial Series, Bst, , Suza, G.S. ad Staub, R.B. (2007), Tw-stage iferece usig data evelpmet aalysis efficiecy measuremets i uivariate prducti mdels, Iteratial Trasactis i Operatial Research, 4, Suza, G.S., Gmes, E.G. ad Staub, R.B. (2009a), Ifluece f ctextual variables: a applicati t agricultural research evaluati i Brazil, Prceedigs Iteratial Data Evelpmet Aalysis Sympsium. Suza, G.S., Mreira, T.B. ad Gmes, E.G. (2009b), A efficiecy apprach fr aalyzig the mar agricultural ecmies, Prceedigs Iteratial Data Evelpmet Aalysis Sympsium. Trgers, W.S. Thery ad Methds f Scalig. Wiley, New Yrk, 958. XLI SBPO Pesquisa Operacial a Gestã d Checimet Pág. 582