2.3. EVALUATION OF THE RESULTS OF A PRODUCTION SIMULATION GAME WITH DIFFERENT DEA MODELS

Transcription

1 .3. Evaluation of the results of a production simulation game with different DEA models Tamás OLTA, udit UZONY-ECSÉS DO: 0.855/dBEM.M07.n0.ch09.3. EVALUATON OF THE RESULTS OF A PRODUCTON SMULATON GAME WTH DFFERENT DEA MODELS Summar Data Envelopment Analsis (DEA) is a method for comparing the efficienc of decisionmaing units when the output of these units is evaluated based on the amount of inputs used. A special application area of DEA is the evaluation of student groups participating in a production simulation game. This paper shows how DEA is used to compare the performance of student groups in the simulation game, and how their results can be evaluated using the efficienc scores. Several DEA models eist to capture the special characteristics of real life operation. Basic models with radial efficienc measures are used to analse the effect of input and output weights, and to separate the proportional decrease of inputs from the independent input reduction possibilities. Slac based measure models are applied to stud the oint effect of proportional and independent input/output changes. Dnamic models are used to stud the change of efficienc over time. This paper compares the results of the applied models and analses the differences. The results show that the application of an assurance regain model is strongl recommended. The presence of negative outputs requires the application of models which can be adapted to negative data. Dnamic models indicate efficienc problems even if overall performances are acceptable. ewords: Data Envelopment Analsis, linear programming, performance evaluation, simulation game ntroduction Data Envelopment Analsis (DEA) is a mathematical programming approach that is used for comparing the efficienc of decision maing units (DMU) such as production and/or service sstems. n contrast to other methods (e.g. ratio methods) used for performance evaluation, DEA is capable of handling multiple inputs and multiple outputs as well. DEA was first introduced b Charnes, Cooper and Rhodes for evaluating nonprofit organizations. n the last few decades DEA has been etensivel investigated, and it became an important research area. Several applications of DEA are reported in the literature both in the service and in the production sector as well. There is no an single DEA model which is alwas the best. Different application environments have generated different evaluation problems thus several variants of DEA models have been developed. n this paper DEA is applied in a higher education contet to compare the performance of student groups in a production simulation game. Different DEA models are proposed to capture some special characteristics of operation. n the following part of this paper first the DEA models applied in the presented research are introduced. Net, the application environment is presented and the important differences between the suggested DEA models are discussed. Finall, conclusions are drawn and the areas of future research are summarised. 09

2 oltai, T. Uzoni-ecsés,. Basic concept and the applied models Charnes, Cooper and Rhodes (978) suggested a linear programming model which compared DMUs using relative efficienc measures. Based on the suggested model relative efficienc analsis, or data envelopment analsis (DEA) became an important research area and a useful tool for performance evaluation. Several applications of DEA models are reported in the literature in the service and in the production sector as well (see for eample Dole and Green, 99; Panaotis, 99; Sherman and Ladino, 995; Marovits-Somogi, Gecse and Boor, 0). A frequentl applied area of DEA is higher education. ohnes (006) compared more than 00 higher educational institutions in England using a nested DEA model. Sinuan-Stern, Mehrez and Barbo (994) analsed the relative efficienc of several departments within the same universit. The first model suggested b Charnes, Cooper and Rhodes (978) can be eplained b an intuitive analog taen from engineering. According to the law of energ conservation, the different tpes of energies can be transformed, but energ cannot be created. n case of a power plant for eample, it is not possible to produce more energ, than the energ content of the fuel used, that is, technical efficienc is alwas lower than. Appling this engineering analog in the area of performance evaluation in operations management it can be stated, that the measure of output is alwas smaller than the measure of input. n the best possible case, the ratio of output measure and input measure is equal to. The output and input measures are calculated as weighted outputs and weighted inputs, and the best possible weight values are looed for a DMU, which is called reference DMU R. Let us assume that number of DMUs are evaluated, when different outputs are observed and different inputs are used. Notations applied in this paper are listed in Table. f (=,,; =,,) are the observed output values of output, and i (i=,,; =,,) are the observed input values of input i for DMU, furthermore v (=,,) and u i (i=,,) denote the output and input weights then the linear programming formulation for finding the most favorable weights for DMU R is as follows, Ma v R / u i i v / u i i,, n i u i, v 0 i,, ;,, () f problem () is transformed to eliminate the ratio of variables, and the weighted input is fied (equal to ) in order to get unique solution for LP problem (), then the primal version of the input oriented, constant return to scale (CRS) model is obtained, that is, 0

3 .3. Evaluation of the results of a production simulation game with different DEA models Ma v R u i i v u i i 0 i u i, v 0,, n i,, ;,, Table : Notation () ndices and parameters - inde of decision maing units (DMUs), =,,, i - inde of inputs, i=, - inde of outputs,, =,,, l - inde of lin flows, r - column inde of the output weight constraints matri, s - column inde of the input weight constraints matri, R - inde of the reference DMU, α - inde of lin flow tpe (Good or Bad). Parameters: - number of DMUs, - number of inputs, - number of outputs, G - number of good lins, B - number of bad lins, i - quantit of input i of DMU, - quantit of input i of DMU in group, i i - quantit of input i of DMU in group, it - quantit of input i of DMU in period t, - quantit of output of DMU, - quantit of output of DMU in group, i i - quantit of output of DMU in group, t - quantit of input of DMU in period t, NP L - lower limit of the ratio of input weights i+ and i, i i, L - lower limit of the ratio of output weights + and, OUT, U - upper limit of the ratio of input weights belonging to input i+ and i, NP i, i U - upper limit of the ratio of output weights belonging to + and, lt - w i + w - P i + P p ir q s OUT, z - quantit of lin l of DMU in period t of tpe α, - weight of input slac i, - weight of output slac, - normalization base of input slac i, - normalization base of output slac, - element of the input weight matri, - element of the output weight matri. ()

4 oltai, T. Uzoni-ecsés,. Table : Notation () - Variables Variables: u i - weight of input i, v - weight of output, λ - dual variable of DMU, λ t - dual variable of DMU, in period t, θ - radial efficienc score, μ R - slac based measure efficienc score of DMU R, ρ R - modified slac based measure efficienc score of DMU R, - s i - vector containing the input surplus values of each DMU, + s - vector containing the output shortage values of each DMU, π r - dual variable of input constraint r, τ s - dual variable of output constraint s. The dual version of problem (), however, has more practical relevance and leads to another interpretation of DEA. According to the dual interpretation an linear combination of the observed output and input values leads to a new and feasible DMU, which ma eist in practice. The production possibilit set is determined b all possible linear combinations of the observed outputs and inputs. f λ (=,,) are the coefficients of the linear combination of output and input values, then the production possibilit set of DMU R can be defined as follows, R i,, i,, f we consider the λ (=,,) coefficients as variables, and a proper obective function is used to get an optimal combination of the output and input values, then the distance from an eisting DMU from the optimal DMUs can be a basis of an efficienc score. The dual version of the input oriented CRS model assumes that all inputs must be decreased to the same proportion (θ), and efficienc is given b the smallest proportion. Consequentl the smallest amount of input necessar to produce the observed output must be determined. The corresponding dual LP model is as follows, Min R i 0,, i,,,, n primal problem (), the optimal value of the weights of inputs and outputs are determined b linear programming. Frequentl, the value of some weights is zero, that is, when the efficienc score is calculated, some inputs and outputs have zero weights. nputs and outputs with zero weights are ignored in the evaluation, which is not alwas (3) (4)

5 .3. Evaluation of the results of a production simulation game with different DEA models acceptable for management purposes. To avoid the problem of zero weights, restrictions which reflect the intention of management can be added to model (). One possible form of weight restriction is when constraints for all possible pairs of inputs and outputs are introduced, that is, OUT v L OUT, U,, v (5) NP u L NP, i i i U i, i i, u i Adding constraints (5) to model () and writing the dual form of the resulting model, the input oriented CRS assurance region (AR) dual model is obtained which is as follows, Min s s q s R r p i r ir, s, r 0,, i,,,, ; n model (), (4) and (6) input and output data can onl be semi-positive. n practice, however, sometimes negative inputs and outputs ma frequentl occur. Several attempts can be found in the literature to cope with negative data when radial efficienc measure is used. n the following part of the paper we use the semi-oriented radial model (SORM) proposed b Emrouznead, Anouze and Thanassoulis (00). n the SORM model all inputs and outputs are separated into two groups. Positive data belong to group and the absolute values of the negative data belong to group. The separated input data of the SORM model are as follows, if 0 () i i i 0 if i 0 (7) 0 if 0 () i i abs if 0 i i and the separated output data of the SORM model are as follows, if 0 () 0 if 0 (8) 0 if 0 () abs if 0 s; r (6) 3

6 oltai, T. Uzoni-ecsés,. Positive outputs and negative inputs are favorable for the decision maer. Consequentl, the ideal DMU is alwas constrained b these values from below, that is, the production possibilit set determined b the positive outputs and negative inputs are as follows,,, R (9) i i,, Negative outputs and positive inputs are unfavourable for the decision maer. Consequentl, the ideal DMU is alwas constrained b these values from above, that is, the production possibilit set determined b the negative outputs and positive inputs are as follows,,, R (0) i i,, Based on constraints (9) and (0), the dual form of the SORM model is as follows, Min R,, R,, i i,, () i i,, 0,, Models (), (4), (6) and () are based on a radial measure of efficienc, that is, all inputs are decreased proportionall b the same ratio. The slac based model (SBM) proposed b Tone (999) uses the difference of the observed values and the best possible linear combination of inputs and outputs. The difference of the actual value and the best possible value is called slac. With the help of production possibilit set (3), all possible slac values of DMU R are given b () if s + indicate the output increase possibilit of output and s - i indicate the input decrease possibilit of input i, that is, s s i R i 4,, i,, ()

7 .3. Evaluation of the results of a production simulation game with different DEA models The slac values epress the distance of a DMU from the best possible DMU. Based on the slac values the following efficienc measure can be used, R i w w s i i s The slac based measure of efficienc proposed b Tone (999) can tae an value between 0 and, and it is based on the weighted average of the normalized input and output slacs. The basis of normalization is the actual value of outputs and inputs in epression (3). The basic DEA concept assumes that all observed inputs and outputs are semi-positive. n case of negative values some modification of the formulation is required to get feasible solution and to eep the efficienc score between 0 and. Silva Portela, Thanassoulis and Simpson (004) proposed a directional distance approach which is based on the range of the possible output and input improvements. Assume that the highest possible output is the highest observed output, and the improvement possibilit is defined b the distance between the highest possible value and the observed value. n this case, the output slac ma change in the following range for reference DMU R, P R ma ( ) R, (4) Similarl, assume, that the smallest possible input is the smallest observed input, and the improvement possibilit is defined b the distance between the observed value and the smallest possible value. n this case, the input slac ma change in the following range for reference DMU R, P min ( i ) i, s (5) Ranges (4) and (5) are called Silver-Portela (S-P) ranges, and can be used for normalizing the slac values in case of negative outputs and/or inputs. Correspondingl, the modified slac based model (MSBM) proposed b Sharp, Meng and Liu (007) applies the following efficienc measure, 5 / / R R (3) s / wi i P i R (6) w s / P Solving models (4), (6) and () in the first phase, a second phase is required. n the second phase a slac maimization model is solved using the optimal efficienc score of phase. Phase is used to chec the strong efficienc conditions, and to obtain the independent input decrease and output increase possibilities (Cooper, Seiford and Tone, 007). n case of the slac based models, this second phase is not required. Several variations of the radial and slac based models eist in the literature (see for eample Sooper, Seiford and Tone, 007). Depending on the main obective of evaluation, input oriented, output oriented or non-oriented models can be used. f the rate of the use of inputs and the rate of the generation of outputs change, then a

8 oltai, T. Uzoni-ecsés,. variable return to scale (VRS) model is appropriate. f information related to input and output weights are necessar, then primal models are preferred. Assurance region models can be used when weight restrictions are introduced b the decision maer. Finall, when the efficienc of several DMUs must be evaluated, and the inputs and outputs of these DMUs are not independent from each other, then networ DEA models are recommended (Tone and Tsutsui, 00). Dnamic DEA models are special networ DEA models. n this case, the performance of the same DMU must be evaluated over several periods; consequentl, the production possibilit set must be defined for each period. A part of the production possibilit set is determined b the observed output and input values of the DMUs in each period t, that is, constraints () must be completed with inde t, as follows, Rt t t t it t,, ; i,, ; 6 t,, T t,, T n dnamic models, the production possibilit set of a given period is also determined b the lin flows (z lt ) which connect the neighbouring periods. f a lin flow has favourable effect on operation, then it is called Good lin (Tone and Tsutsui, 00) and an output tpe constraints must be determined, that is, Good t Good zlt zrt l,, G; t,, T f a lin flow has unfavourable effect on operation, then it is called Bad lin (Tone and Tsutsui, 00) and an input tpe constraints must be determined, that is, Bad t Bad zlt zrt l,, B; t,, T The different periods are connected b the continuit equations in the dnamic models. The optimal linear combination of the lin flows is identical in the neighbouring periods, that is, l (7) (8) (9) t t zlt zlt t,, T ; Good, Bad (0) l n the following part of the paper the performance of the student groups in the production simulation games are evaluated with several DEA models. Radial efficienc is determined with model (4), (6) and (). Slac based efficienc is determined with obective functions (3) and (6). Finall, dnamic efficienc is determined using constraints (7), (8), (9) and (0). Application environment We analsed a production simulation game, which is developed b Ecosim to support education and training in the production management area ( The obective of the game is to simulate production management decision maing in a car

9 .3. Evaluation of the results of a production simulation game with different DEA models engine manufacturing factor. The factor produces three different car engines for five different marets in 7 periods. Each maret has its own demand characteristics. The car engines are assembled from parts on assembl lines operated b worers. For the net production period (ear) each student group must mae sales and mareting, production, investment and financial decisions. After submitting the decisions, the simulation program generates the results of the actual production period. The results are summarized in a production report and in a financial report. Using the results and eperiences of the earlier periods the student groups tr to increase operational performance of the net periods. We used different input oriented DEA models for evaluating the performance of student groups at the end of the seventh period of the simulation game. n all cases we applied a constant return to scale model, because there is not size difference between the DMUs, thus a variable return to scale (VRS) approach is not relevant. Two outputs and four inputs were considered in the analsis. n our previous papers we presented the evaluation of the performance of student groups using different outputs (oltai and Uzoni 0). n this paper, the results of several DEA models addressing various modelling problems are presented. One of the outputs is cumulated production quantit which reflects the effect of production management decisions related to machine and worer capacit, to material requirement planning and to inventor management. The other output is net profit which integrates the effect of mareting, production and financial decisions. The four inputs cumulated number of worers, cumulated number of machine hours, cumulated sum of mone spent on raw materials and cumulated value of credits represent the resources used in the production process. Consequentl, the performance of the production sstem based on these decisions reflects student s nowledge in the related areas. Comparison of the results of the different DEA models The performance of 8 student groups is compared using input oriented CRS, CRS- AR, SORM, SBM, MSBM and dnamic MSBM model. The results are summarized in Table. Column and 3 shows the values of the two outputs applied in the evaluation. These data are properl scaled to avoid numerical problems. Column 4-9 shows the efficienc scores of the different models. n those models, which cannot handle negative data, negative values were substituted b zero. Using the basic input oriented CRS model, 7 student groups have the highest possible efficienc score. The results show that the operation of almost half of the DMUs is efficient. Furthermore, the value of the efficienc score of inefficient groups is close to, which indicate a low discrimination power of the model. n this case, a large number of input and output weights are zero, consequentl, for eample, the profit has insignificant effect on the obtained efficienc scores. Appling weight restrictions (CCR-AR), it can be observed that all groups obtained lower scores. The number of the efficient groups is also reduced, onl groups 3, 7 and 5 remained efficient. We applied 0. for the pairwise relative lower limit of the inputs, and 0.5 for the lower limit of the ratio of outputs. 7

10 oltai, T. Uzoni-ecsés,. Team Output Net profit Table : Efficienc results of DEA models Output Production Quantit CRS CRS- AR SORM SBM MSBM Dnamic MSBM 0,650,70,0000 0,98,0000,00000,0000 0,64 0,097,74,0000 0,809,0000,00000,0000 0,5409 3,874,9,0000,0000,0000,00000,0000 0, ,86,448 0,973 0,8750 0,973 0,036 0,7033 0, ,69,37 0,9579 0,7583 0,9579, ,59 0, ,046,573 0,983 0,8583 0,983 0,0705 0,6846 0,54 7,656,778,0000,0000,0000,00000,0000,0000 8,007,553 0,997 0,95 0,997 0,6730 0,6043 0,886 9,74,977,0000 0,9999,0000,00000,0000 0,7370 0,05,836 0,998 0,935 0,998 0,8890 0,8757 0,6030 0,987,440 0,998 0,9473 0,998 0,7596 0,746 0,636 0,83,466 0,9798 0,8647 0,9798 0,9680 0,6468 0, ,675,368 0,93 0,800 0,93 0,4736 0,5573 0,565 4,79,650,0000 0,9859,0000,00000,0000, ,879,665,0000,0000,0000,00000,0000 0, ,97,487 0,9508 0,8305 0,9508 0,8356 0,564 0, ,667,964 0,9053 0,850 0,9053 0,43 0,0676 0, ,799,553 0,9867 0,873 0,9867 0, ,784 0,564 Source: the authors own table t is proved, that the efficienc score of the SBM models is not greater than the CRS efficienc values (Tone, 999). n addition, a DMU is CCR efficient if it is SBM efficient. Consequentl, CCR efficient student groups remained at the efficient status under SBM evaluation. The SBM score of most of the inefficient groups are lower than the CRS scores. Group 5 has higher efficienc score with SBM than with CRS evaluation. This contradiction indicates that the SBM model can not be applied in this case. Note, that group 5 has negative net profit, consequentl the output values are not semi-positive, and the efficienc scores are theoreticall erroneous. We can also observe large differences between the efficienc scores of the CRS (CRS-AR) and the SBM models. These large differences can be eplained b the fact, that slacs are not reflected in the CRS scores. The CRS model calculate the slac values in the second phase, and these values are considered, when the target quantities are calculated, but are not reflected in the scores. Finall, the last column shows the overall efficienc scores calculated b the dnamic MSBM model. The results show, that onl groups 7 and 4 remained efficient. Moreover the efficienc scores of most of the groups are significantl reduced. A detailed analsis of the period efficiencies showed, that some groups which have good overall performance, ma have efficienc problems in some periods. The MSBM and the SORM models can be used to handle negative data. According to Table. MSBM and SORM selected the same DMUs as efficient, but different target values are recommended. The target values recommended b the CRS, and the MSBM models for a selected student group (Team 0) is presented in Table 3. t can be seen, 8

11 .3. Evaluation of the results of a production simulation game with different DEA models that the MSBM target values indicate a slightl smaller input reduction, than that of the CRS values, but with a higher production quantit. Table 3: Target values of Team 0 Production quantit Net profit No. worers Machine hours Raw materials Debt Original CRS MSBM Source: the authors own table Note, that the SORM efficienc scores are identical with the CRS efficienc scores in Table. This can be eplained b the fact, that Team 5 is the onl team with negative output value. n this special case, the constraint belonging to this unfavorable output does not influence the production possibilit set, and consequentl the efficienc scores. Conclusion This paper compared the results of different DEA models when the performance of student groups in a production simulation game is evaluated. Basic models with radial efficienc measures are used to analse the effect of input and output weights, and to separate the proportional decrease of inputs from the independent input reduction possibilities. Slac based measure models are applied to stud the oint effect of proportional and independent input/output changes. The results show the advantage of the application of the assurance regain model. The presence of negative outputs requires the application of models which can be adapted to negative data. The evaluation of the groups in the static DEA models uses aggregated input and output values. The inputs and outputs in the 7 production periods are simpl cumulated, consequentl, the dnamic behaviour of the groups is not reflected in the results. When appling Dnamic DEA models, the progress of groups during the decision maing process can be analsed and a more detailed picture about the learning process can be obtained. (oltai and Uzoni 03) Analsing the different results provided b the static and dnamic models and using the information provided b the dnamic models about local efficienc problems are an interesting and promising topic for further research. References. Charnes, A., Cooper, W.W., Rhodes, A. (978): Measuring the efficienc of decision maing units. European ournal of Operations Research,, pp Cooper, W.W., Seiford, L.M., Tone,. (007): Data envelopment analsis. Springer Dole,.R. and Green, R.H. (99): Comparing products using data envelopment analsis. Omega nternational ournal of Management Sciences, 9(6), pp

12 oltai, T. Uzoni-ecsés,. 4. Emrouznead, A., Anouze, A.L., Thanassoulis, E. (00): A semi oriented radial measure for measuring the efficienc of decision maing units with negative data, using DEA. European ournal of Operational Research, 00, pp ohnes,., (006): Data envelopment analsis and its application to the measurement of efficienc in higher education. Economics of Education Review, 5(3), pp oltai, T., Uzoni-ecsés,. (0): Data Envelopment Analsis and its Application for Measuring the Performance of Students in a Production Simulation Game. n: llés, Cs.B. (ED) SMEs' Management in the st Centur: Challanges and Solutions. Czestochowa: Publishing Section of the Facult of Management, Czestochova Universit of Technolog, pp oltai, T., Uzoni-ecsés,. (03): Analsis of the learning process in a production simulation game with DEA using cumulative data. n: Misolci Egetem. nnovációs és Technológia Transzfer Centrum (szer.). microcad 03: XXV. microcad nternational Scientific Conference: Section Q: Economic Challenges in the st Centur (Gazdasági ihíváso a XX. században). Misolci Egetem, Paper Q9. 6 p. 8. Marovits-Somogi. R., Gecse, G. and Boor, Z. (0): Basic efficienc measurement of Hungarian logistics centres using data envelopment analsis. Periodica Poltechnica Social and Management Sciences, 9(), pp Panaotis, A.M. (99): Data envelopment analsis applied to electricit distribution districts. ournal of the Operations Research Societ, 43(5), pp Sharp,., Meng, W., Liu, S. (006): A modified slacs based measure model for data envelopment analsis with natural negative outputs and inputs. ournal of Operational Research Societ 57(), pp Sherman, H.D. and Ladino G. (995): Managing ban productivit using data envelopment analsis (DEA). nterfaces, 5(), pp Sinuan-Stern, Z., Mehrez, A. and Barbo, A. (994): Academic departments efficienc via DEA. Computers & Operations Research, (5), pp Tone,. (999): A slacs based measure of efficienc in data envelopment analsis. European ournal of Operational Research, 30 pp Tone,. and Tsutsui, M. (00): Dnamic DEA: A slac-based measure approach. Omega, 38, pp Silva Portela, M.C.A., Thanassoulis, E., Simpson, G. (004): Negative Data in DEA: a directional distance approach applied to ban branches. ournal of Operational Research Societ, 55, pp