Maximal margin classifiers applied to DGA-based diagnosis of power transformers

Transcription

1 Piotr S. SZCZEPANIAK, Marcin KŁOSIŃSKI Institute of Information Technology, Technical University of Łódź Maximal margin classifiers alied to DGA-based diagnosis of ower transformers Abstract. The aer addresses a modern aroach to the roblem of ower transformer diagnosis. The method called suort vector machines enables the creation of an exert system for oil transformer technical condition diagnosis. The system, which is based on real results of chromatograhy of gases dissolved in transformer oil (DGA), erforms better than an internationally acknowledged standard the IEC code. Streszczenie. Przedstawiono nową metodę diagnostyki transformatora mocy bazująca na algorytmie suort vector machine. W systemie bada się chromatograficznie gazy rozuszczone w oleju transformatorowym. (Zastosowanie klasyfikatora do diagnozy gazów rozuszczonych w oleju transformatora mocy) Keywords: ower transformers, DGA, IEC, classification, maximal margin classifiers, SVM Słowa kluczowe: transformator mocy, analiza oleju, klasyfikator, SVM. Introduction to DGA-based diagnosis The analysis of gases dissolved (DGA) in oil is one of the basic methods alied to diagnosing technical condition of transformers. The internationally acknowledged rules for interretation of concentration of gases are reflected in the IEC code [IEC(1979)]. owever, basing on their own exerience, different countries have worked out different transformer diagnosing methods. All those methods base on rigid mathematical deendencies. Data used to evaluate a transformer are defined by three variables (x,y,z) as shown in (1). (1) C x C C y C z C where, C, C, C, and C denote the amount of hydrogen, methane, acetylene, ethylene, and ethane in the gas under examination (in m units arts er one million), resectively. The meaning of these variables is the same as in the IEC code; they reflect the DGA results. Following the IEC code, it has been assumed that there are nine classes describing the state of the transformer: No fault Partial discharge of low energy Partial discharge of high energy Disrutive discharge of low energy Disrutive discharge of high energy Overheating below 150 o C Overheating between 150oC and 300 o C, Overheating between 300oC and 700 o C Overheating over 700 o C. Consequently, every trile enables reasoning about the technical condition of the examined transformer. The ranges of quotient values are coded with three integer numbers {0,1,} (see Table 1.). According to the IEC code, there are nine classes (see Table.), which are roerly defined from an engineering ersective with an additional class containing unrecognized cases. Such exert knowledge can be reresented in the form of logic rules: () If code{d x, d y, d z }, then transformer s state. These are cris logic rules, which are unambiguous and mutually exclusive. Table 1. IEC codes Transformer state code C C Table. Classification according to IEC code quotient C C C Value C C [0; 0,1) [0,1; 1) [1; 3) 1 1 Value > 3 Visually, the IEC coding system creates a searation in three-dimensional sace by introducing a set of disjoint fragments (Fig.1). Of course, the biggest fragment is the one reresenting an unrecognized state. To summarize, to each trile (x, y, z), or to each trile resented in Table one can assign an additional value describing (in an encoded or linguistic way) the technical condition of a transformer. This value, which is figured out on the basis of the insection of an unlugged transformer or an exert oinion, can be actually interreted as a diagnosis of the object s condition. The idea resented in this aer is as follows. aving a sufficient number of reresentative results for a series of transformers, one should be able to find regions related to the above nine classes. The examination of a new DGA result would be then straightforward. So far, the IEC, as well as every other national system of classification, has offered the arbitrary division of the sace. In the resented roosition, the determination of regions related to the nine ) Code 10 is usually interreted as full discharge of high energy, that is, the diagnosis full discharge of low energy is associated with one of the three codes: {101}, {01} or {0}. C C C Without fault Partial discharge of low energy Partial discharge of high energy Disrutive discharge of low energy 1 or 1 or 0 Disrutive discharge of high energy 1 0 Overheating below 150 C Overheating between 150 C and 300 C 0 0 Overheating between 300 C and 700 C 0 1 Overheating over 700 C 0 Unrecognized fault Other codes 100 PRZEGLĄD ELEKTROTECNICZNY (Electrical Review), ISSN , R. 88 NR /01

2 classes of diagnosis deends on the real results of revious examinations erformed on transformers working in real conditions. The regions are determined using the suort vector machines. Fig.. Linearly searable data; two classes In SVM method, it is suggested that the otimal hyerlane should be the one which rovides maximal (symmetric in relation to searating lane) searation margin of training data. It should be emhasized at this oint that the searation is not made using all available data, but only a small art of them (Fig. ). Intuitively, the searation line should be found basing on the nearest data oints. The way we can maximize the margin is the minimization of weight vector norm w which means the minimization of w T w = w i. Consequently, we should minimize w T w (a) Fig.1. Visualization of IEC codes classification Basics of Suort Vector Machines The resented classifiers are Suort Vector Machines, which were first introduced by Vanik et al. [Boser, et al. (199), Vanik (1995,1998)]. When regarded as a maximization of margin distance that searates different classes, SVM method can be called also maximal margin classifier. The weakness of the term classifier is that this method can be also used in regression roblems. There are a growing number of ublications in this field: Schölkof (1998), Schölkof, et al. (1999, 00), Decoste, et al. (00), Lin, et al. (00). Additionally, there are a lenty of Internet resources (htt:// htt:// where one can find also introduction lectures such as Christianini (001), Chen, et al. (001). The aim of this method is to erform some data searation, which simultaneously conforms to tough statistic requirements. This can be achieved by searching for a global minimum of some convex cost function. Since its first ublication, the method has been under develoment and one can find different kinds of it. The rimary task can be defined as follows: Given a set of linearly searable data (Fig.) (3) (x, y ), =1,,..., P, x R n, y { 1,+1}. Construct a searating hyer-lane y(x; w, b) so as the searation margin = w is maximal. ere, w, and b are arameters which determine the hyerlane. subject to y (w T x + b) 1 (b) Task (a), (b) is well known in the otimization theory as square otimization with inequity constraints. Such roblems are usually solved by introducing non-negative arameters, the so called Lagrange multiliers and Lagrange function in the form of: P 1 T T 1 (5) L(w, b, ) = w w - α [ y ( w x b ) 1] Providing that 0. The 1 coefficient is used for convenience reasons only it does not break the generality of the given solution. The function must be minimized for w and b and maximized for. It means that we are looking for a saddle oint. It occurs very often that the data are not linearly searable. To ensure the searability of data, some transformation to a higher dimensional sace can be introduced. Such a transformation should be subject to Cover theorem [Cover (195)] about data searability. This theorem says that sace can be transformed with high robability to higher multi-dimensional sace, in which data are linearly searable. Such a higher-dimensional sace is called feature sace. It is noteworthy that for such high-dimensional saces all calculations become extremely difficult. On the other hand, all calculations in SVM method are done using dot roducts, which means that a secific sort of transformation function can be used, namely kernel function. PRZEGLĄD ELEKTROTECNICZNY (Electrical Review), ISSN , R. 88 NR /01 101

3 Definition: Kernel is a function K such as for all u,v X : () K( u,v ) ( u ), ( v ) where is the real transform from one sace to another. As stated, the non-linear transform does not have to be given exlicitly. On the other hand, all dot roducts can be easily calculated in higher-dimensional sace using the kernel function, without even knowing exactly what the transform is. There are lenty of kernel function tyes, the most often used being: Polynomial: K( u,v ) u,v 1 (7) n set by the author and it remains unchanged during the otimal classifier training. The higher the C value, the bigger the enalty value for errors. As a result, there are fewer wrongly classified oints. A smaller C value lets us decrease the margin and tolerate more incorrect classifications. Diagnosis aving a sufficient number of training data collected for a articular transformer fault tye, one can introduce an exert system able to suort the diagnosis of transformers. This would mean creating a classifier which could categorize new data (x, y, z) based on the teaching atterns from a database. Such a classifier would enable one to introduce a more flexible way for aligning classification tyes with certain transformer tyes. This seems to be a much better aroach than the IEC method. Gaussian (radial basis): 1 (8) K ( u,v ) ex u v r yer tangens: (9) K ( u,v ) tanh u, v In this aer, two kernel functions have been used: linear (that is olynomial with n = 1) and radial with various r values. The method described above may not work if the data are distorted and not linearly searable even in the feature sace. Such data have an aarently good margin, which causes constraints (b) not to be fulfilled. Also the classes very often overla each other. When those constraints are not fulfilled, the value of resective Lagrange coefficients is also increased. The way we can accet such distortions of data is to introduce some tolerance for not fulfilling constraints. In other words, we introduce a way to soften the constraints (soft margin classifiers) [Cortes, et al. (1995), Cristianini, et al. (003), Kecman (001)]. The exceeding value of margins is limited and the limits are set for each examle searately. Instead of fulfilling constraints (b), that is: y (w T x + b) 1 (=1,...,P), one should lace the examles within borders: (w T x + b) 1 for y = +1 ; (w T x + b) 1+ for y = 1. We can formulate two roblems deending on the way we consider tolerance 0 in the quality assessment: ( ) (10a) I: minimize w T w + C II: minimize subject to y (w T x + b) 1. (10b) w T w + C (11a) subject to y (w T x + b) 1, (11b) 0. (11c) C arameter is used to balance the number of incorrect classifications and the size of the margin this arameter is Fig.3. Samle distribution of learning data belonging to nine classes Because real data collected on ower transformers (Fig.3) are distributed in a Cartesian coordinate system, it is difficult to effectively define the boundaries of regions they belong to. Although a art of the data is located in regions that can be quite easily isolated from each other, much of them is groued and artly mixed near the centre of the coordinate system and one of its axes. Additionally, a art of the decision area contains a small number of data (in ractice, some cases occur soradically and that is why few of them are recorded). What might also cause some difficulty is the disroortion between the numbers of data belonging to each category (Table 3), which is characteristic of this kind of technical roblems (in ractice, diagnosis is erformed on transformers that are heavily damaged). This makes the test even more challenging. Table 3. Number of measurements classified according to IEC code Category of technical condition Number of measurements No fault 33 Partial disrution of low energy 11 Partial disrution of high energy Full disrution of low energy 10 Full disrution of high energy 0 Overheating below C 9 Overheating (150 0 C, C Overheating (300 0 C, C 79 Overheating by more than C 9 Unrecognized fault IEC code not alicable 78 Total: 10 PRZEGLĄD ELEKTROTECNICZNY (Electrical Review), ISSN , R. 88 NR /01

4 As shown in Table 3, in the majority of cases we observe overheating by more than C and C. In a considerable number of cases the diagnosis cannot be made. Partial disrutions of energy are not very frequent. There is also quite a low number of transformers with no faults. This can be exlained by the fact that we more often insect those transformers whose erformance is visibly worse, in order to revent a serious fault or total damage. For verification of the method, measurements from 7 network transformers 50 MVA were available. From this set, the reeating and ambiguous cases must be sorted out. After removing them, there remain 391 examles, 15 out of which being unrecognized faults. Alication of the classifier based on the IEC reference to the mentioned data yields general correctness rate of about 5%. owever, in order to test the ability to create a learning machine, the given set should be slit into two sets one for training and one for testing. In Table detailed results for classification are resented. In the first column, different tyes of training datasets are listed and in the remaining columns there are classification results in the form of ercentage of correctly labeled objects for different SVM configurations, and IEC for comarison. The following datasets have been used: (1) All available oints have been used to train the classifier and then it has been tested on the same oint set. () alf of the oints have been taken for a training set whereas the remaining half - for testing (no reeating data). (3) 80% oint have been used for training, 0% for testing, again no reeating data. For the configuration, the following abbreviations have been used: (a) Linear kernel (see 7), C = 10 (b) Linear kernel, C = 50 (c) Radial kernel (see 8), r = 10, C = 50 (d) Radial kernel, r = 50, C = 50 (e) Radial kernel, r = 100, C = 50 (f) IEC Table. Classification results (in %) for different tyes of datasets Dataset (a) (b) (c) (d) (e) (f) (1) () (3) It follows from the above that SVM method yields better correctness in comarison to IEC code. Additionally, it emerges that radial basis kernel function allows achieving much better correctness in comarison to linear kernel. Fig. resents how C coefficient influences the fraction of correctly classified testing vectors in () dataset. As x values the C arameter was changed from 10 to 75 with ste 10, as y there are fractions of correctly classified vectors ( y 0,1 ). In fact, changing the enalty factor value does not influence much the classification correctness. The threshold value is about 30, after which there are virtually no changes in correctness. Because in SVM classifiers with radial kernel have definitely yielded better results, one should examine how exactly that kernel works. Fig. 5 resents the deendencies between radius value in the radial kernel function and the yielded correctness in () dataset. On the x axis there is r value from formula (8) changed from 5 to 75 with ste 5, the y axis contains again fraction of correctly classified vectors. The lower series deicts correctness achieved on a testing set. correctness 0, 0,598 0,59 0,59 0,59 0, arameter C Fig. The influence of C on the classification s correctness correctness 0,9 0,85 0,8 0,75 0,7 0,5 0, arameter r Fig 5. The influence of r on the classification s correctness Fig. IEC (to) versus SVM (bottom) results Obviously, the higher r value, the better is the correctness. On the other hand, higher r causes the classifier to be overtrained, which is tyical for a attern learning rocess. It means that r value must be a comromise between good correctness on training set and the ability to generalize knowledge, which in turn is deicted by correctness on a testing set. Fig. deicts the regions created by SVM as comared to the IEC ones. One can easily notice the difference: while IEC leaves most of the feature sace unrecognized, SVM PRZEGLĄD ELEKTROTECNICZNY (Electrical Review), ISSN , R. 88 NR /01 103

5 classifier introduces classification in the whole sace by extraolating the rules learnt from the training atterns. Eventually, the SVM method enables one to create a classifier which is caable of introducing feature sace searation that is about 15 0 ercent more accurate than the standard IEC aroach Summary The maximal margin classifier enables the creation of an exert system which diagnoses technical condition of an oil transformer basing on results of chromatograhy of gases dissolved in transformer oil (Dissolved Gas Analysis DGA). The system is designed to diagnose small and medium size units for which DGA is an essential source of information on their technical condition. The diagnosis is based on the searation of learning data, which enables the creation of regions reflecting the same state of the transformer. On that basis, new results of DGA can be examined more accurately than in the case of the IEC method. Therefore, a new method for transformer diagnosis suort has been develoed. The method, based on real measure data collected from a articular category of transformers, reflects in a way the rocess of learning from exerience. Of course, the imlementation of the resented system in industry will demand much further research and testing on numerous learning sets, articularly for the categories that are rarely recorded in ractice. The method resented can be also emloyed in other tasks of grouing and classification of hardly searable data. REFERENCES [1] I E C (1979): International Electrotechnical Commission, Interretation of the Analysis of Gases in Transformers and other Oil-Filled Electrical Equiment in Service. Geneva, [] B.E.B o s e r, I.M.Guyon, V.N.Vanik (199): A training algorithm for otimal margin classifier. In: D.aussler (Ed.): 5th Annual ACM Worksho on COLT. Pittsburg, PA, ACM Press, [3] V.N.Vanik (1995): The Nature of Statistical Learning Theory. Sringer-Verlag, Berlin, [] V.N.Vanik (1998): Statistical Learning Theory. Wiley, New York. [5] N.C r i s tianini (001): ICML 01 Tutorial. htt:// [] N.C r i s tianini, J.Shawe-Taylor (003): Suort Vectors and Kernel Methods. In: M.Berthold, D.J.and (Eds.): Intelligent Data Analysis; An Introduction. Sringer-Verlag, Berlin, eidelberg. [7] P.-.Chen, C.-J.Lin, B.Schölkof (001): A tutorial on - Suort Vector Machines. htt:// [8] C.C o r t e s, V.N.V a nik (1995): Suort Vector Networks. Machine Learning, 0, [9] T.M.C o v e r (195): Geometrical and Statistical Proerties of Systems of Linear Inequalities with Alications in Pattern Recognition. IEEE Trans. on Electronic Comuters, 1, [10] D.D e c o ste, B.S c h ölkof (00): Training Invariant Suort Machines. Machine Learning,, [11] V.K e cman (001): Learning and Soft Comuting. The MIT Press, Cambridge, Massachusetts. [1] B. Schölkof, C.J.C.Burges, A.J.Smola (Eds.): Advances in Kernel Methods: Suort Vector Learning. The MIT Press, Cambridge, MA; [13] Y.Lin, Y.Lee, G.Wahba (00): Suort vector machines for classification in nonstandard situations. Machine Learning,, nos.1-3; [1] B. S c hölkof (1997): Suort vector learning. R.Oldenbourg Verlag, München. Doktorarbeit, TU Berlin; htt:// [15] B.S c h ölkof, A.S m ola (00): Learning with Kernels: Suort Vector Machines, Regularization, Otimization and Beyond. MIT Press, 00. [1] M.U l e wi cz (003): Suort Vector Machines with Distortions for andwritten Digits Recognition. In: M.Kurzyński, E.Puchała, M.Woźniak (Eds.), KOSYR 003 Comuter Recognition Systems, Wrocław, Authors: rof. dr hab. inż. Piotr S. Szczeaniak, Technical University of Łódź, Institute of Information Technology, ul. Wólczańska 15, 90-9 Łódź, Poland, iotr@ics..lodz.l; mgr inż. Marcin Kłosiński, Technical University of Łódź, Institute of Information Technology, ul. Wólczańska 15, 90-9 Łódź, Poland, klosinskim@tt.com.l; 10 PRZEGLĄD ELEKTROTECNICZNY (Electrical Review), ISSN , R. 88 NR /01