AUTHOR COPY. A non-linear programming-based similarity reasoning scheme for modelling of monotonicity-preserving multi-input fuzzy inference systems

Transcription

1 Journal of Intelligent & Fuzzy Systems () DOI:./IFS-- IOS Press A non-linear programming-based similarity reasoning scheme for modelling of monotonicity-preserving multi-input fuzzy inference systems Kai Meng Tay a,, Tze Ling Jee a and Chee Peng Lim b a Department of Electronic Engineering, Faculty of Engineering, University Malaysia Sarawak, Malaysia b School of Computer Sciences, University of Science Malaysia, Malaysia Abstract. In this paper, the zero-order Sugeno Fuzzy Inference System (FIS) that preserves the monotonicity property is studied. The sufficient conditions for the zero-order Sugeno FIS model to satisfy the monotonicity property are exploited as a set of useful governing equations to facilitate the FIS modelling process. The sufficient conditions suggest a fuzzy partition (at the rule antecedent part) and a monotonically-ordered rule base (at the rule consequent part) that can preserve the monotonicity property. The investigation focuses on the use of two Similarity Reasoning (SR)-based methods, i.e., Analogical Reasoning (AR) and Fuzzy Rule Interpolation (FRI), to deduce each conclusion separately. It is shown that AR and FRI may not be a direct solution to modelling of a multi-input FIS model that fulfils the monotonicity property, owing to the difficulty in getting a set of monotonically-ordered conclusions. As such, a Non-Linear Programming (NLP)-based SR scheme for constructing a monotonicity-preserving multi-input FIS model is proposed. In the proposed scheme, AR or FRI is first used to predict the rule conclusion of each observation. Then, a search algorithm is adopted to look for a set of consequents with minimized root means square errors as compared with the predicted conclusions. A constraint imposed by the sufficient conditions is also included in the search process. Applicability of the proposed scheme to undertaking fuzzy Failure Mode and Effect Analysis (FMEA) tasks is demonstrated. The results indicate that the proposed NLP-based SR scheme is useful for preserving the monotonicity property for building a multi-input FIS model with an incomplete rule base. Keywords: Fuzzy inference system, similarity reasoning, analogical reasoning, fuzzy rule interpolation, non-linear programming, monotonicity property, failure mode and effect analysis. Introduction The Fuzzy Inference System (FIS), which is based on the concept of fuzzy set theory, fuzzy production rules, Corresponding author. Kai Meng Tay, Department of Electronic Engineering, Faculty of Engineering, University Malaysia Sarawak, Malaysia. kmtay@feng.unimas.my. and fuzzy reasoning, has been successfully applied to undertake a variety of problems, e.g., control [,, ], decision making [], selection [, ], assessment [, ], queuing [], fault diagnosis [], and approximation problems []. In general, there are two types of FIS models [, ], viz., First Inference Then Aggregate (FITA) and First Aggregate Then Inference (FATI). For FITA models, a crisp value is first -//$. IOS Press and the authors. All rights reserved

2 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling determined, and then the output estimate is obtained by aggregating the crisp values of the compatibility fuzzy rules. For FATI models, an aggregated fuzzy set is constructed from the rule consequents of the compatible fuzzy rules. Then, the output estimate is obtained by using a suitable defuzzification technique. In this paper, we study a class of FIS models that preserves the monotonicity property and that exhibits a better approximation property than those from the conventional FIS models. Our focus is on the zero-order Sugeno FIS model, which is a type of FITA model. Consider an FIS model, y = f (x,x,..., x i,..., x n ), that fulfils the condition of monotonicity between its output, y with respect to its i th input, x i, within the universe of discourse. In other words, y monotonically increases or decreases as x i increases, i.e., f (x,x,..., xi,..., x n) f (x,x,..., xi,..., x n)orf (x,x,..., xi,..., x n) f (x,x,..., xi,..., x n), respectively, for xi <xi. The importance of this line of study has been highlighted in a number of publications [,,,,,, ]. Several reasons have been provided to justify the importance of the monotonicity property of FIS models, which include (i) many real-world systems and control problems obey the monotonicity property [, ]; (ii) it is an important property such that the output of an FIS model is valid for undertaking comparison and decision making problems [,, ]; (iii) in the case whereby the number of data samples is small, it is important to fully exploit the monotonicity property as an additional qualitative knowledge/information for designing improved system identification and/or modelling procedures [, ]. From the literature, a number of studies on the monotonicity property have been reported. Examples include a set of sufficient conditions reported in [] for an FIS model to preserve the monotonicity property. The sufficient conditions are a set of mathematical conditions derived with the assumption that the first derivative of an FIS model is always greater than or equal to zero, or less than or equal to zero, for a monotonicity increasing or decreasing function, respectively. According to the sufficient conditions, two criteria (at the antecedent and consequent parts of a rule) are essential to obtain a monotonicity-preserving FIS function. For a zeroorder Sugeno FIS model, a set of comparable fuzzy sets that fulfils a particular mathematical inequality is required at the rule antecedent part. A monotonicityordered rule base is required at the rule consequent part. These criteria are applicable to the Sugeno-type of FIS models. The sufficient conditions are useful mathematical conditions that can be deployed as part of the FIS modelling procedure, i.e., the governing equations. From the literature, there are only a few articles that address the problem of designing monotonicity-preserving FIS models []. As an example, the sufficient conditions are investigated to design monotonicity-preserving FIS models in [, ]. The sufficient conditions have also been extended to other FIS models, e.g., a hierarchical FIS model [], a type-two FIS model [], an FIS model with fuzzy rule interpolation [], and an FIS model with a learning algorithm [, ]. In our previous studies, the sufficient conditions have been applied to the Failure Mode and Effect Analysis (FMEA) methodology [] and education assessment []. Recent advances in FIS modelling focus on the use of Similarity Reasoning (SR) as a solution to the incomplete rule base problem. Two typical SR schemes are Analogical Reasoning (AR) [] and Fuzzy Rule Interpolation (FRI) []. Obtaining a complete rule base is sometimes infeasible, especially for multi-input FIS models, owing to the large number of rules required. As an example, for the real case study on Failure Mode and Effect Analysis (FMEA) presented in section, a complete fuzzy rule base needs to have a total of rules. A fuzzy rule consists of an antecedent and a consequent. For an incomplete rule base, some consequents are unknown or missing. A conventional FIS model assumes that the unknown consequents are zero. This assumption may not always be true or appropriate; and this is the so-called tomato classification problem []. An SR scheme considers an antecedent with unknown consequent as an observation, and it deduces a conclusion (as a prediction of the consequent) for the observation based on the incomplete rule set. While the SR research is popular, a search in the literature reveals that the use of SR in monotonicity-preserving FIS modelling receives little attention. In our previous study [], we have reported a preliminary analysis on the use of FRI in monotonicity-preserving FIS models. It has been pointed out that conventional FRI that attempts to deduce each FIS rule consequent separately, with a simple weighted average function, may not be a direct solution to build a monotonicity-preserving multi-input FIS model []. This is because FRI faces the difficulty in obtaining a set of monotonically-ordered conclusions, with response to a set of observations, which are comparable among themselves as well as comparable with the existing rules in the incomplete rule base (as in section ).

3 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling The main objective of this study is to design and develop a monotonicity-preserving multi-input FIS model based on SR. It is assumed that fuzzy sets at the antecedent part are comparable, and they fulfil the sufficient conditions (as in section..). The sufficient conditions are exploited to be part of the FIS and SR modelling procedure. Besides, the incomplete rule base is assumed to follow a monotonic order. Two conventional SR schemes (i.e., AR and FRI) that attempt to deduce each FIS rule consequent separately with a simple weighted average function are analyzed. It is shown that these SR schemes are not sufficient to build a monotonicity-preserving multi-input FIS model, owing to the difficulty in obtaining a set of monotonic-ordered conclusions. Thus, a new SR scheme is proposed. The SR scheme is formulated as a constrained optimization problem. It consists of an objective function to be optimized under a set of inequality constraints. We further solve the constrained-optimization-based SR scheme with a Non-Linear Programming (NLP) method [,, ]. Specifically, the sequential quadratic programming algorithm, i.e., Quasi-Newton [], is adopted. To demonstrate the usefulness of the proposed NLPbased SR scheme, a case study on FMEA is examined. It is shown that the fuzzy Risk Priority Number (RPN) model (a three-input FIS model that requires the monotonicity property) suffers from the combinatorial rule explosion problem [], as it is tedious to collect a full set of fuzzy rules for the fuzzy RPN model. The proposed NLP-based SR scheme serves as a solution to this problem. The organization of this paper is as follows. In section, the background on FIS models, the sufficient conditions, and SR is described. In section, the use of two conventional SR schemes for building monotonicity-preserving multi-input FIS models is analyzed. In section, the proposed NLP-based SR scheme is explained and demonstrated with a numerical example. In section, applicability of the proposed NLP-based SR scheme to FMEA is evaluated, with the results analyzed. Concluding remarks are presented in section.. Background.. The representative value (Rep) of a fuzzy set For a fuzzy set A F(X), its representative value, Rep(A), carries important information with respect to the overall location, or the most typical location of the fuzzy set in the X domain. Rep(A) is a numerical value, which is obtained by a projection of A using a mathematical function/operator. It can be viewed as a simplified representation of a fuzzy membership function. Various defuzzification operators [] (e.g., centroid of gravity, mean of max, bisector of area, the smallest of max, the largest of max, etc.), center point, mean of suprema and infima of the α-cut of a fuzzy set, can be used to obtain Rep(A). The representative value of a single-dimensional fuzzy set can be extended to an n-dimensional fuzzy set. The n-dimensional fuzzy set in the X,X,... X n domain is represented with n numerical values. Each numerical value represents the most typical location of the n-dimensional fuzzy set when it is projected to the respective domain. The representative value plays an important role in FIS modelling. It is used as part of the FIS model to represent the most typical location of a fuzzy set at the rule consequent domain. In FRI, it is used as a guideline to derive the intermediate rules. It is also a simplified and explicit measure to compare the global feature of different FRI techniques in the uniform coordinate system [,, ]... The zero-order Sugeno fuzzy inference system and the sufficient conditions The zero-order Sugeno-type of FIS model and the sufficient conditions are explained in the following sections.... The zero-order Sugeno fuzzy inference system Fuzzy production rules for an n-input FIS model can be represented as follows: R j,j,...,j n : If (x is A j ) AND (x is A j )... AND (x n is A j n n ), THEN (yisb j,j,...,j n ) where j i M i. The product function is the AND operator in the rule antecedent. The compatibility grade, or known as the firing strength, of each fuzzy rule, i.e., R j,j,...,j n, is defined as µ j (x ), µ j (x ).. µ j n n (x n ). For the x i domain, its membership functions are µ i (x i), µ i (x i),..., and µ M i i (x i ). The output is obtained by using a weighted average of the representative value, b j,j,..., j n, with respect to its compatibility grade, as in Equation (). Equation () is a zero-order Sugeno FIS model.

4 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling y = f ( x) = jn=mn j n=... j =M j =M j = µ j ) (x ) µ j (x ).. µ j n n (x n ) b j,j,...,j n jn=mn j n=... j ( =M j =M j = j = µ j ) () (x ) µ j (x ).. µ j n n (x n ) j = (... The sufficient conditions The sufficient conditions of an FIS model (as in Equation ()) are derived based on dy/dx i ordy/dx i, for monotonically increasing or decreasing conditions, respectively [], using partial differentiation and the quotient rule. The sufficient conditions return a weighted addition series, whereby all components in the weighted addition series are assumed to be always greater than or equal to zero, or less than or equal to zero for monotonically increasing or decreasing conditions. The sufficient conditions that are able to preserve the monotonicity property of an FIS model are as follows. ( Condition. At the rule antecedent, p dµ i (x)/dx)/ µ p i (x) ( dµ q i (x)/dx)/ µ q i (x), where p>q. Note that (dµ (x)/dx) / µ (x) is the ratio between the rate of change in the membership degree and the membership degree itself. In this study, the Gaussian membership function, G (x : c, σ) = e [x c] /σ, is considered. The derivative of a Gaussian membership function with respect to x is G (x) = ((x c)/σ ) G (x). Note that (dµ (x)/dx) / µ (x) for a Gaussian membership function, i.e., (G (x)/g(x) ), returns a linear function, i.e., E (x) = G (x)/g(x) = (/σ )x + (c/σ ). Condition. At the rule consequent, b j,j,...,j n = p,...,j n b j,j,...,j n =q,...,j n or b j,j,...,j i = p,...,j n b j,j,...,j i =q,...,j n for dy/dx i or dy/dx i, respectively. This suggests that a monotonic rule base is required. If Conditions and are satisfied, the monotonicity property is satisfied. In addition, Condition indicates the importance of a monotonically-ordered rule base for a monotonicity-preserving FIS model... A review on similarity reasoning as a Conclusion Predictor The background and a review pertaining to SR schemes are presented as follows. where b j,j,..., j n = Rep (B j,j,..., j n ).... Background of similarity reasoning SR is viewed as an application of qualitative reasoning in FIS []. An example of SR from [] is: R: If pressure is high Then volume is small R: If pressure is low Then volume is large Therefore: If pressure is medium Then volume is (w small + w large), where w = sup (high medium), and w = sup (high medium). From the above example, R and R are two fuzzy rules from a rule base. Note that pressure is high is the antecedent and volume is small is the consequent of R. On the other hand, pressure is medium isan observation, and SR is used to deduce the conclusion of pressure is medium, i.e., w small + w large. A number of SR schemes have been proposed to allow a conclusion of an observation (in the form of a fuzzy set) to be deduced or predicted, based on a fuzzy rule base (database). In this study, SR is viewed as a computing paradigm, as shown in Fig.. An observation (in the form of a fuzzy set) acts as an input to the computing paradigm. The observation is compared with the antecedent part of each fuzzy rule in the database, and a similarity measure is produced for each fuzzy rule. The similarity measure represents how similar a fuzzy rule is with respect to the observation. There are many ways how this similarity measure can be derived. In [, ], several overlapping-based similarity measures are reviewed, compared, and analyzed. In [, ], a class of distance-based similarity measures is proposed. Figure depicts an observation and an antecedent of a fuzzy rule. In this study, sup Observation Antecedent, i.e., the overlapping degree of the observation and the antecedent, is used as the similarity measure. The representative values of the observation and the antecedent are labeled as Rep(Observation) and Rep(Antecedent), respectively. Their distance is defined as Rep(Observation) Rep(Antecedent), and the distance based similarity measure is /( Rep(Observation) Rep(Antecedent) ).

5 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling Observation The collection of fuzzy rules (database) Similarity measure Fuzzy rules selection Aggregation of conclusions Conclusion (fuzzy set, Center point, width, etc.) Fig.. A similarity reasoning paradigm. Fig.. Similarity measure between an observation and an antecedent of a fuzzy rule. Fuzzy rules from the database are then selected for aggregation of the conclusion. Various rule selection methods are available. In [], a threshold is introduced to select fuzzy rules with a similarity measure above the threshold. In [], the n closest fuzzy rules from the observation are selected for FRI. The importance of the ordering criteria in the selection of fuzzy rules has also been pointed out in [, ]. Next, the consequents of the selected fuzzy rules are aggregated. A mathematical function is adopted so that the property (or properties) of the conclusion can be obtained from the consequents. The similarity measure is used as an indication of the degree to which the property (or properties) of the consequent of each selected fuzzy rule contributes to the conclusion []. In [], the center point and the width of the conclusion are aggregated using a distance-based similarity measure via a weighted average function. In [], the conclusion is aggregated using a similarity measure via a weighted addition function. An observation (labeled as A m, where m =,,,..., n empty ) is considered. In stage (), the similarity measure of A m and each antecedent of the fuzzy rules, A l, S(A m,a l) is determined. The similarity measure for AR is S(A m,a l) = sup A m A l, while for FRI, it is S(A m,a l) = / Rep (A m ) Rep(A l).for a multi-input FIS model, the normalized Euclidean distance is adopted. In stage (), n select fuzzy rules with the highest similarity measure are selected. For FRI, n select fuzzy rules with the shortest fuzzy distance are selected. Note that an ordering criterion can also be included. In stage (), a conclusion is deduced. SR schemes usually attempt to predict each conclusion separately. For a zero-order Sugeno FIS model, as in Equation (), the representative value for the conclusion of A m can be deduced using Equation (). l=nselect Rep (Bm ) = l= S(A m,a l) Rep (B l ) l=nselect l= S(A m,a () l)... Generalization of similarity reasoning An n-input FIS model with n available fuzzy rules, (R,R,..., R navailable ), in an incomplete rule base, where R l : A l B l (l =,,,.., n available ), is considered. There are n empty observations (A, A,..., A n empty ), where n empty >. The conclusions for the n empty observations are B,B,..., B n empty, respectively. SR schemes can be used to predict the missing rules in three stages, i.e., () similarity measure (S), () fuzzy rule selection (f select ), and () inference of conclusion (f inference ). It is also possible to generalize SR as an input-output mathematical function as in Equation (). Rep (B m ) = f inference (f select (R,R,..., R navailable,a m ) ) () Each conclusion (in the form of the representative value) is deduced separately using the above three stages. The same procedure is repeated for n empty times.

6 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling. Similarity reasoning for modelling of monotonicity-preserving multi-input fuzzy inference systems In this section, the use of SR for modelling of monotonic multi-input FIS models is described, as follows... Analysis A monotonic n-input FIS model, where n>, is considered. Figure illustrates an example of an n-input FIS modelling problem, y = f (x,x,..., x i,..., x n ). Two of its inputs, x u and x v are further depicted. According to the monotonicity property, y should increase/decrease as x u and/or x v increases. The assumptions of our analysis are as follows. () The membership functions of each input satisfy Condition. For the example in Fig., the membership functions of x u (µ u,µ u,µ u and µ u ) and x v (µ v,µ v,µ v and µ v ) are generated according to Condition. () The rule base is incomplete, i.e., with only n available rules. Referring to Fig., R, R, R, R, and R are part of the n available rules. The observations are A and A. () Condition is satisfied among the n available rules. For the example in Fig., it is necessary that Rep (B) Rep (B), Rep (B) Rep (B), and Rep (B) Rep (B) in order to make a valid comparison among R, R, R, R, and R. Condition suggests an ordering criterion is needed for n available rules and n empty observations. For the example in Fig., Rep(B) Rep(B ) Rep(B ) Rep(B). With the use of SR (as explained in section.), this condition may not be satisfied always. Referring to Fig., Rep(B ) Rep(B)is violated if a selected rule with a high similarity measure with respect to A (e.g., R) has a relatively low representative value for its consequent, as compared with those of A. Similarly, Rep(B ) Rep(B ) is violated if a selected rule with a high similarity measure with respect to A (e.g., R) has a relatively high representative value for its consequent, as compared with those of A... A numerical example and simulation A monotonic two-input FIS model, z = f (x, y), is considered. It is known that the relationship between f (x, y)and its inputs, x and y, observe the monotonically increasing dynamics. Inputs x and y consist of five Gaussian membership functions, mf x, mf x, mf x, mf x, and mf x, as shown in Fig. a, and mf y,mf y,mf y,mf y and mf y, as shown in Fig. b, respectively. These membership functions satisfy Condition. They can be projected using the ratio suggested by Condition ((dµ i (x) dx) /µ i (x). Fig. a and b show the projections of the membership functions of x and y, respectively. A complete rule base should have rules. The output domain, z, consists of membership functions, namely mf z,mf z,mf z,mf z and mf z, with representative values,,,, and, respectively. Fig.. Similarity reasoning for a monotonic multi-input FIS model.

7 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling (a) (b) Fig.. (a) Membership functions for x, and (b) Membership functions for y (a) Fig.. (a) Projection for Membership functions for x, and (b) Projection for Membership functions for y. In this example, suppose there are available rules (as summarized in Fig. ) and observations/conclusions. Note that the available rules fulfill Condition. A rule matrix, as shown in Fig., can be used to represent this problem. For example, R from Fig. is represented as R : mf z, and is located in the cell with coordinate (mf x,mf y ), as in Fig.. Observation A, with coordinate (mf x, mf y ) is mapped to an unknown conclusion, B, whereby B needs to be predicted. Two SR schemes (explained in..), i.e., AR and FRI, are used to deduce the consequents associated with the observations. With the deduced consequents, the two-input FIS model, z = f (x, y), is built. Figures and depict the surface plots of z versus x and y using AR and FRI, respectively. Notice that nonmonotonic surface plots are obtained. This is owing to non-monotonically ordered conclusions are deduced by AR and FRI. For example, using AR, rep(b ) is predicted to be. and rep(b ) to be.. Using FRI, rep(b ) is predicted to be. and rep(b )to be.. The above analysis suggests that using SR (which is based on a weighted average) and deducing each conclusion separately may not produce a monotonicitypreserving FIS model. It does not explain the situations (b) : If is and is, Then is : If is and is, Then is : If is and is, Then is : If is and is, Then is : If is and is, Then is : If is and is, Then is : If is and is, Then is : If is and is, Then is Fig.. Fuzzy rules in the database. whereby a set of monotonically-ordered conclusions cannot be obtained (Condition ), as follows: Situation : A deduced conclusion of an observation may not be comparable with the available fuzzy rules. Situation : It is difficult to deduce a set of conclusions (for a set of observations) which are comparable among each other.. A proposed non-linear programming based similarity reasoning scheme Instead of deducing each consequent separately as in Equation (), an alternative that attempts to deduce the conclusion set simultaneously with a search and optimization procedure, g, is proposed. It is summarized in Equation (). ( ) Rep(B ),Rep(B ),..., Rep(B n empty ) ( ) = g R,R,..., R navailable,rep(a ),Rep(A ),..., Rep(A n empty ) ()

8 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling Fig.. A rule matrix for the numerical example. N.. Problem formulation A novel formulation of SR for monotonicitypreserving multi-input FIS models as a constrained optimization problem is proposed, as follows: Minimize f (b,b,..., b n empty ), where b = rep(b) Subjected to Constraint #. Each conclusion is comparable with the rule base collection, i.e., b m b m b m, whereby b m is the highest possible representative value for the consequent of the observation, b m is the lowest possible representative value for consequent of the observation. As shown in Fig., for observation A, its conclusion is B, whereby b b b. Fuzzy rules which are expected to have higher and lower consequents are highlighted in darker-grey and lighter-grey, respectively. Hence, b = min (rep(mf z), rep(mf z ), rep(mf z )), and b = max (rep(mf z), rep (mf z )). #. Conclusions are comparable among themselves, i.e., a set of monotonically-ordered conclusions exists, i.e., b m b m b m. As shown in Fig., rep(b ) rep(b ) rep(b ) rep(b ). Constraints # and # are a set of inequality constraints. In this paper, we examine the use of a proximity measure function which needs to Y X Fig.. Surface plot of z versus x and y using AR. be minimized in the proposed SR scheme. Each conclusion is predicted with either AR or FRI, and a set of reference consequents is obtained. A reference conclusion for Rep(B ) is denoted as Rep(B ) R. For a set of candidates, denoted by (Rep(B ) c,rep(b ) c,..., Rep(B n empty ) c ), its proximity measure is expressed with a root mean square function, as in Equation (). Proximity measure = z nempty m= (b mr b ), mc where b = rep(b) () The function is a measure of closeness between the candidate and the reference consequent. The lower the proximity measure, the closer the candidate to the reference consequent... The search procedure From the problem formulation in section., SR for monotonicity-preserving multi-input FIS models can be solved with a Non-Linear Programming (NLP) [, ] method. Alternatively, a population-based stochastic optimization method, e.g., Genetic Algorithm (GA) [] or Particle Swarm Optimization (PSO) [], can be

9 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling used. In this paper, we focus on the use of NLP since it is straightforward and easy to implement, as follows.... Non-linear programming NLP is a method for solving a system with equality and/or inequality constraints, over a set of unknown real variables. It has an objective function to be maximized or minimized, whereby some of the constraint(s) or the objective function is/are nonlinear [, ]. It has been applied to many problems, which include procurement, management, break-even analysis, and logistic [, ]. For the NLP-based SR scheme, Constraints # and # are the inequality constraints. In this paper, a sequential quadratic programming of NLP with a quasi- Newton updating method [] is adopted. It is one of the popular and effective methods to solve constrained non-linear optimizations problems ) []. The objective function, f (b,b,..., b n empty, to be minimized is shown in Equation (). ) Objective function = f (b,b,..., b n empty nempty = z (bnr b nc ) () n= Sequential quadratic programming solves a series of quadratic programming sub-problem by iteration. It starts with a predefined initial set of bn. At each iteration, an approximation is made of the Hessian of the Lagrangian function using a quasi-newton updating method []. A quadratic programming sub-problem is generated, and its solution is used to form a search direction for a line search procedure. The iteration is terminated when the stopping criteria are met. In short, each iteration consists of three major steps, as follows: a) Generating a Hessian matrix: a quasi-newton approximation of the Hessian of the Lagrangian function; b) Generating and solving a quadratic programming sub-problem; c) Performing line search: solution from step (b) is used to generate a new set b n. Further information of the method can be found in []... Results from the numerical example In this section, the results for the numerical example in section. using the NLP-based SR scheme are presented. Figures and depict the surface plots of z versus x and y using the optimized conclusions set (b,b,..., b n empty ), with AR and FRI as the reference consequents, respectively. Notice that monotonic surface plots are observed. Thus, the proposed NLP-based SR scheme is able to provide a solution for constructing monotonicity-preserving FIS models.. A case study on failure mode and effect analysis In this section, the application of the proposed model is demonstrated with a real case study on FMEA in a semiconductor manufacturing plant, as follows... The FIS-based risk priority number model A search in the literature reveals that the use of advanced computing techniques in quality and reliability studies is not new [,, ]. In this domain, FMEA N Y X Fig.. Surface plot of z versus x and y using FRI.

10 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling.. N... Y Fig.. Surface plot of z versus x and y using the proposed NLP-based SR scheme using the reference consequents from AR. N Y Fig.. Surface plot of z versus x and y using the proposed NLP-based SR scheme using the reference consequents from FRI. is a popular problem prevention methodology that can be interfaced with many quality and reliability models []. It is a systemized group of activities intended to recognize and to evaluate the potential failures of a product/process and the associated effects []. FMEA identifies actions which can reduce or eliminate the chances of the potential failures from recurring. Conventional FMEA uses a Risk Priority Number (RPN) model to evaluate the risk associated with each failure mode. The RPN is a product of three risk factors, i.e., Severity (S), Occurrence (O), and Detect (D) (RPN = S O D). In this case study, a fuzzy RPN model is used as an alternative to overcome some of the weaknesses of the conventional RPN model []. In the fuzzy RPN model, the product function is replaced by an FIS model. The inputs (S, O, and D risk factors) and output (RPN score) are represented with membership X X functions. From the literature, a number of FIS-based RPN models have been successfully applied to FMEA. Examples include an auxiliary feed water system and a chemical volume control system in a nuclear power plant [, ], an engine system [], a semiconductor manufacturing line [], and a fishing vessel []. Various improvements to FIS-based RPN models have also been proposed. In [], a fuzzy rule-based Bayesian reasoning method for prioritizing failures in FMEA has been reported. An FIS-based RPN model using the grey relation theory has also been suggested []. In our previous studies [,, ], it has been argued that an effective FIS-based RPN should satisfy the monotonicity property. In [], we presented an FMEA framework with an FIS-based RPN and the sufficient conditions, and showed that the proposed method acts an effective, reliable, and practical method to preserve the monotonicity property among S, O, D and RPN.

11 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling Table The scale table for severity Rank Linguistic terms Criteria Very high (liability) Failure will affect safety or compliance to law High (reliability/reputation) Customer impact. Major reliability excursions Moderate (quality/convenience) Impacts customer yield. Wrong package/par/marking Low (special handling) Yield hit, cosmetic None (unnoticed) Unnoticed However, to the best of our knowledge, the use of SR in FMEA is new. Besides, investigations on handling an incomplete rule base in FMEA are lacking. Hence, we attempt to address these two issues in this study. Similar to the conventional RPN model, the FISbased RPN model takes S, O, and D as the inputs, and produces an RPN score as the output. The three input factors are estimated by domain experts in accordance with a scale from to based on a set of commonly agreed evaluation criteria. Tables summarize the evaluation criteria, which are used in a semiconductor manufacturing plant, for S, O, and D ratings, respectively. The membership functions of S, O, and D can be generated based on the criteria in Tables respectively. Figures depict the fuzzy membership functions for S(µ S ), O(µ O ), and D(µ D ), respectively. As an example, referring to Fig., the second membership function of S, i.e., µ S, with a linguistic label of Low represents S ratings from to, RPN = f RPN (S, O, D) which correspond to Yield hit, Cosmetic as in Table. The same scenario applies to Fig., e.g., the Moderate membership, i.e., µ O, represents O ratings from to, which correspond to Once/week, Several/month as in Table. In Fig., the High membership function, i.e., µ D, represents D ratings from and, which correspond to Controls are able to Detect within the same machine/module as in Table. The output of the FIS-based RPN model, i.e., the RPN score, varies from to. In this case study, it is divided into five equal partitions, with five fuzzy membership functions of B, i.e., Low, Low Medium, Medium, High Medium, and High, respectively. The corresponding scores of b (the representative value of the output membership functions) are assumed to be the point whereby the membership value of B is. = MS MO MD a= b= MS MO a= b= Table The scale table for occurrence Rank Linguistic terms Criteria Very high Many/shift, many/day High Many/week, few/week Moderate Once/week, several/month Low Once/month Very low Once/quarter Remote Once ever Hence, b is,.,.,., and, respectively. A fuzzy rule base is a collection of knowledge from the experts in the If-Then format. Considering S, O, and D, and their linguistic terms, the fuzzy rule base has ( (S) (O) (D)) rules in total using the grid partition approach. Figure shows two fuzzy rules collected from the domain experts (maintenance engineers). In this study, the zero-order Sugeno FIS is used to produce the RPN score, i.e. c= µa S (S) µb O (O) µc D (D) ba,b,c MD c= µa S (S) µb O (O) µc D (D) ()... The monotonicity property It is argued that an FIS-based RPN model needs to satisfy the monotonicity property. This is because the input attributes (i.e., S, O, and D ratings) are defined in such a way that the higher the rating, the more critical the situation is. The output (i.e., the RPN score) is a measure of the failure risk. The monotonicity property is important to allow a valid comparison among all failure modes to be made. For example, consider two failure modes with input sets [,, ] (representing [S, O, and D]) and [,, ]. The RPN score for the second failure mode should be higher than, or at least equal to, that of the first. The prediction is deemed illogical if the RPN model yields a contradictory result. This can be explained by referring to Tables. Let the two failure modes have the same S and O ratings of, but

12 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling.. None Low Moderate High Very High.. Remote.... Very Low Low Fig.. The membership function of Severity. Moderate.... Fig.. The membership function of Occurrence. High Very High Very High High Moderate ow Very Low EXtremely Low Fig.. The membership function of Detect. Table The scale table for Detect Rank Linguistic terms Criteria Extremely low No control available Very low Controls probably will not Detect Low Controls may not Detect excursion until reach next functional area Moderate Controls are able to Detect within the same functional area High Controls are able to Detect within the same machine/module Very high Prevent excursion from occurring with different D ratings of and, respectively. The failure mode with D of ( Controls are able to Detect within the same functional area ) represents a better control mechanism than that of D of ( Controls may not Detect excursion until reach next functional area. ). Thus, the RPN score for [,, ] should be lower than that of [,, ]. The monotonicity property states that when D increases, the RPN score should increase, or at least be maintained or, in other words, it should not decrease.

13 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling Rule If Severity is Very High and Occurrence is Very High and Detect is Extremely Low then RPN is High. Rule If Severity is Very High and Occurrence is Very High and Detect is Very Low then RPN is High Fig.. An example of two fuzzy production rules. Define the Scale tables for Severity, Occurrence and Detect Yes Severity Membership functions generation (Condition ) FMEA users Evaluate the probability of each causes to occur (Occurrence ranking) Occurrence Membership functions generation (Condition ) Fuzzy rule selection Expert Knowledge Collection (Condition ) The proposed Monotonicity-preserving Similarity Reasoning scheme Study about the process/product and divide the process/product to subprocesses/components Determine all potential failure mode of each component/process Determine the effects of each failure mode Determine the root causes of each failure mode List current control/prevention of each cause Evaluate the efficiency of the control/prevention (Detect Ranking) Detect Membership functions generation (Condition ) Evaluate the impact of each effect (Severity ranking) FIS based RPN model FIS based-rpn calculation Correction required No End Fig.. The proposed FMEA procedure with an FIS-based RPN coupled with a monotonicity-preserving NLP-based SR scheme... The proposed FMEA procedure In our previous study [], the sufficient conditions have been applied to an FIS-based RPN model. In this paper, a monotonicity-preserving SR scheme is further incorporated as part of the FMEA procedure. Figure depicts a flow chart for the proposed FMEA procedure with an FIS-based RPN model

14 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling High Moderate Low None Very High Fig.. Projection of Membership functions for Severity. Very High High - - Moderate - Low - Very Low - Remote - Fig.. Projection of Membership functions for Occurrence. EXtremely Low Very Low Low Moderate - High Very High - coupled with a monotonicity-preserving SR scheme. Condition is used to generate membership functions of S, O, and D. Figures show the fuzzy membership functions of S, O, and D that satisfy Condition. Figures are the projections of these membership functions using Condition, i.e., (dµ i (x)/dx)/µ i (x)), for S, O, and D respectively. A rule selection procedure is first conducted to select a number of fuzzy rules to form an initial (incomplete) rule base. The validity of the selected fuzzy rules is checked using Condition, whereby a monotonically-ordered rule base is expected. A monotonicity-preserving SR scheme is further included to predict the missing fuzzy rules, in order to form a complete rule base. This approach, therefore, reduces the laborious process of Fig.. Projection of Membership functions for Detect. acquiring a large number of fuzzy rules (i.e., in this case study) that are needed to form a complete rule base before the FMEA framework can be used... Experiments To validate the proposed scheme, a series of experiments with data/information collected from a semiconductor manufacturing process of the Flip Chip Ball Grid Array (FCBGA) products is conducted. FCBGA is a low cost semiconductor packaging solution which utilizes the Controlled Collapse Chip Connect technology, or known as Flip Chip (FC), for its die to substrate interconnection []. FC was initiated at

15 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling the early s to eliminate the expanse, unreliability, and low productivity of the manual wire bonding process []. In this paper, the wafer mounting process in FCBGA manufacturing is studied. In the wafer mounting process, a wafer is mounted on a plastic tape that is attached to a ring. It ensures that dies from the wafer remain firmly in place during the wafer dicing process. In the experiment, the proposed FIS-based RPN model coupled with the FMEA procedure in Fig. is deployed under the following scenarios: () The fuzzy rule base is complete (all the fuzzy rules are selected) and SR is not required; () The fuzzy rule base is incomplete (% and % of the fuzzy rules are randomly selected) and without SR; () The fuzzy rule base is incomplete (% and % of the fuzzy rules are randomly selected) and two SR methods, i.e., AR and FRI, are used to deduce the missing fuzzy rules; () The fuzzy rule base is incomplete (% and % of the fuzzy rules are randomly selected), and the monotonicity-preserving SR scheme (as explained in section ) is used to deduce the missing fuzzy rules. Note that when SR is not used, the conclusions are assumed to be zero.... Results Table summarizes the failure risk evaluation, ranking, and prioritization results using the traditional and FIS-based RPN models for the wafer mounting process. Columns Sev (Severity), Occ (Occurrence), and Det (Detect) show the three attribute ratings describing all the failure modes. Failure risk evaluation and prioritization outcomes based on the traditional RPN model are shown in columns RPN and RPN rank, respectively. As an example, in Table, failure mode represents broken wafer, which leads to yield loss, and is given a severity score of (refer to Table ). This failure happens because of drawing out arm failure, and because it rarely happens, it is assigned an occurrence score of (refer to Table ). In order to overcome this failure, software enhancement is taken as a corrective action. Owing to the corrective action taken is effective, which is able to eliminate the root cause of failure; a detect score of is given (refer to Table ). Using the traditional RPN model, an RPN score of is obtained, with the lowest RPN ranking (RPN rank = ). Columns FIS-based RPN model (%), FISbased RPN model (%), and FIS-based RPN model (%) show the failures risk evaluation results using the FIS-based RPN models with % (complete rule base), %, and % fuzzy rules, respectively. Under FIS-based RPN model (%), sub-columns FRPN and Experts Knowledge indicate the failure risk evaluation outcomes and the linguistic terms assigned by the domain experts for the complete fuzzy rule base. Based on the example given, failure mode is tagged with a fuzzy RPN score of., and is assigned a linguistic value of Low, with b of.. Under FIS-based RPN model (%) and FISbased RPN model (%), the results with incomplete fuzzy rule bases, i.e., with % and % available fuzzy rules, respectively, are reported. Sub-columns Fuzzy RPN (Without SR), Fuzzy RPN (AR) and Fuzzy RPN (FRI) present the evaluation results without SR, with AR, and with FRI, respectively. Referring to the example given, fuzzy RPN scores of. (without SR),. (with AR), and. (with FRI) are obtained for % fuzzy rules. For % fuzzy rules, fuzzy RPN scores of. (without SR),. (with AR), and. (with FRI) are produced, respectively. Sub-columns Fuzzy RPN (Monotonic AR) and Fuzzy RPN (Monotonic FRI) report the evaluation results of the FIS-based RPN model with the proposed NLP-based SR scheme (by using AR and FRI to predict the reference conclusions, respectively). Again, for the given example, fuzzy RPN scores of. (with monotonic AR) and. (with monotonic FRI) are obtained. For % fuzzy rules, a fuzzy RPN score of. is obtained for both monotonic AR and FRI predictions.... Analysis of the monotonicity property In this section, the monotonicity property of the FISbased RPN models is analyzed. Based on Table, with the complete fuzzy rule base, fuzzy RPN scores for failure modes,, and ([,, ], [,, ], and [,, ], for [S, O, D], respectively) are.,. and.. Correlating between the O ratings and the fuzzy RPN scores, the monotonicity property is satisfied. For % fuzzy rules without SR, the fuzzy RPN scores are.,., and., respectively, for failure modes,, and. The scores are close to zero because all three failure modes fall within the region whereby the fuzzy rules are missing and the conclusions are undefined; therefore they are mapped to zero since no SR is applied. This is the so-called tomato classification problem. The use of AR and FRI overcomes this problem to a certain extend. With AR, the fuzzy RPN scores are.,., and., respectively. With FRI, the fuzzy RPN scores are.,., and., respectively.

16 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling Table Failure risk evaluation, ranking, and prioritization results with the traditional RPN, as well as the FIS-based RPN and with NLP-based SR scheme for the wafer mounting process Failures Inputs ranking RPN RPN FIS-based RPN model (%) FIS-based RPN models (%) FIS-based RPN models (%) mode rank Fuzzy Experts Fuzzy Fuzzy Fuzzy Fuzzy RPN Fuzzy RPN Fuzzy RPN Fuzzy Fuzzy Fuzzy RPN Fuzzy RPN RPN Knowledge RPN (without RPN RPN (monotonic FRI) (without (AR) RPN AR) (monotonic (FPR) SR) (AR) (FRI) AR) FRI) SR) (FRI) AR) FRI) Sev Occ Det Linguistic term b. Low Low Low Low Low Low Low Medium Low Medium Low Medium Low Medium Low Medium Low Medium Low Medium Low Medium Low Medium Medium Medium

17 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling RPN RPN - RPN Occurrence Detect Fig.. Surface plot for a complete fuzzy rule base. Occurrence Detect Fig.. Surface plot for % fuzzy rules and without SR. Occurrence Detect Fig.. Surface plot for % fuzzy rules and with AR.

18 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling RPN RPN Occurrence Detect Fig.. Surface plot for % fuzzy rules and with FRI. Occurrence Detect Fig.. Surface plot for % fuzzy rules and with AR predictions used as reference conclusions in the NLP-based SR scheme. RPN Occurrence Detect Fig.. Surface plot for % fuzzy rules and with FRI predictions used as reference conclusions in the NLP-based SR scheme.

19 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling RPN RPN RPN Occurrence Detect Fig.. Surface plot for % fuzzy rules and without SR. Occurrence Detect Fig.. Surface plot for % fuzzy rules and with AR. Occurrence Detect Fig.. Surface plot for % fuzzy rules and with FRI.

20 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling RPN Occurrence Detect Fig.. Surface plot for % fuzzy rules and with AR predictions used as reference conclusions in the NLP-based SR scheme. RPN Occurrence Detect Fig.. Surface plot for % fuzzy rules and with FRI predictions used as reference conclusions in the NLP-based SR scheme. For the case of % fuzzy rules without SR, the fuzzy RPN scores are.,., and., respectively. The scores are not close to zero because all three failure modes fall within the region whereby the fuzzy rules are available. However, the fuzzy RPN score for failure mode is relatively low. This is because some of its neighboring fuzzy rules are missing. As such, the weighted average method returns a relatively low fuzzy RPN score. Again, the use of AR and FRI improves the situation to a certain extend. The fuzzy RPN scores are.,., and. with AR, and.,., and. with FRI, respectively. Notice that while AR and FRI improve the predictions for overcoming the tomato classification problem, the monotonicity property is not satisfied. This is because both AR and FRI do not deduce conclusions that fulfill Condition. The proposed NLP-based SR scheme serves as a solution. By using the NLP-based SR scheme with AR predictions as reference conclusions, the fuzzy RPN scores are.,., and., respectively, for % fuzzy rules. Similarly, the fuzzy RPN scores are.,. and., respectively, while the use of FRI predictions as reference conclusions. For % fuzzy rules, the fuzzy RPN scores are.,. and., as well as.,. and., respectively, with AR and FRI predictions as reference conclusions. In summary, the monotonicity property is satisfied with the proposed NLP-based SR scheme.... Surface plots of the FIS-based RPN models The monotonicity property can also be observed easily by using the surface plot. Figures and depict

21 K.M. Tay et al. / NLP-based similarity reasoning for monotone fuzzy modelling the surface plots of RPN versus O and D, with S fixed to, for the complete fuzzy rule base and for % fuzzy rules without SR, respectively. A monotonic surface is shown in Fig.. By increasing S or D from to, the RPN scores either increase or remain constant. But, note that the rule collection process involved is tedious as all rules have to be defined beforehand. On the other hand, the surface plot is non-monotonic in Fig.. This is because there are areas whereby the respective conclusions are undefined and are, therefore, mapped to zero. This problem can be solved by using AR and FRI. Figures and depict the resulting surface plots. While smoother surface plots are obtained, they do not satisfy the monotonicity property, as highlighted in Figs. and. Figures and depict the surface plots of RPN versus O and D, with S fixed to, for the monotonicity-preserving SR scheme, i.e., with AR and FRI predictions used as reference conclusions, respectively. Both surface plots are smooth, and they obey the monotonicity property. Figures depict the surface plots of RPN versus O and D, with S fixed to, for % fuzzy rules without SR, with AR, and with FRI, respectively. Without SR, the surface plot is non-monotonic, as there are areas in which the conclusions are undefined and are assumed to be zero. Again, AR and FRI can improve the situation, but without satisfying the monotonicity property, as highlighted in Figs.. Figures and depict the surface plots of RPN versus O and D, with S fixed to, for monotonicitypreserving SR scheme with AR and FRI predictions used as reference conclusions, respectively. Again, both surface plots are smooth and they obey the monotonicity property.. Summary In this paper, we have shown that the use of AR and FRI to predict each rule consequent separately and the use of a simple weighted average function may not be a direct solution for constructing monotonicitypreserving multi-input FIS models. An alternative based on SR is therefore proposed. In the proposed SR scheme, an NLP-based search procedure is adopted to search for a set of conclusions that obey the sufficient conditions with the minimum proximity measure. A numerical example is first presented to clarify the proposed NLP-based SR scheme. Then, the effectiveness of the proposed scheme is demonstrated using an FIS-based RPN model in FMEA. The proposed scheme is effective in preserving the monotonicity property of the FIS-based RPN model and, at the same time, in overcoming the combinatorial explosion problem in FMEA rule collection. As such, the NLP-based SR scheme is useful for undertaking the incomplete fuzzy rule base problem in constructing monotonicity-preserving multi-input FIS models. In addition to NLP, further work is currently underway to employ GA [] or PSO [] as a search procedure for this problem. Besides, other proximity measure functions are investigated in order to improve the robustness of the monotonicitypreserving multi-input FIS models. Acknowledgments The authors gratefully acknowledge the Fundamental Research Grant Scheme (FRGS/()// () and No. ), the Exploratory Research Grant Scheme (ERGS///TK/UNIMAS//), and USM Research University Grant (No. ) for supporting this research work. References [] P. Baranyi, L.T. Kóczy and T.D. Gedeon, A generalized concept for fuzzy rule interpolation, IEEE T Fuzzy Syst () (),. [] J.B. Bowles and C.E. Peláez, Fuzzy logic prioritization of failures in a system failure mode effects and criticality analysis, Reliab Eng Syst Safe () (),. [] E.V. Broekhoven and B.V. Baets, Monotone Mamdani Assilian models under mean of maxima defuzzification, Fuzzy Set Syst () (),. [] E.V. Broekhoven and B.D. Baets, Only smooth rule bases can generate monotone Mamdani Assilian models under centerof-gravity defuzzification, IEEE T Fuzzy Syst () (),. [] O. Cordon, F. Herrera and A. Peregrin, Applicability of the fuzzy operators in the design of fuzzy logic controllers, Fuzzy Set Syst () (),. [] M.R. Emami, I.B. Turksen and A.A. Goldenberg, A unified parameterized formulation of reasoning in fuzzy modeling and control, Fuzzy Set Syst () (),. [] A. Escobet, A. Nebot and F.E. Cellier, Fault diagnosis system based on fuzzy logic: Application to a valve actuator benchmark, J Intell Fuzzy Syst () (),. [] D.E. Goldberg, Genetic algorithms in search, optimization, and machine learning, Addison-Wesley Professional,. [] A.C.F. Guimarães and C.M.F. Lapa, Effects analysis fuzzy inference system in nuclear problems using approximate reasoning, Ann Nucl Energy () (),. [] A.C.F. Guimarães and C.M.F. Lapa, Fuzzy FMEA applied to PWR chemical and volume control system, Prog Nucl Energ () (),.