A Fast Approximation Algorithm for Set-Membership System Identification

Transcription

1 A Fast Approximation Algorithm for Set-Membership System Identification J.A. Castaño F. Ruiz J. Régnier Pontificia Universidad Javeriana, Bogotá, Colombia. Université de Toulouse, INPT, CNRS, LAPLACE, France. Abstract: The Set Membership nonlinear identification method is a flexible technique, suitable for the identification of non-linear systems where no information on the system structure is available. This method generates a non-parametric model, embedded on the identification data set, with optimality properties and bounds on the possible values the variable can assume. However, the complexity of the model grows with the product of the input dimension and the size of the data set. This is problematic when a big data set is employed. In this paper, the nearest point approximation to the exact Set Membership model, found in literature, is analyzed and a novel approximation is proposed, whose complexity does not depend on the size of the data set. Guaranteed bounds on the worst-case approximation error are given. Two examples, considering simulated and experimental data, illustrate the validity and applicability of the obtained results. Keywords: System Identification, Set Membership, Nonlinear systems. 1. INTRODUCTION Non-linear system identification is one of the most active research areas in control engineering. Novel identification algorithms have been proposed in recent years, all of them oriented to the construction of a more general theory of non-linear identification. The emerging methods differ on the a priori information required for the identification and the hypotheses made on the system to be identified. For example, model structure, function characteristics and parametrization. The application areas of nonlinear identification are multiple and so are the properties required on the method employed for the identification and on the characteristics of the resulting model. The selection of a method not wellsuited to the application can lead to a low performance of the solution or even infeasibility of the implementation. The selection of a particular method among the others should be based on the specific needs of the application. For example, low computational complexity in the identification phase or in the simulation task Jiang et al. (9), high flexibility of the approximating function Canale et al. (9), low parameters vector dimension Hassouna and Coirault (), etc. However, most, if not all, of the recently proposed identification methods are oriented to the analysis of mathematical properties of the algorithms, without any concern on the computational cost of the resulting algorithms. Artificial neural networks are a classical method for nonlinear system identification, Chu et al. (1989); Yu and Li 1 Part of this study is in the framework of French- Colombian cooperation, with support of the ECOS program (ECOSNord/COLCIENCIAS-ICETEX / Project No. C1P1). (1). They offer strong approximation properties and efficient training algorithms are available for the identification phase, but their main drawback is the non-convexity of the related optimization problems and the lack of results about optimality and guaranteed error after network training.interconnectedmodels,suchaswienerandhammerstein, have also been reported in literature, Espinoza et al. (5); Hassouna and Coirault (); Chen et al. (5),thesemodelsarehighlystructured,assumingseries and/or parallel interconnections of dynamic linear blocks and (usually static) nonlinear functions, such that structure limits the class of systems that can be represented by these models. Orthogonal functions are widely used in nonlinear system identification, their main drawback is the exponential complexity grow as the dimension of the functiondomainincreases,perezandtsujii(1991). Efforts havebeenmadeinordertolimitthemodelcomplexity,bai (8); Bai et al. (7), but most of there results apply only to FIR (Finite Impulse Response) systems, making them difficult to apply in practice. A different framework to make inferences from data measurements on non linear systems is the Set Membership formulation Milanese and Novara (9). This method applies to the problems of identification, prediction and filtering, and allows to compute optimal or almost optimal inferences with finite. In particular, a solution to the system identification problem was proposed in Milanese and Novara (4). The formulation requires only Lipschitz continuity of the function to be estimated and assumes additive noise with limited amplitude. The generated model is non-parametric, been embedded in the data. The algorithm generates an optimal estimate of the system and tight bounds on the possible values the variable can assume. However, the complexity of the Copyright by the International Federation of Automatic Control (IFAC) 441

2 model, in the sense of the computational effort required to evaluate the value for a new input vector, grows with the product of the input dimension and the size of the data set. In this paper, the computational complexity of the Set Membership nonlinear identification method and the nearest point approximation, proposed in Canale et al. (9) are analyzed. It is shown that despite the nearest point approximation lowers the computational complexity, it remains proportional to the input dimension and data set size. A new approximation based on the evaluation of the optimal Set Membership estimate on a uniform grid of the function domain is proposed, the complexity of this algorithm grows linearly with the input dimension only, while the memory demand is high. A bound on the approximation error for a given grid size is provided, allowing to select the grid resolution. It is shown that a similar result can not be achieved with the nearestpoint approach, because the evaluation of the worst-case approximation error leads to a non-convex optimization problem. The proposed methods to approximate Set Membership models can be applied not only for system identification but also in areas such as non-linear filtering Milanese et al. (9) and time series prediction Milanese and Novara (5). The paper is organized as follows. In Section II, the Set Membership nonlinear identification method is presented. Section III reviews the Nearest point approximation, describes the grid approach, and evaluates their computational complexity and guaranteed worst-case approximation error. Section IV illustrates the validity of the results on a simulated example and on experimental data related to the electric model of a Excimer UV lamp. The final conclusions are drown in the last Section.. SET MEMBERSHIP IDENTIFICATION In this section, the nonlinear Set Membership identification method given in Milanese and Novara (4) is briefly described. First, consider a the system in NARX (Nonlinear AutoRegressive with exogenous input) structure, described as y(t) = f o (w(t)) (1) where w(t) = [y(t 1),...,y(t n y ),u(t 1),...,u(t n u )], w(t) R n,n = n y +n u A SISO (Single Input-Single Output) system is assumed without loss of generality. Assume the function f o that describes the system is unknown, but a set of measurements of y and w is available. The objective is to find an estimate ˆf of f o. Consider a set of measurements ỹ(k) = f o (w(k))+d(k) () for k = 1,,...,N. Where d(k) is the measurement error and d(k) ǫ. The function f o that describes the system is unknown but the following information is available f o F. = { f C 1 (W) : f γ, w W } (3) where f (w) denotes the gradient of f(w) and x is the Euclidean norm. In this identification method, it is important to find the Feasible System Set (FSS). This set is formed by all functions in F consistent with prior information and measurements. FSS. = {f F : ỹ(k) f(w(k)) ǫ,k = 1,,...,N} (4) If prior assumptions are true, FSS and f o FSS. In the following, prior hypotheses are assumed validated. The optimal estimate of f o (w) is given by where f c (w). = f u(w)+f l (w) (5) f u (w) = min 1 k N (ỹ(k)+ǫ+γ w w(k) ) (6) f l (w) = max 1 k N (ỹ(k) ǫ γ w w(k) ) (7) From Theorems, 5 and 7 in Milanese and Novara (4), it follows that f u (w) and f l (w) are optimal bounds for f o (w). f u (w) and f l (w) are Lipschitz continuous on W. f c is optimal for any L p (W) norm, with p [1, ]. where the optimality criterion is: f opt = arginf sup f ˆf ˆf f FSS.1 Set Membership Model Complexity It is important to note that the Set Membership algorithm produces a non-parametric model. That is, there is not a parameter vector or explicit function that represents the system. To evaluate the optimal for a new regressor value x, the following algorithm must be executed. First, find the distance between x and each one of the regressors w(k) in the data set. Then, for each regressor w(k), this distance is projected from the corresponding ỹ(k)+ǫ, with gradient γ, to x. In order to generate the bound f u (x) in (6), it is necessary to compare each projection with the other ones, and to select the minimum one. A similar procedure is performed to obtain f l (x). Finally, the optimal estimation is evaluated as the mean value between the upper and lower bounds (5). This algorithm must loop through the data set and evaluate the distance x w(k) N times, where x R n. Then, this algorithm has computational complexity O(nN). This method needs memory enough to save the identification data. The data set file is composed by [n + 1,N] elements, then, if each sample is saved in standard floating point format, the memory required is 4(n+1)N bytes. p 4411

3 3. MODEL APPROXIMATIONS In order to improve the computational performance of the Set Membership identification method, some approximations have been proposed, that reduce the computation time and keep good estimation precision. One of these methods is the Nearest point approach, Canale et al. (9). In this Section, the performance of the Nearest point approach is analyzed and a new approximation, named Net Approach, is proposed. The quality of an approximation f of the optimal estimate f c is measured by the guaranteed difference between the estimates f (w) and f c (w) for all w W. This worst case error is defined as: Definition 1. Approximation error of a function f : E a (f ) = sup f (w) f c (w) w W 3.1 Nearest Point Approximation The Nearest Point (NP) method generates a model with lower computational complexity than the original Set Membership model and bounded approximation error. For a given regressor x, the NP estimate of f o (x) is: f NP = ỹ(i) (8) where i is the index of the data set element such that x w(i) = min j=1,,...,n x w(j) Result 1. For a given regressor value x, the NP approach guarantees the following bounds for f o (x): ỹ(i) ǫ γ x w(i) f o (x) ỹ(i)+ǫ+γ x w(i) Proof:Since prior information on f o and noise d is assumed true, then FSS and ỹ(i) ǫ f l (w i ) f u (w i ) ỹ(i) + ǫ. Then, result follows directly from the fact that f u and f l are Lipschitz continuous with constant gradient γ, Milanese and Novara (4). Corollary 1. The approximation error of f NP is: E a (f NP ) = sup γ(min i=1,,...,n w w(i) ) w W Remark: Note that the evaluation of E a (f NP ) is a nonconvex optimization problem, then only upper bounds on the worst-case approximation error can be found using nonlinear optimization techniques. The nearest point method is implemented using an algorithm with complexity O(nN), that is, the same complexity of the original SM method. Even though the complexity does not change, the number of operations is reduced because it is necessary to compare only the distance x w(j) between elements in the data set, avoiding the evaluation of the bounds f l (x) and f u (x). The memory size required by this method is the same of the original SM method. 3. Net Approximation The main drawback of the nearest point approach is that the search for the closest point requires to loop through all the data set. In order to avoid the search, the idea in the net approach is to evaluate the optimal estimate f c (w) in a uniform grid covering all the function domain, and to use the grid as a new data set for the nearest point algorithm. This approach requires more memory than other approximations but it reduces the computation time. In order to keep the presentation simple, let us assume that each regressor element is bounded in a closed interval [ w i, w i ], then the domain of f o is a hyperbox in R n, centered at the origin, with face size w i. Let us introduce the following set of points: n Φ M = i=1 where is the Cartesian product, Φ i = { w i +δ i (j.5) : j = 1,...,M i }, δ i = w i M i and M i is the number of points taken in the i th dimension. Φ i The set Φ M has M = n i=1 M i elements. For a given regressor value x, the NET estimate of f o (x) is: f NET = f c (w ) (9) where w = arg min w Φ M x w Remark: The set of values f c (w ) must be stored in the system memory during the design phase. Result. For a given regressor x, the NET approach guarantees the following bounds on f o (x): f l (w ) γ x w f o (x) f u (w )+γ x w Proof: f u and f l are optimal upper and lower bounds on f o (w ), then f l (w ) f o (w ) f u (w ). Then, result follows directly from the fact that f u and f l are Lipschitz continuous with constant gradient γ. Corollary. The approximation error of f NET is: ( n ) 1/ E a (f NET ) = γ i=1 That is, the approximation error is the product of the maximum gradient γ and the greatest distance from a grid vertex to a point in the regressor domain. The net approximation is implemented using an algorithm with complexity O(n). The number of operations depends only on the regressor dimension and, basically, the procedure is to find a position in the grid. This reduces the processing time, and therefore, it is possible to use the method in higher speed applications. Furthermore, it is feasible to evaluate the approximation error because the maximum distance from a regressor value x and its nearest point in the grid is known. The memory size required to store the net can be calculated using the number of points contained in the set Φ M. δ i 441

4 If each sample f c (w ) is saved as a single precision floating point number, then a memory space of 4M bytes is needed to store the set. Nowadays, the cost and size of memory chips have lowered to less than USD/Gbyte, so an increase in memory size, is not a prohibitive modification in embedded systems. 4. EXAMPLES In this section, two examples illustrate the properties of the net approach. First, a simulated example is described, then the electrical model of an excimer lamp is identified from experimental data. 4.1 Simulated example The FIR (Finite Impulse Response) system (1) has been simulated for an input sequence u generated as white uniform noise with dynamic rank [ 1,1]. The measurement noise v is white uniform noise with v(k).4. N = data have been generated as identification data set. y(k)= u(k 1) +5u(k 5)... u(k 4) 5sin(u(k ))+v(k) (1) Using the validation procedure in Milanese and Novara (4), the selected prior information is ǫ =. and γ = 6. The Set Membership identification and its approximations have been evaluated on a new data set with N = 5, generated with system (1). The estimation quality has been measured with the following function: FIT = ( 1 norm(y ŷ) norm(y) mean(y) ) 1% (11) Table 1 shows performance of the Set Membership (SM), Nearest point (SMNP) and Net (SMNA) approximations. Estimation quality (FIT), Computation Time (CT), maximum estimation error (maxe) and memory usage (MU) are presented. Figures 1, and 3 show the simulation results for the different methods. To generate the net approach, M i = 1 points per each regressor dimension were used. Then, the net contains 1 4 = points. The approximation error guaranteed by the net approach, according to Corollary is: E a (f NET ) =.7 Table 1. Methods Comparison. Set Membership (SM), Nearest point (SMNP) and Net (SMNA) approximations performance on the validation data set. Method FIT % CT maxe MU SM ms.66 3 KB SMNP ms KB SMNA µs MB Figure 4 shows a histogram of the approximation error for the Nearest Point method. In this case, the upper bound E a (f NP ) can not be found because it is required to solve a non-convex optimization problem. In the simulation results, the maximum observed approximation error is Fig. 1. Set Membership method. Continuous red line: measured. Dashed black line: Optimal estimate. Continuous blue lines: optimal upper and lower bounds Fig.. Nearest point approach. Continuous red line: measured. Dashed black line: Nearest point estimate. Continuous blue line: optimal estimate Fig. 3. Net approach. Continuous red line: measured. Dashed blue line: Net estimate. Figure 5 shows a histogram of the approximation error for the Net method. In this case, the upper bound E a (f NP ) is.7. In the simulation results, the maximum observed approximation error is.697. From Figure 5, it can be seen that most of the points have an error lower than half the guaranteed upper bound. 4. Electric Model for a Excimer UV Lamp Excimer lamps are a novel technology for the production of high performance UV radiation, Díez et al. (7). 4413

5 Data Density Fig. 4. Histogram of the approximation error for the Nearest point method. To generate the net approach, M i = 31 points per each regressor dimension were used. Then, the net contains 31 4 = 9351 points. The approximation error guaranteed by the net approach, according to Corollary is: E a (f NET ) =.78 Table. Methods Comparison. Set Membership (SM), Nearest point (SMNP) and Net (SMNA) approximations performance on the validation data set. Method FIT % CT maxe MU SM ms.343 KB SMNP ms.64 KB SMNA µs KB data density Fig. 5. Histogram of the approximation error for the Net method. Efficient power sources have been developed in order to maximize the radiated power of the lamps, Flórez et al. (1). For this purpose, electrical models of the lamp are needed. The electric model for such a complex system is estimated form experimental data, usually adjusting an electric circuit that emulates the lamp behavior, the problem with this approach is that the resulting identification problem is highly non-convex. A Black-box model of the current-voltage relation of the lamp has been estimated using the Set Membership method. A data set with N = 3 has been employed. Thefollowing NARXmodel has beenemployed: y(k) = f(u(k 3),u(k 4),u(k 5),y(k 1))+v(k) (1) where y(k) is the instant current voltage, u(k) is the applied current and v(k) is the measurement noise. The voltage and current signals have been properly scaled to a dynamic rank of [-1,1]. Using the validation procedure in MilaneseandNovara(4),theselectedpriorinformation is ǫ =.65 and γ =.6. The model has been validated on a data set not used for the estimation, formed by N = Table shows performance of the Set Membership (SM), Nearest point (SMNP) and Net (SMNA) approximations. Estimation quality (FIT), Computation Time (CT), maximum estimation error (maxe) and memory usage (MU) are presented. Figures 6, 7 and 8 show the simulation results for the different methods Fig. 6. Set Membership method. Continuous red line: measured. Dashed black line: Optimal estimate. Continuous blue lines: optimal upper and lower bounds Fig. 7. Nearest point approach. Continuous red line: measured. Continuous black line: Nearest point estimate. Continuous blue line: optimal estimate. Figure 9 shows a histogram of the approximation error for the Nearest Point method. In this case, the upper bound E a (f NP ) can not be found because it is required to solve a non-convex optimization problem. In the simulation results, the maximum observed approximation error is.3. Figure1showsahistogramoftheapproximationerrorfor the Net method. In this case, the upper bound E a (f NP ) is.78. In the simulation results, the maximum observed approximation error is.57. From Figure 1, it can be seen that most of the points have an error lower than half the guaranteed upper bound. 4414

6 Fig. 8. Net approach. Continuous red line: measured. Dashed black line: Net estimate. Continuous blue line: optimal estimate. Data density error Fig. 9. Histogram of the approximation error for the Nearest Point method. Data density Fig. 1. Histogram of the approximation error for the Net method. 5. CONCLUSIONS In this work, an approximation to the Set Membership Nonlinear identification method has been presented. This approach allows to reduce the algorithm complexity from O(nN) to O(n) and to evaluate guaranteed bounds on the approximation error. Two examples had been presented that confirm the advantages of the proposed method, reducing the simulation time more that two orders of magnitude with respect to the optimal Set Membership and the Nearest point algorithms. The main requirement to implement the net approach is to increase the memory resources, but this is not a limiting factor nowadays in embedded systems. REFERENCES Bai, E. (8). Non-parametric nonlinear system identification: A data-driven orthogonal basis function approach. Automatic Control, IEEE Transactions on, 53(11), Bai, E., Tempo, R., and Liu, Y. (7). Identification of IIR nonlinear systems without prior structural information. Automatic Control, IEEE Transactions on, 5(3), Canale, M., Fagiano, L., and Milanese, M. (9). Set membership approximation theory for fast implementationofmodelpredictivecontrollaws. Automatica,45(1), Chen, S., Hong, X., Harris, C.J., and Wang, X. (5). Identificationofnonlinearsystemsusinggeneralizedkernel models. Control Systems Technology, IEEE Transactions on, 13(3), Chu, R., Shoureshi, R., and Tenorio, M. (1989). Neural networks for system identification. In American Control Conference, 1989, Díez,R.,Salanne,j.P.,Piquet,H.,Bhosle,S.,andZissis,G. (7). Predictivemodelofadbdlampforpowersupply design and method for the automatic identification of its parameters. Eur. Phys. J. Appl. Phys., 37(3), doi:1.151/epjap:717. Espinoza, M., Suykens, J.A.K., and De Moor, B. (5). Kernel based partially linear models and nonlinear identification. Automatic Control, IEEE Transactions on, 5(1), Flórez, D., Díez, R., Hay, A.K., Perilla, D., Ruiz, F., and Piquet, H. (1). Programmable current converter synthesis for the evaluation of uv radiation of excimer lamps. In Proceedings of the IEEE Andescon 1. Hassouna, S. and Coirault, P. (). Identification of volterra kernels of non-linear systems. In Systems, Man, and Cybernetics, IEEE International Conference on, volume 5, vol.5. Jiang, C., Qiu, T., Zhao, J., and Chen, B. (9). Gross error detection and identification based on parameter estimation for dynamic systems. Chinese Journal of Chemical Engineering, 17(3), Milanese, M. and Novara, C. (4). Set membership identification of nonlinear systems. Automatica, 4(6), Milanese, M. and Novara, C. (5). Set membership prediction of nonlinear time series. Automatic Control, IEEE Transactions on, 5(11), Milanese, M. and Novara, C. (9). Set membership methods in identification, prediction and filtering of nonlinearsystems. InProceedings of the 15th IFAC Symposium on System Identification Saint-Malo, France. Milanese, M., Novara, C., Hsu, K., and Poolla, K. (9). The filter design from data (FD) problem: Nonlinear set membership approach. Automatica, 45(1), Perez, H. and Tsujii, S. (1991). A system identification algorithmusingorthogonalfunctions. Signal Processing, IEEE Transactions on, 39(3), Yu, W. and Li, X. (1). Some new results on system identification with dynamic neural networks. Neural Networks, IEEE Transactions on, 1(),