A NONLINEAR SOURCE SEPARATION APPROACH FOR THE NICOLSKY-EISENMAN MODEL

6th European Signal Processing Conference EUSIPCO 28, Lausanne, Switzerlan, August 25-29, 28, copyright by EURASIP A NONLINEAR SOURCE SEPARATION APPROACH FOR THE NICOLSKY-EISENMAN MODEL Leonaro Tomazeli Duarte an Christian Jutten GIPSA-lab, INPG-CNRS, 46 Avenue Félix Viallet, Grenoble, France leonaro.uarte@gipsa-lab.inpg.fr, christian.jutten@gipsa-lab.inpg.fr ABSTRACT In previous wors [7, 8], we propose source separation methos for a simplifie version of the Nicolsy-Eisenman NE moel, which is relate to a chemical sensing application. In the present paper, we provie a metho able to eal with the complete NE moel. Basically, such a moel can be seen as a composition of a non-iagonal nonlinear transformation followe by a iagonal nonlinear transformation, i.e. a set of component-wise functions. The basic iea behin the evelope technique is to estimate the parameters of these two stages in a separate fashion by using a prior nowlege of the sources, namely the fact that one of the sources is constant uring a certain perio of time. Simulations attest the viability of the propose technique.. INTRODUCTION The problem of blin source separation BSS concerns the retrieval of an unnown set of source signals by using only samples that are mixtures of these original signals. There is a great number of methos for the situation in which the mixing process is linear. However, it seems that in some applications the linear approximation is not suitable an, as a consequence, it becomes necessary to esign source separation methos that tae the nonlinearity of the moel into account. The main problem in the nonlinear case is that the ubiquitous tool for performing source separation, the inepenent component analysis ICA, oes not wor in a general situation [, 2]. In other wors, there are cases in which the recovery of the statistical inepenence, which is the very essence of ICA, oes not guarantee the separation of the sources when the mixture moel is nonlinear. In view of this limitation, a more reasonable approach is to consier constraine mixing systems as, for example, post-nonlinear PNL mixtures [2] an linear-quaratic mixtures []. In recent papers [3, 8], the problem of BSS in a particular class of nonlinear systems relate to a chemical sensing application was investigate. For instance, we evelope in [7, 8] source separation methos for estimating the concentrations of ions in a solution when the valences of these ions are ifferent. In our first wors, we consiere a simplifie version of the mixing moel that is pertinent only in the situations where the sensors present a Nernstian response [3]. In the present wor, by relying on an aitional assumption on the sources, we exten our solution to the complete moel, which permits us to wor in more realistic situations. Our wor will be presente as follows: in Section 2 we introuce the problem. After that, in Section 3, we expose the basics of Leonaro Tomazeli Duarte woul lie to than the National Council for Scientific an Technological Development CNPq-Brazil for funing his PhD research. our metho. In Section 4, simulations are carrie out in orer to illustrate the viability of the proposal. Finally, in Section 5, we state our conclusions an perspectives. 2. PROBLEM STATEMENT In analytical chemistry, a relevant problem is to estimate the evolution of the concentrations of several ions in a solution. An inexpensive an practical approach is base on the use of potentiometric sensors, such as glass-electroes an ionsensitive fiel-effect transistors ISFET. Basically, the sensing mechanism in this sort of sensor is ue to the epenence of the potential ifference between two electroes on the concentration of a target ion [9]. Unfortunately, there is a pronounce lac of selectivity behin this principle, i.e. the generate potential ifference in the sensor can also epen on other interfering ions present in the solution. A very common approach to eal with the interference problem is to perform a calibration of the chemical sensor by using samples from a nown atabase [9]. However, there are two major ifficulties in such an approach: the acquisition of training samples may be time-emaning an 2 this calibration proceure must be performe from time to time ue to the sensors rift. In this context, an approach foune on array of chemical sensors an on BSS methos emerges as an attractive alternative, since, in this case, the estimation of the evolution of the concentrations can be one by using only the sensors responses, that is, without conucting supervise learning with a nown ata set. Although very general, BSS techniques usually require a parametric escription of the mixing moel. In our case, this information can be foun on the classical moel of Nicolsy- Eisenman NE [9], which provies a simple an yet aequate escription of the potentiometric sensors. Accoring to this moel, the response of the i-th sensor is given by: z i z x i t = c i + i log s i t+ a i j s j t j, j, j i where t correspons to the time inex; s i t an s j t enote the concentration of the target ion an of the concentration of the j-th interfering ion, respectively. z i an z j enote the valence of the ions i an j, respectively. The selective coefficients a i j moel the interference process; c i an i are constants that epen on some physical parameters. Note that when the ions have the same valence, then the moel can be seen as a particular case of the class of PNL systems, as escribe in [3]. In this wor, we are intereste in the situation where the valences are ifferent z i z j. In view of the toughness of the resulting moel, we investigate only the case with two sources. Also, the parameters c i may be not consiere, since

6th European Signal Processing Conference EUSIPCO 28, Lausanne, Switzerlan, August 25-29, 28, copyright by EURASIP they represent concentrations. Finally, assumption 4 state a nowlege of the ions valences. Actually, this information is available since the main interfering ions, an so their valences, are nown in avance. Figure : The Nicolsy-Eisenman moel. they only introuce an ambiguity with respect to the sources mean value. Therefore, the obtaine mixing moel can be escribe as: x t = log x 2 t = 2 log p t p 2 t = log s t+a 2 s 2 t = 2 log s 2 t+a 2 s t, 2 where = z /z 2. As illustrate in Fig., this mixing moel is compose of two stages, being the first one a particular nonlinear mapping that epens on the valences of the target ions an the secon one a pair of nonlinear component-wise functions parametrize in i. As it will be clarifie in the sequel, the propose metho treats these two stages in a separate way. 3. PROPOSED METHOD A natural strategy to perform BSS on the moel 2 is to aopt the two-stage separating system shown in Fig.. The first stage is compose of exponential component-wise functions expx i /i, whereas the secon one shoul provie a non-iagonal mapping able to counterbalance the effects introuce by the first stage of the mixing system. In [7, 8], we investigate source separation methos for this first stage an we assume that the component-wise functions were nown in avance. In fact, this situation is realistic only when the sensors have a Nernstian response, given that in such a case the parameters i are nown in avance. In the sequel, we will present a brief review of our previous wors. Then, we will etail the main contribution of this paper which is relate to the estimation of the componentwise functions, i.e., of the parameters i. With this new step, it becomes possible to evelop a complete source separation metho for the moel escribe in 2. During our evelopment, the following assumptions are consiere: Assumption The sources are statistically inepenent; Assumption 2 The sources are positive an boune, i.e, s i t [Si min,si max ], where Si max > Si min > ; Assumption 3 The system expresse in 2 is invertible in the region given by [S min,s max ] [S2 min,s2 max ]; Assumption 4 is nown an taes only positive integer values; In the context of the ion sensing application, assumption is equivalent to consier that there is no chemical reaction between the ions. Concerning the assumptions 2 an 3, our separation methos woul wor in a preefine range of concentrations, as is usual in commercial sensors. Furthermore, it is quite natural to consier positive sources, since 3. Review of our solution for a simplifie version of 2 When the component-wise functions in 2 are nown in avance, the mixing moel becomes p t = s t+a 2 s 2 t p 2 t = s 2 t+a 2 s t. 3 In orer to retrieve s i, we aopte [7] the following recurrent networ as separating system: y n+ = y 2 n+ = p t w 2 y 2 n p 2 t w 2 y n, 4 where the vectors w = [w 2 w 2 ] T an yn = [y n y 2 n] T enote the parameters to be ajuste an the system outputs at time n, respectively. For a given sample of the mixtures [p t p 2 t] T, an for a given value w, the system outputs are obtaine after the convergence of the ynamics 4. In orer to ajust the parameters w in 4, we propose [8] the following learning rule that minimizes the mutual information of the vector y: w w µe { y w T β y y, 5 where µ enotes the learning rate, β y y is the score function ifference vector associate with the ranom variable y an is the Jacobian of 4 with respect to w. y w 3.2 An approach for inverting the component-wise functions Now, let us turn our attention to the complete version of the NE moel. The following parametric moel can be consiere in the first stage of the separating system e t = exp x t e 2 t = exp x2, 6 t where i are the parameters to be ajuste. A first natural approach to aapt the separating system woul be to evelop an ICA algorithm to fin the parameters {, 2,w 2,w 2. In other wors, we coul see a set of parameters that optimizes a functional associate with the statistical inepenence, such as the mutual information. However, it woul be tough to evelop a graient rule in this case given that the separating system is compose of a nonlinear ynamic system an, therefore, there woul be a ris of instability uring the learning phase. Besies this practical problem, there is a crucial theoretical point that shoul be aresse: is the consiere mixing moel separable, i.e., is the recovery of the statistical inepenence enough to assure source separation? We o not have such an answer, but recent stuies [2] suggest that for nonlinear transformations lie the one ealt with here 2, it is necessary to consier prior information other than inepenence to achieve source separation. 2

6th European Signal Processing Conference EUSIPCO 28, Lausanne, Switzerlan, August 25-29, 28, copyright by EURASIP In view of the aforementione problems, we believe that it is more reasonable to consier a two-step approach, in which the component-wise functions are firstly estimate an, then, the recurrent system is traine. The price to be pai is that we nee the following aitional assumption: Assumption 5 There is, at least, a perio of time where one, an only one, of the sources has zero-variance. In view of assumption 5, if we consier, for instance, that s t taes a constant value S where S [Si min,si max ] in a given time winow, then, accoring to 2, we have p t = S + a 2 s 2 t p 2 t = s 2 t+a 2 S. 7 From this expression, it is not ifficult to see that p t = S + a 2 p 2 t a 2 S. 8 Hence, uner such an assumption, 8 is a polynomial of orer in the p, p 2 plane. Our iea is base on this fact, in the sense that this polynomial function is lost after the application of the component-wise functions an, thus, we may invert the log functions by searching a set {, 2 that restores a polynomial in the e,e 2 plane. In the sequel, we shall etail this iea. Firstly, let us escribe the mapping between the p, p 2 an e,e 2 planes. By consiering 2 an 7, one can rewrite 6 as: e t = exp e 2 t = exp logp t 2 logp 2 t 2 = p t 2 = p 2 t 2. 9 The next step of our stuy is to fin the relation between the ata in the e,e 2 plane. As state by 8, there is a polynomial relation in the p, p 2 plane, which can be expresse by p t = i= ϕ i p 2 t i, where the coefficients ϕ i can be etermine by the binomial expansion of 8. After a straightforwar evelopment consiering 9 an, the following expression is obtaine: [ ] 2 e t = ϕ i e 2 t 2 i. i= As state above, our initial iea to fin an 2 is base on the recovery of a polynomial curve of orer in the e,e 2 plane. In orer to verify the viability of our approach, we nee to investigate the following question: for what values of an 2 the function is a polynomial of orer? At a first sight, it is clear that when the optimum solution =,2 = 2 is achieve, then that expression results in a polynomial curve in the e,e 2 plane. However, there is a particular situation where a polynomial curve is obtaine although the mapping 9 is still nonlinear. In fact, when S is null, then we can see from 8 that all the coefficients except ϕ in are null. In this situation, the following solution = D,2 = D 2, where D is a constant, also gives a polynomial of orer although oes not correspon to our esire solution. Inee, one of the reasons behin assumption 2 is exactly to avoi this situation by consiering only positive sources. To implement our iea, we must efine a way to chec if a set of points in the e,e 2 plane correspons to a polynomial of orer. This can be one by efining a cost function as the mean square of the resiuals resulting from the regression of the set of sample {e t,e 2 t t= N being N the number of samples through a polynomial of orer. In mathematical terms, this cost function can be expresse as min e t, 2 t i= α i e 2 t i 2, 2 where α i correspon to the i-th regression coefficient. In orer to gain more insight, let us substitute in 2, which gives min, 2 t [ i= ] 2 2 ϕ i e 2 t 2 i α i e 2 t i. 3 i= One may note that this expression is a nonlinear function with respect to the parameters {, 2. Moreover, accoring to equation, for a given sample at time t there is an unerlying relation between e t an e 2 t, which in turn maes the regression coefficients α i nonlinearly epenent on the parameters to be optimize. As a consequence, it becomes ifficult to obtain the erivatives of this function an, therefore, to evelop a graient-base optimization metho. An alternative approach to optimize 2 can be foun on the so-calle evolutionary techniques. Briefly, this class of algorithms performs a searching proceure that is base on the notion of population, i.e., it eals with a set of possible solutions iniviuals. At each iteration, some perturbation mutation an recombination, for instance is introuce on the population an a group of iniviuals is selecte to continue in the population of the next iteration usually, the selecte iniviuals are the ones with higher fitness, which is calculate through the cost function. The major benefit brought by an evolutionary metho to our problem is that no information about the erivatives is neee, since the selection stage is base only on the evaluation of the cost function, which, for our problem, is a straightforwar tas. 3.2. Detection of zero-variance perios The iea escribe in the last section wors uner the assumption that there is a time winow in which one of the sources oes not vary. Eviently, if a blin scenario is envisage, then one shoul be able to etect this silent perio. A possible way to perform this tas is to consier the problem from a geometric stanpoint. Given that the mixing moel is invertible an the sources are suppose boune, the borers of the istribution in the x,x 2 plane correspons to the situation in which one of the source is constant. Therefore, at least in an ieal situation, we coul etect the silent perios by estimating the borers of the istribution of the mixtures, in the same way as performe in [2]. Note that this

6th European Signal Processing Conference EUSIPCO 28, Lausanne, Switzerlan, August 25-29, 28, copyright by EURASIP.8 Mutual Information 2 3 4 5 6 7 8 9 Source.5 2 3 4 5 6 7 8 9 Source 2.5 2 3 4 5 6 7 8 9 Figure 2: Mutual information between the mixtures a an corresponing sources b c. strategy wors even when the assumption 5 is not met. Unfortunately, this proceure is ifficult to implement since it emans a very accurate estimate of the borers, which may be ifficult to achieve when the number of samples is small. A secon approach to search the silent perios is base on the fact that when one of the sources is constant, each sensor response correspons to a eterministic function of the same ranom variable, i.e., x = g s 2 an x 2 = g 2 s 2. Therefore, we can argue that such situation is the one with maximum nonlinear correlation between the sensors an, given that, we may try to ientify the silent perios by searching time winows for which a measure of correlation is maximize. This iea has alreay been evelope for linear source separation [6] an, in this case, the silent perios are foun by observing the secon-orer correlation measure. In our case, we eal with a nonlinear moel an, as consequence, a measure able to etect nonlinear epenences must be employe. With this purpose in min, we consier the mutual information between the mixing signals. The mutual information of two continuous ranom variables lies on the interval Ix,x 2 < +, being zero when x an x 2 are statistically inepenent, an tening to infinity when there is a eterministic relation between these variables. Therefore, we can fin the silent perios by looing at the time winows for which the mutual information is maximize. In fact, it seems more practical to maximize a normalize version of the mutual information efine as ςx,x 2 = exp 2Ix,x 2, given that its maximum value is one an occurs when there is a eterministic relationship between x an x 2. To illustrate the iea of the last paragraph, we present in Fig. 2 the evolution of the normalize mutual information between the sensors response estimate through a time winow of length 5 an the respectively sources. Note that the maximum of the mutual information occurs exactly for time winows containing a constant source. 3.3 Description of the complete algorithm We can summarize the complete separation algorithm in Tab.. Concerning the first step, we aopte the mutual information estimator propose in [4]. As alreay iscusse, the optimization of 2 in the secon stage can be carrie out through evolutionary methos. In this wor, we chose Table : Algorithm. Detection of silent perios Estimate the mutual information between the mixtures x an x 2 for a moving-time winow. Select the time winow in which the mutual information is maximum. 2. Estimation of the component-wise functions. For the selecte time winow, minimize expression 2 with respect to an 2. 3. Training the recurrent networ Determine the parameters w i j of 4 through the algorithm 5. The inputs of the recurrent networ are e i. [Ca 2+ ] M [Na + ] M.8.6.4.2.8.6.4.2 2 4 6 8. 2 4 6 8 Figure 3: Sources. the opt-ainet algorithm. This evolutionary metho has been proven to be efficient in signal processing applications see [], for instance. In aition to its robustness to local minima, the opt-ainet only nees zero-orer information, which, as iscusse before, is a very interesting feature for our problem. The technical etails of this metho can be foun in [5]. 4. SIMULATION RESULTS To assess the performance of the algorithm escribe in Tab., we simulate the problem of etecting the ions Ca 2+ an Na + through an array of two sensors each one has a ifferent ion as target. For that, we consier the parameters a 2 =.79 an a 2 =.4, which were taen from [3]. Also, we have assume that both sensors have a perfect Nerstian response [9], i.e., =.29 an 2 =.258. The efficacy of the obtaine solution for each source was quantifie accoring to the following inex: E{s 2 i SIR i = log E{s i y i 2. 4 Thus, SIR =.5SIR + SIR 2 efines a global inex. Regaring the parameters of the algorithm, a set of samples was consiere. The etection of the silent perios is performe by estimating the mutual information for a winow of a length of 5 samples. Actually, it is ifficult to achieve a reliable estimation of the mutual information with

6th European Signal Processing Conference EUSIPCO 28, Lausanne, Switzerlan, August 25-29, 28, copyright by EURASIP Ca 2+ sensor mv Na + sensor mv estimate [Ca 2+ ] M estimate [Na + ] M.3.4.5.6.7.8 2 4 6 8.4.6.8. 2 4 6 8.8.6.4.2.8.6.4.2 Figure 4: Mixtures. 2 4 6 8. 2 4 6 8 Figure 5: Retrieve sources. a smaller number of samples. Finally, concerning the training of the recurrent networ, the number of iterations was 5 an the learning rate was µ =.2. Uner the escribe scenario an with the two sources shown in Fig. 3, our metho has achieve the following performance SIR = 4.52, SIR 2 = 38.27 an SIR = 39.39. Such a result attests that our proposal oes well in this case, as can be confirme by looing at the mixing signals in Fig. 4 an at the retrieve sources in Fig. 5. 5. CONCLUSIONS AND PERSPECTIVES In this wor, we have propose a source separation metho for the NE moel, which is relate to a chemical sensing application. Uner the assumption that one of the sources oes not vary uring a perio of time, it became possible to estimate the component-wise functions, which correspon to the secon stage of the NE moel. By joining this stage to our previous solutions, we coul efine a complete separation framewor. In orer to verify the efficacy of our proposal, we conucte some experiments consiering a set of parameters taen from the literature. The obtaine solutions highlight that our metho is a promising one to the application in min. There are several perspectives for this wor. A first one is to investigate the extension of our metho for the situation with more than two sources. Also, we intent to conuct some real experiments to investigate the applicability of our metho in a real problem. By proceeing this way, we will be able to aress several questions in more etails as, for instance, the stuy of the noisy case, starting with the problem of moeling the noise in this sort of application. REFERENCES [] R. R. F. Attux, M. Loiola, R. Suyama, L. N. e Castro, F.J. Von Zuben, an J. M. T. Romano. Blin search for optimal wiener equalizers using an artificial immune networ moel. EURASIP Journal on Applie Signal Processing, 23:74 747, 23. [2] M. Babaie-Zaeh, C. Jutten, an K. Nayebi. A geometric approach for separating post non-linear mixtures. In Proc. EUSIPCO 22, Toulouse, France, pages 4, 22. [3] G. Beoya, C. Jutten, S. Bermejo, an J. Cabestany. Improving semiconuctor-base chemical sensor arrays using avance algorithms for blin source separation. In Proc. of IEEE SICON 24, New Orleans, USA, pages 49 54, 24. [4] G.A. Darbellay an I. Vaja. Estimation of the information by an aaptive partitioning of the observation space. IEEE Transactions on Information Theory, 454:35 32, May 999. [5] L. N. e Castro an F. J. Von Zuben. Learning an optimization using the clonal selection principle. IEEE Transactions on Evolutionary Computation, 6:239 25, 22. [6] Y. Deville an M. Puigt. Temporal an time-frequency correlation-base blin source separation methos. part i: Determine an uneretermine linear instantaneous mixtures. Signal Processing, 87:374 47, 27. [7] L. T. Duarte an C. Jutten. Blin source separation of a class of nonlinear mixtures. In Proc. of ICA 27, Lonon, UK, pages 4 48, 27. [8] L. T. Duarte an C. Jutten. A mutual information minimization approach for a class of nonlinear recurrent separating systems. In Proc. of the IEEE MLSP 27, Thessalonii, Greece, pages 22 27, 27. [9] P. Fabry an J. Fouletier, eitors. Microcapteurs chimiques et biologiques. Lavoisier, 23. [] S. Hosseini an Y. Deville. Blin separation of linearquaratic mixtures of real sources using a recurrent structure. In Proc. of the IWANN 23, Menorca, Spain, pages 289 296, 23. [] A. Hyvärinen an P. Pajunen. Nonlinear inepenent component analysis: existence an uniqueness results. Neural Networs, 2:429 439, 999. [2] C. Jutten an J. Karhunen. Avances in blin source separation BSS an inepenent component analysis ICA for nonlinear mixtures. International Journal of Neural Systems, 4:267 292, 24. [3] Y. Umezawa, P. Bühlmann, K. Umezawa, K. Toha, an S. Amemiya. Potentiometric selectivity coefficients of ion-selective electroes. Pure an Applie Chemistry, 72:85 282, 2.