Statistical-Mechanical Approach to Probabilistic Image Processing Loopy Belief Propagation and Advanced Mean-Field Method

Transcription

1 Statistical-Mechanical Approach to Probabilistic Image Processing Loopy Belief Propagation and Advanced Mean-Field Method Kazuyuki Tanaka and Noriko Yoshiike 2 Graduate School of Information Sciences, Tohoku University Aranaki-aza-aoba 09, Aoba-ku, Sendai , Japan Abstract The framework is presented of Bayesian image restoration for multi-valued images by means of the multi-state classical spin systems. Hyperparameters in the probabilistic models are determined so as to maximize the marginal likelihood. A practical algorithm is described for multi-valued image restoration based on loopy belief propagations in probabilistic inference. Loopy belief propagations are equivalent to the Bethe approximation and the pair approximation of cluster variation method in statistical mechanics. Introduction Bayesian approach to image processing is one of powerful methods for probabilistic image processing[, 2, 3, 4]. In Bayesian approach to image processing, some probabilistic models which have been proposed as physical models for some magnetic materials in statistical mechanics are useful as aprioriprobabilistic models. It seemingly seems that image processing and magnetic materials have no concern with each other. However, both of them are defined on a regular lattice and are composed with many elements relating. In magnetic materials, many spins are arrayed on a regular lattice and the spins locally interact with each other. In image processing, images are constructed from pixels and the pixels are arrayed on a regular lattice. In order to process an image in a certain purpose, a filter is designed. Many image processing filters usually have structures in which the output of each pixel is determined from the configuration of the neighboring pixels. Recently, many researchers have investigated to design new image processing system with paying attention to the similarity[5, 6]. When we formulate image processing as a probabilistic model, some ideas in statistical-mechanical models are very useful. Although the probabilistic models are usually massive ones and it is difficult to treat analytically, we are able to construct some approximate algorithms by using advanced mean-field methods in statistical mechanics. We expect to pioneer a new paradigm of computational art in image processing by using some statisticalmechanical techniques proposed to study nature in material science. Advanced mean-field methods are powerful ones for massive probabilistic models and have been applied to the probabilistic algorithms[7]. Moreover, some computer scientists are interested in the similarity between advanced mean-field methods and loopy belief propagations that have investigated in order to construct probabilistic inference systems in artificial intelligence. An extremum condition for minimizing an approximate free energy in the advanced mean-field method is sometimes equivalent to an update rule of loopy belief propagation[8, 9]. Recently, advanced mean-field methods have been applied also to Bayesian image processing[0, ]. In Bayesian image processing, we have to determine hyperparameters. Some hyperparameters are corresponding to interaction parameters or other model parameters. In practice, it is common for the hyperparameters to be determined so as to maximize a marginal likelihood[4, 2, 3]. In some probabilistic models treated in the statistical-mechanics, some types of phase transitions occur at a critical point. We have the first-order and the second-order phase transitions as familiar types of phase transitions. In the first-order phase transition, the first differentiate of the free energy is discontinuous at a critical point. kazu@statp.is.tohoku.ac.jp, URL: kazu/index-e.html 2 yoshiike@statp.is.tohoku.ac.jp, URL: yoshiike/

2 In the second-order phase transition, while the first differentiate of the free energy is always continuous, the second differentiate of the free energy is discontinuous at a critical point. If the probabilistic model adopted as an aprioriprobability distribution has the second-order phase transition, the free energy is differentiable at any value of hyperparameter. On the other hand, if the probabilistic model has the first-order phase transition, the free energy is not differentiable at a critical point of hyperparameter. The marginal likelihood is expressed in terms of free energies of an aprioriprobability distribution and an a posteriori probability distribution. This fact means that the marginal likelihood is not differentiable at a critical point of hyperparameter in an aprioriprobability distribution. In the present paper, we review the framework of the loopy belief propagation in image processing based on a variational principle for the minimization of an approximate free energy of the probabilistic model defined on a square lattice. Bayesian image processing is formulated by using Bayesian statistics and the maximization of a marginal likelihood and by adopting Q-state Potts model and Q-Ising model as the aprioriprobability distribution. The free energy in the Q-state Potts model for Q 3 isnot differentiable at a value of hyperparameter when we treat it by using advanced mean-field methods. The maximization of the marginal likelihood is achieved by using the loopy belief propagation. Part of the present work is the research collaboration with Professor D. M. Titterington, of the Glasgow University and Professor J. Inoue of the Hokkaido University. 2 Bayesian Image Analysis and Hyperparameter Estimation We consider an image on a square lattice Ω {i} such that each pixel takes one of the gray-levels Q = {0,, 2,,Q }, with0andq corresponding to black and white, respectively. The intensities at pixel i in the original image and the degraded image are regarded as random variables denoted by F i and G i, respectively. Then the random fields of intensities in the original image and the degraded image are represented by F {F i } and G {G i }, respectively. The actual original image and degraded image are denoted by f = {f i } and g = {f i }, respectively. The probability that the original image is f, Pr{F = f}, is called the aprioriprobability of the image. In the Bayes formula, the a posteriori probability Pr{F = f G = g}, that the original image is f when the given degraded image is g, is expressed as Pr{F = f G = g} = Pr{G = g } F = f Pr{F = f} Pr{G = g F = z}pr{f = z}, z where the summation z is taken over all possible configurations of images z = {z i }. The probability Pr { G = g F = f } is the conditional probability that the degraded image is g when the original image is f and describes the degradation process. In the present paper, it is assumed that the degraded image g is generated from the original image f by changing the intensity of each pixel to another intensity with the same probability p, independently of the other pixels; the probability that the intensity is unchanged is therefore Q p, which constrains p to lie in the range 0 p /Q. The conditional probability distribution associated with the degradation process when the original image is f is Pr{G = g F = f} = Pr{G = g F = f,p} = p δ fi,g i + Q p δ fi,g i, 2 where δ a,b is the Kronecker delta. Moreover, the aprioriprobability distribution that the original image is f is assumed to be Pr{F = f} =Pr{F = f α} = exp αuf i,f j, 3 Z PR α where B is the set of all the nearest-neighbor pairs of pixels on the square lattice Ω and Z PR α isa normalization constant. By substituting Eqs.2 and 3 into Eq., we obtain Pr{F = f G = g,α,p} = Z PO α, p 2

3 p δ fi,g i + Q p δ fi,g i exp αuf i,f j, 4 where Z PO α, p is a normalization constant. In the maximum marginal likelihood estimation approach, the hyperparameters α and p are determined so as to maximize the marginal likelihood Pr{G = g α, p}, where Pr{G = g α, p} z Pr{G = g F = z,p}pr{f = z α}. 5 We denote the maximizer of the marginal likelihood Pr{G = g α, p} by α and p. Thus α, p = arg maxpr{g = g α, p}. 6 α,p The marginal likelihood Pr{G = g α, p} is expressed in terms of the free energies F PO α, p ln Z PO α, p and F PR α ln Z PR α as follows: ln Pr{G = g α, p} =ln Z PO α, p ln Z PR α = F PO α, p+f PR α. 7 Given the estimates α and p, the restored image f = { f i } is determined by f i =arg max f i Q Pr{F i = f i G = g,α,p}. 8 Here Pr{F i = f i G = g,α,p} is a marginal probabilities defined by Pr{F i = f i G = g,α,p} z δ fi,z i Pr{F = z G = g,α,p}, 9 This way of producing a restored image is called maximum posterior marginal estimation. 3 Loopy Belief Propagation In the framework given in the previous section, we have to calculate the marginal probability distributions Pr{F i = f i G = g,α,p},andthefreeenergiesf PO α, p andf PR α. Since it is hard to calculate the marginal probability distributions and the free energies exactly, we apply the loopy belief propagation to the above probabilistic models given by Pr{F = f G = g,α,p} and Pr{F = f α}. Next we summarize the simultaneous equations to be satisfied by marginal probability distributions, namely P i f i z δ fi,z i P z, 0 in the loopy belief propagation for a probability distribution P f defined by ψ i f i φ ij f i,f j P f = ψ i z i z. φ ij z i,z j Here φ ij ξ,ξ andψ i ξ are always positive for any values of ξ Q and ξ Q. The detailed derivation is similar to that given in [4, 4, 5] and here we merely state the results of the variational calculation for an approximate free energy in the loopy belief propagation 3. The probability distribution is rewritten as P f = Φ ij f i,f j Φ i f i, Z Z Φ i f i Φ j f j z Φ i z i Φ ij z i,z j Φ i z i Φz j, 2 3 In the statistical mechanics, the loopy belief propagation has been often referred to as the Bethe approximation[6] or the cluster variation method[7, 8, 9]. 3

4 where Φ i f i ψ if i ψ i ζ, Φ ψ i f i φ ij f i,f j ψ j f j ijf i,f j ψ i ζφ ij ζ,ζ φ j ζ. 3 Now we introduce the Kullback-Leibler divergence D[Q P ] between the given probability distribution P f and a probability distribution Qf can be written in terms of F[φ] as follows: D[Q P ] z Qz Qzln = P z z Qz lnqz lnp z. 4 By using the normalization Qz =, 5 z the Kullback-Leibler divergence D[Q P ] can be rewritten as D[Q P ]=F[Q]+lnZ, 6 where F[Q] z Qz lnqz lnφ i z i + lnφij z i,z j lnφ i z i lnφ j z j. 7 The Kullback-Leibler divergence D[Q P ] is equal to zero if Qf =P f. Moreover, it is valid that F[P ] = lnz. In statistical mechanics, F[P ] = lnz is referred to as a free energy of the probabilistic model given in Eq.2. We consider a probability distribution Qf expressed by where Qf = Q i f i Q ij f i,f j Q i f i Q j f j, 8 Q i f i z δ fi,z i Qz, Q ij f i,f j z δ fi,z i δ fj,z j Qz. 9 We remark that Q ij f i,f j =Q ji f j,f i. By substituting Eq.8 to the factor lnqz in Eq.6 with Eq.7, the Kullback-Leibler divergence D[Q P ] can be given as follows: D[Q P ]= Qz lnq i z i + lnqij z i,z j lnq i z i lnq j z j z lnφ i z i + lnφij z i,z j lnφ i z i lnφ j z j +lnz. 20 By applying the normalization conditions Q i ζ =, Q ij ζ,ζ =, 2 and the definition of marginal probability of Qf given in Eq.9, the expression of the Kullback-Leibler divergence D[Q P ] given in Eq.20 can be rewritten as follows: D[Q P ]= Q i ζlnq i ζ 4

5 + Q ij ζ,ζ lnq ij ζ,ζ Q i ζlnq i ζ Q j ζlnq j ζ Q i ζlnφ i ζ + Q ij ζ,ζ lnφ ij ζ,ζ Q i ζlnφ i ζ Q i ζlnφ i ζ +lnz. 22 Hence, the Kullback-Leibler divergence D[Q P ] is expressed only in terms of marginal probabilities {Q i,q ij,} as follows: D[Q P ]=F[{Q i,q ij,}]+lnz, 23 where and F [ {Q γ γ Ω B} ] D[Q i Φ i ]+ D[Q ij Φ ij ] D[Q i Φ i ] D[Q j Φ j ], 24 D[Q i Φ i ] Qi ζ Q i ζln, D[Q ij Φ ij ] Q ij ζ,ζ Qij ζ,ζ ln Φ i ζ Φ ij ζ,ζ. 25 From the definitions of marginals {Q i,q ij,}, we see that the following conditions are valid: Q i ζ = Q ij ζ,ζ, j c i,. 26 Here c i {j } is the set of all the nearest-neighbor pixels of i. The conditions are often referred to as reducibility conditions in the cluster variation method. In the standpoint of the cluster variation method[4], the marginal probabilities of P f are approximately determined so as to minimize the right-hand side of Eq.23 under the constraint conditions 2 and 26 as follows: {P i,p ij,} { Q i, Q ij,} { arg min D[Q P ] Q i ζ = Q ij ζ,ζ,j c i,, {Q i,q ij,} Q i ζ =, } Q ij ζ,ζ = =arg min {F [ {Q γ γ Ω B} ] Qi ζ = Q ij ζ,ζ,j c i,, {Q i,q ij,} Q i ζ =, } Q ij ζ,ζ =. 27 By introducing the Lagrange multipliers to ensure the constraint conditions 2 and 26 L [ {Q γ γ Ω B} ] F [ {Q γ γ Ω B} ] λ ij,i ζ Q i ζ Q ij ζ,ζ j c i λ i Q i ζ λ ij Q ij ζ,ζ, 28 and by taking the first variation of L [ {Q γ γ Ω B} ], the expressions of the marginal probabilities { Q i ξ, ξ Q} and { Q ij ξ,ξ,ξ Q,ξ Q} can be given as follows: ψ i ξ /3 exp 3 {k k c Q λ i} ik,iξ i ξ =, 29 φ i ζ /3 exp {k k c λ i} ik,iζ 3 5

6 Q ij ξ,ξ = φ ij ξ,ξ exp λ i ξ+λ j ξ. 30 φ ij ζ,ζ exp λ ij,i ζ+λ ij,j ζ The Lagrange multipliers {λ ij,i,λ ij,j } are determined so as to satisfy the reducibility conditions 26 with Eqs.29 and 30. The approximate value of the free energy F[P ]isgivenas k c i\j F[P ] F [ { Q γ γ Ω B} ]. 3 By replacing λ ij,i ξ byexp λ ij,i ξ = ψ i ξ µ k i ξ in Eqs.29 and 30, and by substitute Eqs.26 and 33, we obtain the simultaneous equations for the sets of marginal probabilities {P i ξ, ξ Q} and the free energy F ln Z as follows: P i ξ ψ i ξ µ k i ξ k c i ψ i ζ, 32 µ k i ζ k c i F[P ] lnz 3 ψ i ζ ln µ k i ζ k c i ln ψ i ζφ ij ζ,ζ ψ j ζ µ k i ζ k c i\j l c j\i µ l j ζ, 33 where µ j i ξ = φ ij ξ,ζψ j ζ µ k j ζ k c j\i φ ij ζ,ζψ j ζ µ k j ζ. 34 k c j\i By setting ψ i ξ =p δ ξ,gi + Q p δ ξ,gi and φ ij ξ,ξ =exp αuξ,ξ, we obtain the marginal probability Pr{F i = ξ G = g,α,p} and the free energy F PO α, p asp i f i andf, respectively. By setting ψ i ξ =andφ ij ξ,ξ =exp αuξ,ξ, we obtain the free energy F PR α asf. Though these forms may not be so familiar to some physicists, lnµ i j ξ corresponds to the effective field in the conventional Bethe approximation. In probabilistic inference, the quantity µ i j ξ is called a message propagated from i to j. Equations 34 have the form of fixed-point equations for the messages µ i j ξ. In practical numerical calculations, we solve the simultaneous equations 34 by using iterative methods. For various values of the hyperparameters α and p, we obtain the marginal probability distributions Pr{F i = f i G = g,α,p}, the free energies F PO α, p andf PR α and search for the optimal set of values, α, p, that maximize the marginal likelihood Pr{G = g α, p} numerically. 4 Numerical Experiments In this section, we report some numerical experiments. The optimal set of values of hyperparameters, α, p, are determined by means of maximum marginal likelihood estimation and the loopy belief propagation given in Secs.2 and 3. To evaluate restoration performance quantitatively, fifteen original images f are simulated from the aprioriprobability distribution 3 for the Q-state Potts model and the Q-Ising model, respectively. We produce a degraded image g from each original image f by means of the degradation process 2 for Q p =0.3. By applying the iterative algorithm of the loopy belief propagation to each degraded image g, we obtain estimates of the hyperparameters p and α and the restored image f for each degraded image g. ForQ = 4, one of the numerical experiments is shown in Figs. and 2, respectively. In order to show the achievement of the maximization of marginal likelihood, we give in Figs. 3 and 4, also p- 6

7 a b c Figure : Image restoration based on the Q-state Potts model Q = 4. The original image f is generated by the aprioriprobability distribution 3. a Original image f α =.25. a Degraded image g Q p =0.3. b Restored image f obtained by the loopy belief propagation Q p =0.2246, α =.0050, df, f =0.557, SNR = db. a b c Figure 2: Image restoration based on the Q-Ising model Q = 4. The original image f is generated by the aprioriprobability distribution 3. a Original image f α = b Degraded image g Q p =0.3. c Restored image f obtained by the loopy belief propagation Q p =0.275, α =0.6202, df, f =0.702, SNR =4.647 db. and α-dependences of ln Pr{G = g α, p} corresponding to the image restorations shown in Figs. and 2, respectively. In order to show the achievement of the maximization of marginal likelihood, we give in Figs. 3 and 4, also p- andα-dependences of ln Pr{G = g α, p} corresponding to the image restorations shown in Figs. 3 and 4, respectively. From the fifteen degraded images g and the corresponding restored images f, we calculate the 95% confidence intervals for the hyperparameters, p and α, and the values of the Hamming distance df, f, df, f δ Ω fi, fi, 35 and the improvement in the signal to noise ratio, SNR, SNR 0 log 0 f g 2 f f 2 db. 36 These results showed that combining the loopy belief propagation with maximum marginal likelihood estimation gives us good results for hyperparameter estimations for the Q-state Potts model and the Q-Ising model. Though the Q-state Potts model has the first-order phase transition, we can estimate the hyperparameters α and p from the maximization of the marginal likelihood. We then performed numerical experiments based on the artificial images in Figs. 5, 6 and 7. The artificial images f in Figs. 5a, 6a and 7a. are generated from the 256-valued standard images Mandrill, Home and Girl by thresholding. Degraded images g, produced from the original images f with Q p =0.3, are shown in Figs. 5b, 6b and 7b. The image restorations created by means of the iterative algorithm using the loopy belief propagation for the Q-state Potts model are shown in 7

8 a α = Q p b.0..2 Q p.3 = α Figure 3: Maximization of the marginal likelihood Pr{G = g α, p} with respect to p and α, whichis corresponding to the image restorations shown in figure. a p-dependence of the logarithm of marginal likelihood per pixel, Ω ln Pr{G = g α, p}, in setting α = b α-dependence of the logarithm of marginal likelihood per pixel, Ω ln Pr{G = g α, p}, in setting Q p = a α = Q p b.0..2 Q p.3 = α Figure 4: Maximization of the marginal likelihood Pr{G = g α, p} with respect to p and α, whichis corresponding to the image restorations shown in figure. a p-dependence of the logarithm of marginal likelihood per pixel, Ω ln Pr{G = g α, p} in setting α = b α-dependence of the logarithm of marginal likelihood per pixel, Ω ln Pr{G = g α, p}, in setting Q p = Figs. 5c, 6c and 7c. The image restorations created by means of the iterative algorithm using the loopy belief propagation for the Q-Ising model are shown in Figs. 5d, 6d and 7d. The image Girl has many spatially-smooth parts, whereas the image Home has many spatially-flat parts and the image Mandrill has many spatially-changing parts. However, the Q-Ising model gives us better results than the Q-state Potts model. 5 Concluding Remarks In the present paper, we investigated the loopy belief propagation algorithm for probabilistic image restorations when the Q-state Potts model or the Q-Ising model is adopted as the aprioriprobability distribution. Our algorithm can be determined the hyperparameters so as to maximize the marginal likelihood for each degraded image. Now we proceed the investigations for more generalized loopy belief propagation algorithm based on a cluster variation method[8, 9, 7, 8, 9]. Although the expectation-maximization EM algorithm is usually applied to maximize the marginal likelihood, we maximize it by calculating the approximate value of marginal likelihood directly. It is important to construct in the approximate algoritghm for the maximization of the marginal likelihood by combining the expectation-maximization algorithm[3] with the Bethe approximation or the cluster variation method. On the other hand, we now consider the application of the probabilistic image processing algorithm to object segmentations for gesture recognition systems[20]. Moreover, we have applied the loopy belief propagation not only to image processing but also to probabilistic inference and have extended it to the algorithm to calculate correlations between 8

9 a b c d Figure 5: Image restoration for an artificial 4-valued image f generated from the 256-valued standard image Mandrill obtained by thresholding Q = 4. a Original image f. b Degraded image g Q p =0.3. c Restored image f by Q-state Potts model Q p = , α =.0050, df, f = , SNR = d Restored image f by Q-Ising model Q p = , α =0.6236, df, f =0.2406, SNR =4.08. a b c d Figure 6: Image restoration for an artificial 4-valued image f generated from the 256-valued standard image Home obtained by thresholding Q = 4. a Original image f. b Degraded image g Q p = 0.3. c Restored image f by Q-state Potts model Q p =0.28, α =.0050, df, f =0.500, SNR = d Restored image f by Q-Ising model Q p =0.2827, α =0.7274, df, f = 0.249, SNR = any pairs of nodes by combining the cluster variation method with a linear response theory[2]. Acknowledgements The author is grateful to Professor T. Horiguchi of the Tohoku University, for valuable discussions. This work was partly supported by the Grants-In-Aid No for Scientific Research from the Ministry of Education, Culture, Sports, Science and Technology of Japan. References [] D. Geman, Random Fields and Inverse Problems in Imaging, Lecture Notes in Mathematics, no.427, pp.3-93, Springer-Verlag, New York, 990. [2] R. Chellappa and A. Jain eds, Markov Random Fields: Theory and Applications, Academic Press, New York, 993. [3] S. Z. Li, Markov Random Field Modeling in Computer Vision, Springer-Verlag, Tokyo, 995. [4] K. Tanaka, Statistical-mechanical approach to image processing, J. Phys. A: Math. Gen., vol.35, no.37, R8-R50, [5] H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction, Oxford University Press, Oxford, 200. [6] H. Nishimori and K. Y. M. Wong, Statistical mechanics of image restoration and error-correcting codes, Phys. Rev. E, vol.60, no., pp.32-44,

10 a b c d Figure 7: Image restoration for an artificial 4-valued image f generated from the 256-valued standard image Girl obtained by thresholding Q = 4. a Original image f. b Degraded image g Q p = 0.3. c Restored image f by Q-state Potts model Q p =0.222, α =.03, df, f =0.290, SNR = d Restored image f by Q-Ising model Q p =0.2805, α =0.7880, df, f = 0.098, SNR = [7] M. Opper and D. Saad edited, Advanced Mean Field Methods Theory and Practice, MIT Press, 200. [8] J. S. Yedidia, W. T. Freeman and Y. Weiss, Generalized belief propagation, In T. Leen, T. Dietterich and V. Tresp, editors, Advances in Neural Information Processing Systems 3, pp , 200. [9] J. S. Yedidia, W. T. Freeman and Y. Weiss, Constructing free energy approximations and generalized belief propagation algorithms, Mitsubishi Electric Research Laboratories Technical Report, Technical Report Number TR , August 2002 available at [0] K. Tanaka and T. Morita, Cluster variation method and image restoration problem, Physics Letters A, vol.203, no.2-3, pp.22-28, 995. [] Y. Weiss and W. T. Freeman, Correctness of belief propagation in Gaussian graphical models of arbitrary topology, Neural Computation, vol.3, no.0, pp , 200. [2] K. Tanaka and J. Inoue, Maximum likelihood hyperparameter estimation for solvable Markov random field model in image restoration, IEICE Transactions on Information and Systems, vol.e85-d, no.3, pp , [3] J. Inoue and K. Tanaka, Dynamics of the maximum likelihood hyper-parameter estimation in image restoration: Gradient descent versus expectation and maximization algorithm, Physical Review E, vol. 65, no., Article No. 0625, pp.-, [4] K. Tanaka, Theoretical study of hyperparameter estimation by maximization of marginal likelihood in image restoration by means of cluster variation method, IEICE Transactions, vol.j83-a, no.0, pp.48-60, 2000 in Japanese; translated in Electronics and Communications in Japan, Part 3, vol.85, no.7, pp.50-62, 2002 in English. [5] K. Tanaka, Maximum Marginal Likelihood Estimation and Constrained Optimization in Image Restoration, Transactions of Japanese Society for Artificial Intelligence, vol.6, no.2, pp , 200. [6] C. Domb, On the theory of cooperative phenomena in crystals, Advances in Physics, vol.9, no.35, 960. [7] T. Morita, General Structure of the distribution functions for the Heisenberg model and the Ising model, J. Math. Phys. vol.3, no., pp.5-23, 972. [8] T. Morita, Cluster variation method and Möbius inversion formula, Journal of Statistical Physics, vol.59, no.3/4, pp , 990. [9] T. Morita, Cluster variation method for non-uniform Ising and Heisenberg models and spin-pair correlation function, Prog. Theor. Phys., vol.85, no.2, pp , 99. [20] N. Yoshiike and Y. Takefuji, Object segmentation using maximum neural networks for gesture recognition systems, Neurocomputing, vol.5, pp , [2] K. Tanaka, Probabilistic inference by means of cluster variation method and linear response theory, IEICE Transactions on Information and Systems, vol.e86-d, no.7, 2003, to appear. 0