Precoding and Signal Shaping for Digital Transmission. Robert F. H. Fischer Copyright 0 2002 John Wiley & Sons, Inc. ISBN: 0-471-22410-3 Appendix A Wirtinger Calculus T he optimization of system parameters is a very common problem in communications and engineering. For example, the optimal tap weights of an equalizer should be adapted for minimum deviation (e.g., measured by the mean-squared error or the peak distortion) of the output signal from the desired (reference) signal. For an analytical solution, a cost function is set up and the partial derivatives with respect to the adjustable parameters are set to zero. Solving this set of equations results in the desired optimal solution. Often, however, the problem is formulated using complex-valued parameters. In digital communications, signals and systems are preferably treated in the equivalent complex baseband [Fra69, Tre7 1, Pro0 11. For solving such optimization problems, derivation with respect to a complex variable is required. Starting from well-known principles, this Appendix derives a smart and easily remembered calculus, sometimes known as the Wirtinger Calculus [FL88, Rem891. 405
406 WIRTINGER CALCULUS A.l REAL AND COMPLEX DERIVATIVES First, we consider a real-valued function of a real variable: f : IR 3 x t+ y = f(x) E IR. (A.l.l) The point xopt, for which f(x) is maximum is obtained by taking the derivative of f with respect to x and setting it to zero. For xopt the following equation has to be valid: (A. 1.2) Here we assume f(x) to be continuous in some region R, and the derivative to exist. Whether the solution of the above equation actually gives a minimum or maximum point has to be checked via additional considerations or by inspecting higher-order derivatives. Analogous to real functions, a derivative can be defined for complex functions of a complex variable as well: f: C 3 zt+ w =f(z) E c (A.1.3) (A.1.4) The above limit has to exist for the infinitely many series {zn} which approach ZO, i.e., lim z, = 20. If f (z) exists in a region R C C, the function f(z) is called n+ 03 analytic, holomorphic, or regular in R. In the following, the relations between real and complex derivatives are discussed. A complex function can be decomposed into two real functions, each depending on two real variables x and y, the real and imaginary parts of z: f(z) = f(x +j y) 2 u(x, y) + ju(x, y), z = x +j y. (A.1.5) It can be shown that in order for f(z) to be holomorphic, the component functions u(x, y) and u(x, y) have to meet the Cauchy-Riemann differential equations, which read (e.g., [FL88, Rem891): (A.1.6a) (A. 1.6b) The same considerations are valid for minimization.
WlRTlNGER CALCULUS 407 The complex derivative of a holomorphic function f(z) can then be expressed by the partial derivatives of the real functions u(x, y) and ~ (x, y): (A.1.7) The complex derivative of a complex function plays an important role in complex analysis-in communications it has almost no significance. In fact, a more common problem is the optimization of real functions, depending on complex parameters. Complex cost functions are of no interest, because in the field of complex numbers no ordering (relations < and >) is defined and thus minimization or maximization makes no sense. A.2 WlRTlNGER CALCULUS As already stated, we have to treat real functions of one or more complex variables. Thus, let us now consider functions f : Q: 3 z =x+j y c-) w = f(z) = u(zly) E IR. (A.2.1) Since ~ (x, y) = 0 holds (cf. (A.lS)), f(z) generally is not holomorphic. A real function would only be regular if, according to (A. 1.6), - 0 and &@& 8Y = - 0 are valid. But this only holds for a real constant, and hence can be disregarded. The straightforward solution to the optimization of the above function is as follows: Instead of regarding f(z) as a real function of one complex variable, we view f(z) = u(z, y) as a function of two real variables. Thus optimization can be done as for multidimensional real functions. We want to find which requires f(z) -+ opt. E u(x,y) -+ opt., WX,Y) I dx - 0 and ~ du(z,y) I(). 8Y (A.2.2) In order to obtain a more compact representation, both of the above real-valued equations for the optimal components xopt and yopt can be linearly combined into one complex-valued equation: (A.2.3) where, for the moment, a1 and a2 are arbitrary real and nonzero constants. Equations (A.2.2) and (A.2.3) are equivalent (and hence, of course, result in the same solution) because real and imaginary part are orthogonal. As already stated, this procedure is mainly intended to get a compact representation.
408 WlRTlNGER CALCULUS Writing real part and imaginary part of z = x + j y as the tuple (z, y), we can define the following differential operator: (A.2.4) This operator can, of course, also be applied to complex functions (A. 1.5). This is reasonable, because real cost functions are often composed of complex components, e.g., f(z) = 1zI2 = z. z* fl(z). f ~(z), with an obvious definition of fl,z(z) E C. Note, z* 2 x - j y denotes the complex conjugate of z = x + j y. The remaining task is to chose suitable constants a1 and u2. The main aim is to obtain a calculus that is easily remembered and easy to apply. As will be shown later, the choice a1 = $ and a2 = -$ meets all requirements. To honor the work of the Austrian mathematician Wilhelrn Wirtinger (1 865-1945) who established this differential calculus, we call it Wirtinger Calculus. Definition A. 1 : Wirfinger Calculus The partial derivatives of a (complex) function f (z) of a complex variable z = 2 + j y E C, 5, y E R, with respect to z and z*, respectively, are defined as: and - af a 1. a.f dz --,-.a.f - 2 (ax By) (A.2.5) (A.2.6) A.2.1 Examples We now study some important examples. First, let f(z) = cz, where c 6 C is a constant. Derivation of f(z) yields and + j "(>;J ") = 1 (c + j (j c)) = 0. (A.2.8) az* 2 2 Similarly, for f (z) = cz*, we arrive at: az 2 dx -j -Jy)) = (c- j (-j c)) = 0, ay (A.2.9)
WlRTlNGER CALCULUS 409 and +jdc(xd~jy)) = -(c+j(-jc)) 1 =c. (A.2.10) dz* 2 dx 2 Next, we consider the function f = zz* = 1zI2 = x2 + y2. Here the derivatives read: a a 1 d(x2+ y2) - + y2)) = (22- j2y) = zz* z*, (A.2.11) = - ( -j and az 2 dx d 1 d(x2+y2) - zz* = - ( dz* 2 dx ay +j dy To summarize, the correspondences in Table A.l are valid. y2)) = f (2x + j 2y) = z. (A.2.12) Table A. I Wirtinger derivatives of some important functions CZ CZ* ZZ* C 0 0 c z* z Note that using the Wirtinger Calculus differentiation is formally done as with real functions. Moreover, and somewhat unexpected, z* is formally considered as a constant when derivating with respect to z and vice versa. It is also easy to show that the sum, product, and quotient rules still hold. For example, given f(z) = fl(z). f2(z), we obtain a -f1(z).f2(z) = az - j afl(.)f2(z) - j j l(z)~) dy (A.2.13)
410 WIRTINGER CALCULUS Finally, for f(z) = h(g(z)) 5 h(w), g : C ++ C, the following chain rules hold [FL88, Rem891: A.2.2 Discussion The Wirtinger derivative can be considered to lie inbetween the real derivative of a real function and the complex derivative of a complex function. Rewriting (A.2.5) and (A.2.6), we arrive at: af- = o dz* (A.2.15a) (A.2.15 b) On the one hand, equation (A.2.15a) states that for holomorphic functions the Wirtinger derivative with respect to z agrees with the ordinary derivativeof a complex function (cf. (A.l.7)). On the other hand, (A.2.15b) can be interpreted in the way that holomorphic functions do not formally depend on z*. Contrary to the usual complex derivative, the Wirtinger derivative exists for all functions, in particular nonholomorphic ones, such as real functions. Since both operators and & are merely a compact notation incorporating two real differential quotients, they can be applied to arbitrary functions of complex variables. For nonholomorphic functions, $ # 0 usually holds, and thus either the derivative with respect to z or z* can be used for optimization. The actual cost functions determines
GRADIENTS 4 1 1 which one is more advantageous; if quadratic forms are considered, the operator is preferable. To summarize, it should again be emphasized, that, because of its compact notation, Wirtinger Calculus is very well suited for optimization in engineering. It circumvents a separate inspection of real part and imaginary part of the cost function. Because of the simple arithmetic rules-mostly it can be calculated as known from real functions-the Wirtinger Calculus is very clear. A.3 GRADIENTS For the majority of applications the cost function does not only depend on one, but on many variables, e.g., we have f : C" 3 z = [zl, z2,..., znit c-) w = f(z) E IR. For optimization, all n partial derivatives with respect to the complex variables z1 = 2 1 + jy1 through z, = 2, + jy, have to be calculated. Usually, these derivatives are again combined into a vector, the so called gradient: A which, in the optimum, has to equal the zero vector 0 = [0, 0,..., 0IT. Wirtinger Calculus is especially well suited for such multidimensional functions, because here only with a great effort can the real part and the imaginary part be separated and inspected independently. Using the above definitions of the partial derivatives ((A.2.5) and (A.2.6)), we arrive at simple arithmetic rules, now expressed using vectors and matrices. A.3.1 Examples We now again study some important examples. First, let f(z) = ctz = c:=l c,z, or n f(r) = ctz* = c,=l czz:, respectively, with c = [c~, c2,..., c,it and c, constant. It is easy to prove the following properties for gradients: -' T c z=c, ' T -c z*=o, az 8% L T az* c z=o, Z c T z*=c. az* (A.3.2) *Here we use column vectors, but the same considerations also apply to row vectors.
4 f 2 WlRTfNGER CALCULUS Finally, considering the quadratic form f (z) = zhmz, where M is a constant n x n matrix, derivation results in: d T - zhmz = (zhm) 8% az* and for f(z) = zhz we arrive at: -' z H z=z*, 8% a - zhmz = MZ d H z z=z. dz* To summarize, the correspondences in Table A.2 are valid. (A.3.3) (A.3.4) Table A.2 Wirtinger derivatives (gradients) of some important functions. A.3.2 Discussion The gradient with respect to the Wirtinger derivatives, is related to the gradient (A.3.5) which is frequently used, e.g., in [Hay961 by or since f(z) is real-valued, af (z) (Vf(Z))* = 2-. 8% (A.3.7) The first disadvantage of definition (A.3.6) compared to Wirtinger Calculus is that an undesired factor of 2 occurs. In particular, if the chain rule is applied 1 times, the result is artificially multiplied by 2l. Second, the calculus is not very elegant, because the gradient of, e.g., f (z) = ctz is V(cTz) = 0, but that of f(z) = ctz* calculates to V(cTz*) = 2%. Hence, because of its much clearer arithmetic rules, we exclusively apply the Wirtinger Calculus.
GRADIENTS 413 REFERENCES [FL881 [Fra69] [Hay961 [Pro011 [Rem89] [Tre711 W. Fischer and 1. Lieb. Funktionentheorie. Vieweg-Verlag, Braunschweig, Germany, 1988. (In German.) L. E. Franks. Signal Theory. Prentice-Hall, Inc., Englewood Cliffs, NJ, 1969. S. Haykin. Adaptive Filter Theory. Prentice-Hall, Inc., Englewood Cliffs, NJ, 3rd edition, 1996. J. G. Proakis. Digital Communications. McGraw-Hill, New York, 4th edition, 200 1. R. Remmert. Funktionentheorie 1. Springer Verlag, Berlin, Heidelberg, 1989. (In German.) H. L. van Trees. Detection, Estimation, and Modulation Theory-Part 111: Radar-Sonar Signal Processing and Gaussian Signals in Noise. John Wiley & Sons, Inc., New York, 1971.