Matrix Calculus and Kronecker Product

Transcription

1 Matrix Calculus and Kronecker Product A Practical Approach to Linear and Multilinear Algebra Second Edition

2 This page intentionally left blank

3 Matrix Calculus and Kronecker Product A Practical Approach to Linear and Multilinear Algebra Willi-Hans Steeb University of Johannesburg, South Africa Yorick Hardy University of Johannesburg, South Africa Second Edition World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI

4 Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore USA office: 27 Warren Street, Suite , Hackensack, NJ UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. MATRIX CALCULUS AND KRONECKER PRODUCT (2nd Edition) A Practical Approach to Linear and Multilinear Algebra Copyright 2011 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. ISBN ISBN Printed in Singapore.

5 Preface The Kronecker product of matrices plays an important role in mathematics and in applications found in theoretical physics. Such applications are signal processing where the Fourier and Hadamard matrices play the central role. In group theory and matrix representation theory the Kronecker product also comes into play. In statistical mechanics we apply the Kronecker product in the calculation of the partition function and free energy of spin and Fermi systems. Furthermore the spectral theorem for finite dimensional Hermitian matrices can be formulated using the Kronecker product. The so-called quantum groups rely heavily on the Kronecker product. Most books on linear algebra and matrix theory investigate the Kronecker product only superficially. This book gives a comprehensive introduction to the Kronecker product of matrices with a large number of applications. In chapter 1 we give a comprehensive introduction into matrix algebra. The basic definitions and notations are given in section 1.1 and the basic operations are given in section 1.2. Matrices are closely associated with linear equations. This connection is described in section 1.3. The trace and determinant of square matrices are introduced and their properties are discussed in section 1.4. The eigenvalue problem plays a central role in physics. Section 1.5 is devoted to this problem. Section 1.6 presents the Cayley-Hamilton theorem which is very useful for computing functions of matrices. Projection matrices and projection operators are important in Hilbert space theory and quantum mechanics. They are also used in group theoretical reduction in finite group theory. Section 1.7 discusses these matrices. In signal processing Fourier and Hadamard matrices play a central role for the fast Fourier transform and fast Hadamard transform, respectively. Section 1.8 is devoted to these matrices. Transformations v

6 vi Matrix Calculus and Kronecker Product of matrices are described in section 1.9. The invariance of the trace and determinant are also discussed. Finite groups can be represented as permutation matrices. These matrices are investigated in section Various useful matrix decompositions are introduced in section The pseudo inverse for all matrices is defined in section The vec operator describes an important connection between matrices and vectors. This operator is also important in connection with the Kronecker product. Section 1.13 introduces this operator. The different vector and matrix norms are defined in section The relationships between the different norms are explained. We also describe the connection with the eigenvalues of the matrices. Section 1.15 discusses the approximation of a matrix by a lower rank matrix. Sequences of vectors and matrices are introduced in section 1.16 and in particular the exponential function is discussed. The Gram-Schmidt orthonormalization technique is very important in matrix algebra. This technique is described in section Groups and matrices are studied in section A number of their properties are given. Section 1.19 introduces Lie algebras. The commutator is the basic operation of Lie algebras and is central in quantum mechanics. Commutators and anti-commutators are described in section The exponential function of a square matrix is useful in many applications, for example Lie groups, Lie transformation groups and for the solution of systems of ordinary differential equations. In section 1.21 methods for calculating functions of matrices are described. Sections 2.1, 2.2 and 2.3 in chapter 2 give an introduction to the Kronecker product. In particular, the connection with matrix multiplication is discussed. In section 2.4 permutation matrices are discussed. Section 2.5 is devoted to the trace and determinant of a matrix and their relation to the Kronecker product. The eigenvalue problem is studied in section 2.6. We calculate the eigenvalues and eigenvectors of Kronecker products of matrices. We consider projection matrices and the Kronecker product in section 2.7. Fourier and Hadamard matrices are important in spectral analysis, such as fast Fourier transforms. These matrices are introduced in section 2.8 and their connection with the Kronecker product is described. The direct sum and the Kronecker sum are studied in section 2.9 and Some matrix decompositions which are specific to Kronecker product structures are described in section Section 2.12 is devoted to the vec-operator and its connection with the Kronecker product. Groups and the Kronecker product are investigated in sections 2.13 and In particular the matrix representation of groups is described. In section 2.15 the relation between

7 Preface vii the Kronecker product, the commutator and the anticommutator is investigated. The inversion of partitioned matrices is discussed in section Approximation of matrices by a nearest Kronecker product is the topic of section In chapter 3 we study applications in statistical mechanics, quantum mechanics, Lax representation and signal processing for the Kronecker product. First we introduce Pauli spin matrices and give some applications in section 3.1. The Pauli group, Clifford groups and Bell group are discussed in section 3.2. Applications in quantum theory are given in section 3.3. We assume that the Hamilton operator is given by a Hermitian matrix. We investigate the time evolution of the wave function (Schrödinger equation) and the time evolution of a matrix (Heisenberg equation of motion). The eigenvalue problem of the two point Heisenberg model is solved in detail. The one dimensional Ising model is solved in section 3.5. Fermi systems are studied in section 3.6. We then study the dimer problem, which is a combinatorical problem in section 3.7. The two dimensional Ising model is solved in section 3.8. In section 3.9 the one dimensional Heisenberg model is discussed applying the famous Yang-Baxter relation. Quantum groups are discussed in section Section 3.12 describes the connection of the Kronecker product with the Lax representation for ordinary differential equations. Signal processing and the Kronecker product is discussed in section Section 3.14 describes Clebsch-Gordan series. Section 3.15 considers the connection between the Kronecker product and Braid-like relations. One application of the nearest Kronecker product is image compression. An example method for this type of image compression is presented in section The Kronecker product can be used to find fast transforms. Section 3.16 examines the fast Fourier transform in terms of the Kronecker product. The tensor product can be considered as an extension of the Kronecker product to infinite dimensions. Chapter 4 gives an introduction into the tensor product and some applications. The Hilbert space is introduced in section 4.1 and the tensor product in section 4.2. Sections 4.3 and 4.4 give two applications. In the fist one we consider a spin-orbit system and the second one a Bose-spin system. For the interpretation of quantum mechanics (system and the measuring apparatus) the tensor product and Kronecker product is of central importance. We describe this connection in section 4.5. Section 4.6 introduces the universal enveloping algebra.

8 viii Matrix Calculus and Kronecker Product Chapter 5 presents a number of computer algebra implementations in SymbolicC++ and Maxima of examples and concepts from the previous chapters. In most sections a large number of examples and problems serve to illustrate the mathematical tools. The end of a proof is indicated by. The end of an example is indicated by. If you have any comments about the book please send them to us at: steebwilli@gmail.com yhardy@uj.ac.za Web page: The authors regularly update a collection of problems in matrix algebra and multilinear algebra. The problem collections can be found at the web site: The International School for Scientific Computing (ISSC) provides certificate courses for this subject. Please contact the authors if you want to do this course or other courses of the ISSC.

9 Contents Preface Symbol Index 1. Matrix Calculus Definitions and Notation Matrix Operations Linear Equations Trace and Determinant Eigenvalue Problem Cayley-Hamilton Theorem Projection Matrices Fourier and Hadamard Matrices Transformation of Matrices Permutation Matrices Matrix Decompositions Pseudo Inverse Vec Operator Vector and Matrix Norms Rank-k Approximations Sequences of Vectors and Matrices Gram-Schmidt Orthonormalization Groups Lie Algebras Commutators and Anti-Commutators Functions of Matrices v xiii ix

10 x Matrix Calculus and Kronecker Product 2. Kronecker Product Definitions and Notations Basic Properties Matrix Multiplication Permutation Matrices Trace and Determinant Eigenvalue Problem Projection Matrices Fourier and Hadamard Matrices Direct Sum Kronecker Sum Matrix Decompositions Vec Operator Groups Group Representation Theory Commutators and Anti-Commutators Inversion of Partitioned Matrices Nearest Kronecker Product Applications Spin Matrices Pauli Group, Clifford Groups and Bell Group Application in Quantum Theory Thermodynamics One Dimensional Ising Model Fermi Systems Dimer Problem Two Dimensional Ising Model One Dimensional Heisenberg Model Hopf Algebras Quantum Groups Lax Representation Signal Processing Clebsch-Gordan Series Braid-like Relations Fast Fourier Transform Image Compression

11 Contents xi 4. Tensor Product Hilbert Spaces Hilbert Tensor Products of Hilbert Spaces Spin and Statistics for the n-body Problem Exciton-Phonon Systems Interpretation of Quantum Mechanics Universal Enveloping Algebra Computer Algebra Implementations 283 Bibliography 301 Index 305

12 This page intentionally left blank

13 Symbol Index empty set A B union of the sets A and B A B intersection of the sets A and B N the set of positive integers: natural numbers Z the set of integers Q the set of rational numbers R the set of real numbers R + nonnegative real numbers C the set of complex numbers F field A algebra R n the n dimensional real linear space C n the n dimensional complex linear space i := 1 z complex number R(z) real part of z I(z) imaginary part of z u, v column vectors in C n e 1,n,..., e n,n standard basis in C n 0 zero vector in C n Kronecker product (direct product, tensor product) direct sum K Kronecker sum. norm (, ) scalar product (inner product) in C n tr trace det determinant xiii

14 xiv Matrix Calculus and Kronecker Product r(a) rank of the matrix A I unit matrix (identity matrix), identity operator I n n n unit matrix (identity matrix) 0 n n n zero matrix 0 m n m n zero matrix A T transpose of the matrix A σ permutation P permutation matrix U unitary matrix Π projection matrix [, ] commutator [, ] + anti-commutator δ jk Kronecker delta 1 if j = k and 0 if j k λ eigenvalue G group GL(n, F) group of all n n invertible matrices over the field F L Lie algebra σ spin σ x, σ y, σ z Pauli spin matrices ϵ real parameter c, c Fermi creation and annihilation operators b, b Bose creation and annihilation operators T transfer matrix Ĥ Hamilton operator ω frequency t time N Number of lattice sites J exchange constant Z(β) partition function, β = 1/(k B T )