arxiv:math/ v1 [math.co] 23 Oct 1998

Transcription

1 arxiv:math/9843v [mathco] 3 Oct 998 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS WITH DEFECTS HARALD HELFGOTT AND IRA M GESSEL Abstract We show how to count tilings of Aztec diamonds and hexagons with defects using determinants In several cases these determinants can be evaluated in closed form In particular weobtain solutionstoopen problems andinjamespropp slistofproblemsonenumeration of matchings [] Introduction While studying dimer models P W Kasteleyn [4] noticed that tilings of very simple figures by very simple tiles can be not only plausible physical models but also starting points for some very interesting enumeration problems Kasteleyn himself solved the problem of counting tilings of a rectangle by dominoes He also found a general method (now known as Kasteleyn matrices) for computing the number of tilings of any bipartite planar graph in polynomial time Kasteleyn s method has proven very useful for computational-experimental work but it does not of itself provide proofs of closed formulas for specific enumeration problems We shall see a few examples of problems for which Kasteleyn matrices alone are inadequate By an (abcdef) hexagon we mean a hexagon with sides of lengths abcdef and angles of degrees subdivided into equilateral triangles of unit side by lines parallel to the sides We draw such a hexagon with the sides of lengths abcdef in clockwise order so that the side of length b is at the top and the side of length e is at the bottom We shall use the term (abc) hexagon for an (abcabc) hexagon Thus Figure shows a (343) hexagon An Aztec diamond of order n is the union of all unit squares with integral vertices contained within the region x + y n+ Figure shows an Aztec diamond of order 3 We are interested in tilings of hexagons with lozenges which are rhombi with unit sides and angles of and 6 degrees and tilings of Aztec diamonds with dominoes which are by rectangles In particular we shall examine three problems from James Propp s list of open problems on tilings [] Problem (Propp s Problem ) Show that in the (n nn ) hexagon the central vertical lozenge (consisting of the two innermost triangles) is covered by a lozenge in exactly one-third of the tilings Problem (Propp s Problem ) Enumerate the lozenge-tilings of the region obtained from the (nn+nn+nn+) hexagon by removing the central triangle Problem 3 (Propp s Problem ) Find the number of domino tilings of a (k ) by k undented Aztec rectangle with a square adjoining the central square removed where the a by b undented Aztec Date: July Harald Helfgott s research was funded in part by the Undergraduate Research Program of Brandeis University Ira Gessel s research was supported by NSF grant DMS96456

2 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS Figure Aztec diamond of order 3 Figure (3 4 3) hexagon rectangle is defined as the union of the squares bounded by x + y b + x + y b a y x b+ y x (b+) We have solved these three problems not by using Kasteleyn matrices but by choosing a new approach which while much less general than Kasteleyn matrices is better suited for problems like these three We can summarize our approach as follows: Find the number of tilings of half of a hexagon or half of a diamond with dents at given places This is not new: see [6] and [7] Express the number of tilings of the figure as a whole as a sum of squares of the expressions obtained in the first step The sum s range depends on the defects (missing triangles or squares fixed lozenges or dominos) given in the problem 3 Express the sum of squares as a Hankel determinant 4 Evaluate the Hankel determinant using continued fractions or Jacobi s theorem C Krattenthaler has been working on these problems at the same time as us together with M Ciucu [5] and S Okada [9] The solution to Problem in [5] is literally orthogonal to ours: Ciucu and Krattenthaler slice the hexagon vertically rather than horizontally More generally Fulmek and Krattenthaler [8] have counted tilings of an (nmnnmn) hexagon that contain an arbitrary fixed rhombus on the symmetry axis that cuts through the sides of length m Krattenthaler and Okada s solution [9] to Problem and Krattenthaler s solution [6] to Problem are much like ours in steps and Thereafter they are based on identities for Schur functions not Hankel determinants The work of Krattenthaler and his coauthors and our work thus complement each other From Tilings to Determinants First we note that a necessary and sufficient condition for an (abcdef) hexagon to exist is that the parameters be nonnegative integers satisfying a d c f e b The number of upward pointing triangles minus the number of downward pointing triangles in an (abcdef) hexagon is a d Then since every lozenge covers one upward pointing triangle and one downward pointing triangle an (abcdef) hexagon can be tiled by lozenges only if a d and this implies that that the hexagon is an (abc) hexagon Moreover if we remove a d upward pointing triangles from an (abcdef) hexagon with a d then the remaining figure will have as many upward pointing as downward pointing triangles Definition A (kqk) upper semi-hexagon is the upper half of a (kqk) hexagon having sides kqkq + k ie a symmetric trapezium A (kqk) lower semi-hexagon is defined similarly A (kqk) dented upper semi-hexagon is a (kqk) semi-hexagon with k upward pointing triangles removed from the side of length q +k (Figure 4 shows a (343) dented upper semi-hexagon with

3 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS Figure 3 Dented 3 by Aztec rectangle Figure 4 Dented (3 4 3) semi-hexagon dents at positions 4 and 6) It will be convenient to use the term semi-hexagon for an upper semi-hexagon Note that a (kqk) semi-hexagon is the same as a (kqkq + k) hexagon so removing k upward pointing triangles leaves a region with as many upward as downward triangles Definition AnabybdentedAztecrectangleistheunionofthesquaresboundedbyx+y b+ x+y b a y x b y x (b+) with the squares in positions r < r < < r b removed from the side given by y x b (see Figure 3) Before proceeding with our results on tilings we first note some facts about the power sums j + j + +m j that we will need later on We omit the straightforward proofs For any integer m and any nonnegative integer j we define Sm j by j + +m j if m > ; Sm j if m ; ( ) j+( j + j + +( m ) j) if m < where we interpret as Lemma The numbers Sm j have the following properties: () For any integers p and q with p q p j +(p+) j + +q j Sq j Sj p () S j for all j and Sj for j > (3) For m > S j m ( )j+ S j m (4) For m Sm j is given by the exponential generating function Sm j x j j! ex (e mx ) e x j (5) Sm j is a polynomial in m of degree j + with leading coefficient /(j +) Next we prove two known results First we have a closed expression for the number of tilings of semi-hexagons with given dents first stated in this form in [6] This is equivalent to a well-known result on the enumeration of Gelfand patterns as noted in [6] or on column-strict plane partitions (See Knuth [5 exercise 3 p 7; solution p 593] for a proof similar to ours) Lemma The number of tilings of a (kqk) semi-hexagon with dents at positions r < < r k < q +k is T kqr (r j r i ) V k i<j<k

4 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 4 where V n!! n! <i<j n (j i) Proof We proceed by induction on k For the case k there is only one tiling no matter where the solitary dent is Hence the lemma holds for k Let us now assume the lemma holds for k Suppose we have a tiling of a (k +qk +) with dents at r < < r k < q +k + If we remove the bottom layer of lozenges from the dented side we obtain a tiling of a (kqk) semi-hexagon with dents at t < < t k q + k r i t i < r i+ For every such tiling of a (kqk) semi-hexagon with dents at those places there is exactly one tiling of the dented (k q k) semi-hexagon Hence T k+qr r i t i<r i+ T kqt V k r i t i<r i+ V k V k S j V k S j r i t i<r i+ i<j<k t j i k r i+ Sj r i k r i+ Sj r k (t j t i ) where Sm j j + j + + m j In the second line of our calculations we can see that since i<j<k (t j t i ) depends only on the differences between the t i s T k+qr depends only on the differences between the r i s not on their actual values (It is also easy to see this combinatorially) Hence it is sufficient to prove the formula in the case r By Lemma S j m Sj is a polynomial in m of degree j + with leading coefficient /(j +) that vanishes at Thus we can reduce the determinant S j r i+ Sj k to r j+ i+ /(j +) k by elementary column operations Hence r j+ i+ j + r j+ V k T k+qr V k r r r k V k r r r k V k V k k i+ k r j i+ k i<j<k+ i<j<k+ (r j r i ) (r j r i ) since we assumed that r Then by our observation the formula holds for all values of r 3 Tilings of Dented Aztec Rectangles Definition 3 AnabybdentedAztecrectangleistheunionofthesquaresboundedbyx+y b+ x+y b a y x b y x (b+) with the squares in positions r < r < < r b removed from the side given by y x b (see Figure 3) An a by b undented Aztec rectangle is an a by b+ dented Aztec rectangle with all squares on the side given by y x b removed

5 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 5 Our next result counts dented Aztec rectangles Another proof can be found in [7] Just as tilings of dented hexagons correspond to Gelfand patterns in [7] it is shown that tilings of dented Aztec rectangles correspond to monotone triangles and in this context a proof of the formula can be found in [8] Lemma 3 The number of tilings of an a by b dented Aztec rectangle with dents at r r b a is A abr b(b ) V b i<j<b (r j r i ) where V n!! n! Proof We proceed by induction on b First we note that if r i r i+ for some i then the lemma asserts that A abr which is correct Although of no interest in itself this case will be necessary for the induction If b there is only one tiling no matter where the one dent is (In general the number of dents has to be equal to b for the dented Aztec rectangle to be tileable) Hence the lemma holds for b Let us now assume the lemma holds for b Suppose we have a tiling of an a by b + Aztec rectangle with dents at r < < r b a If we remove all dominoes with one or two squares on the dented long diagonal and the adjacent short diagonal we obtain a tiling of an a by b Aztec rectangle with dents at t < < t b a where r k t k r k+ For every such tiling of an a by b Aztec rectangle with dents at those places there are m tilings of the a by b+ dented Aztec rectangle where m is the cardinality of {k : r k < t k < r k+ } Next we show that this implies A ab+r l {} b r k t k l k <r k+ A abt () This follows from the fact that if r k < t k < r k+ then r k t k l k < r k+ if l k is either or but if r k t k then this inequality holds only for l k and if r k t k+ it holds only for l k Thus the number of different possible values of l corresponding to a given sequence r is m where m is the cardinality of {k : r k < t k < r k+ } Moreover if for some l {} b t satisfies r k t k l k < r k+ for all i then we must have t t t b so all terms A abt that occur in either have t < < t b or are zero; in either case they are covered by the induction hypothesis Hence A ab+r A abt l {} b r k t k l k <r k+ b(b ) V b b(b ) V b b(b ) V b l {} b r k +l k t k <r k+ +l k i<j<b l {} b r k +l k t k <r k+ +l k l {} b t j i b S j r i++l i Sj r i+l i b (t j t i ) ()

6 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 6 where Sm j j + j + +m j Now if u(ijk) is any function defined for ij < b k then since a determinant is a linear function of its rows we have u(ijl i ) b u(ij)+u(ij) b l {} b Thus A ab+r b(b ) S j r V i+ +Sj r i+ (S j r i +Sj r i ) b b b(b ) (S j r V i+ +Sj r i+ ) (S j r +Sj r ) b By () we can see that since i<j<b (t j t i ) depends only on the differences between the t k s A ab+r depends only on the differences between the r k s not on their actual values Hence we may assume that r Since S j m +Sj m (Sj +Sj ) Sj m +Sj m Sj is a polynomial in m of degree j + with leading coefficient /(j+) that vanishes at we can reduce the determinant (S j r i+ +Sj r i+ ) S j b to r j+ i+ /(j +) b by elementary column operations Hence A ab+r b(b ) V b r j+ i+ j + (b+)b r j+ i+ V b b b (b+)b r r r b V b (b+)b r r r b V b (b+)b V b i<j<b+ r j i+ b i<j<b+ (r j r i ) (r j r i ) b 4 From hexagons to Determinants We now compute the number of tilings of a (kqk) hexagon with restrictions on where vertical lozenges may cross the horizontal symmetry axis Proposition 4 Let L be a subset of {k+q } Then the number of tilings of a (kqk) hexagon in which the set of indices of the vertical lozenges crossing the q+k-long symmetry axis is a subset of L is k k l i+j l L Proof We first recall that by the Binet-Cauchy theorem [ p 9] if M is any k by n matrix and M t is its transpose then the determinant of MM t is equal to the sum of the squares of the k by k minors of M

7 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 7 The number of tilings of a (kqk) hexagon in which the indices of the vertical lozenges crossing the q +k-long symmetry axis are r < r < < r k is clearly Tkqr ( r i k ) Vk j Thus the number of tilings to be counted is the sum of Tkqr over all r < r < < r k where each r j is in L Now suppose that the elements of L are l < l < < l n and let M be the k by n matrix (lj i) i<k j<n Then by the Binet-Cauchy theorem r Tkqr MM t Vk k k l i+j l L Note that since the numbers T kqr depend only on the differences of the r i the determinant in Proposition 4 depends only on the differences of the elements of L; thus we may shift all the elements of L by the same amount without changing the determinant This observation will be useful later on: Lemma 5 For any finite set L of numbers and any number u k l i+j k (l+u) i+j l L l L Proposition 6 The number of tilings of a (kn+ kkk +n kk +) hexagon with a triangle removed below the center of the horizontal line dividing the two hemispheres is ( n ) k V k V k (+( )i+j ) l i+j+ Proof If we cut such a tiled hexagon into two partsby a horizontalline through the middle vertices and then remove the lozenges that are bisected by this line we obtain a tiling of a (kn+ kk) upper semi-hexagon with dents at points r < r < < r k and a tiling of a (k+n kk+) lower semi-hexagon with dents at points r < r < < r k and at the center Since the formula in Lemma depends only on the differences among the r i we can make zero lie on the center of horizontal line dividing the two hemispheres of the hexagon Thus we have n r < r < < r k n r i The number of tilings of the upper semi-hexagon is (r j r i ) V k i<j<k and the number of tilings of the lower semi-hexagon is (r j r i ) V k i<j<k l i<k Hence the number of tilings of the hexagon for given n r < < r k n is ( r i k V k V j ) r i ( r j / r i j k ) k V k V k i<k (Note that this vanishes whenever r j for some j) r i

8 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 8 Now let M be the k by n+ matrix ( j j i ) i<k n j n Then by the Binet-Cauchy theorem the number of tilings of the hexagon is ( rj ) / r i j k MM t V k V k V k V k n r < <r n n k V k V k l l i+j n l n n k V k V k (+( )i+j ) l i+j+ l 5 From Aztec rectangles to determinants For our next result we use the following lemma which is analogous to the Binet-Cauchy theorem Lemma 7 Let U (u ij ) be a k by k matrix with rows indexed from to k and columns from to k For each k-subset A of { k } let U A be the k by k minor of U corresponding to the rows in A and all columns and let Ā be the complement of A in { k } Then U A U Ā k u ij k u i+j k A {k } A k Proof The lemma is a direct consequence of a result of Propp and Stanley [ Theorem ] More precisely the lemma follows from their result when we sum over all possibilities for A (As noted by Propp and Stanley their result is a special case of a theorem of Sylvester [4]) Proposition 8 The number of tilings of an a by b undented Aztec rectangle where a < b a+ and b k+ with squares with indices r < r < < r b a missing from a diagonal of length a+ going through the central square is k +a k ( j<i<k+ a (r i r j ) i<a k j<k+ a ) t j t i r j i a k t j i+ a k where t < t < < t a k are the elements of { a} {r r r k a } Proof Every tiling of the undented Aztec rectangle with missing squares can be subdivided into a tiling of two a by k + dented Aztec rectangles with sets of dents disjoint from each other and from {r r r k a } Let T { a} {r r r k a } and let t < t < < t a k be the elements of T Then the number of tilings of the undented Aztec rectangle with missing squares is k(k+) k j<i<k+ a (r i r j ) i<a k j<k+ a t i r j P T t t P P a k t <t (t t ) (t t ) t t T P t <t

9 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 9 which written with determinants instead of products is k(k+) ( ) (r i r j ) t i r j k j<i<k+ a i<a k j<k+ a P T P a k p j i a k q j i a k where p < p < < p a k are the elements of P and q < q < < q a k are the elements of T P Applying Lemma 7 yields the theorem We can prove the following proposition in exactly the same way Proposition 9 The number of tilings of an a by b undented Aztec rectangle a < b a + b k with squares with indices r < r < < r b a missing from a diagonal of length a + touching the central square is k k+a V k V k ( j<i<k+ a (r i r j ) i<a k+ j<k a ) t j t i r j i a k t j i+ where t < t < < t a k are the elements of { a} {r r r k a } a k 6 Computing Determinants of Aztec Rectangles: A Special Case We can now solve Problem 3 using Proposition 9 with a k b k r k The number of tilings is ( k k+a ) t j t i r j i a k t j i+ a k V k V k i<a k+ j<k a k +k V k V k ( i<k +k k (k )!k! V k V k k +k V k V k k +k V k V k For k q we have (t j t j ) j <j <k j <j <q j <j <k j <j <k (j j ) j j <q ( q 4 q (q ) ) ) t j t i (k ) i t j i k t j i+ k (t j t j ) (t j t j ) j <j <q k j <j <k j <j <k ((q ++j ) (j )) t j i+ k (t j+ t j+) (t j+ t j+) ( (q ++j ) (q ++j ) ) 3 5 (q ) q (q +) q (q +3) q (4q )

10 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS and j <j <k (t j+ t j+) j <j <q ((j +) (j +)) j <q j <q j <j <q ((q +j ) (j +)) ( q 4 q 3 (q 4))( q 4 q (q )) 3 5 (q ) q (q +) q (4q 3) ((q +j ) (q +j )) For k q + we have (t j t j ) j <j <k j <j <q (j j ) j <q j <q+ j <j <q+ ((q ++j ) (j )) ( q 4 q (q ))( q 4 q (q)) ((q ++j ) (q ++j )) 3 5 (q ) q (q +) q (q +3) q (4q +) and (t j+ t j+) j <j <k ((j +) (j +)) ((q ++j ) (q ++j )) j <j <q j j<q j <j <q ((q ++j ) (j +)) ( q 4 q (q )) 3 5 (q ) q (q +) q (q +3) q (4q ) Therefore for k q the number of tilings is (q) +q q q 4q 5 4 4q 9 (q ) 3 and for k q + the number of tilings is (q ) q (q +) q (q +3) q 3 (4q 3) 3 (4q ) (q+) +(q+) q q 4q 3 4 4q 7 (q ) 5 (q) (q ) q (q +) q (q +3) q (4q ) 3 (4q +)

11 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 7 Computing Determinants: Hexagons In this section we solve Propp s Problem and more generally we count tilings of a (m nm ) or (mn m)hexagon with a vertical lozenge at the center (A (kqk) hexagon has a central vertical lozenge if and only if k +q is odd) Lemma The number of tilings of a (m nm ) hexagon with a vertical lozenge in the center is Vm (+( ) i+j )S i+j m+n m The number of tilings of a (mn m) hexagon with a vertical lozenge in the center is Vm (+( ) i+j )S i+j m+n m Proof By Proposition 4 the number of tilings of a (m nm ) hexagon is m m+n l By Lemma 5 this determinant is equal to m+n m l i+j (+( ) i+j )S i+j l m n+ (+( ) i+j )S i+j m l i+j m+n +δ i+j m m+n m + (+( ) i+j )S i+j m+n m where δ k is if k and is otherwise It also follows from Proposition 4 that the number of tilings of a (m nm ) hexagon that do not have a vertical lozenge in the center is m l m+n l m+n By Lemma 5 this determinant is equal to m l i+j m n+ l m+n l m l i+j (+( ) i+j )S i+j m+n m We find the number of tilings that do have a vertical lozenge in the center by subtracting from the total number of tilings the number of tilings that do not have a lozenge in the center The formula for (mn m) hexagons is derived similarly As a first step in evaluating the determinants in Lemma we evaluate the determinant S i+j k p It is interesting to note that this determinant was evaluated by Zavrotsky [7] in the course of his research on minimum square sums and we follow his proof Lemma S i+j p k V k 4 (p k +) (p ) k p k (p+) k (p+k ) k V p+k V p k Vk 4 Vp k where S i p i + i + +p i and V p!! p!

12 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS Proof (Zavrotsky [7]) If p is a positive integer wecan expressthe matrix (Sp i+j ) k as the product of a k by n matrix and an p by k matrix as in the proof of Proposition 4 Since the rank of an p by k matrix is at most p the rank of the matrix (Sp i+j ) k is at most p Moreover this holds also for p Now let (a ij (λ)) k be a matrix whose entries are polynomials in λ It is known [9 p 7] that if for some value λ of λ the matrix (a ij (λ )) k has rank at most m where m k then a ij (λ) k is divisible by (λ λ ) k m as a polynomial in λ By Lemma there is a polynomial Sλ i in λ whose value at λ p is Si p Since the rank of (S i+j m )k is at most m S i+j λ k is divisible by (λ m) k m for m k Since S m i ( ) i+ Sm i when i > it follows that the rank of (S m) i+j k is at most one more than the rank of (S i+j m )k ; ie at most m Thus S i+j λ k is divisible by (λ+m) k m for m k Since Sλ i S is a polynomial in λ of degree i+ i+j λ k is a polynomial in λ of degree k Hence is equal to a constant times S i+j λ k (λ k +) (λ ) k λ k (λ+) k (λ+) k (λ+k ) Since Sλ i has leading coefficient /(i + ) and by [3 p 45] the determinant /(i+j +) k (a Hilbert determinant) is equal to Vk 4 /k we may compare leading coefficients and that the constant is Vk 4 /k Corollary The number of tilings of a (kqk) hexagon is k+q V q k+q k In particular the number of tilings of a (m nm ) hexagon is k V 4m+n 3 n m m+n V 4m 3 and the number of tilings of a (mn m) hexagon is V 4m+n n m+n V 4m m Proof By Proposition 4 and Lemma 5 the number of tilings of a (kqk) hexagon is Vk S i+j k+q k The result then follows from Lemma It is also possible as shown in [6] to derive the formula for the number of tilings of an (abc) hexagon directly from Lemma Next we prove a general theorem on Hankel determinants that allows us to evaluate the determinants in Lemma Proposition 3 Let {a i } i be a sequence and let H s (k) a(i+j+s)/ k

13 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 3 for k with H s () where we take a i to be if i is not an integer Define λ k inductively by so that λ H () a and for k Then Proof Define M r (k) by It is easy to see that Then H (k +) λ k+ λ k λ k λ k H (k )H (k +) H (k) k/ H (k) λ H (k +) j j i M r (k) a i+j+r k λ i λ i (3) H r (k) M r ( k/ )M r+ ( k/ ) (4) H (m+) H (m) M (m+) M (m) H (m+) H (m+) Thus it suffices to prove (3) for k m Since (3) holds for k to prove it for even k we need only show that for m H (m) H (m+) H (m ) m H (m ) λ i λ λ i Using (4) we may write the identity to be proved as i M (m) M (m+) M (m ) λ λ 3 λ m (5) M (m) λ λ λ m To prove (5) we use Jacobi s identity [ pp ] (M (m)) M (m)m (m)+m (m+)m (m ) Dividing both sides by M (m)m (m+) we may rewrite Jacobi s identity as M (m) M (m+) M (m ) M (m) M (m) M (m)m (m+) (6) To complete the proof we need to express the right side of (6) in terms of the λ i We have M (m) M (m ) H (m ) H (m ) λ H (m ) m H (m 3) H (m 3) λ m λ m 3 H (m 4) λ M (m ) m λ m 3 M (m ) Since M () and M () λ this gives and thus M (m) M (m ) λ λ λ m M (m) λ m (λ λ ) m (λ m 3 λ m )

14 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 4 Similarly we can show that M (m) (λ λ ) m (λ m 4 λ m 3 ) (λ m λ m ) Making these substitutions in the right side of (6) yields (5) completing the proof Note There is a simple combinatorial proof of Proposition 3 in which the determinant is interpreted as counting nonintersecting paths; see Viennot [5 Chapter IV] We now apply Proposition 3 to evaluate the determinant (+( ) i+j )Sp i+j k Proposition 4 The determinant (+( ) i+j )Sp i+j is equal to p+ p+k+ p k V 4 k p k+ where (a) j a(a+) (a+j ) k/ j k ( ) j (5 4 ) j ( p) j (p+) j () j ( 4 ) j ( 3 +p) j ( p) j Proof Let us set a i Sp i +δ i Then { Sp i a i/ +δ i/ if i is even if i is odd (+( )i )Sp i +δ i With the notation of Proposition 3 the determinant to be evaluated is a(i+j)/ k a(i+j+)/ k H (k) Thus by Proposition 3 we can express the value of this determinant in terms of the values of the corresponding determinants H (k) We have p (+( ) i )Sp i +δ i l i l p so by Lemma 5 the determinant H (k) a(i+j)/ k is equal to S i+j may be evaluated by Lemma which gives k Vp V k H (k) p+kp k V 4 p+ k This determinant Therefore with λ k as in Proposition 3 we have λ a S p p+ and for k > Thus by Proposition 3 we have λ k k 4 k/ H (k) λ H (k +) p+ where (a) j a(a+) (a+j ) j (p+k +)(p k +) (k )(k +) j i λ i λ i p+k+ p k V 4 k p k+ k/ j ( ) j (5 4 ) j ( p) j (p+) j () j ( 4 ) j ( 3 +p) j ( p) j We can now combine Lemma with the determinant evaluation of Proposition 4 to count tilings of hexagons with a vertical lozenge in the center:

15 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 5 Theorem 5 The number of tilings of a (m nm ) hexagon with a vertical lozenge in the center is V 4m+n 3 n Vm m ( ) j (5 4 ) j ( m n) j (m+n) j (m+n )Vm+n V 4m 3 () j j ( 4 ) j ( +m+n) j ( 3 m n) j and the number of tilings of a (mn m) hexagon with a vertical lozenge in the center is V 4m+n n Vm m ( ) j (5 4 ) j ( m n) j (m+n) j (m+n )Vm+n V 4m () j j ( 4 ) j ( +m+n) j ( 3 m n) j To finish the solution of Propp s Problem we need only evaluate the sum in Theorem 5 in the case m n To do this we use the Wilf-Zeilberger (WZ) method [6] Lemma 6 Proof Let n i ( ) i (5 4 ) i( n) i (n) i () i ( 4 ) i( +n) i( 3 n) i 4n 3 We want to prove that ( Q(ni) ) i (5 4 ) i( n) i (n) i 4n () i ( 4 ) i( +n) i( 3 n) i n Q(ni) 3 Since this identity is clearly true for n it is sufficient to prove that n i i n Q(n+i) Q(ni) for n > To apply the WZ method we must first find a function U(ni) such that With the help of Maple we find that if we set i U(ni+) U(ni) Q(n+i) Q(ni) (7) U(ni) i (i+ 4n)(+4n) ( 8n +4n i +i+ ) (4i+)(i++4n)(i n)(i n)(n+)n Q(ni) then(7)issatisfied (OncewehavethisformulaforU(ni)theverificationof(7)isstraightforward) Next we sum identity (7) on i from to n and add Q(n+n) to both sides The left side telescopes and we get Q(n+n)+U(nn) U(n) n i n Q(n+i) Q(ni) (8) But U(n) and we can easily check that Q(n+n)+U(nn) Thus the left side of (8) is hence so is the right side i

16 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 6 Note The sum in Lemma 6 is a partial sum of a special case of Dougall s very-well-poised 5 F 4 () sum [4 p 5 eq (3)]: if the upper limit of summation were n instead of n we would have a special case of Dougall s theorem It is interesting to note that in Ciucu and Krattenthaler s solution of Propp s Problem they used an analogous evaluation of a partial sum of the Pfaff-Saalschütz theorem [5 eq ()] We can now finish our solution to Propp s Problem : Theorem 7 In a (n nn ) or (nn n) hexagon the two central triangles are covered by a lozenge in exactly one-third of the tilings Proof We compare the result of setting m n in Corollary with the result of setting m n in Theorem 5 and evaluating the sum by Lemma 6 8 Computing More Determinants By Proposition 6 evaluating the Hankel determinant (+( ) i )Sn i+j+ k will solve Propp s Problem To do this we use the close connection between Hankel determinants and continued fractions that was implicit in our proof of Proposition 3 The following lemma is equivalent to [ Thm 7] Lemma 8 Let {a i } i be a sequence and suppose that the generating function for the a i has the continued fraction Then a i x i i where we take a r to be if r is not an integer λ λ x λ x λ 3x a (i+j)/ k λ k λk λ k By Lemma 8 if we can find the continued fraction for (+( ) j )Sn j+ x j/ Sn i+ x i j then we can evaluate the corresponding Hankel determinant The continued fraction in question is given by the following formula which we prove in the next section Proposition 9 i S i+ n x i i µ µ x µ x

17 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 7 where µ n(n+) µ i i 4i+ (n+i+)(n i) i µ i+ i+ 4i+ (n+i+)(n i) Now from Proposition 6 Lemma 8 and Proposition 9 we obtain the solution to Propp s Problem : Theorem The number of tilings of a (kn+ kkk+n kk+) hexagon without the central triangle is (n q)(n q +) 5 n 4q+ (n+) 4q+ (n+q +) (q)(q+) 3 8(q )+ 5 8(q )+ (q +) for k q + (n q +) 3 n 4q (n+) 4q (n+q) 3 (q )(q) 3 8(q ) 5 8(q ) (q ) 6 for k q 9 Proof of the continued fraction In this section we prove the continued fraction of Proposition 9 The exponential generating function for (+( ) j )S j n is by Lemma (+( ) j )Sn j x j j! ex (e nx ) e x e x (e nx ) e x j (enx )(e x e nx ) e x sinh n n+ xsinh x sinh x Now let L be the linear operator on formal power series defined by ( x i ) L u i u i x i i! We note that L(f(x)) has the formal integral representation i L(f(x)) x i f(t)e t/x dt obtained by performing the integration term by term If F(x) L(f(x)) then this formula may be written as a Laplace transform F(/z) z f(t)e tz dt and this is the form in which it is most often seen in the literature on continued fractions The continued fraction we need is given by the following formula in which n need not be an integer The case in which n is a nonnegative integer is clearly equivalent to Lemma 9

18 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 8 Lemma where L ( sinh n n+ xsinh sinh x x ) ( n+ ) x µ x µ x µ i i 4i+ (n+i+)(n i) µ i+ i+ 4i+ (n+i+)(n i) Lemma is one of several continued fractions for this function given by Lange [7 pp 59 6] (A closely related continued fraction for the same function was given by Stieltjes [3]) For completeness we give here a self-contained proof: Lemma Let f f f be formal power series in x with nonzero constant terms and let c c be constants such that for each k Then for each m Proof Equation (9) is equivalent to Iterating this formula gives f k f k c k x f k+ (9) f m f m f m f m Taking the limit as n yields the lemma c m x c m+x f k f k c k x f k+ f k j c m x c m+ x c m+n x f m+n+ f m+n Proof of Lemma Let E L so that ( ) E a j x j x j a j j! and suppose that with f k as in Lemma g k E(x k f k ) Multiplying the recurrence (9) by x k and using the fact that if u(x) is divisible by x then ( ) u(x) E d x dx E( u(x) ) j

19 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 9 we find that (9) is equivalent to We now consider the case of Lemma in which dg k dx g k c k g k+ () c i i 4i+ (n+i+)(n i) c i+ i+ 4i+ (n+i+)(n i) We shall express a solution of recurrence () in terms of the hypergeometric series defined by ( ) a b F c z (a) n (b) n z n n!(c) n where (u) n u(u+) (u+n ) We claim that a solution of recurrence () is ( g k (ex ) k k+ e nx k+ ) n F k! k + ex ( k F + k ) n k + ex () The verification that g i defined by () really does satisfy () is a straightforward but tedious computation using the formula ( ) d a b dz F c z ab ( ) a+ b+ c F c+ z together with the contiguous relations for the hypergeometric series [ p 558] (This computation was done with the help of Maple) It is not hard to show that g We now evaluate Using the easily verified fact that we find that Thus f and g (e x )e nx F ( n F ( β ) z n ) ex ( z(β ) g e nx (e nx )(e (n+)x ) n(n+)(e x ) f x L(g ) ( n+ ) L x F ( n ( z) β ) ex sinh n n+ x sinh n(n+) sinh x ( sinh n n+ x sinh sinh x x ) () Substituting these values of f and f into the case m of Lemma and multiplying both sides by ( ) n+ x completes the proof of Lemma ) x

20 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS It is clear from the recurrence () and the value of g that g k is a rational function of e x and e nx Although we won t need it we can give an explicit formula that expresses g k in this form by applying to () the formula F ( m+ β k + ) z ( ) k k! [( ) m( z)k m β m!z k ( β) k m ) z F ( m β β m+k ( k) m ( β) k k m i ] (β k) i ( k +m) i z i i!( k) i for k m which can be proved by equating coefficients of powers of z on both sides (Note that () is the case m k ) Acknowledgments We would like to thank Christian Krattenthaler for telling us about the status of his work on these problems and James Propp for his assistance at various times during the writing of this paper References [] M Abramowitz and I A Stegun Handbook of Mathematical Functions Dover New York 97 [] G E Andrews The Theory of Partitions Encyclopedia of Mathematics and Its Applications Vol Addison- Wesley Reading Massachusetts 976 [3] J W Archbold Algebra Pitman Paperbacks Bath97 [4] W N Bailey Generalized Hypergeometric Series Cambridge University Press London935 [5] M Ciucu and C Krattenthaler The number of rhombus tilings of a symmetric hexagon which contain the central rhombus J Combin Theory Ser A to be published [6] H Cohn M Larsen and J Propp The shape of a typical boxed plane partition submitted to the New York Journal of Mathematics [7] N Elkies G Kuperberg M Larsen and J Propp Alternating-Sign matrices and domino tilings J Algebraic Combinatorics (99) 3 and 9 34 [8] M Fulmek and C Krattenthaler The number of rhombus tilings of a symmetric hexagon which contain a fixed rhombus on the symmetry axis I Ann Combin (998) 9 4 [9] R A Frazer W J Duncan and A R Collar Elementary Matrices and Some Applications to Dynamics and Differential Equations Cambridge University Press 947 [] F R Gantmacher The Theory of Matrices Volume Chelsea New York 959 [] P Henrici Applied and Computational Complex Analysis Volume Wiley New York 974 [] W B Jones and W J Thron Continued Fractions: Analytic Theory and Applications Encyclopedia of Mathematics and Its Applications Vol Addison-Wesley Reading Massachusetts 98 [3] W Jockusch and J Propp Antisymmetric monotone triangles and domino tilings of quartered Aztec diamonds J Algebraic Combinatorics to be published [4] P W Kasteleyn The statistics of dimers on a lattice I The number of dimer arrangements on a quadratic lattice Physica 7 (96) 9 5 [5] D E Knuth The Art of Computer Programming Vol 3: Sorting and Searching Addison-Wesley Reading Massachusetts 973 [6] C Krattenthaler Schur function identities and the number of perfect matchings of holey Aztec rectangles preprint [7] L J Lange Continued fraction representations for functions related to the gamma function in Continued Fractions and Orthogonal Functions: Theory and Applications ed S Clement Cooper and W J Thron M Dekker 994 pp [8] W H Mills D P Robbins and H Rumsey Jr Alternating sign matrices and descending plane partitions J Combin Theory Ser A 34 (983) [9] S Okada and C Krattenthaler The number of rhombus tilings of a punctured hexagon and the minor summation formula Adv Appl Math to be published [] J Propp and R Stanley Domino tilings with barriers preprint

21 ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS [] J Propp Twenty open problems in enumeration of matchings online document 996: propp/openpsgz [] C Radoux Calcul effectif de certains déterminants de Hankel Bull Soc Math Belg Sér B 3 (979) [3] T J Stieltjes Sur quelques intégrales définies et leur développement en fractions continues Ouvres Complètes Vol P Noordhoff Groningen 98 pp Originally published in Quarterly Journal of Math 4(89) [4] J J Sylvester On a certain fundamental theorem of determinants Philosophical Magazine (4) (85) 4 45 [5] X G Viennot Une théorie combinatoire des polynômes orthogonaux généraux Conference Notes Université du Québec à Montréal 983 [6] H S Wilf and D Zeilberger Rational functions certify combinatorial identities J Amer Math Soc 3 (99) [7] A Zavrotsky El Gesseliano (Spanish) Notas de Matematicas no 73 Universidad de los Andes Facultad de Ciencias Departamento de Matematica Merida Venezuela 985 Mathematics Department Brandeis University Waltham MA address: hhelf@csbrandeisedu gessel@mathbrandeisedu