ENUMERATING CURVES IN CALABI-YAU THREEFOLDS CONTENTS

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS LECTURES BY JUN LI, NOTES BY TONY FENG CONTENTS 1. Introduction 2 Part 1. Moduli 6 2. Moduli spaces 6 3. Moduli of sheaves via GIT 11 Part 2. Deformation and obstruction theory 21 4. Obstruction theory 21 5. Construction of virtual cycles 26 6. Deformation theory 33 Part 3. Gromov-Witten Theory 37 7. Constructing Gromov-Witten invariants 37 8. K3 surfaces 39 9. The Yau-Zaslow Theorem 47 1

2 LECTURES BY JUN LI, NOTES BY TONY FENG 1. INTRODUCTION This course is about the enumerative geometry of curves in Calabi-Yau threefolds. Definition 1.0.1. A Calabi-Yau threefold is a 3-dimensional projective variety X over such that K X = 3 T X = X is trivial and h 1 ( X ) = 0. Today we ll explain three examples that illustrate the type of questions that we re interested in. 1.1. Counting rational curves. Let X = Q 5 be a quintic hypersurface in 4. This is a Calabi-Yau threefold, by the adjunction formula for triviality of K and the Lefschetz hyperplane theorem for triviality of H 1 ( X ). We are interested in the space Rat d (Q 5 ) of rational curves of degree d in Q 5. We can think of such a curve as a map F : 1 4 given by degree d polynomials and factoring through Q 5 : 1 Q 5 F Now, a map 1 4 is given by [f 0,..., f 4 ] with f i H 0 ( 1(d )) = d +1, so the F are parametrized by f = (f 0,..., f 4 ) 5d +5 which do not have common zeros (which is what happens in general). The map F factors through Q 5 if F (f 0,..., f 4 ) = 0 H 0 ( 1(5d )) = 5d +1 (by Riemann-Roch). We have constructed a map 4 H 0 ( 1(d )) 5 F H 0 ( 1(5d )) sending f F (f ). Then F 1 (0) should be a union of 4-dimensional cones. Note that if f : 1 Q 5, then for any α Aut( 1 ) we also have that f α factors through Q 5. So each f : 1 Q 5 contributes to a 4-dimensional subvariety in F 1 (0) (three from automorphisms and one from scaling), which leads to the following conjecture. Conjecture 1.1.1 (Clemens). For very general Q 5 4, the space Rat d (Q 5 ) is discrete for all d 1. The classical method is to study the incidence scheme Φ d = {(C, F ) C F 1 (0),C Rat d ( 4 )} and more precisely to study its irreducibility and fibers of the obvious projection Φ d π Ratd ( 4 ). Theorem 1.1.2. If d 11, then Φ d is irreducible. For d 12, then Φ d is reducible. Irreducibility implies Clemens conjecture TONY: [why?], but reducibility doesn t disprove it.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 3 1.2. Discreteness of curves in CY 3-folds. Curves in a CY 3-fold are expected to be discrete. Let X be a CY 3-fold and C X a smooth curve. We can form the Hilbert scheme Hilb P X, with respect to the Hilbert polynomial P(m ) = χ( C (m)) = n deg H C + (1 g ) (there is always an ample line bundle H on X implicitly fixed). Then Hilb P X parametrizes curves in X with Hilbert polynomial P. Assume g 2 for simplicity. We want to calculate dim T [C ] Hilb P X. A first-order deformation of a smooth subcurve C in X gives (tautologically) an abstract first-order deformation of C in moduli, so we get a map dim T [C ] Hilb P X Def(C ). What is the kernel? If the complex structure of C doesn t change, then the deformation must be given by a vector field, so the kernel of this map is H 0 (T X C ). Also, there exists Def(C ) δ H 1 (T X C ) (which requires some thinking to see!). We claim that we have an exact sequence 0 H 0 (T X C ) dim T [C ] Hilb P X Def(C ) δ H 1 (T X C ) Exercise 1.2.1. Construct the arrows in this exact sequence, and prove that it is exact. It is expected that δ is surjective, so we can read off dim T [C ] Hilb P X = dim Def(C ) + h0 (T X C ) h 1 (T X C ). By Riemann Roch and well-known fact, this is (3g 3)+deg T X C +rank T X C (1 g ), and deg T X C = 0 since X is Calabi-Yau, so we see that this is 0. Remark 1.2.2. If the curves in X are discrete, then we can enumerate the number of curves of genus g and degree d in X. This hinges on δ being surjective. To get that to be the case, you can try to get this by varying the quintic in H 0 ( 4, (5)). This works for some small choices of g and d, but it is hard to arrange uniformly. 1.3. Enumerating rational curves in a K3 surface. Definition 1.3.1. A K3 surface is a 2-dimensional Calabi-Yau manifold. Let X = S be a K3 surface. Pick L Pic(S) and C L a smooth curve. Then the adjunction formula (and triviality of K S ) tells us that g a (C ) = 1 2 L2 + 1 Applying Riemann-Roch for surfaces and using that T C N C S = 2 T X = X, and deg N C X = C C = L 2 we also find that dim L = 1 2 L2 + 1. We expect that {curves with at least k nodes in L } has codimension k, hence (expected) dimension dim L k. So the number of rational nodal C L, which have 1 2 L2 +1 nodes, should be finite. (There s a better argument, showing that L must contain a rational curve, and rational curves cannot form a family.)

4 LECTURES BY JUN LI, NOTES BY TONY FENG Theorem 1.3.2 (Chen). For general S, we have Pic(S) = [L] and rational C L are nodal. Yau-Zaslow found a way to count this finite number. For smooth C L and C an invertible sheaf, we can form the 1-dimensional sheaf ι on S. Then FIGURE 1.3.1. Pushforward of a line bundle from a curve in S. P ι (n) = χ(ι L n ) = χ( H n C ) = n deg L C + deg + (1 g ). The point is that this is a linear polynomial. Suppose we assume that deg + (1 g ) is an odd number, and let s even assume that it is 1. Define S (P) = {stable shaves of S -modules, χ((n)) = p(n)}. There are some very bad 1-dimensional sheaves (Figure 1.3), and the stability condition is intended to rule them out. Theorem 1.3.3. S (P) is smooth (using that P(0) = 1). There is a map S (P) π Chow(S) (this is the Chow variety of curves in S, not the Chow group!) sending supp(). By the assumptions on the Picard group, this must land in L. Yau and Zaslow made the very interesting observation that if C L is a nodal curve with geometric genus > 1, then e (π 1 [C ]) = 0. Exercise 1.3.4. Prove this. [Hint: If C is a smooth curve, then Pic 0 (C ) is an abelian variety. Then Pic 0 (C ) acts on π 1 [C ] by tensoring a vector bundle with a line bundle on C. Use this to show that e (π 1 [C ]) = 0. In fact, we don t need the full abelian variety; it s enough to have a torus action.]

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 5 FIGURE 1.3.2. A sheaf on S which is composed of the pushforward of a line bundle curve plus a skyscraper sheaf over a point. The condition of stability rules out these sorts of bad sheaves. Therefore, e (M X (P)) = C L rational e (π 1 ([C ])). Assume that S is general. Then any rational C in L are necessarily nodal. Exercise 1.3.5. It is an exercise to show that e (π 1 [C ]) = 1 in such cases. [Hint: π 1 ([C ]) consists of on 1 plus identifications over two pairs of fibers, say P P and Q Q. This is the same as giving two separate identifications, hence two 1 each with two points glued, which have Euler number 1. ] Theorem 1.3.6. e ( S (P)) can be calculated. Theorem 1.3.7 (Yau-Zaslow). If n g = # Rat L (S), where 1 2 L2 + 1 = g, then n g q g = (1 q n ) 24. n=1 These ideas actually come from string theory. 1.4. Course plan. The plan of the course is to study curves in Calabi-Yau threefolds. One could first attempt to do this by classical algebraic geometry. It turns out that this leads to a lot of tricky issues about the space of curves. However, we shall see that if certain genericity is met (as in the above examples), then we can get good answers. The genericity amounts to saying that the counting is a topological enumeration. To achieve this in algebraic geometry, we use virtual cycles of certain moduli spaces. For this we need the moduli spaces to be proper. Here is a sampler of the moduli spaces we re interested in:

6 LECTURES BY JUN LI, NOTES BY TONY FENG Moduli of stable maps: g (X,α) = {C sm g C X as subscheme f X f [C ] = α H 2 (X,)} g (X,α). {C sm g X [C ] α H 2 (X,)} Hilb P X Another compactification of curves as subschemes: any such curve gives a structure exact sequence 0 C X X C 0 and we can consider the data [ X C ] in the derived category and compactify. The first moduli leads to GW invariants, the second leads to DT invariants, and the third leads to PT invariants. Part 1. Moduli 2. MODULI SPACES We re going to sketch constructions of three types of moduli spaces. One is a moduli space of framed objects, i.e. objects equipped with an embedding in an ambient space. This includes, for instance, the Hilbert schemes. Slightly more generally, we can consider moduli of sheaves, which leads to the Quot schemes. Another kind of moduli space is that of unframed objects, e.g. the moduli space of curves g. This was first studied by Mumford, using the strategy of quotienting a moduli space of framed objects by automorphisms. The third kind of moduli spaces are the stacks, which go beyond the realm of schemes. 2.1. Moduli problems. Example 2.1.1. Consider all quotient vector spaces m P where dim P = k < m. The space of such is parametrized by Gr(k, m ). This is one of the first examples of moduli spaces. Example 2.1.2. (Hilbert schemes) Let X n be projective and h a polynomial. Let H = n (n) X. We consider {Z X P Z = h} where P z (n) = χ( Z H n ) is the Hilbert polynomial. Usually there is no dispute about what the closed points are, but it is not necessarily clear how to define the scheme structure. Sometimes, it is appropriate to define a nonreduced scheme structure. Grothendieck taught us that scheme structure is captured by considering not just closed points but all scheme-valued points. This leads us to consider the moduli functor sending : Schemes Sets opp S Z S Z closed Z S flat over S P Zs =h for all s S.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 7 This is functorial: if S S, then a family over S can be pulled back to a family over S. Exercise 2.1.3. Carefully write down a moduli functor for other moduli problems. Definition 2.1.4. We say that M is the fine moduli space of if there exists a natural transformation = Hom(, M ): Schemes Sets opp. Here Hom(, M )(S) = Hom(S, M ). In particular this implies that (M ) = Hom(M, M ), so there should be a family M corresponding to 1 M Hom(M, M ). This M is called the universal family. By definition, any family T (T ) = Hom(T, M ) corresponds to some ρ : T M. T ρ By the definition of 1 M, we have that is canonically isomorphic to the pullback of via ρ. Example 2.1.5. The fine moduli space for Example 2.1.1 is Gr(k, m ). 2.2. Quot schemes. Let Z be a closed subscheme of a projective variety X (with a given ample line bundle (1)). Then Z is defined by its ideal sheaf Z X, which fits into a fundamental exact sequence M 0 Z X Z 0. The subsheaf is completely determined by the quotient sheaf, and it turns out to be better to think about quotients. So we are going to construct a moduli space for quotient sheaves X with χ = h, where χ (n) = χ( (n)). Definition 2.2.1. A quotient sheaf of a fixed vector bundle over X is a surjection of sheaves. If is a surjection over X, we can push it forward via a closed embedding ι : X n. Any quotient sheaf ι over n is obtained from = ι. Using this fact we can punt our problems to projective space. Definition 2.2.2. Fix a projective variety (X, (1)), a polynomial h, and a sheaf of X - modules. We define a moduli functor Quot h : Schemes Setsopp sending S p X flat over S P s =h for all s S. Example 2.2.3. For X = pt, we are consider quotients as in Example 2.1.1. Example 2.2.4. If X is a projective scheme and = X, then we are parametrizing quotients [ X ]. Theorem 2.2.5. The moduli functor Quot h has a projective fine moduli space, called the Quot scheme.

8 LECTURES BY JUN LI, NOTES BY TONY FENG Outline of proof. Let T be a scheme. Suppose we have an object [px ] Quoth (T ). We re going to apply a standard trick of twisting by a higher power of (1). First some notation: we have a line bundle (1) on X, and we denote Consider the exact sequence We can twist by µ to obtain p X (µ) := p X p X (µ). 0 p X 0. 0 (µ) p X (µ) (µ) 0. We can then push forward to T to get a long exact sequence 0 p T (µ) p T p X (µ) p T (µ) R 1 p T (µ).... For a given, we have that R 1 p T (µ) = 0 for large enough choice of µ. That this choice can be made uniformly in is highly non-obvious, and is the key ingredient in the proof. Also note that p T p X (µ) = H 0 (X,(µ)) T. Now we use the (crucial!) flatness assumption. One characterization of being flatover T is that X T is flat over T if for µ 0, p T (u ) is a locally free sheaf of T -modules. (This uses Serre s vanishing theorem for ample line bundles plus cohomology and base change.) Therefore, from the original quotient [p X ] Quoth (T ) we get a point [H 0 (X,(µ)) T p T ] in a certain Grassmannian, namely Gr(H 0 (X,(µ)), h(µ))(t ). This globalizes to a map ξ ψ µ (ξ): Quot h Gr µ := Gr(H 0 (X,(µ)), h(µ)). This embeds the moduli functor in a Grassmannian, and we want to show that Quot schemes are cut out as a closed subscheme of the Grassmannian. There is a significant issue outstanding here, which is that we need to argue that we can find a single µ that works for all ξ Quot h (T ). For dimension 1, this is clear by Riemann-Roch. In general, one tries to slice the dimension down and use induction. We will not give the proof. The problem is equivalent to showng that Quot h () is bounded in an appropriate sense. To be precise, this should mean that it is parametrized by a finite type scheme. Lemma 2.2.6. The map ψ µ : Quot h Gr µ is injective. (It doesn t follow immediately that ψ µ is actually an embedding, but we ll show that later.) Proof. Suppose that ξ 1 = [ 1 ] and ξ 2 = [ 2 ] are two elements of Quot h () such that ψ µ ( 1 ) = ψ µ ( 2 ). Then we want to show that ξ 1 = ξ 2. In other words, we have that H 0 ( 1 (µ)) = H 0 ( 2 (µ)) as elements of Gr µ. Then we want to argue that 1 = 2. Well, we have the exact sequences and 0 1 1 0 0 2 2 0.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 9 We have that (µ) is generated by global sections for µ sufficiently large, i.e. H 0 ( (µ)) X (µ). The key point is that we can also assume for µ large that 1 (µ) is generated by global sections, and that this can be arranged uniformly in 1. This implies that 1 (µ) is the image of H 0 ( 1 (µ)) X H 0 ((µ)) X (µ), and similarly for 2. 0 H 0 ( 1 (µ)) H 0 (X,(µ)) H 0 ( 1 (µ)) 0 0 H 0 ( 2 (µ)) H 0 (X,(µ)) H 0 ( 2 (µ)) 0 So we have shown that the equality of global section implies (for large enough µ) that 1 (µ) = 2 (µ), which implies that 1 = 2. This shows (granting µ exists with all desired vanishing) that ψ h Hom(, Gr µ ) is injective. Next, we define closed subscheme Q µ Gr µ such that for all ρ : T Gr µ, ρ factors through T ρ Q µ Gr µ if and only if there is a ξ Quot h (T ) such that ρ = ψ µ(ξ), i.e. Quot h Hom(, Gr µ ) = Hom(,Q µ ) Recall that the map ψ µ was constructed by starting with and twisting and pushing it forward to ξ = [0 p X 0] Quoth (T ) 0 p T (µ) p T p X (µ) p T (µ) 0 The key trick is to apply this to the universal family on Gr µ =: Gr. That is a vector bundle equipped with a natural surjection 0 W X Gr 0. where W = H 0 (X,(µ)). Taking global sections, we get (for large enough µ) an exact sequence 0 K = H 0 ( (µ)) W H 0 ( (µ)) 0. A map X Gr is determined by the information of a surjection of bundles over X W X, which we can suppose has kernel. Then for our choice of µ, the image of K W X (µ) is (µ) (µ). The point is that this recovers the subsheaf (µ) (µ).

10 LECTURES BY JUN LI, NOTES BY TONY FENG Proposition 2.2.7. There exists a locally closed Q Gr µ such that Quot P Hom(, Gr µ ) = Hom(,Q) Proof. A T -point of Quot P Hom(, Gr µ) is given by a surjection p X. We have shown how to choose a uniform µ so that this gives a T -point of Gr µ : Let (µ) be the image of W X Gr p X (µ). X Gr W X Gr p X (µ). The quotient sheaf P(µ) := px (µ)/ (µ) is not flat over all of Gr (as expected). What we expect is that for some closed subset Q Gr, all closed points ω Q Gr satisfy H 0 (P X ω (µ)) = P(µ) and H i >0 (P X ω (µ)) = 0. Exercise 2.2.8. Consider the map of sheaves given by 3 2 2 2 (a 1, a 2, a 3 ) (x a 1 + y 2 a 2,x a 3 ). Check that the loci where the rank of the quotient is equal to 0, 1, 2 are locally closed subschemes. Lemma 2.2.9. Let Z be a scheme. For any vector bundles V 1, V 2 and ϕ : Z (V 1 ) Z (V 2 ) and any r, there exists a unique maximal locally closed Z r Z such that the cokernel of φ Zr O Z (V 1 ) Zr Z (V 2 ) Zr is a rank r locally free sheaf of Zr -modules. This is a determinantal construction. This lemma plus a technical argument implies what we want. We re going to skip the details. The idea is that you push all calculations to the Grassmannian. It only remains to show that Q Gr µ is proper. This will imply that Q is projective and since Quot P Hom(,Q) is an equivalence. This proves that QuotP has a fine moduli space. Proof. We ll use the valuative criterion for properness, which says that if C = C {pt}, then a map C Q can be extended to C Q: C C Q

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 11 A map C Q is equivalent to an element of Quot P (C ), which is the data of a surjection px over X C. The game is to extend this to a flat surjection of sheaves px over X C. Let = ker px over X C, which fits into a short exact sequence. 0 p X 0. Let = {s p X s X C }. The key point is that defining this as a subsheaf of p X on X C makes it coherent and flat. Exercise 2.2.10. Prove the flatness. Example 2.2.11. Here is an example of what can go wrong if you extend too naïvely. If t = 0, then you could take C C k O 0. This is not a flat extension. We have finally established (modulo some details): Theorem 2.2.12. Hilb P X is represented by a projective scheme. 3. MODULI OF SHEAVES VIA GIT 3.1. Stable sheaves. Suppose (X, H) is a projective scheme. Let V be a holomorphic vector bundle on X. Then its sheaf of sections X (V ) is locally free. This construction induces an equivalence {Vector Bundles/X } = {locally free X -modules}. If ι : Z X and is locally free over Z, then ι is a sheaf of X -modules. We have X / Z = ι Z. Dimension of a sheaf. We want to quantify a notion of dimension for sheaves. Example 3.1.1. X / Z is supported on Z X, and so should have the same dimension as Z. Definition 3.1.2. We say that is supported on Z if for all f I Z X, multiplication by f on is 0. f 0 Example 3.1.3. If = k [x, y ]/(x, y 2 ) then is supported on V (x, y 2 ), which has dimension 0. Definition 3.1.4. Let be a sheaf and s a non-zero section. Then we define dim s = dim Im( X s } = dim supp(s). Example 3.1.5. Let X be a smooth variety and a sheaf. The torsion subsheaf is tor := {s dim s < dim X }. The quotient / tor is torsion-free.

12 LECTURES BY JUN LI, NOTES BY TONY FENG Definition 3.1.6. A sheaf is of pure dimension r if for any non-zero s, we have dim s = r. Definition 3.1.7. If has pure dimension r, then its Poincaré polynomial is P (n) := χ( H n ) = a r n r +... which has degree r. We also define the monic normalization Example 3.1.8. If dim X = 1, then by Riemann-Roch, and p (n) := P (n) a r = n r +.... P (n) = n(rank)deg H + χ() p (n) = n + χ() 1 rank deg H. We can finally give the definition of stable sheaves. Definition 3.1.9 (Simpson). A /X is stable if and only if it is of pure dimension r and for any subsheaf, we have p (n) p (n) i.e. p (n) < p (n) for sufficiently large n. Proposition 3.1.10. (i) The collection of all stable sheaves which are pure of fixed Poincaré polynomial P is bounded. (ii) If is stable, then Hom(, ) =. Proof. We give only the proof of (ii). Suppose you have α with non-zero. (Replacing α be α c Id, we can ensure that the map is not surjective.) Then there is a kernel, which fits into the short exact sequence 0 α 0. Since the Hilbert polynomials are equal, we must have non-zero. Since is stable, p p. Let be the cokernel of, which fits into 0 0 Then by the definition of stability we also have p p, P + P = P. It is an exercise to show that these equations are incompatible. A consequence of the first part is: Corollary 3.1.11. There is a uniform µ 0 such that for as in the hypotheses of the proposition, H 0 ((µ)) X (µ) for all µ µ 0.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 13 We want to make a moduli space of sheaves. You always run into trouble when making a moduli space of objects with jumping automorphisms. For stable bundles we at least understand the automorphisms, which are as small as possible. 3.2. Moduli of stable sheaves. We want to construct a moduli space for sheaves with P = P for a fixed polynomial P. Pick a large µ such that H 0 ((µ)) X (µ) for all stable. Then can try to construct a moduli space of stable sheaves by embedding it in a quot scheme via [] [H 0 ((µ)) X (µ)]. (3.2.1) Here the Quot scheme in question is Quot P(µ) X (). For N = P(µ), let N X Q be the tautological surjection onto the universal bundle, corresponding to the map Gr Gr. The map (1) depends on a choice of isomorphism H 0 ( (µ)) = P(µ). The ambiguity in the choice of basis is measured by GL(N ). In fact we can shrink the automorphisms to SL(N ). Therefore, the map takes points to SL N -orbits: one to SL(N ) orbit { stable, P = P} Q. The action of SL n on [ N ] is by pre-composition. The image of ϕ inside Q is an SL(N )-invariant open subscheme. The idea is to define a moduli space of stable sheaves by taking the quotient with respect to the SL(N )-action. 3.3. Geometric invariant theory. We re going to discuss a simple version of GIT for G = SL(n,). Let V be a G -representation over, which we can think of in terms of a map G GL(V ). Then G acts on (V ) = (V 0)/. The goal is to construct a quotient scheme V /G, which is morally the space of G -orbits. Here is the key observation. We have an embedding V H 0 ( (l)) for l > 0, and action of G on H 0 ( V (l)). The invariant functions on this space are H 0 ( V (l)) ) G. If l is large, then H 0 ( V (l)) becomes large and we might hope to find enough G -invariant functions to find an embedding. (H 0 ( V (l)) ) G V H 0 ( V (l)) The crux of the matter is then to study G -invariant sections in H 0 ( (l)). Definition 3.3.1. Let v V 0 represent [v ] V. We say that (1) [v ] is semistable if G v does not contain 0, (2) [v ] is stable if G v is closed in V and Stab G (v ) <. (3) [v ] is weakly stable if G v is closed.

14 LECTURES BY JUN LI, NOTES BY TONY FENG Lemma 3.3.2. Let X 1, X 2 V be two disjoint G -invariant closed subsets. Then there is s (Sym V ) G such that s X1 = 0 and s X2 = 1. Proof. Let I (X 1 ) be the ideal of polynomials vanishing on X 1 and define I (X 2 ) similarly. We know that I (X 1 ) + I (X 2 ) contains the element 1, so we can find 1 = f 1 + f 2 (3.3.1) with f 1 I 1 and f 2 I 2, which implies that f 1 X1 = 0 and f 1 X2 = 1, and f 2 X2 = 0 and f 2 X1 = 1. Then we apply a Reynolds operator R : {polynomials} {G -invariant polynomials}. At least over characteristic 0, this exists for any reductive group. (For reductive groups, GIT always amounts to careful applications of the Reynolds operators.) Definition 3.3.3. A Reynolds operator is a G -equivariant map R : W i W G i (1) functorial: W 1 φ W 2 which is R W1 G (2) R(αf ) = αr(f ) for G -invariant α. R W G 2 What is the construction of the Reynolds operator? You take the maximal compact subgroup K of G, whose tangent space complexifies to g. Then there is a Haar measure µ on K, and you can apply Weyl s trick to: R(f )(x ) = K f (g x)d g. This is obviously K -invariant. It follows that it is even G -invariant, since this is equialent to being g-invariant, and g is the complexification of Lie(K ). Applying this to (2) above, we obtain R(1) = R(f 1 ) + R(f 2 ) where R(f 1 ) X1 = 0 and R(f 1 ) X2 = 1, and R(f 2 ) X2 = 0 and R(f 2 ) X1 = 1. Lemma 3.3.4. Let v V be semistable. Then there exists l and an s H 0 ( (l)) G such that s (v ) 0. Let V l = H 0 ( (l)) be the space of homogeneous polynomials of degreen l on V. Proof. Take X 1 = G v and X 2 = {0}. By the definition of semistability, they are disjoint. Apply the preceding lemma to find s Sym(V ) G such that s G v = 1 and s (0) = 0. Write s = f 1 + f 2 +... + f l +... with f l V G l. So there exists l such that f l(v ) 0.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 15 Let V s s be the set of semistable points in V {0} and U s s be the image of V s s in V. Then we know that for all [v ] U s s, there exists f V G l with f (v ) 0. Lemma 3.3.5. There exists l 0 such that when l 0 divides non-zero l, for any v V s s there exists s V G l such that s (v ) 0. Proof. Take R = l>0 H 0 ( (l)) = V l. Then R G = l>0 V G l. It suffices to show that R is finitely generated as an algebra over. Let S + be the subset of G -invariant homogeneous polynomials of positive degree. We consider S + Sym(V ), the ideal generated by G -invariant polynomials of positive degree. It is finitely generated, say be f 1 g 1,..., f n g n where the f i are homogeneous and G -invariant. We claim that f 1,..., f N generate R G. To see this, consider h R G with vanishing constant term. Then h S + [V ], and we can write N h = f i (g i h i ). i =1 We can assume that deg(g i h i ) < deg h. Using that f i are homogeneous of positive degree, we have N h = R(h) = f i R(g i h i ) where the R(g i h i ) are G -invariant. The result follows by induction. Now we have constructed U s s i =1 Φ l (H 0 ( V (l)) ) G. We claim that the maximal domain of Φ l is U s s. Suppose [v ] U s s and suppose that Φ l can be extended to [v ]. Therefore, G v s 1 l (1) for some s l (H 0 ( V (l)) ) G, since we can find a homogeneous polynomial taking value 1 on the image of v, and then by invariance it takes that value everywhere. But 0 is in the closure of the orbit by definition of not being semistable. The morphism Φ l : U s s (H 0 ( V (l)) ) G factors through Proj R G, where R G = d 0 H 0 ( (l)) G. Φ l U s s (H 0 ( V (l)) ) G Proj R G An important issue is how we know that U s s U s s //G is surjective. This had better be a property of a quotient map. It is easy to show that its image is dense, so we should check the valuative criterion for properness. This cannot be checked at the level of U s s, since that is not proper! A sequence of points can go off to in the G -direction, so e somehow need to use the G -action to renormalize it.

16 LECTURES BY JUN LI, NOTES BY TONY FENG Proposition 3.3.6. We have (1) Φ l is surjective. (2) Φ l induces a bijection from weakly stable G -orbits to closed points in Proj R G. Proof. We need to quote the semistable replacement theorem, which says: Let p C be a pointed smooth curve (C is really the spectrum of a DVR). Let φ : C p U s s be a morphism. Then there exists a finite base change f : C C and σ: C p G such that extends to C U s s. (φ f ) σ = σ(x) φ(x): C p U s s 3.4. The numerical criterion. There is an effective method to characterize (semi)stable points. Again, we re going to study the special case G = SL n. Definition 3.4.1. A one-parameter subgroup is a map λ: G. There is a basis w 1,..., w n of V such that λ can be diagonalized as w λ(t ) i r 1 r 2... r n and r i = 0. Theorem 3.4.2 (Numerical criterion). For all v V \ 0, (1) v is stable if and only if for all one-parameter subgroups λ, limλ(t ) v =. t 0 (2) v is semistable if and only if for all λ, limλ(t ) v 0. t 0 = t r i w i for The point is that lim t 0 λ(t ) v is in V and not in G v if it exists. So if G v is closed (as it must be in the stable case), then this limit cannot exist. In the semistable case, it must not be 0. Proof. Suppose that v is not semistable. Then G v contains 0. We re going to unravel the valuative criterion for properness to see that there is a oneparameter subgroup with limit 0. The point is that if 0 is the limit of some curve, then it is the limit of the orbit of a one-parameter subgroup. For R a DVR with fraction field K, and t a uniformizing parameter, we consider diagrams Spec R G v Spec K Assume that Spec K G v is represented by ρ : Spec K G, say ρ(t ) = a i j (t ) SL(N, K ). G v

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 17 Then there exist A 1, A 2 SL(R) such that t a 1c 1 (t ) A 1 ρa 1 2 =... t a N cn (t ) with the c i (t ) R. We can then further modify to assume that all the c i are 1. Suppose limρ(t )v = 0. Since ρ(t )V = ρ(t )A 1 2 A 2v, if we pick λ to be the one-parameter subgroup λ = A 2 ρ(t )A 1 2 (note that there is no A 1 in the definition) then limλ(t )v = 0. t 0 Apply the numerical criterion to [ n V φ P] Gr( n V, m ). For λ a one-parameter subgroup in SL(n,) we can find a basis e 1,..., e n of n such that e λ(t ) i = t a i e i for a 1... a n. We want to show that lim t 0 ξ λ(t ) 0. The homogeneous coordinates of ξ in terms of a basis w 1,..., w l of V are [φ(e i 1 ω j1 ) φ(e i 2 ω j2 )... φ(e i d ω jd )] λ(t ) 0 d P =. The one-parameter subgroup scales this by t a 1+...+a d. We need to verify that for all 1- parameter subgroups λ, we have ξ λ (t ) 0. So we need to find bases e i 1,..., e i d and ω j1,...,ω jd such that φ(e i 1 ω j1 ) φ(e i 2 ω j2 )... φ(e i d ω jd ) 0 and a i 1 +... + a i d 0. If you think about this problem, you ll realize that it the key is to consider the subgroups Im φ(span{e 1,..., e i } V ) P. Theorem 3.4.3 (Mumford). Let λ be a one-parameter subgroup with e 1,..., e n a choice of diagonalizing basis. Let E i = Span(e 1,..., e i ). Let H i = φ(e i V ) P. Then if and only if for any k < n, limξ λ(t ) = t 0 k < m dim H k dim P. 3.5. Application to moduli of stable sheaves. Let X be a vector bundle of rank 2 on a smooth curve. Given [H 0 ((µ)) X ], we get H 0 ((µ)) X (k ) (k ) Taking global sections, we get a map of vector spaces [H 0 ((µ)) H 0 ( X (k )) H 0 ( (k + µ))] such that the SL(N ) orbit of [ n V P] is well-defined. So we want to make sense of an embedding to the quotient of Gr by SL(N ). To do this, we need to check that the point [ n V P] of Gr is SL(n)-stable. Mumford s criterion shows that this is the case

18 LECTURES BY JUN LI, NOTES BY TONY FENG provided that: for any basis e 1,..., e n H 0 ((µ)) = n and E i = Span(e 1,..., e i ), H i = φ(e i V ), we have for any 1 k < m k < n dim H k dim P. Okay, so fix e 1,..., e n and k [1, n 1]. We need to estimate h i := dimφ(span(e 1,..., e i ) V ). Let i (µ) = Span(e 1,..., e i ) Γ((µ)). The point is that Im Φ({e 1,..., e k } V ) H 0 ( i (k + µ)) H 0 ((k + µ)). Since H i H 0 ( (k + µ)), Riemann-Roch shows There are two cases. h i (Riemann-Roch number of i (k + µ)) + h 1 ( i (k + µ)). (1) If h 1 ( i (µ)) = 0, then we get that i h i is at most χ( i (µ)), so if d = deg X (1) then i χ( i (µ)) dim H i dim H i = χ( i (µ)) χ( i (µ + k )) When is stable we have deg i rank i = µd + deg i + rank i (1 g ) d (µ + k ) + deg i + rank i (1 g ). < deg, and you can check that this gives the rank correct inequality. We have used that for k 0, dim H i = h 0 ( i (µ + k )). (2) If H 1 ( i (µ)) 0 and rank = 2, then this computation requires some H 1 inserted in. The essential case is rank i = 1. This is only possible if the degree is negative. The point is that this makes a huge gap in the inequality deg i < deg rank i rank. The point is that rank one subsheaves are not boundd, but the subsheaves of degree bounded from below is a bounded set. Theorem 3.5.1. If is stable, then for k µ 0, H 0 ((µ)) H 0 ( X (k )) H 0 ( (k + µ)) n V P is SL(n)-stable in the Quot scheme.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 19 3.6. Coarse moduli spaces. Definition 3.6.1. A scheme M is a coarse moduli space of : Sch Set if there is a transformation Hom(, M ) such that (1) Hom(k, M ) = (k ) for algebraically closed k, (2) For all N and Hom(, N ) there is a unique ρ : M N such that the diagram commutes. Hom(, M )! Hom(, N ) The uniqueness of the coarse moduli space is clear from the fact that it possesses a universal property. The point is that in constructing moduli spaces, you sometimes have to shrink into order to get something separated. So M is the space involving the least shrinkage. Theorem 3.6.2. The moduli of stable sheaves has a coarse moduli space. Proof. For us is the moduli functor of stable sheaves, so a point [] (k ) is the data of [ n = H 0 ( (µ)) X (µ)]. We can view this as an SL(N )-orbit in Quot P n X is to construct the map (Quot P ) n s s //G = Proj X which is moreover stable. The main issue H 0 ( Quot (k )) G. For all schemes S, and ξ := [ X S] (S) we need to construct a functorial map φ ξ : S (Quot P ) n s //G. X Covering S by affines S α, we get π S α π Sα α (µ) α (µ). A choice of framing π S α π Sα α (µ) = n X S α defines a point in Quot(S). Different choices of frames will lead to different points of the Quot scheme, but they differ by an action of G. Therefore, the composition map to the GIT quotient is well-defined: S α Quot P n X k Quot P //G n X This allows us to define the map on the level of the S α, and then patch them. s s

20 LECTURES BY JUN LI, NOTES BY TONY FENG In general, the stable locus is open in the semistable locus, which is compact. We define M X (c i ) = (Quot P ) n s //G to be the moduli of stable sheaves with fixed Poincaré X polynomial (Chern class), and M X (c i ) = (Quot P n X ) s s //G to be its compactification. The map Quot s s Quot s s //G is a good quotient, meaning that Quot s s has a G - invariant affine cover by G -invariant open affines U, so that U//G can simply be defined by Spec Γ( U ) G. For ξ in the stable locus Quot s //G, the pre-image is a closed orbit. This fails in for points which are semistable but not stable. For that reason, M X (c i ) is the moduli of S-quivariant classes of semistable sheaves. In terms of the Harder- Narasimhan filtration, there exists a unique filtration 0 = 0 1... n = such that k +1 / k is stable and p k +1 / k is decreasing. If is semistable, then there exists a filtration 0 = 0 1... n = such that k +1 / k is stable and p k +1 / k = p. Then Gr() := k +1 / k is unique, and we say that Gr( ) = Gr( ). 3.7. Moduli of stable maps. Let X be a projective scheme (smooth). We define where the condition ( ) means that M g,n (X, d ) = {f : (C, p 1,..., p n ) X : ( )}/ (1) f is a morphism such that deg f ( X (1)) = d (or more generally, that f [C ] = α H 2 (X,)). (2) (C, p 1,..., p n ) is a pointed nodal projective curve of genus g. (3) # Aut( f ) <, where automorphisms α are commutative diagams (C, p 1,..., p n ) α = X (C, p 1,..., p n ) Example 3.7.1. Consider the case n = 0. Then for [f : C X ] M g (X, d )(k ), the condition that f is stable is equivalent to := ω C f X (1) being ample. For such an f, we have for µ 0 ι µ : C H 0 ( µ ) and thus (ι µ, f ): H 0 ( µ ) X.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 21 Upon choosing a frame for H 0 ( µ ), we have H 0 ( µ ) = N. Then we can consider Hilb P N X ρ : C N X : C nodal, genus g ρ p1 N (1) =ω C p2 X (1) C p 2 ρ : C X degree d =: W. This is defined up to a choice of frame, which should be cancelled out by the action of SL(N + 1). So we would like, as before, to set M g (X, d ) := [W /G = SL(N + 1)]. The problem is that, unlike the situation with sheaves, it is difficult to find a G -invariant affine open cover. What works is to pass to an étale cover of [W /G = SL(N + 1)], which gets into the territory of Deligne-Mumford stacks. Part 2. Deformation and obstruction theory 4.1. Initial examples. 4. OBSTRUCTION THEORY Example 4.1.1. Here s a toy example. Consider the equations f 1 = z 2 f 2 = 2z 2 + z 3 f 3 = z Let X i := (f i = 0) 1. We can detect the local differences in the X i abou 0 by mapping in from thickened neighborhoods. Consider first mapping in from first-order neighborhoods : Spec k [t ]/t 2 Spec k Let β 1 : z t be a map Spec k [t ]/t 2 1. Then β 1 (f 1) = β 1 (f 2) = 0, while β 1 (f 3) 0. Therefore, β 1 factors through X 1 and X 2 but not X 3. These β (f i ) are preliminary versions of obstruction classes. We can continue the story to third order. Let β 2 be the lift z t. Spec k [t ]/t 3 β 1 X 1 Spec k [t ]/t 2 β 1 β 2 Spec k X 1 Then β 2 (f 1) = t 2 β 2 (f 2) = 2t 2, which tells us that X 1 and X 2 look different to third order.

22 LECTURES BY JUN LI, NOTES BY TONY FENG Example 4.1.2. Now here s a slightly more complicated example. Let f = (z 3 1, z 2) Γ( 2 2 ). We let X := (f = 0) 2. Consider lifting to first order: Spec k [t ]/t 2 Spec k β 1 X 2 Suppose β 1 is defined by z 1 t and z 2 0. Then β1 (f ) = 0, i.e. the obstruction vanishes and we can extend. What about the next level? Spec k [t ]/t 3 Spec k [t ]/t 2 β 1 β 2 Spec k X 2 Suppose β 2 is defined by z 1 t and z 2 t 2. Then β 2 is an extension of β 1. But β 2 (f ) 2 (t 2 ) is non-zero so β 2 doesn t factor through X. An obstruction class should vanish if and only if some extension is possible, and indeed one extension in this case is z 1 t + t 2 and z 2 0. The natural home for the obstruction class is not simply the ideal sheaf. In our example, cnosider the complex T O 2 d f 2 2 O. Suppose we tensor with the ideal sheaf (t 2 ). Then we get T O 2 (t 2 ) d f 2 2 O (t 2 ) (t 2 ) β 2 (f ) 0 0 The upshot is that it is not really the ideal that witnesses the obstruction, but the complex above. 4.2. The obstruction class. Given an S-scheme X S, suppose we can embed X into a smooth W S: X W S Then we can take J = I X W and L 1 X /S := [J /J 2 Ω W /S X ] D b ( X ). Note that H 0 (L 1 X /S ) = Ω X /S (from the usual exact sequence of differentials). Proposition 4.2.1. The complex L 1 X /S S is well-defined.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 23 Proof. We need to prove independence of the embedding. Suppose we have two such X W 1, X W 2. We can put them together: W 1 X W 1 W 2 W 2 which induces a map of complexes [J 1 /J 2 1 Ω W1 /S X ] [J 12 /J 2 12 Ω W1 S W 2 /S X ]. One can then check that this is a quasi-isomorphism, using Ω W1 S W 2 /S X = ΩW1 /S Ω W2 /S. We want to use this to handle intrinsic obstruction classes and lifting problems. For instance, we might want to analyze formal smoothness of X at p X. For (A,m) a local Artin k -algebra and I an ideal with mi = 0, the formal smoothness of X S is concerned with solving a lifting problem Spec A/I ϕ A/I X Spec A ϕ A S The idea is that Spec A is a little thickening of Spec A/I. Theorem 4.2.2. There is a canonical obstruction class such that ob(ϕ A/I, A, I ) Ext 1 A/I (L 1 X /S, I ) (1) ob(ϕ A/I, A, I ) = 0 if and only if ϕ A exists, (2) if ϕ A exists then the set of all ϕ A is an Ext 0 A/I (L 1 X /S, I )-torsor.

24 LECTURES BY JUN LI, NOTES BY TONY FENG Proof. How to construct this class? Embed X in a smooth W /S. Then we have some lift ϕ to W. Spec A/I ϕ A/I X W ϕ Spec A S S This induces ϕ : W A and we know that J = I X /W maps to I, so J 2 I 2 = 0. So ϕ descends to a map J /J 2 ϕ I. This is part of a map ε of elements of D b ( X ): deg 1 0 I [1] := [I 0] ε( 1) L 1 X /S := [J /J 2 δ ε(0) Ω W /S X ] The map ε can be interpreted as an element in Hom D b ( X ) (L 1 X /S, I [1]) = Ext1 (L 1 X /S, I ). We define this to be the obstruction class ob(ϕ A/I, A, I ) (although we have not yet shown that it is well-defined). Let s ignore the dependence on the embedding. What happens if we choose a different lift ϕ : Spec A W? Then we get a different class ε Ext 1 (L 1 X /S, I ). We claim that there exists η: Ω W /S X I such that ε ε = η δ. This will show that the two maps are homotopic in the derived category. The two lifts ϕ and ϕ coincide on the subscheme Spec A/I. That means that the difference should be in the cotangent sheaf. More precisely, Grothendieck s definition of conormal sheaf Ω W is as follows: for ι : W W W the diagonal embedding, we set Ω W := ι (I /I 2 ). In our case, the two lifts put together give a map Φ: Spec A ϕ,ϕ W W which agree on Spec A/I, so the pullback map W S W A sends I to I. Since I 2 I 2 = 0, this descends to I /I 2 = Ω W /S I. This defines η. Now we prove (1). First suppose that ϕ A exists. Then we can pick ϕ = ϕ A. Then J = 0, so trivially ε = 0.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 25 Conversely, suppose that ε = 0. So Ext 1 (L s 1 X /S, I ) = Hom D b (L 1 X /S, I [1]) = H 0 ((L 1 X /S ) I [1]) = H 1 ((L 1 X /S ) I ) To compute (L 1 X /S ) in the derived category, we need to take a resolution for L 1 X /S : [J /J 2 ] Ω W /S X ] [... 2 1 Ω W /S X ] Then (L 1 X /S ) = [ 0 1 2...] and we have a class ε in the first cohomology of 0 I 1 I... The fact that [ε] = 0 H 1 ( I ), representing the homomorphism ε: 1 J /J 2 I, tells us that ε Im(d 1 ), i.e. is induced from some η Hom(Ω W /S X, I ). Using this η, we get a map ϕ : Spec A W factoring through X. We skip (2). 4.3. Perfect obstruction theories. Definition 4.3.1. We define the property ( ) as follows: D( X ) has property (*) if (1) H i >0 (F ) = 0, (2) H 1 (F ) and H 0 (F ) are coherent sheaves. If F satisfies ( ), we furthermore say that F is perfect if locally (in an appropriate topology) F = [ a a +1... b ] are locally free sheaves of X -modules. If F is perfect, we furthermore say that F is perfect of perfect amplitude [ 1, 0] if F = [ 1 0 ]. Definition 4.3.2. A perfect (relative) obstruction theory of X /S consists of such that φ : E L 1 X /S D( X ) (1) E is perfect of perfect amplitude contained in [ 1, 0], (2) H 0 (φ) is an isomorphism and H 1 (φ) is surjective. Remark 4.3.3. A general obstruction theory replaces perfect with ( ).

26 LECTURES BY JUN LI, NOTES BY TONY FENG What is an obstruction theory intuitively? It arises from studying a lifting problem Spec A/I ϕ A/I X Spec A ϕ A S Obstruction theory defines an obstruction class ob(ϕ A/I, A, I ; E ) Ext 1 (E, I ) such that (1) ϕ A exists if and only if ob(ϕ A/I, A, I ; E ) = 0, and (2) if ϕ A exists then the set of such is an Ext 0 (E, I )-torsor. Given φ : E L 1 X /S, we define ob(ϕ A/I, A, I ; E ) by the diagram ob(ϕ A/I, A, I ) Ext 1 (L 1 X /S, I ) ob(ϕ A/I, A, I ; E ) Ext 1 (E, I ). We claim that (2) implies that Ext 0 (L 1 X /S, I ) = Ext 0 (E, I ). The injectivity follows from the fact that the thing is an isomorphism on H 0. Also (2) implies that the map Ext 1 (φ, L) is injective, because H 1 is surjective. We re skipping the proofs. 5. CONSTRUCTION OF VIRTUAL CYCLES 5.1. The normal cone. Suppose we have a vector bundle E W of rank m over a smooth variety of dimension n. Suppose X W is a subvariety defined by s = 0, where s is a section of W. Then (1) dim x X n m for all x X. (2) If dim X = n m, then [X ] = c top (E ) A n m W. (3) Even if dim X > n m, we can define c top (E, s ) A n m X and we set [X ] vir := c top (E, s ) A n m X viewing n m is the expected dimension of X. MacPherson s deformation to the normal cone. This c top (E, s ) is defined via To explain this, consider the section t 1 s : W E as t 0. More precisely, consider the limiting behavior of the graph Γ t 1 s. Then limγ t 1 s =: N X /W E X. t 0 Explicitly N X /W is the fiber over 0 1 of Γ t 1 s E 1. Then we define t 0 c top (E, s ) := 0! E X [N X /W ] A X. The point is that in general, you can take a cycle in the vector bundle and move it by rational equivalence to something vertical, and then intersect with the 0 section.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 27 Let J := I X W. Then we have a complex [J /J 2 Ω W X ] D b (X ). This admits a map from [ d s X Ω W X ], which is the identity on Ω W X and the map. We claim that this map in the derived category L 1 X /S := [J /J 2 Ω W X ] is a perfect obstruction theory. E := [ X d s Ω W X ] Note that another way of describing the normal cone is as C X /W = Spec X J n /J n+1. n 0 There is an obvious surjection Sym (J /J 2 ) J n /J n+1 n 0 inducing a closed embedding C X /W Spec X Sym (J /J 2 ). If I have an obstruction theory given by = [ 1 0 ] φ L 1 X /S 1 J /J 2 then we get an embedding 0 Ω W X Spec X Sym (J /J 2 ) Spec X Sym 1 =: E 1 This depends on X W and φ. We want to get rid of these dependencies. Go back to the definition of L 1 X /S. We have Spec X Sym (J /J 2 ) Spec X Sym (Ω W /S X ) = T W /S X. We can think of T W /S X as a group over X, which acts on Sym (J /J 2 ). So we can take the quotient as an Artin stack over S, which fits into maps Definition 5.1.1. We define [C X /W /T W /S X ] [Spec Sym(J /J 2 )/T W /S X ] [E 1 /T W /S X ]. (5.1.1) [E 1 /T W /S X ] = h 1 /h 0 ((E ) ). It depends only on E. We define [Spec Sym(J /J 2 )/T W /S X ] to be the intrinsic normal sheaf X /S.

28 LECTURES BY JUN LI, NOTES BY TONY FENG Finally, we define [C X /W /T W /S X ] to be the intrinsic normal cone X /S. So diagram (3) expresses inclusions cone stack into abelian cone stack into vector bundle stack. Suppose D( X ) is of class ( ), i.e. h >0 ( ) = 0 and h 1 ( ) and h 0 ( ) are coherent. What is h 1 /h 0 (( ) )? Take a resolution where the i are locally free. Then and we get 0 ker d 1 1. Definition 5.1.2. We define = [... 2 1 0 ] ( ) = [ 0 1...] h 1 /h 0 ( ) = [Spec X Sym ker d 1 /F 0 ]. The point is that X /S is canonically isomorphic to h1 /h 0 ((L 1 X /S ) ). If E is perfect with perfect amplitude, then h 1 /h 0 (E ) = [E 1 /E 0 ]. Lemma 5.1.3. The stack h 1 /h 0 ( ) is well-defined and has property ( ). 5.2. Crash course on derived categories. We write Sh( X ) for the category of sheaves of X -modules. This is an abelian category, meaning that there the familiar operations of kernels, cokernels, direct sums. From Sh( X ) we can construct C ( X ), the category of complexes valued in Sh( X ), whose objects are complexes of X -modules d i d i +1... i i +1 i +2... and morphisms are morphisms of complexes. Shifts. Given a complex deg 1 0 we can form a shifted complex E = [ 1 0 ] deg 2 1 0 E [1] = [ 1 0 ] Definition 5.2.1. A triangulated category is category (plus some extra structure) equipped with a collection of distinguished triangles A B C +1 A [1]

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 29 satisfying certain axioms. (For the details, we recommend Kashiwara s Derived categories and sheaves.) The most important one is that one gets a long exact sequence in cohomology... H i (A ) H i (B ) H i (C ) H i +1 (A )... Intuitively, these play the role of short exact sequences of sheaves. Example 5.2.2. For f : A B in C ( X ), we get a distinguished triangle A B MC (f ) +1 A [1] The mapping cone MC (f ) is the total complex formed from the double complex... A i 1 A i A i +1... as (check your sign connections).... B i 1 B i B i +1...... A i B i 1 A i +1 B i... Definition 5.2.3. We can form the homotopy category K ( X ), whose objects are the same as those of C ( X ) but morphisms are quotiented out by the equivalence relation of homotopy: f 1 f 2 : A B if there exists η: A i +1 B i such that f 1 f 2 = η δ + δ η. A i δ A i +1 η B i δ B i +1 Note that this implies H (f 1 ) = H (f 2 ): H (A ) H (B ). Definition 5.2.4. Let N K ( X ) be the objects A with H(A ) = 0. The derived category D( X ) is K ( X )/N. Given A D( X ) and B D( X ), a morphism ϕ : A B in D( X ) is the same as a diagram quasi-isom E quasi-isom A B (A quasi-isomorphism is a map f : A B such that H i (f ): H i (A) = H i (B) is an isomorphism.)

30 LECTURES BY JUN LI, NOTES BY TONY FENG Given E = [E 1 E 0 ] D( X ) with E i locally free, we can form the stack h 1 /h 0 (E ) = [E 1 /E 0 ]. We claim that this is well-defined as an Artin stack, i.e. independent of the presentation. Proof. We work locally. If E = [E 1 E 0 ] = [E 1 E 0 ], then by definition we can dominate both complexes by quasi-isomorphisms quasi-isom [F 1 F 0 ] quasi-isom [E 1 E 1 ] [E 1 E 0 ] We claim that F 1 can be chosen to be locally free. First we make F 1 E 1 surjective. We just take the direct sum with something that surjects onto the cokernel of F 1 E 1 : A F 1 A F 0 E 1 E 0 Similarly we can also make F 0 E 0 surjective. So we have reduced to assuming that both maps F 1 E 1 and F 0 E 0 are surjective. F 1 F 0 E 1 E 0 Since we are working locally, we can split these surjections. So we can assume F 1 = A 1 E 1 and F 0 = A 0 E 0, with locally free kernel B 0 E 0. B 0 E 0 F 1 A 0 B 0 E 1 E 0 Then we replace F 1 with the fibered product. This shows that h 1 /h 0 ([E 1 E 0 ] ) = h 1 /h 0 ([F 1 F 0 ] ). Consider C X h 1 /h 0 (L 1 X ). What is the geometry of C X x for x X?

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 31 Example 5.2.5. Suppose that X has surface singularities. Suppose it is inside a threedimensional W. Suppose X W 0 W. Then C X /W x = CX /W0 x N W0 /W x. There is an action of T W0 X on C X /W0 x and of T W x on C X /W0 x, and the quotients are canonically identified: CX /W X CX /W0 =. T W X T W0 X 5.3. The cotangent complex. Let A B be a map of rings. View this as a map of schemes Spec B X Spec A We construct a canonical resolution P B as follows: S... A[A[B]] A[B] ε B Elements of A[B] are a n 1...n k i 1...i k [α i 1 ] n 1...[α i k ] n i k. The map ε is by deleting the brackets. Elements of A[A[B]] are sums of things with two layers of brackets. The two maps are determined by which layer of bracket to delete. This forms a simplicial complex, Ω P /A. Definition 5.3.1. We define the cotangent complex to be the resolution L X :=... Ω Pk /A Pk,ε B Ω Pk 1 /A Pk 1,ε B... Theorem 5.3.2. Suppose that π: X S is a relative DM morphism. Then there exists L X /S D( X ) such that Also (1) H >0 (L X /S ) = 0, (2) H 1 (L X /S ) and H 0 (L X /S ) are coherent, (3) H 0 (L X /S ) = Ω X /S. (4) L 1 X /S is the truncated cotangent complex. is distinguished triangle. π L S L X L X /S +1

32 LECTURES BY JUN LI, NOTES BY TONY FENG 5.4. Virtual cycles. Suppose X S is a DM stack. We can form L X /S. Suppose we have an étale covering U S. We can find an S-embedding U W into some smooth W S. Then we can form the intrinsic normal cone C intr U = [C U/W /T W U ] h 1 /h 0 (L 1 U/S ). Such a collection (by the canonical nature of the intrinsic normal cone C intr ) descends to C intr X /S h1 /h 0 (L 1 X /S ) = h 1 /h 0 (L X /S ). For what follows, it is useful to recall the construction of the top chern class earlier, for X W cut out by a section s of a vector bundle E. Assume that there is a perfect obstruction theory E φ X L X /S. Then we have an embedding h 1 /h 0 (L 1 X /S ) h 1 /h 0 (E ). Suppose E [E 1 E 0 ] C ( X ), where E i are locally free. Then we get The map [C intr X /S ] h1 /h 0 (E ) = [E 1 /E 0 ]. φ : E 1 h1 /h 0 (E ) is flat, so we have a flat pullback φ [C intr X /S ] A (E1 ). Then we define [X ] intr φ X = 0! [ φ [C intr E1 X /S ]] A rank E 1 rank E 0 (X ). Example 5.4.1. Let Y be a smooth projective scheme. Consider X = M g,n (Y,α) We also have a universal curve over X : S = M g,n X π f Y Theorem 5.4.2 (Illusie). There is an obstruction theory (Rπ f T Y ) L X /S. Lemma 5.4.3. Rπ f T Y is quasi-isomorphic to [ 0 1 ] D( X ) with 0, 1 locally free.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 33 6. DEFORMATION THEORY 6.1. Classical approach to obstruction theory. Let be a locally free sheaf on X. Def() is the space of 1st-order deformations of. ob() ist he space of obstructions to deforming. A first-order deformation looks like an extension from A/I on X Spec A/I to some on X Spec A, where mi = 0. 0 A X X Spec A/I X Spec A Definition 6.1.1. We define the obstruction space Ob( 0 ) = H 2 (X, 0 0 ) = Ext2 ( 0, 0 ). Theorem 6.1.2. There is an obstruction class ob( A/I, AI ) Ob( 0 ) k I such that an extension exists if and only if ob( A/I, AI ) vanishes. Furthermore, if Pic X is smooth then we can take Ob( 0 ) = Ext 2 ( 0, 0 ) 0 (corresponding to the traceless subsheaf). Proof. We cover X by open affines X α trivializing α := Xα Spec A/I. α X Spec A/I X α Spec A/I We begin by choosing arbitrary extensions of α to α over X α Spec A. We also choose arbitrary extensions of the transition functions f βα : α β, which satisfy the cocycle condition, to f βα H 0 (X αβ Spec A, α β ). We know that f αβ f γβ f βα H 0 (X αβγ Spec A, α α ) is the identity modulo I, so f αβ f γβ f βα H 0 (X αβγ, α α I ). But H 0 (X αβγ, α α I ) = H 0 (X αβγ, Hom( α, α ) A I ). Using that I 2 = 0, you check that δ f is a closed 2-cycle in C 2 ({X α }, Hom(, ) A I ). We define ob( A/I, A, I ) := [δ f ] H 2 (X, Hom(, )) I. Exercise 6.1.3. Prove that ob( A/I, A, I ) = 0 if and only if the choices f αβ can be modified to satisfy the cocycle condition. Furthermore, if Pic(X ) is smooth then ob( A/I, A, I ) ker(ext 2 ( 0, 0 ) I tr H 2 (X, X ) I )

34 LECTURES BY JUN LI, NOTES BY TONY FENG Why? The formation of the obstruction class is compatible with taking determinants, so we have a map Ext 2 trace X A/I (,) I Ext 2 X A/I (det, det) I ob( A/I, A, I ) ob(det A/I, A, I ) The smoothness of Pic(X ) implies that there are no obstructions to deforming line bundles, so that Ext 2 X A/I (det, det ) I = 0. Remark 6.1.4. More generally, X is smooth then one can still form a determinant in the Grothendieck group by taking a resolution of by vector bundles and defining det = (det i ) ( 1)i. 0 n... 0 0 Remark 6.1.5. The same sort of argument shows that Def() = H 1 (X, End()). Theorem 6.1.6. With choices of identification Def( 0 ) = n and Ob( 0 ) Ext 2 X ( 0, 0 ) = m, there exists f [[z 1,..., z n ]] m such that (f = 0) is the germ of the moduli of sheaves at [ 0 ]. Locally the tangent space to the moduli space looks like Def( 0 ), and the theorem is telling us that the moduli space itself is cut out by equation coming from the obstruction space. 6.2. Modern obstruction theory. In modern terms, an obstruction theory is a morphism φ X : E L X in the derived category such that H 0 (φ X ) is an isomorphism and H 1 (φ X ) is surjective. Let X be a smooth projective variety, c a fixed total Chern class, and the moduli space of stable sheaves with Chern class c. This is quasi-projective. Remark 6.2.1. might not have a universal family, i.e. it may only be a coarse moduli space. This is basically coming from the fact that even stable sheaves have nontrivial automorphisms. However, we can always cover by étale maps U such that there is a tautological family X U. We can then consider the pushforward to : X U Atiyah class. We explain the construction of the Atiyah class. Let Y is a scheme and a sheaf of Y -modules. In the classical version, with Y smooth, you consider π U U 0 I /I 2 Y Y /I 2 Y Y /I 0

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 35 We know that I /I 2 = i Ω Y and Y Y /I = i, so we can rewrite the above sequence as 0 i Ω Y Y Y /I 2 i 0 So the class of this extension is an element of Ext 1 (i Y, i Ω Y ) = Hom(i Y, i Ω Y [1]) i.e. a map (in the derived category) i Y i Ω Y [1]. If p 1, p 2 are the two projection maps Y Y Y, then this induces p 1 (p 2 E i Y ) p 1 (p 2 E i Ω Y [1]) This is a map E E E Ω Y [1], which defines a class A(E ) Ext 1 (E, E Ω Y ), which is the Atiyah class. (See an article...atiyah class... by Huybrechts and Thomas.) Theorem 6.2.2. The Atiyah class induces an obstruction theory φ : π U (Hom(, ) 0 ) [ 1] L 1 U. Proof. We apply the preceding discussion to Y = X U. We upgrade everything by replacing Ω by the cotangent complex (since we are dealing with potentially singular Y ). Embed Y W for some smooth W and let J = I Y W. Using the conormal exact sequence Ω W /Y = (J /J 2 ) Y Ω W Y Ω Y 0 and we obtain (after some calcluation) Then we get 0 i Ω Y Y Y i Y 0 i Y = [i Y (J /J 2 ) I W Y Y Y Y ]. i Y 1 Y [1] = [i Y (J /J 2 ) I W /I 2 W Y Y = i W Ω W Y 0] We can string these together into a map in the derived category [i Y (J /J 2 ) I W Y Y Y Y ] [i Y (J /J 2 ) I W /I 2 W Y Y = i W Ω W Y Y 0] Then we get p 1 (( Y i Y 1 Y [1]) p 2 ) Ext1 Y (, 1 Y ). Apply this to Y = X U, over which we have a tautological family. Then we get an Atiyah class A() Ext 1 X U (, 1 X U ). We have From the projection we get a map 1 X U = π X Ω X π U 1 U. Ext 1 X U (, 1 X U ) Ext1 X U (, π U 1 U ) = Ext1 X U (Hom(,),π U 1 U ).

36 LECTURES BY JUN LI, NOTES BY TONY FENG Then applying Verdier duality gives Ext 1 X U (Hom(,),π U 1 U ) = Ext 2 U (π U (Hom(,) π X ω X ), 1 U ). Tracing through the isomorphisms, the Atiyah class corresponds to a class A() in (Hom(, ) π X ω X, 1 ), which corresponds to a map U π U (Hom(,) π X ω X )[2] 1 U 6.3. Calabi-Yau threefolds. Theorem 6.3.1. If X is a Calabi-Yau threefold and c is a total Chern class, then := X (c) s s is projective and we have a virtual cycle [ ] vir A vir.dim ( ) where vir. dim = dim T [] = dim Ext 1 (E, E ) 0 dim Ext 2 (E, E ) 0 = 0 by Serre duality. The Donaldson-Thomas invariant is deg vir. Let X be a smooth projective variety. Let = M g,n (X,α) for α H 2 (X,). We have a universal curve f X [f : C X ] M g,n [C ] We then get a relative obstruction theory (according to Illusie) (Rπ (f Ω X )) M g,n (X,α)/M g,n In this case the virtual dimension is vir. dim( ) = n + (3 dim X )(g 1) + c 1 (T X ). Given a perfect obstruction theory E L 1 X, we have the intrinsic normal cone C X h 1 /h 0 ((E ) ). α Definition 6.3.2. We define [ ] vir = 0! [C X ] A (X ).

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 37 Part 3. Gromov-Witten Theory 7. CONSTRUCTING GROMOV-WITTEN INVARIANTS 7.1. Summary of virtual cycles. The picture is always that we have X W cut out by a section s of some vector bundle E W. We have a normal cone C X /W E X and we define a virtual class [X ] vir = 0! E [C X /W ] = c top (E, s ) A dim W rank E (X ). In this situation, we have an obstruction theory E := [E X Ω W X ] L 1 X = [J /J 2 Ω W X ] We pretend that E is a two-term complex [E 1 E 0 ], which one should imagine as σ being [T W X E X ]. In particular, for x X we imagine H 0 (E x ) = T x X and H 1 (E x ) = Ob(x). The map (L 1 X ) (E ) induces a map of the intrinsic normal cone We define [X ] vir = 0! E 1 [C X,E1 ]. Then C X h 1 /h 0 (E ) = [E 1 /E 0 ]. dim[x ] vir = dimc X + rank E 0 rank E 1. To figure out dimc X, we need to recall the construction of the intrinsic normal cone. The construction is by embedding X W and forming [C X /W /T W X ]. So dimc X /W = dim W, hence dimc X = dim W dim W = 0. 7.2. Moduli of stable maps. We consider f X M g (X,α) [f : C X ] We have a relative obstruction theory M g,n [C ] It may be more intuitive to dualize (Rπ (f Ω X )) M g,n (X,α)/M g,n M g,n (X,α)/M g,n Rπ (f Ω X ) [E 0 E 1 ]

38 LECTURES BY JUN LI, NOTES BY TONY FENG Consider the intrinsic normal cone C M g (X )/M g h 1 /h 0 ( ), which fits into a diagram M g (X )/M g C M g (X )/M g h 1 /h 0 ( ) M g (X )/M g h 1 /h 0 (Rπ f T X ) = [E 1 /E 0 ] E 1 To calculate the dimension of the normal cone, choose smooth embedding M g (X ) W over M g. (This M g is really an Artin stack since automorphisms may become infinite after forgetting the map, but that s okay because we are just doing a local calculation.) Then we have C M g (X )/M g = C M g (X )/W /T W /M g (We should think of M g as being a Deligne-Mumford stack, even though it s technically an Artin stack.) Then dim[m g (X,α)] vir = dim M g + dim E 0 dim E 1 = dim M g + χ(f T X ) = dim M g + dim X (1 g ) + deg f T X = dim M g + dim X (1 g ) + α c 1 (T X ) = 3(g 1) + dim X (1 g ) + c 1 (T X ) α = (3 dim X )(g 1) + c 1 (T X ) α. TONY: [still not sure ifthe second equality is BS or not] Conclusion: the dimension of the virtual cycle is dim[m g (X,α)] vir = (3 dim X )(g 1) + c 1 (T X ) α. (7.2.1) 7.3. Example: the quintic threefold. Let X = (x 5 1 +... + x 5 5 = 0) 4 be a quintic threefold. We view it as being cut out by a section s H 0 ( 4, (4)). By adjunction we see that K X = X, and by the standard Chern class calcluation we see that c 1 (T X ) = 0. Putting also dim X = 3 into the dimension formula (4), we see that [M g (X, d )] vir A 0 (M g (X, d )). By composing with X 4, we have a canonical map M g (X, d ) M g ( 4, d ).

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 39 Over M g ( 4, d ) we have a universal curve, which admits a map f to 4. f 4 π M g ( 4, d ) We can pull back (5) on 4 to and then push forward to M g ( 4, d ). This gives a bundle π f 4(5) on M g ( 4 ). Also, the section s pulls back to a global section f s H 0 (, f (5)). In these terms, a point [u : C 4 ] M g ( 4, d ) comes from M g (X, d ), i.e. f factors through X, if and only if u s = 0. So M g (X, d ) is cut out in M g (X, d ) by f s. The key point is that when g = 0, the stack M g ( 4, d ) is smooth and π f (5) is locally free. In this case g = 0 we have Kontsevich defined dimπ f (5)) = dim M g ( 4, d ) = 5d + 1. [ 0 (X, d )] vir := c 5d +1 (π f (5), f s ) A 0 (M 0 (X, d )). As far as we know, it is an open question whether the virtual cycle [M 0 (X, d ) vir ] defined using the tangent complex Rπ f M 0 (X,d )/M 0 X T X agrees with Kontsevich s construction. 8. K3 SURFACES 8.1. The period map for K3 surfaces. Let S be a K3 surface. (It is a deep theorem that all such are diffeomorphic.) An example is a quartic surface in 3. Then where H 2 (S,) = U 3 E 8 ( 1) 2 2 1 1 2 1 E 8 = 1 2 1......... and E 8 ( 1) = E 8. In particular, we have rank H 2 (S,) = 22. We have the Hodge decomposition H 2 (S,) = H 2,0 (S) H 1,1 (S) H 0,2 (S). By the Dolbeault Theorem, H 2,0 (S) = H 0 (S, K S ) = since K S is trivial. That tells us that the Hodge numbers are h 2,0 = h 0,2 = 1 and h 1,1 = 20. Let V = U 3 E 8 ( 1) 2 and consider V = V. Pick an isomorphism V = H 2 (S,). Then we have the line ω S H 2 (S,) V.

40 LECTURES BY JUN LI, NOTES BY TONY FENG We consider the corresponding point [ω S ] V. We know that (ω S,ω S ) = 0 since ω S ω S is a (4, 0) form, and (ω S,ω S ) > 0. Definition 8.1.1. We define the period domain M V = 21 to be M = C µ (µ,µ)=0 (µ,µ)>0. In particular, the first condition imposes one complex condition and the second is open, so dim M = 20. Suppose that we have a family X of K3 surfaces, and X 0 = S X X 0 = S 0. Then there is a canonical isomorphism H 2 (X t,) = H 2 (S,), since X deformation retracts to S. This induces H 2 (X t,) = H 2 (S,) = V. The line [ω Xt ] defines a period map V factoring through M : M Theorem 8.1.2 (Local Torelli theorem). For any K3 surface X 0, there is a family X 20 such that the induced period map X 20 σ M V σ: 20 M is a local biholomorphism onto an open subset of M. A cohomology class β H 2 (X,) then β = c 1 (L) if and only if β H 1,1 (X,) := H 1,1 (X,) H 2 (X,). Since the generator of the 2, 0 class is ω S, the algebraic classes in S are precisely ω S H 2 (S,) (since H 2 (S,) is a real subspace, any class of it in ω S automatically lies in ω S ). Therefore, if X t is a deformation of S, then β H 1,1 (S,) ceases to be (1, 1) to first order if d d t (ω t,β) t =0 0. Pick β H 1,1 (S,) so β P.D. H 2 (S,) = H 2 (X,) such that β ceases to be (1, 1) to first order when deforming t.

ENUMERATING CURVES IN CALABI-YAU THREEFOLDS 41 FIGURE 8.1.1. Depiction of algebraic classes in H 2 (X,) as lattice point in an orthogonal complement. Proposition 8.1.3. For such a choice of β, the map M g (X 0,β) M g (X,β) is an isomorphism. Proof. The two moduli problems are the same, since any map from a proper algebraic curve C X is contracted by the composition to, hence lies in a fiber. By the choice of β, the only fiber admitting algebraic classes is X 0, even to first order. 8.2. Fundamental cycles of K3 surfaces. We calculated that dim[m g (X,α)] vir = (3 dim X )(g 1) + c 1 (T X ) α which is 0 for Calabi-Yau threefolds. For K3 surfaces, K X = X and the formula tells us that dim[m g (X,α)] vir = g 1. Theorem 8.2.1 (Folklore). For X a K3 surface and any α, [M g (X,α)] vir = 0. Proof. The key idea is that the virtual cycle [M g (X,α)] vir is deformation invariant, i.e. invariant under deformations of (X,α). In other words, if we have a disk S with a basepoint 0, and a deformation X For α H 2 (X,) H 2 (,), we get a map 0 S M g (,α) S because any map [u : C X ] is necessarily contracted by the composition with projection to S, hence factors through the fiber over some point in S. So there exists [Z ]

42 LECTURES BY JUN LI, NOTES BY TONY FENG FIGURE 8.2.1. The surface X 0 inside X. Any proper curve C mapping to X must land in X 0. A (M g (,α)) such that [M g (X s,α)] vir = s! [Z ]. Here we are using a general Gysin map: W Y Y W Y If X is K3 and α H 2 (X,). If M g (X,α) = then obviously the virtual cycle is 0. If M g (X,α) is non-empty then the Poincaré dual class α P.D. lies in H 1,1 (X,) H 2 (X,), since it is algebraic. We can use the period map to deform the K3 to become nonalgebraic, so that α P.D. / X 1,1 (X,) H 2 (X,). X = 0 t Y 0 t We now change notation. Suppose X S is a family of projective smooth K3 surfaces over an affine curve S. Then K X is trivial, so X is a (local) Calabi-Yau threefold. We have X 0 the fiber over 0 S. For α H 2 (X 0,) such that (1) α PD H 1,1 (X 0,); we want to deform so that nearby α is not algebraic, evidenced by not being a (1, 1) class, so we also ask that (2) for any s 0 S, α / H 1,1 (X s,). In fact we will demand even more strongly that α / H 1,1 (X s,) to the first order deformation of X s. This implies M g (X 0,α) = M g (X,α). Obstruction theory furnishes a map Rπ f T X, and similarly for X 0. By M g (X,α)/M g functorialiy, these are compatible: M g (X 0,α)/M g M g (X,α)/M g Rπ f T X0 Rπ f T X