Dynamic Kidney Exchange

Transcription

1 Dynamic Kidney Exchange M. Utku Ünver y University of Pittsburgh First Draft: March 2007 Second Draft: October 2007 Abstract As the kidney exchange programs are being established, one important question that remains to be answered is how should the exchanges be conducted in a dynamically evolving patient pool. We study the kidney exchange problem as a dynamic matching problem, in which patient and her paired-donor arrive under a Poisson arrival process. We associate a constant unit cost of waiting in the pool for each patient, given that the alternative is continuous dialysis. We characterize the dynamically optimal two-way and unrestricted kidney exchange rules that minimize the total cost associated with the exchange pool under certain assumptions. Under the optimal two-way matching rule, all exchanges are conducted as soon as they become available. On the other hand, under the optimal unrestricted matching rule, depending on the arrival frequencies of di erent pairs, one of the two threshold rules can be optimal. In the rst threshold rule, B blood-type patient and A blood-type donor pairs (B-A type) are exclusively used to be matched with reciprocal A-B type pairs instead of possibly larger exchanges, as long as the number of B- A type pairs does not exceed some threshold number. The other threshold rule is the symmetric version of the rst one and is obtained by treating the number of A-B type pairs as the state variable instead of the B-A type. Keywords: Kidney exchange, matching, dynamic programming, market design, time preferences JEL Classi cation Numbers: C78, C70, D78, C61 I would like to thank especially Tayfun Sönmez, and also David Abraham, Murat Fad lo¼glu, Fikri Karaesmen, Onur Kesten, David Kaufman, Fuhito Kojima, Alvin E. Roth, Insan Tunali, and Neşe Yildiz, participants at SAET Conference at Kos, Matching Workshop at Barcelona, Boston College, Carnegie Mellon, and Koç for comments and suggestions. I am grateful to the NSF (National Science Foundation) for nancial support (Award Number: SES ). All errors are my own responsibility. y Address: Department of Economics, University of Pittsburgh, 4528 Wesley W. Posvar Hall, 230 S. Bouquet St., Pittsburgh, PA, 15260, USA. uunver@pitt.edu. Phone: (1) Url: 1

2 1 Introduction There are more than patients waiting for a kidney transplant in the United States. In 2005, only transplants were conducted, 9800 from deceased donors and 6570 from living donors, while new patients joined the deceased donor waiting list and 4200 patients died while waiting for a kidney. 1 Although there is a substantial organ shortage, buying and selling an organ is illegal in many countries in the world, therefore donation is almost the only source for kidney transplants. Especially in the last decade, the increase in the number of kidney transplants came from utilization of live donors, who are mostly relatives, friends, or spouses of the patients and are willing to donate one of their kidneys. However, still many of the living donors cannot be utilized, since the potential donor may not be able to donate to her loved one due to blood-type incompatibility or tissue rejection. The medical community has proposed innovative ways to utilize these living donors through live donor kidney exchanges (Rapaport, 1986). In a live donor kidney exchange, recipients with incompatible donors swap their donors if there is cross-compatibility. Since 1991, kidney exchanges have been done mostly in an ad-hoc manner in di erent countries in the world. Live donor kidney exchanges accounted for at least 10% of all live donor transplants in Korea and in the Netherlands in 2004 (see Park et al, 2004 and de Klerk et al, 2005). The medical community has ethically endorsed the practice of live donor kidney exchanges (Abecassis et al, 2000). Unlike Korea and Netherlands, there is no national system to oversee kidney exchanges in the United States. Di erent transplant centers around the country have recently started kidney exchange programs. For example, New England Program for Kidney Exchange (NEPKE) is an initiative of the transplant centers in New England together with economists (see Roth, Sönmez and Ünver, 2005b), and Alliance for Paired Donation (APD) is an initiative of Dr. Michael Rees at Ohio Medical College and the authors of the above studies. APD has already committed a large number of transplant centers all around the U.S. to participate. There are some other regional kidney exchange programs around the country. In many of these programs, a major objective has been conducting as many transplants as possible. Though, one question has frequently arisen in the implementation stage: How often and how should the exchange be run? Roth, Sönmez, and Ünver (2004, 2005a) have recently proposed mechanisms to organize kidney exchanges in a Pareto e cient and dominant strategy incentive compatible fashion under di erent constraints on exchange sizes and preferences of the recipients for a static recipient population. The above studies addressed the matching aspect of the problem, however they did not consider the dynamic aspects of the exchange pool evolution. 2 From a more general perspective, in the 1 According to SRTR/OPTN national data retrieved at on 2/27/ In the operations research literature, Zenios (2002) considers a dynamic model with only two types of patient-donor pairs and di erent outside options. In this model pairs arrive continuously over time but not in a discrete process like ours. Moreover, since there are only two types of patients-donor pairs and the outside options are di erent, this model is substantially di erent from ours. Our model addresses the matching aspect of the dynamic kidney exchange problem. 2

3 matching literature in economics, although there is signi cant amount of work on mechanism design in static environments, there is almost no study that studies matching or exchange in dynamically changing populations. 3 A recent paper by Abdulkadiro¼glu and Loertscher (2006) inspects the dynamic preference formation in house allocation problems. There is also a literature initiated by Roth and Vande Vate (1990) and Blum, Roth and Rothblum (1997) that inspects whether random repeated myopic transactions will converge to stable matchings in two-sided and other matching markets. 4 The current study is a rst step in this direction in closing this gap. We consider a dynamically evolving population of recipients and their donors, who arrive over time through a stochastic Poisson process. In this setting, we characterize the dynamically optimal kidney exchange mechanism under di erent institutional constraints and assumptions. We consider two constraints in our design following the medical literature. First, in the medical literature, Gjertson and Cecka (2000) and Delmonico (2004) pointed out that a recipient is indi erent between two live donor kidneys as long as they are compatible with the recipient. We adopt this assumption in our current study and assume that the preferences of the recipients have three indi erence classes as being matched to a compatible kidney, being unmatched, and being matched to an incompatible kidney. 5 the dynamic setting. We also introduce time preferences of recipients in Second, the practice of kidney exchange has started by conducting exchanges that include only two recipients and their incompatible donors. More complicated exchanges that include three or more recipients (and their donors) will rarely take place at least initially, because all transplants of a single exchange cycle should take place simultaneously. Otherwise, some donors can potentially back out after their recipients receive transplants given the legal constraint that it is illegal to force a donor to sign a contract that will commit him for donation. Nevertheless, Roth, Sönmez and Ünver (2007) have shown that larger exchanges, especially three-way exchanges including three recipientdonor pairs will substantially increase the gains from exchange. In real life, occasionally larger size exchanges are also being conducted. In the current paper, we separately derive optimal dynamic matching rules that conduct two-way exchanges and unrestricted exchanges. We consider the dynamic problem from the point of view of a central health authority which incurs a social cost for each recipient waiting in the exchange pool. The health authority is trying to minimize the current value of the total cost. We make two assumptions in the derivation of the 3 There is a vast economics literature on the allocation or exchanges of indivisible goods initiated by Shapley and Scarf (1974), Roth and Postlewaite (1977), Roth (1982), Abdulkadiro¼glu and Sönmez (1998, 1999), Papai (2000), Ergin (2000), Ehlers (2002), Ehlers, Klaus and Papai (2002), Kesten (2006), Sönmez and Ünver (2005, 2006). None of these work focuses on the stochastic dynamic problem, although Ergin (2000), Ehlers, Klaus and Papai (2002), Sönmez and Ünver (2006) inspect the problem with static exchange rules under varying population sizes. 4 In the economics literature, there are studies that have studies optimal mechanisms in dynamic settings. Some examples of such papers are Jackson and Palfrey (1998), which studies optimal bargaining mechanisms in a dynamic setting, and Skreta (2006), which studies optimal dynamic mechanism design when the designer cannot commit to a mechanism in the future. 5 We had the same assumption in Roth, Sönmez and Ünver (2005a). 3

4 optimal two-way matching rule. We assume that in the long run, there is an arbitrarily large number of underdemanded types of pairs, whose donors blood types are not compatible with their recipients blood types. We also assume that there is no tissue rejection between donors and recipients of two distinct pairs. Under these two assumptions, we show that an interesting characteristic of an optimal two-way matching rule is that it conducts the maximum number of exchanges as soon as they become available, that is, there is no need of sacri cing one or more currently feasible exchanges for the sake of conducting future exchanges at the steady state (Theorem 1). However, this theorem no longer holds when larger exchanges are feasible, and we derive the optimal unrestricted matching rule as a threshold matching rule under one additional assumption. (Theorem 2, Theorem 3 and Remark 2). A threshold rule relies on threshold values, which are related to the number of A blood-type recipient and B blood-type donor pairs (A-B type pairs), B blood-type recipient and A blood-type donor pairs (B-A type pairs) and potentially on the existence or absence of pairs with A blood-type recipients and donors (A-A type pairs) and B blood-type recipients and donors (B-B type pairs). Depending on the frequencies of arrival of such pairs one of the two types of threshold rules is optimal. In the rst possible solution, the optimal rule conducts the maximum size exchanges as soon as they become available as long as there are no B-A type pairs. However, if there are some B-A type pairs already available in the exchange pool, and their number does not exceed a threshold number, then the health authority should not use the B-A type pairs other than matching A-B type pairs and avoid involving them in larger exchanges that do not have A-B type pairs. Only when the stock of B-A type pairs exceeds the threshold number, the health authority should conduct the largest possible exchanges as soon as they become available and possibly use B-A type pairs in exchanges without A-B type of pairs. The second possible solution is just the symmetric version of the rst solution, and instead treats A-B type pairs as the stock variable. We observe that the rst type of rule is optimal using the real-life blood distribution probabilities, and this rule takes the number of B-A type pairs as the possible stock variable. Under di erent pair arrival rates and market interest rates, we compute the optimal rule. Additionally, we conduct policy simulations and observe that the gains under the optimal unrestricted rule are signi cantly higher than those under the optimal two-way rule. 2 Background 2.1 Medical Constraints on Kidney Transplants Before a donor is deemed compatible with a recipient, two tests should be done: blood-type compatibility test and tissue-type compatibility test (or crossmatch test). There are four blood types, O, A, B, or AB. An O blood type recipient can only receive a transplant from an O donor, an A blood type recipient can only receive a transplant from an O or an A donor, a B blood type recipient can only receive a transplant from an O or B donor, and an AB blood type recipient can receive a 4

5 transplant from all donors. A recipient and a donor are blood-type compatible if the donor can feasibly donate a kidney to the recipient due to their blood types. A recipient cannot always receive a kidney from a blood-type compatible donor due to tissue-type incompatibility. There are 6 proteins on human DNA that determine the tissue type of a person. Some tissue types can be rejected by a recipient s immunological system. A formal test should be done by mixing the blood of the donor and the recipient for testing tissue-type incompatibility prior to the transplant. If antibodies form in the recipient s blood against the donor s tissue antigens then we say that there is positive crossmatch between the recipient and the donor, meaning that the donor and the recipient are tissue-type incompatible. A donor is tissue-type compatible with a recipient, if there is negative crossmatch between them. A donor is compatible with a recipient if he is both blood- and tissue-type compatible with the recipient. A recipient can receive a kidney only from compatible donors. 2.2 Logistic Constraints on Exchange Size Donations are considered as gifts under the current laws in most of the countries. In the U.S., under the National Organ Transplant Act (NOTA) of 1984, buying or selling an organ is illegal and considered as rst degree felony. World Health Organization repeatedly condemned the practice of organ buying and selling. Abecassis et al. (2000) stated a consensus statement that exchanges are medically ethical. However, a donor cannot be paid, forced to commit, or contracted for donation. Hence, all transplants in an exchange should be carried out at the same time, since otherwise a donor can potentially back out after his paired recipient receives a transplant. This creates a physical limitation on the sizes of exchanges that can be carried out. It looks like initially exchanges will be two-way exchanges, since it requires the least number of transplant teams to be present simultaneously. Therefore, we will rst restrict our attention to matching rules that only allow two-way exchanges. Roth, Sönmez and Ünver (2005a) formulated the static two-way exchange problem and found deterministic and probabilistic Pareto-e cient and strategy-proof two-way matching rules. Later Roth, Sönmez and Ünver (2007) and Saidman et al (2005) demonstrated that for a static kidney exchange problem, most of the gains from trade come from two-way exchanges and also showed that the marginal gains of three-way exchanges are quite large and non-negligible. Later we extend our analysis to the case when larger exchanges are feasible. 3 The Dynamic Exchange Model 3.1 Exchange Pool In our model, we consider the recipients who have live donors. We assume that each recipient has a single live donor. A pair i consists of a recipient R i and her live donor D i. Let B = fo, A, B, ABg be the set of blood types. Since each person can be in one of the four blood types, there are 16 5

6 di erent blood type permutations for a pair. We call each of these permutations a pair type. The type of a pair is denoted as X-Y where X, Y 2 B and X is the blood type of the recipient and Y is the blood type of the donor in the pair. Let T = B B be the set of pair types. For any pair type X-Y 2 T, let q X-Y be the probability of a random pair being of type X-Y. We refer to q X-Y as the arrival probability of pair type X-Y 2 T. We have P X-Y 2T q X-Y = 1. For any X-Y2 T, we refer to Y-X as the reciprocal pair type of X-Y. Type X-Y 2 T is blood-type-compatible if the recipient of the pair is blood-type compatible with the donor of the pair. Let T B be the set of blood-type compatible pair types. We have T B = fo-o, A-O, A-A, B-O, B-B, AB-O, AB-A, AB-B, AB-ABg: A pair is tissue-type compatible, if the donor and the recipient of the pair are tissue-type compatible with each other. A pair is compatible if the pair is both blood- and tissue-type compatible. We assume that pairs arrive over time with a stochastic (discrete) Poisson arrival process in continuous time. Let be the arrival rate of the pairs, i.e. the expected number of pairs that arrive per unit time. Once a pair arrives, if it is not compatible, it becomes available for exchange. If it is compatible, the donor immediately donates a kidney to the recipient of the pair and the pair does not participate in exchanges. Exchange pool is the set of the pairs which arrived over time and whose recipient has not yet received a transplant. Let p c be the positive crossmatch probability that determines the probability that a donor and a recipient will be tissue-type incompatible. Let p X-Y denote the pool entry probability of any arriving pair type X-Y2 T. Since blood-type incompatible pairs always join the exchange pool, we have p X-Y = 1 for any X-Y2 T nt B. Since blood-type-compatible pairs join the pool if and only if they are not tissue-type compatible, we have p X-Y = p c for any X-Y2 T B. Let p = P X-Y2T p X-Yq X-Y be the expected number of pairs that enter the pool for exchange per unit time interval. 3.2 Preferences Each recipient has preferences over donors and time of waiting in the pool. Compatible donors are preferred to being unmatched - an option which is denoted by being matched with her paired incompatible donor D i - and, in turn being unmatched is preferred to being matched to incompatible donors. Gjertson and Cecka (2000) point out that, each compatible live donor kidney will last approximately the same as long as the donor is not too old and in relatively in good health. Following this study, we assume that each compatible donor is indi erent for a recipient. Moreover, time spent in the exchange pool is another dimension in the preferences of recipients such that waiting is costly. We will model the waiting cost through a xed cost. 6

7 3.3 Kidney Exchanges An exchange is a list of pairs (i 1 ; i 2 ; : : : ; i k ) for some k 2 such that for any ` < k; donor D i` donates a kidney to recipient R i`1, and donor D ik donates a kidney to recipient R i1. A matching is a set of exchanges such that each pair participates in at most one exchange. A matching or an exchange is individually rational if it never matches a recipient with an incompatible donor. We only consider individually rational exchanges and matchings in this paper. From now on, when we talk about an exchange or a matching, it will be individually rational. A matching is maximal if it matches the maximum number of pairs possible at an instance of the pool. A (dynamic) matching rule is a dynamic procedure such that at each time t 0 it selects a (possibly empty) matching of the pairs available in the pool. Once a pair is matched at time t by a matching rule, it leaves the pool and its recipient receives the assigned transplant. Let N A (t) be the total number of pairs arrived until time t. If a matching rule is executed (starting time 0), N (t) is the total number of pairs matched by rule. There are N R (t) = N A (t) N (t) pairs available at the pool at time t. 3.4 Dynamically Optimal Rules There is a health authority which oversees the exchanges. For each pair, we associate waiting in the pool with a monetary cost and we assume that the unit time cost of waiting for a transplant by undergoing continuous dialysis is equal to c > 0 for each recipient. The alternative option of a transplant is dialysis. A patient can undergo dialysis continuously. It is well-known that receiving a transplant causes the patient to resume a better life. Also health care costs for dialysis are more than those for transplantation in the long term. We model all the costs associated with undergoing continuous dialysis by the monetary constant unit time cost c. Suppose that the health authority implements a matching rule. For any time t, the current value of expected cost at time t under matching rule is given as 6 E t C (t) = = Z 1 t Z 1 t ce t N R () e ( t) d ce t N A () N () e ( t) d, where is the market interest rate. For any time ; t such that > t, we have E t N A () = p ( t)n A (t), where the rst term is the expected number of recipients to arrive at the exchange pool in the interval [t; ] and the second term is the number of recipients that arrived at the pool until time t. Therefore, we can rewrite E t C (t) as E t C (t) = Z 1 t 6 E t refers to the expected value at time t. c( p ( t) N A (t) E t N () )e ( t) d. 7

8 Since R 1 t e ( t) d = 1 and R 1 t ( t) e ( t) d = 1 2, we can rewrite E t C (t) as E t C (t) = cp 2 N A (t) Z 1 t ce t N () )e ( t) d. (1) Only the last term in Equation 1 depends on the choice of rule. The previous terms cannot be controlled by the health authority, since they are the costs associated with the number of recipients arriving at the pool. We refer to this last term as the exchange surplus at time t for rule and denote it by We can rewrite it as ES (t) = Z 1 t ce t N () e ( t) d: ES (t) = Z 1 t = cn (t) c E t N () N (t) N (t) e ( t) d Z 1 t c E t N () N (t) e ( t) d: The rst term above is the exchange surplus attributable to all exchanges that have been done until time t and at time t and the second term is the future exchange surplus attributable to the exchanges to be done in the future. The central health authority cannot control the number of past exchanges at time t either. Let n () be the number of matched recipients at time by rule ; and we have N (t) = X <t n ()! n (t) : We focus on the present and future exchange surplus which is given as fes (t) = cn (t) Z 1 t c E t N () N (t) e ( t) d: (2) A dynamic matching rule is optimal if for any t, it maximizes the present and future exchange surplus at time t given in Equation 2. We look for steady state solutions of the problem independent of initial conditions and time t. We will de ne a steady-state formally. If such solutions exist, they only depend on the current state of the pool (de ned appropriately) but not on time t or the initial conditions. 4 Dynamically Optimal Two-way Matching Rules In this section, we derive the dynamically optimal two-way matching rule. A two-way exchange is an exchange involving only two pairs. A matching is a two-way matching if all exchanges in the matching are two-way exchanges. It will be useful to introduce the following concepts about two-way 8

9 exchanges. We say that two pairs i and j are mutually compatible, if donor D i is compatible with recipient R j and donor D j is compatible with recipient R i. We say that two pairs i and j are mutually blood-type compatible, if donor D i is blood-type compatible with recipient R j and donor D j is blood-type compatible with recipient R i. The following observations state out the important individual rationality constraints that need to be respected in our characterization of optimal matching rules: Observation 1: A pair of type X-Y 2 fo-a, O-B, O-AB, A-AB, B-ABg can participate in a twoway exchange only with a pair of its reciprocal type Y-X or type AB-O. Observation 2: A pair of O-O, A-A, B-B, AB-AB, A-B or B-A can participate in a two-way exchange only with its reciprocal type pair or a pair belonging to some of the types among A-O, B-O, AB-O, AB-A, AB-B. Observation 3: A pair of type X-Y 2 fa-o, B-O, AB-O, AB-A, AB-Bg can not participate in a two-way exchange with a pair of not only its own type (and possibly some other types in the same set), but also some types among O-A, O-B, O-AB, A-AB, B-AB, O-O, A-A, B-B, AB-AB, A-B, B-A, as well. Based on these observations, we partition the set of types into four sets T O ; T U, T S and T R as follows: We refer to the set T O = fa-o, B-O, AB-O, AB-A, AB-Bg as the set of overdemanded types. These types are not only blood-type-compatible, but they can also save pairs of many other types in a two-way exchange, as stated in Observation 3. We refer to the set T U = fo-a, O-B, O-AB, A-AB, B-ABg as the set of underdemanded types, which can participate in a two-way exchange with their reciprocal types or type AB-O pairs, as stated in Observation 1. Underdemanded types can only be matched with their reciprocal types or type AB-O pairs, who are overdemanded types by Observation 1. We refer to the set T S = fo-o, A-A, B-B, AB-ABg as the set of self-demanded types. Self-demanded types can only be matched with the same type pairs or overdemanded type pairs, as stated in Observation 2. We refer to the set T R = fa-b, B-Ag as the set of reciprocally demanded types. Reciprocally demanded types can be matched with their reciprocal types as well as some overdemanded types, as stated in Observation 2. 9

10 Throughout this section we will maintain two assumptions. The rst one is motivated by the medical data. In the long run, this assumption will likely hold. We assume that: Assumption 1 (Long-Run Assumption): Under any dynamic matching rule, at steady state, there is an arbitrarily large number of underdemanded pairs from each pair type in the exchange pool. We will comment on the weakness of this assumption, later on. That is, we will show that regardless of the used two-way matching rule, this assumption will hold under realistic arrival probabilities for the pairs. We will state a limit assumption on limits of exchange (see Roth, Sönmez and Ünver 2007). Assumption 2 (Limit Assumption): No recipient is tissue-type incompatible with the donor of another pair. Recipients can be tissue-type incompatible with their own donors and p c is the probability for that to happen. This ensures that blood-type compatible pairs arrive at the pool. On the other hand, recipients will never be tissue-type incompatible with donors of other pairs under Assumption 2. This assumption will give us an idea about the limits of dynamic optimization. Under Assumption 2, two pairs can participate in a two-way exchange if and only if they are mutually blood-type compatible. Note that average tissue-type incompatibility (positive crossmatch) probability is reported as p c = 0:11 by Zenios, Woodle and Ross (2001). We assume that there is positive crossmatch possibility within a pair to make sure that blood-type compatible pairs arrive at the exchange pool. As Zenios, Woodle and Ross (2001) also point out, a female recipient who has been pregnant is 3 times more likely to have tissue-type incompatibility against her own husband than a random donor. This is caused by the antibodies already formed in the recipient body during pregnancy against the tissue type of her husband. Thus, Assumption 2 has also support from real-life for certain types of pairs. We are ready to state Theorem 1. Theorem 1 Let dynamic matching rule be de ned as a rule that matches only X-Y type pairs with their reciprocal Y-X type pairs, immediately when such an exchange is feasible. Then, under Assumptions 1 and 2, rule is a dynamically optimal two-way matching rule. Moreover, a dynamically optimal two-way matching rule matches the maximum possible number of recipients in the exchange pool at any moment in time at the steady state. Let dynamic two-way matching rule be de ned as in the hypothesis of Theorem 1; that is, for any arriving incompatible pair of any type X-Y 2 T, rule matches this pair immediately with an existing Y-X type pair if such a mutually compatible pair exists in the pool, and does not perform any exchanges, otherwise. 10

11 We will prove Theorem 1, after proving that rule matches the maximum number of pairs possible in any given time interval under Assumptions 1 and 2. Proposition 1 Under Assumptions 1 and 2, within any time interval at the steady state, rule matches the maximum number of pairs possible to match under any two-way matching rule. Proof of Proposition 1: Suppose that Assumptions 1 and 2 hold. Suppose that rule is used for the exchange. Consider a time interval > 0 at the steady state. Let t 0 be the start of this time interval and t 1 = t 0 be the end of this time interval. Since each type is matched with its reciprocal type under rule, at the steady state we have, 1. for any type X-Y 2 T U = fo-a, O-B, O-AB, A-AB, B-ABg, by Assumption 1, there will be arbitrarily large number of type X-Y pairs available, 2. for any type X-Y 2 T O = fa-o, B-O, AB-O, AB-A, AB-Bg, since there is an arbitrarily large number of type Y-X pairs available (by Statement 1 above), for any incoming X-Y type pair i there will exist at least one Y-X type pair mutually compatible (even if Assumption 2 did not hold), and rule will immediately match these two pairs, implying no type X-Y pairs will remain available, 3. for any type X-X 2 T S = fo-o, A-A, B-B, AB-ABg, whenever x type X-X pairs are available in the exchange pool, all of these pairs will be mutually compatible with each other by Assumption 2. Rule will match 2b x c of these pairs with each other, implying that there will 2 be 0 or 1 type X-X pair available, 7 4. for any X-Y 2 T R = fa-b, B-Ag, there is either no Y-X pair in the pool or there are nitely many. If there is a Y-X type pair, by Assumption 2, this pair and the incoming pair are mutually compatible and rule will immediately match these two pairs. In the pool, there will be either (1) no X-Y type pair and no Y-X type pair remaining, (2) no Y-X type pair and some X-Y type pairs remaining, or (3) no X-Y type pair and some Y-X type pairs remaining. Clearly, the maximum number of exchanges in the interval [t 0 ; t 1 ] is performed by not conducting any exchanges in interval [t 0 ; t 1 ) and then conducting the maximal exchange at time t 1. Since time interval is nite, Statement 1 above is still valid at any time t 2 [t 0 ; t 1 ], regardless of which exchanges are conducted in [t 0 ; t 1 ). Therefore, by Proposition 1 of Roth, Sönmez and Ünver (2007), the maximum number of transplants that can be conducted in interval [t 0 ; t 1 ] is # = 2 X X -Y2T O n X -Y 2 X X -X2T S b n X -X 2 c 2 minfn A -B; n B -A g; (3) 7 For any real number x, bxc represents the greatest integer less than or equal to x. 11

12 where n X -Y is the number of type X-Y pairs that are available at time t 1, if no exchange has been conducted in interval [t 0 ; t 1 ). Moreover for any X-Y 2 T, this number in Equation 3 can be achieved by matching each X-Y type pair with a reciprocal type pair, as long as it is possible. Next consider the scenario, in which rule is used in interval [t 0 ; t 1 ). Statements 1-4 above are valid for any time t 2 [t 0 ; t 1 ] under rule. Therefore under rule, the number of matched pairs in interval [t 0 ; t 1 ] is 2 P X -Y2T O n X -Y for the overdemanded and underdemanded pairs by Statements 1 and 2, 2 P X -X2T S b n X -X 2 c for the self-demanded types by Statement 3, and 2 minfn A -B ; n B -A g for the reciprocally demanded types by Statement 4, and their sum is exactly equal to # given in Equation 3, completing the proof of Proposition 1. Theorem 1 can be proven using Proposition 1. Proof of Theorem 1: Suppose Assumptions 1 and 2 hold. Fix time at steady state. For any rule and any time t, N (t) N () is the total number of recipients matched between time t and at steady state under rule, and is maximized by the rule by Proposition 1. Since N (t) is ex-post maximized for = for any t, E N (t) N () N () is maximized by =, as well. Moreover, rule conducts the maximum possible number of exchanges at any given point in time as 0 or 2 (permitted by the two-way exchange restriction). Therefore, n () is also maximized by =. These imply ES () = n () R 1 ce N (t) N (t) e (t ) dt is maximized for = ; at the steady state, implying that is an optimal two-way matching rule. Moreover, it conducts the maximum number of transplants at each time, completing the proof of Theorem 1. Note that, since we can de ne the optimal rule independent of the state of the pool, we did not introduce an explicit state space for the pool. It turns out that under unrestricted exchanges, the optimal rule explicitly depends on the state. Next we show that Assumption 1 will hold in the long run under the most reasonable pair-type arrival distributions, thus, it is not a restrictive assumption. Proposition 2 Suppose that p c (q AB-O q X-O ) < q O-X for all X2 fa,bg, p c (q AB-O q AB-X ) < q X-AB for all X2 fa,bg and p c q AB-O < q O-AB. Then, Assumption 1 holds in the long run regardless of the two-way matching rule used. Proof of Proposition 2: By Observation 1, any underdemanded type can only be matched with its reciprocal type of the AB-O type. Since overdemanded type pairs enter the pool if and only if they are incompatible, p c is the probability of this happening. Thus, if the arrival rate of an underdemanded type is higher than the pool entry rate of its reciprocal and AB-O type pairs then in the long run there will be arbitrarily many underdemanded type pairs. 12

13 R 1 D 1 AB O R 2 D 2 O A R 3 D 3 A AB (a) R 1 D 1 AB O R 2 D 2 O B R 3 D 3 B AB (b) Figure 1: Three-way exchanges involving an AB-O type pair and two underdemanded pairs. The hypothesis of the above proposition very mild, and will hold for su ciently small crossmatch probability. Moreover, it holds for the real-life blood frequencies. For example, assuming that the recipient and the paired-donor are blood-unrelated, the arrival rates reported in the simulations section of the paper satisfy these assumptions, when the crossmatch probability is p c = 0:11, as reported by Zenios, Woodle and Ross (2000). 5 Dynamically Optimal Unrestricted Matching Rules In this section, we consider matching rules that not only allow for two-way exchanges, but also larger exchanges as well. Roth, Sönmez and Ünver (2007) have studied the importance of three-way and larger exchanges in a static environment. The marginal e ect of the availability of three-way and four-way exchange technology with respect to the two-way exchange technology can be summarized as follows: An overdemanded AB-O type pair can potentially save two underdemanded type pairs of types O-A and A-AB or O-B and B-AB under a three-way exchange, instead of only one in a two-way exchange (see Figures 1a,b). When the number of A-B type pairs is larger than the number of B-A type pairs in a static pool (that is, type A-B is like an underdemanded type) potentially all B-A type pairs can be matched with A-B type pairs (especially when the population is large). Moreover overdemanded types can save the excess of the type A-B type pairs as follows: Each B-O type pair can potentially save one O-A type pair and one A-B type pair in a three-way exchange (see Figure 2a). Each AB-A type pair can potentially save one A-B type and one B-AB type pair in a three-way exchange (see Figure 2b). 13

14 R 1 D 1 B O R 2 D 2 O A R 3 D 3 A B (a) R 1 D 1 AB A R 2 D 2 A B R 3 D 3 B AB (b) Figure 2: Three-way exchanges involving an A-B type pair, an overdemanded pair, and an underdemanded pair. R 1 D 1 AB O R 2 D 2 O A R 3 D 3 A B R 3 D 3 B AB Figure 3: Four-way exchange using an AB-O type pair, an A-B type pair, and two underdemanded pairs. Each AB-O type pair can potentially save one O-A type pair, one A-B type pair, and one B-AB type pair in a four-way exchange (see Figure 3). 8 We can still match all remaining pairs with their reciprocal types in large populations in two-way exchanges. We can state the following observation motivated by the above results: Observation 3: In an exchange that matches an underdemanded pair, there should be at least one overdemanded pair. In an exchange that matches a reciprocally demanded pair, there should at least be one reciprocal type pair or an overdemanded pair. 5.1 Steady State Analysis Using the above illustration, under realistic blood type distribution assumptions, we will prove that Assumption 1 still holds, when the applied matching rule is unrestricted. Recall that through Assumption 1, we assumed to have arbitrarily many underdemanded type pairs available in the long run states of the exchange pool, regardless of the dynamic matching rule used to achieve the long run. 8 If B-A type pairs are more than A-B type pairs in a static population, the symmetric versions of these three-way and four-way exchanges will be bene cial. 14

15 Proposition 3 Suppose that p c (q AB-O q X-O ) min fp c q Y-O ; q X-Y g < q O-X for all fx,yg = fa,bg, p c (q AB-O q AB-X ) min fp c q AB-Y ; q Y-X g < q X-AB for all fx,yg = fa,bg and p c q AB-O < q O-AB. Then, Assumption 1 holds in the long run regardless of the two-way matching rule used. Proof of Proposition 3: By the above illustration and Observation 1, underdemanded type O-A can only be matched most e ectively in a two-way exchange with its reciprocal type or AB-O, in a three-way exchange using a B-O type pair with the help of an A-B pair (see Figure 2a). Similarly an underdemanded type B-AB can only be matched most e ectively in a two-way exchange with its reciprocal type or AB-O, in a three-way exchange using an A-O type pair with the help of an A-B pair (see Figure 2b). Also the symmetric of these observations hold for O-B and AB-B. An AB-O pair can only be matched with an O-AB pair. Thus if the above inequalities hold, in the long run, there will be arbitrarily many underdemanded type pairs, regardless of the matching rule used. The hypothesis of the above proposition is also very mild, and will hold for su ciently small crossmatch probability p c. Moreover, it holds for the real-life blood frequencies and crossmatch probability. For example, assuming that the recipient and the paired-donor are blood-unrelated, the arrival rates reported in the simulations section of the paper satisfy these assumptions. Thus, we can safely use Assumption 1 in this section, as well. We did not make any assumptions such as the exchange pool patients can die, get a deceased donor transplant, or exit the pool for other reasons over time. However, this is a trivial extension of our current model. We state this with a remark: Remark 1 If the duration of patients availability in the pool is drawn from an exponential distribution and the expected duration is su ciently long, then Assumption 1 will still hold for su ciently small crossmatch probability p c. In this case, the optimization problem will not change, either, although the interpretation of the market interest rate will have to be modi ed. Under Assumption 1, we are ready to formally de ne what we mean by steady state. A steady state of the dynamic kidney exchange problem refers to situations at which the number of underdemanded type pairs is arbitrarily large and potentially continues to increase over time. Next, we characterize the optimal rule at the steady state. In a dynamic setting, the structure of three-way and four-way exchanges discussed above may cause the second part of Theorem 1 not to hold when three-way and larger sizes of exchanges are feasible. More speci cally, we can bene t from not conducting all feasible exchanges currently available, and holding on to some of the pairs which can currently participate in an exchange in expectation of saving more pairs in the near future. We maintain Assumption 2 as well as Assumption 1 in this section. That is, every recipient is tissue-type compatible with donors of other pairs. To characterize the optimal matching rule, we state one other assumption. 15

16 Next, we show that as long as the di erence between A-B and B-A type arrival frequencies is not large, overdemanded type pairs will be matched immediately. The proof of this proposition is given in the appendix. Proposition 4 Suppose Assumptions 1 and 2 hold. If 2q A-B > q B-A and 2q B-A > q A-B, then under the dynamically optimal unrestricted matching rule, overdemanded type pairs are matched as soon as they arrive at the exchange pool. In general, we expect that A-B and B-A type pairs do not arrive with very di erent frequencies. For example, in the U.S., A-B and B-A type pairs arrive with frequencies q A-B = 0:05 and q B-A = 0:03 (c.f. Terasaki, Gjertson and Cecka, 1998) and the hypothesis of Proposition 4 about the arrival rates is satis ed by these probabilities. Therefore, we will state it as an additional assumption. Assumption 3 (Assumption on the Arrival Rates of Reciprocally Demanded Types): We have 2q A-B > q B-A and 2q B-A > q A-B : Under Assumptions 1,2 and 3, by Proposition 4, we will only need to make decisions in situations in which multiple feasible exchanges of di erent sizes are feasible: For example, consider a situation in which an A-O type pair arrives at the pool, while a B-A type pair is also available. Since by Assumption 1, there is an excess number of O-A and O-B type pairs at the steady state, there are two sizes of feasible exchanges, a three-way exchange (for example, involving A-O,O-B, and B-A type pairs) or a two-way exchange (for example, involving A-O and O-A type pairs). Which exchange should the health authority choose? To answer this question, we analyze the dynamic optimization problem under Assumptions 1, 2, and 3. Since the pairs arrive according to a Poisson process, we can convert the problem to an embedded Markov decision process. 9 We need to de ne a state space for our steady state analysis. Since the pairs in each type is symmetric by Assumption 2, the natural candidate for a state is a 16 dimensional vector, which shows the number of pairs in each type available. In our exchange problem, there is additional structure to eliminate some of these state variables. We inspect overdemanded, underdemanded, self-demanded, and reciprocally demanded types separately: Overdemanded types: If an overdemanded pair i of type X-Y 2 T O arrives, by By Proposition 4, pair i will be matched immediately in some exchange. Hence, the number of overdemanded pairs remaining in the pool is always 0. Underdemanded types: By Assumption 1, at steady state there will be an arbitrarily large number of underdemanded pairs. Hence, the number of underdemanded pairs is always 1. Self-demanded types: Whenever a self-demanded pair i of type X-X 2 T S is available in the exchange pool, it can be matched through two ways under an unrestricted matching rule: 9 See Puterman (1994) for an excellent survey of continuous time and discrete time Markov decision processes. 16

17 1. If another X-X type pair j arrives, by Assumption 2, i and j will be mutually compatible, and two-way exchange (i; j) can be conducted. 2. If an exchange E = (i 1 ; i 2 ; :::; i k ), with Y blood-type donor D ik and Z blood-type recipient R i1, becomes feasible, and blood-type Y donors are blood-type compatible with bloodtype X recipients, while blood-type X donors are blood-type compatible with blood-type Z recipients, then pair i can be inserted in exchange E just after i k, and by Assumption 2, the new exchange E 0 = (i 1 ; i 2 ; :::; i k ; i) will be feasible. By Observation 3, a self-demanded type can never save an overdemanded or reciprocally demanded pair without the help of an overdemanded or another reciprocally demanded pair. Suppose that there are n X-X type pairs. Then, they should be matched in two-way exchanges to save 2 n 2 of them (which is possible by Assumption 2). This and the above observations imply that under an optimal matching rule, for any X-X 2 T S, at steady-state there will be either 0 or 1 X-X type pair. Therefore, in our analysis, existence of self-demanded types will be re ected by 4 additional state variables, each of which getting values either 0 or 1. We rst derive the optimal dynamic matching rule by ignoring the self-demanded type pairs, then we will reintroduce the selfdemanded types to the problem and derive the dynamically optimal matching rule. Assumption 4 (No Self-Demanded Types Assumption): There are no self-demanded types available for exchange and q X-X = 0 for all X-X 2 T : Reciprocally demanded types: By the above analyses, there are no overdemanded and self-demanded type pairs available and there are in nitely many underdemanded type pairs. Therefore, the state of the exchange pool can simply be denoted by the number of A-B type pairs and B-A type pairs. By Assumption 2, an A-B type pair and B-A type pair are mutually compatible with each other, and they can be matched in a two-way exchange. Moreover, by Observation 3, an A-B or B-A type pair cannot save an underdemanded pair in an exchange without the help of an overdemanded pair. Hence, the most optimal use of A-B and B-A type pairs is being matched with each other in a two-way exchange. Therefore, under the optimal matching rule, an A-B and B-A type pair will never remain in the pool together, but they will be matched via a two-way exchange. By this observation, we can simply denote the state of the exchange pool by an integer s; such that if s > 0, then s refers to the number of A-B type pairs in the exchange pool, and if s < 0, then jsj refers to the number of B-A type pairs in the exchange pool. Formally s is the di erence between the number of A-B type pairs and B-A type pairs in the pool, and only one of these two numbers can be non-zero. Let S = Z be the state space (i.e., the set of integers). 17

18 5.2 Markov Chain Representation In this subsection, we characterize the transition from one state to another under a dynamically optimal matching rule by a Markov chain when Assumptions 1, 2, 3 and 4 hold: First suppose s > 0, i.e. there are some A-B type pairs and no B-A type pairs. Suppose a pair of type X-Y 2 T becomes available. In this case, three subcases are possible for pair i: 1. X-Y 2 T U = fo-a, O-B, O-AB, A-AB, B-ABg: By Observation 3, in any exchange involving an underdemanded pair, there should be an overdemanded pair. Since, there are no overdemanded pairs available at steady state under the optimal rule, no new exchanges are feasible. Moreover, the state of the exchange pool remains as s. 2. X-Y 2 T O = fa-o, B-O, AB-O, AB-A, AB-Bg: If pair i is compatible (which occurs with probability 1 p c ), donor D i donates a kidney to recipient R i, and pair i does not arrive at the exchange pool. If pair i is incompatible (which occurs with probability p c ), pair i becomes available for exchange. Three cases are possible: (a) X-Y 2 {A-O, AB-B}: Since s > 0, there are no B-A type pairs available. In this case, there is one type of exchange feasible: a two-way exchange including pair i, and a mutually compatible pair j of type Y-X. By Assumption 1, such a Y-X type pair exists. By Proposition 4, this exchange is conducted, resulting with 2 matched pairs, and the state of the pool remains as s. There is no decision problem in this state. (b) X-Y 2 {B-O, AB-A}: Since s > 0, there are A-B type pairs available. There are two types of exchanges that can be conducted: a two-way exchange and a three-way exchange: By Assumption 1, there is a mutually compatible pair j of type Y-X, and (i; j) is a feasible two-way exchange. If X-Y = B-O: By Assumption 1, there is an arbitrary number of O-A type pairs. Let pair j be an O-A type pair. Let k be an A-B type pair in the pool. By Assumption 2, (i; j; k) is a feasible three-way exchange (see Figures 1a). If X-Y = AB-A: By Assumption 1, there is an arbitrary number of B-AB type pairs. Let k be a B-AB type pair. Let j be an A-B type pair in the pool. By Assumption 2, (i; j; k) is a feasible three-way exchange (see Figure 1b). Let action a 1 refer to conducting a smaller exchange (i.e., two-way), and action a 2 refer to conducting a larger exchange (i.e., three-way). If action a 1 is chosen, 2 pairs are matched, and the state of the pool does not change. If action a 2 is chosen, then 3 pairs are matched, and the state of the pool decreases to s 1. (c) X-Y = AB-O: Since s > 0, there are three types of exchanges that can be conducted: a two-way exchange, a three-way exchange, or a four-way exchange: 18

19 By Assumption 1 and Observation 1, for any W-Z 2 T U, there is a mutually compatible pair j of type W-Z for pair i. Hence, (i; j) is a feasible two-way exchange. By Assumption 1, there are pair j of type O-B and pair k of type B-AB such that (i; j; k) is a feasible three-way exchange. Also by Assumption 1, there are pair g of type O-A and pair h of type A-AB such that (g; h; i) is a feasible three wayexchange. (see Figure 1a,b). By Assumption 1, there is an arbitrarily large number of underdemanded pairs independent of the matching rule, therefore, conducting either of these two three-way exchanges has the same e ect on the future states of the pool. Hence, we will not distinguish these two types of exchanges. By Assumptions 1 and 2, a pair h of type B-AB, a pair j of type O-A, and a pair k of type A-B form the four-way exchange (h; i; j; k) with pair i (see Figure 3). Two-way exchange and three-way exchange do not change the state of the pool. Therefore, conducting a three-way exchange dominates conducting a two-way exchange. Hence, under the optimal rule, we rule out conducting a two-way exchange, when an AB-O type pair arrives. Let action a 1 refer to conducting a smaller exchange (i.e., three-way) exchange, and let action a 2 refer to conducting a larger exchange (i.e., four-way). If action a 1 is chosen, 3 pairs are matched,and the state of the pool remains as s. If action a 2 is chosen, 4 pairs are matched, and the state of the pool decreases to s X-Y 2 T R = fa-b, B-Ag: Two cases are possible: (a) X-Y = A-B: By Observation 3, an A-B type pair can only be matched using a B-A type pair or an overdemanded pair. Since there are no overdemanded and B-A type pairs, there is no possible exchange. The state of the pool increases to s 1. (b) X-Y = B-A: By Assumption 2, a feasible two-way exchange can be conducted using an A-B type pair j in the pool and pair i. This is the only feasible type of exchange. Since matching a B-A type pair with an A-B type pair is the most optimal use of these types of pairs, we need to conduct such a two-way exchange and the state of the pool decreases to s 1: Figure 4 summarizes how the state of the pool evolves for s > 0. Note that we do not need to distinguish decisions regarding two-way versus three-way exchanges and three-way versus four-way exchanges. We denote all actions regarding smaller exchanges by a 1 and all actions regarding larger exchanges by a 2. Since the di erence between a smaller exchange and a larger exchange is always 1 pair, i.e., an A-B type pair, whenever the state of the pool dictates that a three-way exchange is chosen instead of a two-way exchange when a B-O or AB-A type pair arrives, then it will also dictate that a four-way exchange will be chosen instead of a three-way exchange when an AB-O type pair arrives. 19