Efficiency in Dynamic Agency Models

Transcription

1 Efficiency in Dynamic Agency Models Anqi Li First Draft: January 2015 This Draft: December 2017 Abstract We examine a dynamic agency model where the agent s hidden action can affect current and future signals. We show that when players interact for a large number of instances, asymptotic efficiency can be attained if the monitoring technology satisfies two general conditions named measure concentration and informativeness, and if the agent can be penalized by reductions in the instantaneous consumption or the future payoff. We use this result to establish a Folk Theorem for discrete-time agency models with patient players, and to identify signal processes that yield asymptotic efficiency in discretized continuous-time agency models. Key words: incentive contract; concentration inequality. Department of Economics, Washington University in St. Louis. anqili@wustl.edu. I am indebted to my advisor Ilya Segal for inspiration, guidance and support. I also thank Marinho Bertanha, Huiyu Li, George Mailath, and seminar participants at Berkeley, UPenn, SAET 2014 and ASSA 2015 for helpful discussions. 1

2 1 Introduction The question of when and how we can achieve near-efficiency has been the central focus of the existing studies on dynamic agency models with moral hazard. So far, various answers to this question have been given, including: Radner (1985), which establishes a positive result in discrete-time models with patient players; Holmstrom and Milgrom (1987), Hellwig and Schmidt (2002), Sannikov (2008) and Sadzik and Stacchetti (2015), which obtain negative results in (discretized) continuous-time models with Brownian motion signals. These studies, albeit insightful ones, consider specific signal processes and limit the agent s actions to affecting only the concurrent signals. In this paper, we examine a T -instant model where the agent s actions can affect both current and future signals, whereas signals exhibit serial independence for any given action profile. We identify two conditions on the monitoring technology, namely measure concentration and informativeness, that ensure the attainability of near-efficiency when T is large, and examine the implications of this result for the efficiency properties of discrete-time and discretized continuous-time agency models. Our analysis exploits concentration inequalities for independent random variables, which have served as a major tool in computer science, information theory, learning theory, etc., but have somehow been overlooked by the dynamic agency literature. Our focus is on McDiarmid s (1989) concentration inequality, which prescribes a lower bound for the probability that a test statistic with bounded differences is concentrated around its mean (hereafter, concentration bound). Measure concentration says that such test statistic can be constructed, and that its concentration bound holds uniformly over all signal-generating action profiles and converges to one as T grows to infinity. Informativeness goes one step further, stipulating that the mean test statistic is informative of the agent s long-term actions, in that actions generating similar mean test statistic to efficient actions are themselves near-efficient. When both conditions are met, we can attain near-efficiency through a simple test contract when T is large. The contract pays a fixed wage at the first T 1 instances, and penalizes the agent at instant T if the sample test statistic falls short of the mean test statistic under the efficient actions minus a vanishing adjusted term. A key step in the proof is to bound the probability that the agent passes the test by sheer luck, as well as his gain from carefully adjusting action choices as information gets revealed 2

3 over time. Our starting observation is that since the monitoring technology exhibits serial independence, it follows that under any optimal decision rule, signals convey no information about each other and hence satisfy the preconditions of McDiarmid s concentration inequality for any given profile of realized actions. Further, since the concentration bound holds uniformly over all action profiles, it suffices to focus on the high probability event where the sample test statistic is concentrated around its mean when T is large. Meanwhile, informativeness stipulates that in this high probability event, the agent s actions be near-efficient in order to pass the test. This suggests that a big penalty for failure suffice to induce near-efficient actions most of the time, from which all the other results will follow. We give two applications of this result. First, we generalize Radner s (1985) Folk Theorem for discrete-time agency models by allowing signals to depend on past actions. Second, we identify signal processes that yield asymptotic efficiency in discretized continuous-time agency models, such as Poisson and Gamma processes, but not Brownian motion with drift. Compared to Poisson and Gamma processes, Brownian motion tends to drift apart from its mean more, suggesting that we keep the performance threshold slack in order to satisfy measure concentration. But this comprises informativeness, which requires that we keep tightening the performance threshold as players interactions become increasingly frequent. This result sheds light on how the principal should choose between different monitoring technologies in real-world situations. 1.1 Literature Review The current paper provides a unifying analysis of the efficiency properties of discretetime and discretized continuous-time agency models. Existing studies on review contracts, including Radner (1981), Rubinstein and Yaari (1983) and Radner (1985), use pointwise convergence theorem to prove efficiency results in discrete-time agency models. Existing discretized continuous-time agency models include, but are not limited to Hellwig and Schmidt (2002) and Sadzik and Stacchetti (2015), both dealing with signal processes that converge in law to Brownian motion with drift. In order to handle persistence, we exploit universal properties of the test statistic, namely measure concentration and informativeness, that apply to all signal-generating action profiles. Our result holds true even if signals can depend on past actions, and 3

4 it enables us to identify new signal processes that attain asymptotic efficiency in discretized continuous-time models. Concentration inequalities have served as a major tool in computer science, probability theory, information theory, learning theory, etc., but have somehow been overlooked by the dynamic agency literature. See Boucheron et al. (2013) for a textbook treatment of this subject matter. For a non-exclusive list of economics applications, see Kalai (2004) and Deb and Kalai (2015) for studies on large games, Al-Najjar and Smorodinsky (2001) for repeated large games, and Jackson and Golub (2012) for learning in social networks. In order to construct the test contract, the principal needs to know the the mean test statistic and the action cost under the efficient action profile, but nothing else. In a related paper on robust dynamic incentive contracting, Chassang (2013) assumes that the output process is unknown to the principal and proposes a limited-liability contract that approximates the performance of a linear contract. The construction exploits Blackwell s (1956) approachability theorem, whose proof itself involves the use of concentration inequalities. Our goal of attaining asymptotic efficiency is different than that of Chassang (2013), and our monitoring technology is more restrictive than his. Further, our construction does not invoke Blackwell s (1956) regret-minimizing algorithm, and it can be made to satisfy limited liability only in limiting cases. The remainder of this paper is organized as follows: Section 2 outlines the model; Section 3 states the main result; Section 4 investigates several applications; Section 5 concludes. See Section A for omitted proofs, as well as the online appendix for additional results. 2 The Model 2.1 Notations In what follows, T denotes an integer number and v T a vector that consists of an ordered list of T objects (v 1,, v T ). The notation for T is occasionally suppressed, as long as this causes no confusion. The Landau notations O( ) and Θ( ) stand for at most the order of and exactly the order of, respectively. 4

5 2.2 Setup Primitives Consider a series of economies indexed by T = 1, 2,, each involves a risk-neutral neutral principal (she) and a risk-averse agent (he) interacting for T instances. At each instant t = 1,, T, the agent takes a hidden action a t in R +, a signal X t is publicly realized, and the principal pays the agent a real-valued wage w t. Throughout t = 1,, T, the agent derives a utility U(w T ) from consumptions and pays a cost C (a T ) for taking actions, whereas the principal earns an expected revenue V (a T ) and incurs a wage expenditure E (w T ). We maintain a few standard assumptions, including (1) the expected revenue depends only on the agent s actions, (2) the lowest action cost is equal to zero, and (3) the cheapest way to compensate the agent is to pay him a fixed wage. With a slight abuse of notation, we will write E(u) as the minimum wage expenditure that confers a utility u to the agent, whose reservation utility at the outset is normalized to zero. We will assume that E(u) is continuous in u. Monitoring technology Define the monitoring technology by the collection of signal processes generated by all T -instant action profiles. For any given action profile a T, let X t be a signal defined on a probability space (S T, Σ T, P T ( a t )), where probability measure P T ( a T ) can depend on the first t actions a t = (a 1,, a t ). Further, let the ( signal process X T = (X 1,, X T ) be defined ) on the product probability T space S T S T, Σ T Σ T, P T ( a t ), and hence signals are serially t=1 independent once actions have been taken into account. The monitoring technology is players common knowledge, though this assumption will prove to be stronger than necessary. 2.3 Statistics Background McDiarmid s inequality Our analysis makes extensive use of McDiarmid s concentration inequality: Lemma 1 (McDiarmid (1989)). Let Z 1,, Z T be independent random variables taking values in a set S. Further, let ϕ be a test statistic defined on Z 1,, Z T 5

6 satisfying ϕ (z1,, z t, z T ) ϕ (z 1,, z t,, z T ) γt, t, z 1,, z t, z t S. Then for all ɛ > 0, and where P {ϕ Eϕ ɛ} α m (ɛ, ϕ), P { ϕ Eϕ ɛ} 2α m (ɛ, ϕ), ( ) α m (ɛ, ϕ) = exp 2ɛ2. T t=1 γ2 t In words, McDiarmid (1989) says that if a test statistic ϕ defined on T independent (but not necessarily identical) random variables has bounded differences γ 1,, γ T, then the probability that it is ɛ-concentrated (resp. ɛ-semi-concentrated) around the mean is bounded below by 1 2α m (ɛ, ϕ) (resp. 1 α m (ɛ, ϕ)) for all ɛ > 0. The term α m (ɛ, ϕ) is hereafter named McDiarmid s concentration bound, or concentration bound for short. Notice that α m (ɛ, ϕ) depends only on ɛ and the Lipschitz coefficient T t=1 γ2 t of ϕ but nothing else. This property, often referred to as the universality of the concentration bound by the literature, plays a crucial role in the upcoming analysis. Facts Take any δ (0, 1) and r > 0, and let = 1/T. The next two test statistics: ϕ 1,T = 1 δ 1 δ T T δ t 1 Z t t=1 and ϕ 2,T = 1 e r 1 e r serve as the basis for the upcoming analysis. T e rt Z t, Lemma 2. Let Z 1,, Z T be independent random variables taking values in a bounded set S R. Then for all ɛ > 0, t=1 6

7 (i) ( α m (ɛ, ϕ 1,T ) = exp (1 + δ)(1 ) δt ) (1 δ)(1 + δ T ) 2ɛ 2, S 2 where S sup s,s S s s ; (ii) when T is large, ( α m (ɛ, ϕ 2,T ) exp 2T ) ɛ2 g(r), S 2 where g(r) 2(1 e r ) 2 r(1 e 2r ). Proof. Omitted. 2.4 Main Assumptions We make three important assumptions. First, in each economy T, there exists a pure action profile that maximizes the social surplus (hereafter, the target action profile): Assumption 1. In each economy T, there exists a pure action profile a T that solves max σ (R T +) E [V (a T ) σ] E (E [C (a T ) σ]). Under Assumption 1, let VT, C T and S T denote the expected revenue, action cost and social surplus generated by the target action profile, respectively, and let η T = inf σ (R T + ) E [V (a T ) σ] V T be the ratio between the worst-case and target-level revenues. These quantities will prove to be useful soon. Second, the monitoring technology satisfies two basic conditions called measure concentration and informativeness: Assumption 2. There exist series of test statistics { ϕ T : ST T R} T =1 reals {ɛ T } T =1 such that (i) each ϕ T has bounded differences and α m (ɛ T, ϕ T ) is vanishing in T ; and positive 7

8 (ii) there exist vanishing series of positive reals {v T } T =1 and {c T } T =1 such that for each T and a T, V (a T ) VT (1 v T ) and C (a T ) CT (1 c T ) if E [ϕ T a T ] E [ϕ T a T ] 2ɛ T. In words, measure concentration (Part (i) of Assumption 2) ensures the existence of test statistics with vanishing concentration bounds, whereas informativeness (Part (ii) of Assumption 2) requires that actions generating similar mean test statistics to target ones be themselves near-efficient. For notation ease, we shall hereafter write the concentration bound as αt m whenever this causes no confusion. The last assumption imposes some regularities on our problem: Assumption 3. (i) V T, C T, S T Θ(1) as T ; (ii) lim T max {α m (ɛ T, ϕ T ), c T } η T = 0; (iii) the agent faces unlimited liability at instant T. Parts (i) and (ii) of Assumption 3 stipulate that the target-level revenue, action cost and social surplus be non-vanishing in T, and that the worst-case revenue should not decrease too fast as T grows to infinity. Part (iii) of Assumption 3 can be relaxed if the T -instant economy is part of an infinite-horizon model where the agent can be severely penalized by reductions in the future payoff instead (e.g., discrete-time model with patient players, discretized continuous-time model). Due to space limitations, we do not fully describe such model here, and refer interested readers to the online appendix for further details. 2.5 Examples Our framework nests (1) discrete-time agency models with patient players and (2) discretized continuous-time agency models. Below we give two illustrative examples along these lines. In each example, we first describe the model setup and then check the validities of our assumptions. Example 1. The horizon lasts T periods and players share a common discount factor δ (0, 1). In each period t, players observe the realization of the concurrent revenue X t, and their payoffs are given by u(w t ) c(a t ) and X t w t. We maintain the standard assumption that u > 0, u < 0, c > 0 and c > 0, while allowing the distribution 8

9 of X t to depend on the first t actions a t. The last assumption generalizes the main condition of Radner (1981) and Radner (1985), namely revenue can depend only on the concurrent action. In the current setting, the social surplus generated by an arbitrary T -period action profile a T is V (a T ) u 1 (C (a T )), where V (a T ) = 1 δ T 1 δ T t=1 δt 1 E [X t a t ] and C (a T ) = 1 δ T 1 δ T t=1 δt 1 c(a t ). Since the function u 1 (C) is continuous and convex, a sufficient condition for Assumption 1 to hold true is that V is continuous and concave. ) To verify the validity of Assumption 2, take ϕ T = ϕ 1,T and ɛ T Θ (T 12 +ξ, where ξ can be any number in ( 0, 1 2). Notice the following: 1. Since the revenue space S is bounded and independent of T, a straightforward application of Lemma 2 shows that lim T lim δ 1 α m (ɛ T, ϕ 1,T ) = Informativeness follows from the fact that the mean test statistic equals the expected revenue. Formally, since E [ϕ 1,T a T ] = V (a T ) for all a T, it follows that E [ϕ 1,T a T ] E [ϕ 1,T a T ] 2ɛ T holds true if and only if V (a T ) V T 2ɛ T. Under the assumption that a T is efficient, any action profile a T satisfying E [ϕ 1,T a T ] E [ϕ 1,T a T ] 2ɛ T must also satisfy C (a T ) u (u 1 (C T ) 2ɛ T ), because otherwise there is a way of generating more than the full surplus, a contradiction. Putting these results together, we see that v T = 2ɛ T and c T = u (u 1 (C T )) C T 2ɛ T, and that lim T v T = lim T c T = It is easy to construct examples that satisfy Assumption 3, e.g., let E [X t a t ] = a t + ρa t ρ t 1 a 1 for some ρ [0, 1) and u be unbounded below. Example 2. Time evolves continuously over [0, 1] and players share a common interest rate r > 0. The economy indexed by T divides [0, 1] into T subintervals of length = 1/T, with the action, signal and wage over [(t 1), t ] being given by a t, X t and w t, respectively. Throughout [0, 1], the agent s utility is 1 e r T 1 e r t=1 e rt [u(w t ) c(a t )] and the principal s profit 1 e r T 1 e r t=1 e rt [a t w t ]. For now, suppose signals are generated by a Poisson process with arrival rate t s=1 ρt s a t over [(t 1), t ]. In particular, ρ can be any number in [0, 1), with ρ = 0 representing the case of no persistence as studied by Myerson (2010). Let Z t be the jump size over [(t 1), t ] and let X t = Z t 1 Zt 1 ρz t 1 1 Zt 1 1. Notice that X t takes values in S = { ρ, 0, 1 ρ, 1}, and that the mean of X t equals approximately the expected revenue over [(t 1), t ], i.e., E [X t a t ] a t, when T is large. 9

10 A casual inspection reveals the isomorphism between the current example and Example 1, namely, 1. Assumption 1 holds true for the same reason as given in Example 1. ) 2. Take ϕ T = ϕ 2,T and ɛ T Θ (T 12 +ξ, where ξ can be any number in ( 0, 2) 1. Since S is bounded and independent of T, it follows from Lemma 2 (ii) that lim T α m (ɛ T, ϕ 2,T ) = 0. Further, the very fact that the mean statistic equals the expected revenue, i.e., V (a T ) = 1 er T 1 e r t=1 e rt a t = E [ϕ 2,T a T ], implies informativeness, for the same reason as given in Example 1. 3 Main Result In this section, we demonstrate how we can achieve asymptotic efficiency through the use of test contract, which consists of (1) a test statistic ϕ T, (2) a threshold E [ϕ T a T ] ɛ T and (3) a pair of wages w T and w T. The contract pays the agent a fixed wage w T throughout t = 1,, T 1. At t = T, if the sample statistic exceeds the threshold, then the agent passes the test and earns the same wage w T as before. Otherwise he fails the test and earns w T instead. The penalty for failure amounts to U (w T,, w T ) U (w T,, w T ) = λc T, where λ can be any number that is greater than one and independent of T. Under any test contract, the agent s decision rule is a collection of state-continent plans σ T = { σ t,t : A t 1 S t 1 T (A) }, where each σ t,t prescribes the instant-t action choice based on the history of actions and signals. Define F = {ϕ T < E [ϕ T a T ] ɛ T } as the event where the agent fails the test. Any optimal decision rule σ T solves min σ T E [C (a T ) + 1 F λc T σ T ], and the contract induces voluntary participation at the outset if U (w T,, w T ) E [C (a T ) + 1 F λc T σ T ] 0. 10

11 Theorem 1. Take any series of economies that satisfies Assumptions 1-3 for each T, as well as any series of test contracts that satisfies the above described properties. Then for each T N, (i) the probability that the agent fails the test satisfies P (F σ T ) 3λαm T + c T λ 1 + c T ; (ii) the ratio between the expected profit generated by the test contract and the full surplus is bounded below by 1 S T As T, { [ ( )] } 3λα VT m 1 v T (1 v T η T ) T + c T E ((1 + λαt m ) CT ). λ 1 + c T (a) P (F σ T ) O (max {αm T, c T }); (b) the ratio in Part (ii) converges to one. Proof. See the Appendix. Theorem 1 prescribes a lower bound for the test contract s profitability in each economy T and shows that this lower bound converges to the full surplus as T grows to infinity. This result holds true independently of the exact details of the signal process and even if signals can depend on past actions. A key step in the proof is to bound the probability that the agent passes the test by sheer luck, as well as his gain from carefully adjusting action choices as information gets revealed over time. Our starting observation is that since the monitoring technology exhibits serial independence, it follows that under any optimal decision rule, signals convey no information about each other and thus satisfy the preconditions of McDiarmid s concentration inequality for any given profile of realized actions. Further, since the concentration bound holds uniformly over all action profiles, it suffices to focus on the high probability event where the sample test statistic is concentrated around its mean when T is large. Meanwhile, informativeness stipulates that in this high probability event, the agent s actions be near-efficient in order to pass the test. This suggests that a big penalty for failure suffice to induce near-efficient actions most of the time, from which all the other results will follow. 11

12 4 Applications Generalization of existing models Theorem 1 suggests that we can attain asymptotic efficiency in (1) the discrete-time model described in Example 1 as players become arbitrarily patient, as well as (2) the discretized continuous-time model with Poisson signals as described in Example 2. The first result generalizes Radner s (1985) Folk Theorem by allowing revenues to depend on past actions. The second result is alluded to by Myerson (2010), which examines the case of no persistence and binary actions, and uses the assumption that high action must be induced all the time to rule out the use of the test contract. 1 A new model Theorem 1 enables us to identify new signal processes that yield asymptotic efficiency in discretized continuous-time models: Example 2 (Continued). In Example 2, now suppose, instead, that signals follow a Gamma process: over each interval [(t 1), t ], jumps (denoted by Z t ) of size in [x, x + dx] arrive according to a Poisson process with intensity ν t (x)dx, where x 0 and ν t (x) = ( t s=1 ρt s a t ) x 1 exp( λx) for some arbitrary ρ [0, 1) and λ > 0. Let κ = 1 0 exp( λx)dx and X t = κ 1 ( Z t 1 Zt 1 ρz t 1 1 Zt 1 1). Notice that X t takes values in a set S = [ κ 1 ρ, κ 1 ] that is bounded and independent of T, and that the expected value of X t equals approximately the expected revenue over [(t 1), ) t ], i.e., E [X t a t ] a t, when T is large. Take ϕ T = ϕ 2,T and ɛ T Θ (T 12 +ξ, where ξ can be any number in ( 0, 2) 1. Results so far suggest that we satisfy both measure concentration and informativeness, and can thus attain asymptotic efficiency when T is large. A negative result The analysis so far assumes the existence of signals that equates the mean test statistic with the expected revenue. The next example, which is adapted from Sadzik and Stacchetti (2015), shows that this assumption itself is not enough to ensure asymptotic efficiency: Example 2 (Continued). In Example 2, now suppose, instead, that the revenue over [(t 1), t ] is X t, where X t N (a t, σ 2 T ). To see that this revenue process cannot 1 The same assumption appears in other dynamic contracting models with Poisson signals, e.g., Biais et al. (2010). However, these models typically involve addition ingredients that make the comparison between our model less straightforward. 12

13 satisfy both measure concentration and informativeness, notice that if the contrary were true, then we could use the test contract as a building block to construct a near-efficient contract in an infinite-horizon model where the agent is protected by limited liability and is penalized by changes in the future payoff beyond instant T (see the online appendix for further details). But this contradicts with the result of Sadzik and Stacchetti (2015), namely the principal s profit is strictly bounded away from the full surplus in the above described environment. To gain intuition into this negative result, take, for example, ϕ T = ϕ 2,T, and notice that ϕ T N (V (a T ), σ 2 /g(r)) when T is large (see Lemma 2 for the definition of g(r)). Thus for any series of positive reals {ɛ T } T =1, we have v T = 2ɛ T and c T = u (u 1 (CT)) 2ɛ CT T, and hence a sufficient and necessary condition for informativeness to hold true is lim T ɛ T = 0. But then we cannot satisfy measure concentration, because lim T P { ϕ T E [ϕ T a T ] < ɛ T } = 0. What is going on here? As T, the discounted sum of revenues converges in law to a Brownian motion with drift. Since Brownian motion tends to drift apart from its mean, we must leave some slackness in the performance threshold in order to satisfy measure concentration. But this comprises informativeness, which requires that we keep tightening the performance threshold as T grows to infinity. 5 Conclusion We conclude by posing open questions for future research. First, it will be interesting to carry out similar exercises in dynamic games with imperfect monitoring. Second, the result on discretized continuous-time models suggests that the principal prefer Poisson or Gamma signals to Brownian motion if she can choose between the various kinds of monitoring technologies. Likewise, Poisson or Gamma signals should be traded at a premium compared to Brownian motion if monitoring technologies can be bought at a price. The empirical merits of these predictions await a careful test. A Proof of Theorem 1 Part (i): We proceed in three steps: Step 1. Since the monitoring technology exhibits serial independence, it follows 13

14 that under any optimal decision rule σt, past signals affect future signals only through future action choices. Thus, for any given profile of realized actions a T, signals convey no information about each other and hence satisfy the precondition of McDiarmid s concentration inequality. Further, since McDiarmid s concentration bound αt m holds uniformly over all possible a T s, it follows that under σt, the probability that the sample test statistic is ɛ T -concentrated around its mean under the true action profile is bounded below by 1 2α m T. Formally, define G = { ϕ T E [ϕ T a T ] < ɛ T } as the event where the sample statistic is ɛ T -concentrated around its mean under true actions. The above discussion suggests that P (G σ T ) = E [E [1 G a T ] σ T ] 1 2α m T. Step 2. Recall the definition of event F: F = {ϕ T < E [ϕ T a T ] ɛ T }, and define π T = P (F G σ T ) as the probability of the event where the sample test statistic is ɛ T -concentrated around its mean under true actions and yet the agent fails the test. Then, P (F c G σ T ) = 1 P (G c σ T ) P (F G σ T ) 1 2α m T π T. Throughout t = 1,, T, the agent s total loss, which includes the action cost and the penalty for failure, is bounded below by π T λct + E [1 }{{} F c G C (a T ) σt ] +P (G c σt ) }{{}}{{} 0. (a) (b) (c) In particular, (a) and (c) constitute lower bounds for the loss in event F G 14

15 and G c, respectively. Further, (b) can be bounded below as follows: E [1 F c G C (a T ) σt ] = E [E [1 F c G C (a T ) a T ] σt ] E [E [1 F c G CT (1 c T ) a T ] σt ] = P (F c G σt ) CT (1 c T ) (1 2αT m π T ) CT (1 c T ), where the first inequality can be obtained by noticing that in event F c G, the very fact that (1) the test statistic is ɛ T -concentrated around its mean, i.e., ϕ T E [ϕ T a T ] < ɛ T, and that (2) the agent passes the test, i.e., ϕ T E [ϕ T a T ] ɛ T, implies ϕ T E [ϕ T a T ] 2ɛ T and hence C (a T ) CT (1 c T ) by Assumption 2 (ii). Putting these results together, we obtain the following lower bound for the agent s loss under σt : π T λc T + (1 2α m T π T ) C T (1 c T ). (A.1) Step 3. If the agent takes a T instead, then his total loss is bounded above by C T + α m T λc T. (A.2) Since (A.1) (A.2), it follows that π T λc T + (1 2α m T π T ) C T (1 c T ) (1 + λα m T ) C T. Dividing both sides of the above inequality by C T and rearranging, we obtain: π T (λ + 2)αm T + (1 2αm T )c T λ 1 + c T. Thus, P (F σ T ) P (F G σ T ) + P (G c σ T ) π T + 2α m T 3λαm T + c T λ 1 + c T, and this completes the proof of Part (i). 15

16 Part (ii): The agent s expected payoff under σt is bounded below by U (w T,, w T ) (1 + λα m T ) C T, suggesting that the following wage expenditure suffices to make him accept the contract at the outset: E ((1 + λα m T ) C T ). Meanwhile, the expected revenue generated by σ T is bounded below by P (F G c σt ) VT η }{{} T + E [1 F c G V (a T ) σt ]. }{{} (a) (b) In particular, (a) π T + 2αT m, and (b) can be bounded below as follows: E [1 F c G V (a T ) σ T ] = E [E [1 F c G V (a T ) a T ] σ T ] E [E [1 F c G V T (1 v T ) a T ] σ T ] = P (F c G σ T ) V T (1 v T ) (1 2α m T π T ) V T (1 v T ), with the first inequality being implied by the fact that in event F c G, combining (1) ϕ T E [ϕ T a T ] < ɛ T and (2) ϕ T E [ϕ T a T ] ɛ T yields ϕ T E [ϕ T a T ] 2ɛ T and hence V (a T ) V T (1 v T ) by Assumption 2 (ii). Together, these results imply the following upper bound for the expected profit: V T [ ( )] 3λα m 1 v T (1 v T η T ) T + c T E ((1 + λαt m ) CT ), λ 1 + c T and this completes the proof of Part (ii). Parts (a) and (b): Under Assumptions 2 and 3, letting T in the above derivation gives the desired result. 16

17 B Online Appendix This section extends the analysis so far to infinite-horizon models. To avoid repetition, we only consider discretized continuous-time models, and leave it to the reader to verify that the result below carries over to discrete-time models with patient players. Time evolves continuously over [0, + ). Over each interval [(t 1), t ], t = 1, 2,, the agent takes a hidden action a t 0, a signal X t is publicly realized, and the principal pays a non-negative wage w t 0. Throughout [0, + ), the agent s utility is r t=1 e rt (u(w t ) c(a t )), and the principal s expected profit r t=1 e rt (a t w t ). We maintain the standard assumptions that u(0) = 0, u > 0, u < 0 and that c(0) = 0, c > 0 and c > 0. We define the efficient action a by the unique interior solution to max a R+ a u 1 (c(a)). Fix any l > 0, ɛ > 0 and λ > 1 independent of T such that the following condition holds true: l λc (a ) e rs ds < ε 0 l e rs ds. Divide [0, + ) into review cycles of length l, i.e., [(k 1)l, kl], k = 1, 2,. Suppose that actions affect only signals of the concurrent review cycle. Suppose, in addition, that for each review cycle k, there exist series of test statistics {ϕ k,t } T =1 and positive reals {ɛ k,t } T =1 that satisfy Assumption 2. Consider the following generalization of the test contract: at t = (k 1)l, if the relationship survives, then the principal pays a flow wage u 1 (c (a ) + ε) throughout review cycle k; at t = kl, if ϕ k,t exceeds E [ϕ k,t a ] ɛ k,t, then the agent passes the test and the relationship continues; otherwise he fails the test and is fired with probability λc(a ) l ε l 0 e rs ds e rs ds after which both players earn zero reservation utilities in the remainder of their lives., Corollary 1. Suppose the environment satisfies the above descriptions. T, Then as (i) the principal s expected profit converges to a u 1 (c(a )) O(ε); (ii) over each review cycle, the agent s expected payoff equals approximately ε, and the probability that he fails the test is vanishing in T. 17

18 Proof. Suppose the agent adopts the optimal decision rule prescribed by Theorem 1 in each review cycle. Then the penalty he pays for failing the test equals approximately the following term when T is large: ( λc(a ) l ε l 0 e rs ds e rs ds ) ε l l e rs ds = λc(a ) e rs ds. 0 The remainder of the proof follows closely that of Theorem 1 and thus is omitted. References Al-Najjar, N. I. and R. Smorodinsky (2001): Large Nonanonymous Repeated Games, Games and Economic Behavior, 37(1), Blackwell (1956): An Analog of the Minimax Theorem for Vector Payoffs, Pacific Journal of Mathematics, 6, 1-8. Biais, B., T. Mariotti, J. C. Rochet and S. Villeneuve (2010): Large Risks, Limited Liability, and Dynamic Moral Hazard, Econometrica, 78(1), Boucheron, S., G. Lugosi and P. Massart (2013): Concentration Inequalities: A Nonasymptotic Theory of Independence, Oxford, United Kingdom: Oxford Unviersity Press. Chassang, S. (2013): Calibrated Incentive Contracts, Econometrica, 81(5), Deb, J., and E. Kalai (2015): Stability in Large Bayesian Games with Heterogeneous Players, Journal of Economic Theory, 157, Hellwig, M. F., and K. M. Schmidt (2002): Discrete-Time Approximations of the Holmstrom-Milgrom Brownian-Motion Model of Inter-temporal Incentive Provision, Econometrica, 70(6), Holmstrom, B., and P. Milgrom (1987): Aggregation and Linearity in the Provision of Intertemporal Incentives, Econometrica, 55(2),

19 Jackson, M., and B. Golub (2012): How Homophily Affects the Speed of Learning and Best-Response Dynamics, The Quarterly Journal of Economics, 127(3), Kalai, E. (2004): Large Robust Games, Econometrica, 72(6), McDiarmid, C. (1989): On the Methods of Bounded Differences, in Surveys in Combinatorics, ed. by J. Siemons, London Mathematical Society Lecture Note Series 141, Cambridge: Cambridge University Press. Myerson, R. B. (2010): Leadership, Trust, and Power: Dynamic Moral Hazard in High Office, Working Paper, University of Chicago. Radner, R. (1981): Monitoring Cooperative Agreements in a Repeated Principal- Agent Relationship, Econometrica, 49(5), (1985): Repeated Principal-Agent Games with Discounting, Econometrica, 53(5), Rubinstein, A., and M. E. Yaari. (1983): Repeated Insurance Contracts and Moral Hazard, Journal of Economic Theory, 30(1), Sadzik, T., and E. Stacchetti (2015): Agency Models with Frequent Actions: a Quadratic Approximation Method, Econometrica, 83, Sannikov, Y. (2008): A Continuous-Time Version of the Principal-Agent Problem, Review of Economic Studies, 75(3),