DYNAMIC COSTS AND MORAL HAZARD: A DUALITY BASED APPROACH. Guy Arie 1

Transcription

1 DYNAMIC COSTS AND MORAL HAZARD: A DUALITY BASED APPROACH Guy Arie 1 Abstract The marginal cost of effort often increases as effort is exerted. In a dynamic moral hazard setting, dynamically increasing costs create information asymmetry. This aer characterizes the otimal contract and hels exlain the oular yet thus far uzzling use of non-linear incentives, for examle in salesforce comensation. The result is obtained using two comlementing dynamic rograms one based on duality and one similar to the standard. The dual rogram is monotonic and sub-modular, roviding stronger results, including a roof for the sufficiency of one shot deviations. Keywords: Dynamic moral hazard, nonlinear incentives, rivate information, dynamic mechanism design, duality, linear rogramming, stochastic rogramming, dynamic rogramming. 1. INTRODUCTION Increasing marginal costs are a standard comonent of economic analysis. In organizational settings, the increase in cost often has a dynamic motivation. A worker icking fruits, for examle, gets tired as the day rogresses. In other settings the task itself becomes harder over time. Sales erformance, for examle, is measured over a eriod, tyically quarter or year. As the quarter rogresses, the agent deletes the easy sales leads and must exert more effort to generate later sales. Sales effort is inherently hard to monitor and ay is often erformance based. If the firm knew the agent s true cost, it would want to increase incentives towards the end of the quarter. This aer characterizes the otimal mechanism for a dynamic moral hazard setting in which the cost increase deends on the agent s rivate information his effort. 1 University of Rochester, Simon School of Business. First version November This version May This aer is based on the first chater of my dissertation at Kellogg. I am indebted to Bill Rogerson, Mark Satterthwaite and esecially Jeroen Swinkels and Michael Whinston for their advice and encouragement throughout this work. I thank many seminar articiants and esecially Eddie Dekel, Matthias Fahn, Paul Grieco, Jin Li, and Michael Raith for helful suggestions. Financial suort from the GM Center at Kellogg is gratefully acknowledged. 1

2 2 GUY ARIE Section 5.2 rovides an examle in which the otimal contract can be imlemented using a known twist on a quota contract and a second examle in which the otimal contract can be imlemented by a slightly sohisticated convex incentive scheme. Joseh and Kalwani (1998) document the oularity of convex and quota based incentive schemes in sales related settings. However, as Prendergast (1999) summarizes: rather remarkably, the theoretical literature has made little rogress in understanding the observed (nonlinear) shae of comensation contracts, desite costs associated with nonlinearities. The same conclusion is echoed in more recent studies, see e.g. Misra and Nair (2009) and Larkin (2007). Thus, the analysis here shows that increasing marginal cost can rovide a relatively simle micro-economic foundation for the oularity of these schemes. The model is the simlest ossible to cature the roblem of rivately increasing costs. A risk neutral agent decides every day whether to exert costly effort. The robability of success (a sale) in the day increases with effort. The cost of effort today is a convex function of ast effort. 1 Effort is unobserved and the rincial can commit to a contract at the outset To see the incentive roblem, suose that the robability of a sale each eriod is if the agent exerts effort and zero otherwise, and that the agent s cost for making the n-th effort is n. If both the rincial and the agent consider only current eriod incentives, a contract aying the agent 2n for a sale in day n is incentive comatible and rovides the agent zero exected utility clearly first best. However, if the agent considers future ayoffs, this contract is no longer incentive comatible. Shirking in the first eriod and then working whenever asked obtains the agent an exected utility of 1 each eriod. By shirking today the agent increases his rents from future work. The otimal contract must account for this additional incentive to shirk. The otimal contract can be informally described as a dynamic quota: the agent starts in an evaluation stage and eventually moves to a comensation stage. In the comensation stage the agent is aid a fixed iece-rate for each sale and works for an additional fixed number of eriods that is indeendent of any new outcomes. In 1 Multile ossible outcomes are considered in section B.1. The contract is essentially the same. 2 Abstracting from the imerfect commitment roblem is a standard assumtion in the dynamic moral hazard literature. See e.g., Rogerson (1985); Sear and Srivastava (1987); Fernandes and Phelan (2000); Clementi and Hoenhayn (2006); DeMarzo and Fishman (2007); Biais, Mariotti, Rochet, and Villeneuve (2010). For a recent examination of the imlications of renegotiation in related settings see Strulovici (2011).

3 DYNAMIC COSTS AND MORAL HAZARD 3 the evaluation stage the agent is rewarded only by changes to the exected fixed iece-rate, the length of the comensation stage, and the quota the agent must meet to enter the comensation stage. If the agent accumulates enough early successes, his comensation er sale later in the quarter will be high. If the agent did not accumulate enough early successes, the contract leads the agent to sto working. Once the agent meets his dynamic quota, his reward is based only on his highest anticiated cost, generating excessive rewards for successful agents, as found in both Misra and Nair (2009) and Larkin (2007). On the other hand, the only way to rofitably rovide such high rewards is to limit the work by unsuccessful agents, resulting in a higher volatility of the work decision towards the end of the work eriod, consistent with the finding in Oyer (1998). Related roblems have been considered in the literature, simlifying various asects of the roblem. One natural simlification is to assume that the game is static. In static models the agent decides on the level of work before observing any outcomes. Innes (1990) show that a bonus for assing a certain threshold rovides otimal rofits and Kim (1997) shows that under certain conditions the outcome is first-best. The examles in section 5.2 confirm this.poblete and Sulber (2012) restrict the iece rate of the contract to at most the revenue from the sale. In that case the otimal contract is a simle quota contract that ays the agent all the revenues above a certain thershold. The static contracts are generally sub-otimal and not incentive comatible in a dynamic setting as the agent has a richer deviation sace. If the agent s marginal cost of effort fixed (i.e. non-increasing), the otimal dynamic contract can be derived using Clementi and Hoenhayn (2006). 3 The contract will eventually either fire the agent without ay or allocate all remaining revenues to the agent. Thus, the otimal dynamic contract if costs are fixed is a form of a quota contract in which the threshold may change in resonse to early outcomes, but the reward for assing the threshold all the remaining revenue is indeendent of history. As a result, once the threshold is met, the continuation is first-best. Increasing 3 Biais, Mariotti, Rochet, and Villeneuve (2010) consider a closely related continuous time model with fixed marginal costs. There, the agent s binary effort choice reduces the robability of a loss. Biais, Mariotti, Rochet, and Villeneuve (2010) however allow the rincial to also choose the size of the roject (roughly equivalent to multilying both the revenue and cost to the agent in the current model). Biais, Mariotti, Rochet, and Villeneuve (2010) fully characterize the otimal contract in which the agent exerts effort at all times. In our setting the otimal contract for maximal effort u to any eriod T is a simle ayment er success based on the eriod T cost. Instead, this aer focuses on characterizing the otimal contract.

4 4 GUY ARIE marginal costs require that the reward for assing the threshold also deends on the history. In articular, the agent is unlikely to obtain all the remaining revenue and the continuation after assing the threshold is likely not first-best. This is because the first-best continuation is worth more to an agent that shirked because of the agent s lower costs. Increasing marginal costs are considered exlicitly only in two eriod dynamic moral hazard roblems. Mukoyama and Şahin (2005) is a direct alication and Ábrahám, Koehne, and Pavoni (2011) extends the technical analysis to various two eriod settings. The economic analysis in Mukoyama and Şahin (2005) however focuses on the case that the agent s effort in the first eriod comlements effort in the second eriod. In such cases the agent s comensation relies more on the second eriod outcome as second eriod incentives also induce effort in the first eriod. Consequently, the economic rediction in Mukoyama and Şahin (2005) is that the first eriod outcome affects the agent s utility less than the second eriod outcome, and may in fact have no effect. In contrast, increasing marginal costs imly that the agent s efforts are substitutes and thus the rediction here is oosite. For more eriods, Fernandes and Phelan (2000) show that the one-shot-deviation (OSD) condition is violated. That is, there are contracts in which the agent has a rofitable multi-eriod deviation but no rofitable single deviation. Because the standard recursive methods assume the OSD condition holds, they cannot be used to characterize the otimal incentive comatible contract. To alleviate this roblem, Mukoyama and Şahin (2005) 4 consider a long term relationshi in which the agent s actions only affect two eriods (the current and the next). DeMarzo and Sannikov (2008) study a setting that is very similar to the one studied here, but consider only the aggregation roblem (as in Holmstrom and Milgrom (1987)). The additional aggregation roblem changes the analysis and results. 5 An earlier aroach to dealing with changing costs, dating back to Berliner (1957) and Weitzman (1980) is to consider any attemt to adjust the otimal contract to the increasing difficulty as a form of ratcheting which should not be rofitable. The intuitive argument is that the agent will always have a rofitable way to game the 4 see also Tchistyi (2006). 5 Other models either focus on dynamic ersistance of rivate information that is not controlled by the agent (Williams (2011)) or on settings in which the surlus deends on the rivate history but the roduction ends after the first success (see Bergemann and Hege (2005), Bonatti and Hörner (2011) and Halac, Kartik, and Liu (2012)).

5 DYNAMIC COSTS AND MORAL HAZARD 5 system. This intuition, echoed in the cited emirical literature, imlies that linear contracts should be otimal. Our analysis shows that this intuition is only half right. If we limit attention to contracts that require the same amount of total effort regardless of early outcomes, as these early aers (imlictly) did, there are, indeed, no gains from ratcheting. However, if we allow contracts to also adjust the required effort, as modern dynamic contracts do, then ratcheting not only increases the rincial s exected rofit, but in some cases (see the second examle of section 5.2) increases also the agent s exected rofit and thus total surlus. Relative to this literature, the current aer s contribution is a characterization of the otimal contract for settings with increasing marginal cost. Allowing this richer roduction setting results in contracts that are consistent with emirical observations. In articular, the contracts in the existing literature eventually rovide the agent a unique iece-rate that generates the first best continuation. That is, a contract in which the agent may receive a 5% comission in some realizations and a 15% comission in another is difficult to exlain using the existing static or dynamic contracts. 6 In ractice, incentive contracts frequently do have several commission levels (see e.g. Joseh and Kalwani (1998) and Larkin (2007)). Moreover, the existing dynamic contracts all redict that once the agent is aid, the ex-ost continuation is first best. While the rediction is hard to test directly, it is difficult to coencile with the observed convex incentive schemes. To confront the challenge of ersistent rivate information, this aer introduces a reformulation of the dynamic contract roblem that is based on duality. This reformulation rovides additional characterization of the otimal contract. In articular, it roves that while some IC contracts may violate OSD, the otimal contract does not. To understand the dynamic dual intuition, suose that before signing the contract, the rincial and agent learn that some third arty will have the ower to sto the contract at history h. The rincial must decide, at this early stage, how much to send now to revent this termination. The dual value of history h is the largest number that the rincial would ay. To determine this dual value, we must consider (a) the exected continuation revenue and (b) the exected cost of incentivizing the agent to exert the required continuation 6 Allowing for rivate information on the agent s side of his initial roductivity introduces a menu of such contracts, while here the single contract offers several comission levels.

6 6 GUY ARIE effort. However, these do not cature all the imlications. If everyone knows ex-ante that the contract will terminate at history h, the utility rovided to the agent starting in h from the otimal contract can no longer rovide incentives (or disincentives) for effort. For examle, in most contracts, the agent is not aid for succcess in the first eriod. Instead, he is rewarded by a better contract following success than following failure. If the agent knows that the third arty will terminate the contract after the first success, giving him zero continuation utility, he may well require ayment for the first success. Thus, the dual value must also consider (c) the effect of the agent s continuation utility on revious histories. The three considerations above are common to all dynamic moral hazard settings. The current setting adds (d) the effect of the agent s continuation cost gain on revious history. That is, if the contract terminates, any rivate information gains to the agent from shirking in the ast to obtain a lower cost today are destroyed. If the agent knows this ex-ante, his incentives to shirk are weaker, reducing the rincial s costs. The total dual value of the history accounts for all four elements (a) (d) above. In contrast, suose the third arty termination threat is comletely unexected by both arties and just aears in history h. In this case the rincial would ay the u to her exected continuation rofit from history h. The value of avoiding this unexected termination is given by the standard dynamic moral hazard value (cf. Sear and Srivastava (1987)). The difference is illustrated best by roosition 4, which shows that the standard value may be negative esecially when dual value is all the remaining surlus. To determine the dual value, the standard romised utility state variable (cf. Sear and Srivastava (1987)) is transformed to the marginal cost to the contract of the agent s utility. This exactly catures (c) above. In addition, a state variable indicating the marginal cost to the contract of the agent s rivate information is used to cature (d) above. Both of these costs are derived by aggregating the shadow rices (multiliers) on the incentive constraints in the receding histories. Intuitively, the agent s utility in history h either relaxes or constraints each of the receding ICs. For a history h that receded h, the IC s shadow rice (along with the coefficients in the IC itself) determines the marginal cost to the contract of changing the utility in h. Aggregating the shadow rice over all the histories that recede h rovides the required state variable for (c). The same intuition rovides the state variable for (d). To rove it is sufficient to consider only OSD, the dynamic dual analysis considers

7 DYNAMIC COSTS AND MORAL HAZARD 7 a change that increases the exected rofits relative to the otimal contract. This change must violate some incentive constraint, otherwise the original contract would not be otimal. If the constraint that is violated is always a one-shot-deviation constraint, then the otimal contract subject only to OSD must be IC. As the dual state variables reflect costs, the dual value is monotonic in each state variable. 7 In addition, in our setting the costs and corresonding state variables are substitutes. This allows roving stronger results than tyical for the otimal contract, and in articular the OSD result. Duality based aroaches are widely used in economic modeling, dating back to Rockafellar (1970). Vohra (2011) extends the analysis of static adverse selection models by analyzing the dual of the classic adverse selection roblem. Marcet and Marimon (2011) and Mele (2011) consider shadow multiliers in a dynamic setting that can be alied to moral hazard roblems. The formal analysis however assumes the OSD assumtion holds. Abraham and Pavoni (2008) use a mixture of a shadow variable and the romised utility to construct a recursive model of savings and consumtion. Their aroach however relies on a numerical rocedure to verify ex-ost that the first order aroach is valid. The methodological contribution of the aer is a recursive dual formulation that is intuitive, tractable, catures additional frictions (the information rent) and can be extended to other settings. In articular, settings without a roof for sufficiency of OSD. Section 2 lays out the dynamic roduction model. Section 3 rovides the dual form and its economic interretation. The otimal contract is characterized in section 4. Section 5 considers some secific extensions and illustrative examles. Section 6 concludes. 7 Standard dynamic moral hazard analysis uses the agent s continuation utility as the state. If the agent s continuation utility is exactly his outside otion, the contract must tyically terminate and the rincial obtains his outside otion. If the agent s continuation utility is very high, the rincial must either give away the firm (in limited liability), or rovide the agent costly insurance. In all cases, the rincial s exected continuation value is highest for some exected agent s continuation utility between the two extremes and is thus non-monotonic. One way around this (with its advantages and disadvantages) is to reformulate the roblem as maximizing the total surlus (see e.g. Clementi and Hoenhayn (2006)).

8 8 GUY ARIE 2. MODEL 2.1. Setu and Primitives There is a rincial and an agent, both risk neutral. Both have an outside otion set to zero. The agent has limited liability i.e. money can only be transferred to the agent. Time is discrete. In each eriod the agent either works or not. The agent s work is costly to the agent and unobservable to the rincial. The cost of effort in a eriod is c n for a commonly known function c : N R + where n denotes the number of actual eriods of work. That is, for the first eriod of work, n is one. For the second eriod of work, n is two, and so on. If the agent shirks in the first eriod, n in the second eriod is still one. The analysis will focus on the case that c n is an increasing and convex function. However, the dual methodology that is develoed does not deend on any of the assumtions on c. Assumtion 1 number, n. The agent s cost, c n, is increasing and convex in the work eriod Both the rincial and the agent observe the outcome of each eriod. 8 To simlify the exosition, a eriod s roduction outcome is either success or failure, denoted by y Y = {0, 1}. Aendix B.1 shows the results extend directly to any countable set of ossible outcomes. The rincial earns a revenue of v from each success (y = 1) and zero from a failure (y = 0). The robability of success (res. failure) in a eriod in which the agent works is (0, 1) (res. 1 ). If the agent does not work, is relaced with 0 [0, ). To revent the rincial from making free rofits, assume the rincial incurs a cost of v 0 for every eriod in which the contract is still active. 9 As costs are increasing, the surlus from working becomes negative after enough effort was exerted. Let N F B denote the maximum number of eriods in which consecutive work increases surlus: N F B = max n : c n v ( 0 ). The increase in costs imlies that an infinite contract is never otimal. The exosition is simlified by assuming that the agent and rincial do not discount the future. 8 This assumtion is common to dynamic moral hazard models. Allowing the agent to deviate by shifting one eriod s outcome to a later generates additional interesting questions also for the standard models. These remain for future research. 9 This assumtion only simlifies the exosition and is without loss of generality.

9 DYNAMIC COSTS AND MORAL HAZARD 9 Aendix B.2 shows that adding a discount factor amounts to a simle accounting exercise. The following observation simlifies the exosition and notation. All formal results without roof in the text are roved in the aendix. Lemma 1 All contracts that are incentive comatible satisfy individual rationality. There is an otimal contract in which: 1. The agent works for at most N F B eriods 2. The required work decision is a stoing decision: if the agent is ever asked not to work, the contract terminates. 3. The agent is never aid in a eriod with a failure or without required work. The first two are standard and the last is a direct outcome of risk neutrality. Note that the second statement does not rule out randomization of the stoing decision. Given lemma 1, the sace of contract relevant ubic histories H is the sace of revious outcomes: H = N F B n=0 Y n. A ublic history h H denotes a sequence of outcomes. Part two of lemma 1 imlies that if the contract is not yet terminated, the agent was asked to work in all ast eriods. However, only the agent knows in which eriods he actually did work and in which eriods he shirked. As the cost to the agent of working in a eriod is a function of the number of eriods in which the agent actually worked in the ast, the only information in the agent s rivate history that is ayoff relevant is the number of ast shirks: Definition 1 number of ast shirks s. The agent s rivate history (h, s) is the ublic history h and the Let n h denote the number of the eriod just after history h. If the agent did not deviate in the ast, his cost of work in this eriod will be c nh. However, cost deends on the rivate history. With a slight abuse of notation let c h s c nh s denote the cost for any history h with ast deviations s and c h c h 0. As the difference in cost between two work eriods will lay an imortant role, let d h s c nh s c nh s 1 denote the

10 10 GUY ARIE cost difference between the current and revious eriods if the agent shirked s times in the ast and d h d h 0. To simlify the notation later on, set d 1 = c 1. The analysis makes extensive use of histories following and receding other histories. Let h = h 1, h 2 denote the history h 1 followed by the history h 2. That is, the sequence of outcomes h 1 haened and then the sequence h 2 haened. For examle, if the current history is h, then the next history will be either h, 1 or h, 0. Say that the history h 1, h 2 follows history h 1 and denote the follows relation by. That is: h h ĥ H : h = h, ĥ. As the set H includes the emty set, h h The Contract There is no loss in considering only contracts that secify for each eriod a work decision and a wage based on the eriod s ublic history. This section defines the contract using a simle transformation of the common decision variables. This transformation will allow a linear formulation of the roblem without affecting the interretation of the resulting contract. Tyically, the contract secifies for each history whether the contract terminates in the corresonding eriod and the resulting wage. Let (1 a h ) denote the robability that the contract is terminated in history h, if h is reached. Let W h denote the wage aid for success in the eriod. The ex-ante robability that the contract would still be active after a success is a a {1}. To linearize the formulation, the contract will use the ex-ante robability that the rincial did not decide to terminate the contract yet instead. This robability, denoted q h, can be defined recursively from any a and vice versa: (2.1) q a, and q h,y q h a h,y The ayment W h is only aid if the contract was not terminated by eriod h. Thus, the exected for success in a history is W h q h, which is again non-linear. However, by lemma 1, there is no loss of generality in having the contract secify the wage for success in the history, conditional on the contract not terminating yet: 10 (2.2) w h q h W h. Definition 2 A contract is a air of functions q, w, with q h secifying for each 10 See Abreu, Milgrom, and Pearce (1991) for a similar linearization.

11 DYNAMIC COSTS AND MORAL HAZARD 11 history h the cumulative robability that the agent will be still be asked to work in the eriod (equations 2.1); and w h secifying the wage for success, weighted ex-ante by the robability q h (equation 2.2). Remark 1 As the agent and rincial are risk neutral, there are infinitely many equivalent ways for the otimal contract to ay the agent w dollars. Paying a dollar for success today is equivalent to aying 1 more dollars for success tomorrow (assuming the agent is asked to work). To remove this technical dulication of the otimal contracts, the analysis assumes the otimal contract makes ayments as early as ossible. That is, from all contracts that secify the same work lan, the otimal contract is the one that ays most to the agent as early as ossible. If the agent comlies with the contract (i.e. works when asked to), the exected continuation rofit for the rincial starting from a history h is given by: (2.3) V h = q h ( 0 ) v w h + V h,1 + (1 ) V h,0. The only deviation from the standard formulation is that q h and w h also include the robability that the contract was terminated at any history before h. The sum ( 0 ) v is the exected revenue from work, the agent s exected ayment is w h and the last two terms are the continuation value. A similar exected value can be defined for the agent. Letting U h be the agent s exected continuation utility from comlying with the contract in all remaining eriods if he never shirked in the ast, we have: U h = w h q h c h + U h,1 + (1 ) U h,0 However, if the agent did shirk in the ast, his exected rofit increases. The aroach introducd by Fernandes and Phelan (2000) suggests defining the agent s exected utility conditional on s ast deviations. We adot a slightly different aroach, identifying searately the agent s extra gains from revious shirks. In articular, let D h s denote the increase in the agent s exceted utility starting in rivate history (h, s) if he would have made one more shirk at some oint in the ast. That is D0 h is the agent s extra gains starting at ublic history h if he would have made exactly one shirk in the ast, D1 h is the agent s gain from making a second shirk in the ast, and so on. As the additional gains are only a result of the cost difference, we have that: D h s = q h d n s + D h,1 s + (1 ) D h,0 s

12 12 GUY ARIE The agent s exected utility conditional on s ast deviations, denoted U h s is: U h s+1 = U h s + D h s with U h 0 = U h. The otimal contract chooses q, w that maximize V subject to incentive comatibility (IC) and individual rationality (IR). As the agent can always choose to sto working IR is imlied by IC and thus will be subsequently ignored Incentive Comatibility To construct the IC constraints, first assume that the OSD condition holds. That is, it is necessary and sufficient that the agent does not have a rofitable single deviation. As in related treatments, I will refer to the relevant constraints to revent one-shot deviations as Local Deviation Incentive Constraints (LDIC). The LDIC for history h requires that whenever the agent considers a one and only deviation, his exected continuation utility is lower than from comlying with the contract in all future eriods ( U h). If the agent deviates in history h, his exected ayment in the eriod is 0 w h. The agent exects to succeed and transition to the history h, 1 with robability 0. As the agent is considering a one-and-only deviation, his exected continuation utility after shirking and succeeding is U h,1 + D h,1 0. Similarly, with robability (1 0 ) the agent exects the contiuation utility U h,0 +D h,0 0. The LDIC is therefore (2.4) (LDIC) w h q h c h + U h,1 + (1 ) U h,0 ( ) ( ) 0 w h + 0 U h,1 + D h,1 0 + (1 0 ) U h,0 + D h,0 0 It is clear that IC imlies LDIC. However, as in Fernandes and Phelan (2000), in the current model, LDIC does not imly IC. Lemma 2 A contract may satisfy all LDIC but violate IC. The roof constructs a contract in which all LDIC hold with equality while the agent s otimal lan is to shirk in the first two eriods. LDIC is not sufficient because some contracts may require more work in some histories that follow a failure than other histories that follow success. As a result, the rivate information (lower costs) gains from shirking and failing may be too large. To rove that the otimal contract subject to LDIC is indeed IC, consider another set,

13 DYNAMIC COSTS AND MORAL HAZARD 13 the Final Deviation Incentive Constraints (FDIC). The FDIC require that whenever an agent who ossibly shirked in the ast considers one final shirk, he refers to follow the contract in all later eriods. Alying the same logic as for the LDIC, obtains the following formulation of the FDIC for rivate history (h, s): (FDIC) (2.5) ( w h q h c h s + U h,1 + ) ( s 1 j=0 D h,1 j + (1 ) U h,0 + ) s 1 j=0 D h,0 j ( 0 w h + 0 U h,1 + ) ( s j=0 D h,1 j + (1 0 ) U h,0 + ) s j=0 D h,0 j The FDIC 2.5 reflects the work/shirk tradeoff for an agent that already shirked s times and is considering a final shirk. Note that the sums go only to s 1 if the agent does not shirk (the to line) and to s if the agent does shirk (the bottom line). The next lemma verifies the relation between FDIC and IC: Lemma 3 If a contract is FDIC it is IC The intuition for the lemma is simly that any rofitable deviation lan must have a rofitable last deviation. However, FDIC is a stricter condition than IC. For examle, it may be that the first deviation was more costly to the agent than the gain from the second deviation. As FDIC is stricter than IC which in turn is stricter than LDIC, the following corollary follows: Corollary 1 If every otimal contract subject to LDIC satisfies FDIC, then every otimal contract subject to LDIC is otimal subject to IC. The roof for sufficiency of LDIC will show that the condition of corollary 1 does in fact hold. For this, we must analyze the FDIC roblem The FDIC Problem Problem 2.6 is the dynamic FDIC roblem. To save on notation, D stands for the vector of D s for all relevant rivate history s values and a suerscrit y denotes the

14 14 GUY ARIE ossible outcomes. ( The shadow cost for each constraint is rovided as well: V n, q, U, D ) (2.6) = max (U y, D y,q,w) 0 q ( 0 ) v w +V (n + 1, q, U 1, D ) 1 + (1 ) V (n + 1, q, U 0, D ) 0 subject to Probability (µ) q q F DIC s n 1 (λ s ) Constraint 2.5 Regeneration U : (γ) U = w qc n + U 1 + (1 ) U 0 Regeneration -D s n 1 (δ s ) D s = qd n s + D 1 s + (1 ) D 0 s The otimal contract solves max V (1, 1, U, D) U 0, D 0 There are three non-standard elements in roblem 2.6. First, the variable q and the accomanying robability constraint. In the standard notation, the uer bound on the robability of work is 1. Here, as q h identifies the ex-ante robability, the uer bound is simly the revious eriod s robability, which may be lower than 1: 0 q 1, and 0 q h,y q h The robability constraint, together with the determination of the next eriod s q enforces this change. The second non-standard element are the FDIC for each rivate history s, instead of just the one LDIC er history. Finally, the regeneration constraints for the shirking gains (D) are added. These are called the threat-keeing constraints in Fernandes and Phelan (2000). The derivation there details the need for these for s = 1 assuming the one-shot-deviation rincile holds. If OSD does not hold, the roblem must have these for all ossible rivate histories, as is the case here. The intuition for the additional regeneration constraints is simle. As the roblem must set some continuation values after deviation in the FDIC, the dynamic roblem must recursively define those values and maintain these just as it does for the original utility in the standard formulation. The Dynamic LDIC roblem is the same as 2.6 with the FDIC only for s = 0 and

15 DYNAMIC COSTS AND MORAL HAZARD 15 the D regeneration constraint only for s = 0. There are several difficulties with the dynamic roblem. First, it is not well defined for all values of U and D. For examle, there is no solution in which U = 0 and D 0 > 0. This comlicates roving even standard results, such as concavity. Second, one cannot rove using this roblem that in the otimal LDIC contract all FDIC are slack, which is critical for the remainder of the analysis. One sub-otimal contract that satisfies LDIC but not FDIC is illustrated in aendix A.2. Formally, the standard line of roof would show that otimality (or first order conditions) imly that the LHS of the FDIC for s + 1 is larger than the LHS of the FDIC for s. Simlifying, this is d n s + ( ( ( Ds 1 Ds 1) 1 +(1 ) D 0 s Ds 1) 0 0 D 1 s+1 Ds) 1 +(1 0 ) ( ) Ds+1 0 Ds 0 The technical challenge here turns out to be that the Ds+1 0 Ds 0 term may violate the inequality if the agent works more after failure than after success, as 1 0 is larger than While the dual analysis below can be used to show that this condition holds in the otimal contract, it will rovide a more direct roof for sufficiency of the LDIC. 3. DUAL FORM The characterization of the otimal contract in section 4 relies heavily on a dual formulation of the roblem that is develoed in this section. Proosition 1 establishes the dynamic dual reresentation of roblem 2.6 is roblem 3.5. The exosition here uses first order conditions and artial derivatives for the formal arguments. This makes exlicit the underlying economics and the (limited) imortance of the linearity of the roblem. For simlicity, the exosition will focus on the LDIC roblem when indicating formal results for history h. Once this is understood, the FDIC dual is a technical generalization. A formal derivation using linear rogramming duality that does not assume differentiability is rovided in the aendix. If differentiability and concavity holds, the Slater Condition (see e.g. Borwein (2005)) can be used instead. In this case, all the develoment in the text alies directly as a roof for roosition 1. Intuitively, consider the otimal contract roblem at history h. The continuation 11 A searate line of roof can be used uniquely for the case that 0 = 0.

16 16 GUY ARIE contract affects the agent s incentives, and the rincial s rofits for the eriods that recede h as well as those that follow it. However, the standard (rimal) recursive value accounts directly only for the continuation rofit. The dual value accounts for the costs and benefits a continuation imoses on all eriods those that recede it as well as those that follow it. The next subsection develos the receding histories cost intuition in detail. To achieve this, the dual analysis leverages two insights that aly to any otimal contract: 1. In any otimal contract, the cost and benefit imosed by any continuation h on history h that recedes it can be infered from history h s ICs and their shadow rices. 2. In any otimal contract, the continuation benefit (or cost) from changing a state variable (e.g. the agent s romised utility) must equal the cost imosed on the receding eriods. Finally, with the comlete effect of the otimal continuation for each history at hand, the standard dual intuition can be alied: The otimal dual value is determined by starting from the best case for the rincial (i.e. agent works for free ) and accounting for the various constraints and costs. The conclusion is the dynamic dual roblem, rovided in roosition Effect on Preceding Histories Dynamic moral hazard analysis finds the best continuation given some fixed ast that is encoded into a state variable. Since Sear and Srivastava (1987), the ast is tyically encoded into the agent s romised utility. In the dynamic dual analysis, we relace the agent s utility as a state variable with the otimal cost to rovide that utility. This state variable allows us to determine directly the otimal continuation utility in history h instead of solving for the otimal continuation given any utility. It is convenient to consider this receding-histories effect as a cost rather than a benefit, this is of course a cosmetic choice and has no formal imlication. In articular, the cost may be negative (i.e. a benefit). Consider a marginal increase to the agent s utility in the second eriod after failure in the first (U 0 ). Suose this increases the continuation rofit in the second eriod by a small ε. Should it be done? To answer this question, we need to also determine the effect on the first eriod rofit. Additional utility after failure decreases the agent s

17 DYNAMIC COSTS AND MORAL HAZARD 17 incentive to work. In articular, if the IC in the first eriod was binding, additional utility after failure imlies that the IC is now violated. For convenience, the LDIC 2.4 for a general history h is reeated here (omitting the h for simlicity): (3.1) (LDIC) w qc + U 1 + (1 ) U 0 0 w + 0 (U 1 + D0) 1 + (1 0 ) (U 0 + D0) 0 A marginal increase in U 0 violates the LDIC at a rate of ( 0 ). Because the agent is indifferent between utility and the ayment for success in the first eriod, the increase in U 0 can be comensated by increasing the agent s ay in the first eriod at the same rate. However, it may also be comensated by increasing the utility after success U 1. The increase in U 1 may increase continuation rofit and thus be more rofitable. It therefore seems as if to determine the correct marginal cost of increasing U 0 we must know the otimal contract after success as well. The first key insight is that in any otimal contract, the correct formalization of the effect of a change in any of the continuation variables is catured by the shadow cost of that eriod s IC and only by it. All that is required to determine the effect of a change in second eriod utility on first eriod rofit is the first eriod IC (or ICs) in which the second eriod variable aears and the correct shadow costs for these ICs. In articular, if λ is the correct shadow cost of LDIC 3.1, an increase in U 0 decreases the otimal value in the first eriod by λ ( 0 ) by construction, regardless of the secific continuation. We can now return to the first question: if a marginal increase in U 0 increases the continuation rofit starting at U 0 by ε and decreases the receding histories rofit by λ ( 0 ) it should be done if and only if (3.2) λ ( 0 ) (1 ) ε. The LHS of 3.2 is the cost imlied on the receding history. The RHS is the change in rofit, accounting for the robability of getting to history U 0. In the otimal contract, the inequality that is equivalent to 3.2 should never be slack in any history. If it is, the contract is imroved by increasing the agent s romised utility. Similar logic imlies that if the inequality is ever violated, the otimal contract is imroved by decreasing the agent s romised utility the cost saving on receding histories is larger than the loss of continuation rofit.

18 18 GUY ARIE This is the second key insight: in any otimal contract, the marginal continuation benefit of increasing the agent s utility must exactly equal the marginal cost imlied on the receding histories. Otherwise, if the former is larger, the otimal contract should romise the agent more utility. If the latter is larger, the otimal contract should romise the agent less. Thus, otimality imlies two insights: cost on receding histories is catured by the IC; and the net effect of any marginal change must be zero when accounting for both the continuation and receding histories. To formalize these insights, observe first that alying the enveloe theorem to roblem 2.6, using the shadow variable γ imlies that 12 (3.3) h : dv du = γ in any otimal contract. That is, the effect of the agent s utility on continuation rofit is γ. The second insight above imlies that in the otimal contract, this is also the marginal cost on the receding eriods of roviding the agent more utility. Again assuming differentiability, and focusing only on the LDIC roblem, the first order conditions for U 1 and U 0 formalize this: dv 1 du + λ ( 0) γ 1 = 0 (1 ) dv 0 du λ ( 0) (1 ) γ = 0 0 The continuation effect (the first term in each), must equal the effect on the current eriod (the second term) and any effect on the revious eriods (which by recursion is catured exactly by γ). In the very first eriod, there are no receding histories and γ = 0. This is equivalent to the standard static condition that the foc is zero. Observe that indeed with the third term exactly zero the equation for U 0 in a two eriod setting is exactly the same as 3.2. The enveloe theorem (equation 3.3) imlies that it is correct to relace dv y du y = γ y. This gives rise to a law of motion for the shadow cost of the agent s utility: γ 1 = γ λ 0 and γ 0 = γ + λ Note again that differentiability is not required for the formal derivation in the aendix but is assumed throughout the exosition for simlification only. Nevertheless, if the original dynamic roblem is diferentiable the develoement here in the exosition rovides a comlete roof of roosition 1, subject to the Slater condition (or another duality restriction).

19 DYNAMIC COSTS AND MORAL HAZARD 19 In any otimal contract, in any history h, the marginal cost on all the receding histories of roviding the agent utility in a history must be γ h. This cost is derived by aggregating all the shadow costs in all the ICs for the histories that receded h, as indicated recursively in the two laws of motion above. Otimality imlies that in any eriod, γ h is also the shadow cost on the regeneration constraint for romised utility. The analysis so far considered only the continuation utility, which is common to our model and all existing models of dynamic moral hazard. Our roblem has another tye of continuation variables the extra rivate gains from shirking D y 0 (D y s in the FDIC). The same logic alies for these as for the continuation utility: Use the IC in all the receding histories to evaluate the receding-history-cost of changing the agent s rivate gains in a eriod (D). In articular, the contribution of each to the IC in every eriod is additively searable. Observe that in the otimal contract, any continuation gains from increasing D must exactly equal the cost imlied on receding histories By the enveloe theorem dv h dd h = δh Alying the enveloe theorem to the first order condition recursively obtains the laws-of-motion: δ 1 = δ + λ 0 δ 0 = δ + λ In the very first eriod, there are no receding histories that can be affected by the change in shirking gains and thus δ = 0. We have shown that the shadow costs on the regeneration constraints, γ and δ here, have an imortant role in the dynamic analysis. These cature the cost of changing their related variable on the receding histories by aggregating the shadow costs of all the receding ICs. In addition, as in every otimal contract, costs must equal benefits, these reflect the benefit of their related variable to the otimal continuation rofit. While the exosition used the LDIC roblem, the only adjustment required for FDIC is to account for the additional ICs. This simly requires adding the related

20 20 GUY ARIE terms. For examle, the foc for U 1 in 2.6 is dv 1 du + n 1 λ 1 s ( 0 ) γ = 0. s= The Dynamic Dual Value Suose that before signing the contract, the rincial and agent learn that some third arty will have the ower to sto the contract at history h with some small robability ε. The dual roblem is to determine the rice er robability unit that the rincial should be willing to ay to reduce ε to zero. This rice is the dual value of the history. From the receding analysis, the answer to this roblem is clear calculate the continuation rofit less the costs imosed on the receding histories. The dual value in a history n with marginal utility and rivate gains costs of γ and δ resectively has the following comonents: 13 The exected revenue increase: v ( 0 ) The marginal cost of the agent s effort c n λ 0 where λ 0 is the shadow cost of the LDIC. For simlicity again the exosition considers only LDICs. The utility cost of the agent s effort on receding histories ( c n ) γ (ayments and their effect on receding histories are considered in the next section). The rivate gains cost of the agent s effort on receding histories, d n δ, reflecting that if the agent would have shirked in the ast, his cost (res. utility) today would have been lower (res. higher) by d n. The continuation dual values µ 1 + (1 ) µ 0, where suerscrits indicate the current eriod s outcome. Formally, this third arty termination is equivalent to an external reduction in the state variable q. The dual value will therefore be catured by µ, the shadow rice on the robability constraint (that is, dv = µ). As q aears with no coefficient in dq the robability constraint, the dual value can be determined using the first order condition with resect to q. Alying recursion, and recalling we only consider the LDIC roblem for which s = 0, obtains: 13 It may be strange that the wage is not considered here directly. Wage is a way of roviding utility. The dual value considers the commitment to rovide the utility, rather than the method of actually roviding it. The wage decision is catured in the next art of the roblem.

21 DYNAMIC COSTS AND MORAL HAZARD 21 (3.4) q foc for LDIC: µ v ( 0 ) c n λ 0 + c n γ d n δ 0 + µ 1 + (1 ) µ 0 The next and final ste verifies that the constraint will bind whenever µ > 0 and determines the eriod shadow costs λ s and the wage The Dual Problem The dual value corresonds to the rice the rincial is willing to ay to revent ex-ante, before roosing the contract, a forced termination in history h. Generally, the dual solution of such maximization roblems starts from the best case outcome and reduces the value to satisfy all the constraints. The shadow cost of effort by an agent that shirked s times in the ast is λ s and in our setting, duality therefore imlies that the foc 3.4 will bind and λ s will be selected to minimize the RHS of the foc. Focusing on the LDIC case with only one shadow cost, λ 0, the contract must find the worst λ 0. If the agent would require no incentives for effort (λ 0 = 0), the dual value would be highest and the rincial would ay highly to revent termination. Unfortunately, the agent requires any incentives he can get. The dual value should therefore the minimum of the right hand side of foc 3.4. In addition, as the rincial can always simly agree to the termination, the dual value can never be negative. Finally, the rincial has one more tool at her disosal to curb the agent s required incentives: ayment for success. In articular, aying the agent relaxes the LDIC at a rate of λ ( 0 ) at a marginal cost to the rincial of. In addition, aying increases the agent s exected utility at a rate of, with an imlied marginal cost to the rincial γ. The otimal contract increases ayment to the oint that relaxing the LDIC is less rofitable than the cost. That is, LDIC wage constraint: λ ( 0 ) + γ. Problem 3.5 is therefore the FDIC dynamic dual (the LDIC dynamic dual is rovided in section 4 after roving that LDIC are sufficient). The dynamic dual roblem finds the shadow costs that minimize the dynamic dual value, subject to the marginal cost state variables derived above with their laws of motion and the wage constraint. The only difference from the LDIC dual is that all the FDICs and their shadow costs λ s must be considered (again using to denote vectors of length n 1). The non-

22 22 GUY ARIE trivial effect of the FDIC is in the law of motion for the information rent. In addition to the rent from the additional cost difference that can be generated this eriod ( 0 λ s as in the LDIC), the last term in the law of motion catures the rent in revious histories that was required to revent the agent from shirking then and lanning to shirk again now. The other changes from LDIC to FDIC are straightforward. 14 (3.5) ( µ n, γ, ) δ = max [ ( 0, min λ 0 µ n, γ, δ, ) ] λ s.t. ( µ n, γ, δ, ) n 1 λ = v ( 0 ) + c n γ (c n s λ s + d n s δ s ) s=0 +µ (n + 1, γ 1, ) δ 1 + (1 ) µ (n + 1, γ 0, ) δ 0 n 1 wage constraint: ( 0 ) λ s + γ s=0 utility costs: γ 1 = γ 0 n 1 s=0 information rent: δs 1 = δ s + 0 λ s 0 λ s ; and γ 0 = γ n 1 j=s+1 δs 0 = δ s λ s ( stoing condition: µ N F B + 1, γ, ) δ = 0 λ j n 1 j=s+1 λ j and n 1 λ s s=0 Proosition 1 Problem 3.5 is the dual of roblem 2.6. In articular µ (1, 0, 0) = max V (1, 1, U, D) = max V (1, 1, U) and U,D 0 U 0 the dual choice variables in every history are the FDIC shadow costs λ s in both standard roblems The roof roceeds by forming the original roblem as a linear rogram, construct- 14 In articular, for both the wage constraint and utility-cost law of motion relace λ 0 with the sum n 1 s=0 λ s. For the objective, each λ s multilies the correct cost given the imlies ast number of shirks and the same for each δ s.

23 DYNAMIC COSTS AND MORAL HAZARD 23 ing it s dual and then alying dynamic rogramming. If the rimal dynamic roblem is known to be concave, differentiable and feasible, the Slater condition (see e.g. Borwein (2005)) can be used instead, following the stes outlined in the text above. The stoing condition reflects the fact that the rincial can always simly agree to terminate the contract. The more comlicated notation is required only to accomodate the s > 0 rivate histories. Because the state sace reflects costs, this formulation has desirable roerties, namely convexity and monotonicity. Intuitively, higher costs are always bad and matter less as costs increase. In contrast, it is well known that in the standard formulation higher utility for the agent may well be required to increase overall efficiency and the rincial s rofits. Thus, the standard dynamic moral hazard roblem is nonmonotonic in its state variable (the agent s romised utility). These roerties are exloited below to rove the one-shot-deviation (OSD) result, and later characterize the otimal contract. Proosition 2 The following hold for the dual roblem 3.5: µ (n, γ, δ) is convex in (γ, δ). ˆµ (n, γ, δ, λ) is convex in λ for every γ, δ. µ (n, γ, δ) decreases in γ and δ s for any s > 0 The first roerty is the mirror image of the concavity roerty of the standard dynamic moral hazard roblem. The last roerty does not have a arallel roerty in the standard formulation. The continuation value for any eriod is almost never monotonic in the agent s romised utility (the standard state variable). In contrast, the state variables γ, δ reflect the entire costs and the dual value µ reflects the full value of the sub-game starting at the history for the rincial. As higher costs reduce the value, the second result is obtained. 4. THE OPTIMAL CONTRACT 4.1. Sufficiency of LDIC This subsection establishes that it is sufficient to consider only local deviations (LDIC). The economic reason that only LDIC should bind in the otimal contract aears simle. If the agent did shirk in the ast, his costs this eriod are lower (c n is increasing) and the effect of his shirking on future costs is lower (d n is increasing). Therefore, shirking in the ast lowers the incentives to shirk and LDIC should be sufficient.