Random Inspections and Periodic Reviews: Optimal Dynamic Monitoring.

Transcription

1 Random Inspections and Periodic Reviews: Optimal Dynamic Monitoring. Online Appendix: Discrete Time Model November 4, 27 In this appendix we consider a discrete version of the model. We show that the solution in the discrete time version has a similar form to the one in the continuous time model and converges to the continuous time policy when the time between periods goes to zero. Remember that the original continuous time problem is τ sup F e rs ux θ s ds+e rτ MU,x θ τ dfτ subject to τ e r+λs τ q dfs. Suppose that the the principal can only monitor at real time τ {,,2,...}. Let δ = e r and β = e λ. That is, δ is the one period discount factor and β is the one period transition probability. With some abuse of notation, let s define the utility function in period t to be ux t = e rs ũ x t e λs +ā e λs ds, where ũ is the flow payoff in the original continuous time version of the model. Notice that we have that x t+ = βā+βx t. Let s denote the realized payoff of monitoring in period t by and notice that V t = Vt where V t = Vτ = τ t δ n ux n +δ t MU,x t, n= e rs ũx θ s ds+e rτ MU,x θ τ

2 We can now write the discrete time version of the problem as max pt V tp t subject to k βδ k q p t+k, t t p t = Let βδ t ψ t be the Langrange multiplier of the constraint at time t. The Langrangian is L = V t p t + ψ t βδ t+k βδ t q p t+k t= t= k We can get replace p = t p t and write the primal optimization problem as max pt t= V t V p t subject to q+ t= βδ t p t βδ t+k βδ t q p t+k, t k= t= p t The Lagrangean for this problem is Lp,η,ψ = qψ +η + T V t V η + βδ t ψ + From here, we get that for all t it must be the case that t= V t V η + βδ t ψ + t k= t ψ k βδ t βδ q k p t k= ψ k βδ t βδ k q, which means that the dual of the optimization problem in is min qψ +η subject to V t V η + βδ t ψ + t k= ψ k βδ t βδ k q, t η, ψ t t 2 We start with the following Lemma characterizing policies that keep the incentive compatibility 2

3 constraint binding. Lemma. Let { t } t be given by if t t t = p if t = q t p + if t +, βδq = q βδt + βδ t qβδ t + + = q βδ βδ t q βδq βδ t qβδ t +, then the incentive compatibility constraint is binding at t = and t + and slack for < t. Alternatively, let {p t} t be given by p t = q βδ βδq q βδq t, then the incentive compatibility constraint is binding at all t. Proof. If the incentive compatibility constraint is binding for t + then it must be the case that k βδ t+k βδ t q p t+k = for all t +. Supposethat p t = α t t p +. Replacing in the incentive compatibility constraint at time + we get k βδ k q α k = which means that and solving for α we get βδα = q α, α = q βδq. 3

4 Next, we determine p and p +. The incentive compatibility at time zero requires that q+ βδ t p t = q+ t= = t= βδ t p t q = βδ t p + = βδ t q + = t= βδ t p t t= + t= + βδ t p t q where we have used the fact that p = t= + p t. We have that Moreover, we have that t= + t= + p t = p + q k= βδq βδq = p + q βδ βδ t p t = p +βδ t + t= + k = p +βδ t + βδq βδ βδ t βδ t p t βδ q βδq k Replacing in the IC constraint we get p + = q βδ βδq βδ t q βδ t qβδ t + We have the following Proposition characterizing the optimal policy Theorem. Let t = max{t : βδ t q}. If V t has a maximum at t t then the optimal policy is deterministic monitoring at the maximum. Otherwise, the optimal policy is the following:. The optimal policy is p t if V V βδ βδ q p t p V t V, t= 4

5 in which case the incentive compatibility constraint is binding at all times. 2. The optimal policy is p t if βδ V V βδ βδq n= V 2 V βδ βδq 3. The optimal policy is t, < t t if V V βδ βδq n= n= p +n p V +n V p +n p V +n V βδ t V V βδ t βδq n= p +n V + V βδ βδq n= +n, V +n V V +n V p +n V t +n V, 4. The optimal policy is if for all t t if t < t q βδ t+ if t = t p t = βδ t βδ βδ t q if t = t+ βδ t βδ if t > t+, V t+ V t > βδ βδq p t t+n p t V t+n V t t n= Proposition. In the limit, when, the optimal policy converges to the one in the continuous time model. Proof. First, take the limit of the policy in Lemma. First, we consider the case with. Let τ = t and replace the expressions for δ and β. First, we get that p τ / τ / = q e r+λτ + e r+λτ qe r+λτ + τ / + = q e r+λ e r+λτ q e r+λ q e r+λτ qe r+λτ +, p τ / 5

6 so taking the limit when we get Second, we get that for τ τ Notice that p τ / τ/ = lim pτ / τ / = qer+λτ lim pτ / τ / + =. q e r+λ τ τ q / e r+λ q q q p τ / τ / + e r+λ τ τ q / e r+λ q t q q + = + q q e r+λ so taking the limit we get that for any t + with τ = t lim p τ / n/ = er+λτ q n τ/ q τ τ τ / q e r+λ q e r+λτ q qe r+λ, e m s τ m dτ = er+λτ q e m τ τ q which verifies that the policy converges to the on in the continuous time model. Similarly, when = the policy is given by p τ/ = q e r+λ e r+λ q e r+λ q q τ/ so for τ = t we get lim p n = n t τ e m s τ m dτ = e m τ τ Finally, notice that for the final case in Theorem we have that p t+p t+ =, τ = t r+λ log q so we have monitoring with probability at time τ. Finally, we can verify that the condition V τ / V τ / βδ βδq V τ / + V τ / βδ βδq n= n= pτ / τ / +n p τ / τ / pτ / τ / +n p τ / τ / V τ / +n V τ / V τ / +n V τ /, 6

7 converges to V τ = r +λ q E[Vτ τ > τ ] Vτ. Proof Theorem The case in which V t reaches a maximum for some t t is trivial as in this case all the constraints are slack, so we only need to consider the case in which V t is increasing for t t. In this case, the proof relies on the theory of weak duality for infinite dimensional linear programming problems Anderson and Nash, 987, Theorem 2.. The policy in Theorem is feasible for the primal problem. We conjecture that the optimal policy takes this form, and construct Lagrange multipliers that are dual feasible. By weak duality, the value of the dual given these Lagrange multipliers provides and upper bound on the value of the primal. We then verify that the value of the dual corresponds to the value of the primal given our conjectured policy which establishes the optimality of our conjectured policy. Remark. The multipliers that we construct in the proof may fail to be bounded. However, because we are using weak duality, boundedness of the multipliers is not required, and we only need the multipliers to be members of the algebraic dual. Unlike in the case of strong duality, weak duality holds for problem defined on the algebraic dual, so there is no need to postulate a topological structure for the optimization problem Anderson and Nash, 987, Chapter 2. We start with some Lemmas that will be useful in the computations. Lemma 2. If u is convex then ux t+ βux t is increasing. Proof. Since β >, the inequality we are claiming is equivalent to Now, using twice the definition of x t, we have: ux t +β ux t++ β +β ux t. 3 +β x t+ + β +β x t = βx t + βa+βx t β In other words, x t is a weighted average of x t+ and x t. Note that the weights are the same as in 3 and hence convexity of u implies that 3 indeed holds. Lemma 3. MU,x t βmu,x t = βmu,ā The algebraic dual corresponds to all the linear functional while the topological dual is the subset of the algebraic dual corresponding to the bounded linear functionals. = x t 7

8 . Construction of the Lagrange Multipliers Let be the the first date at which there is monitoring with positive probability. The objective is to construct multipliers when the incentive compatibility constraint is binding after +. Let s define F t V t V η + βδ t ψ + t k= ψ k βδ t βδ k q, whichcorrespondtothelhsoftheconstraint at timetinthedualproblem. Clearly, themultipliers η,ψ are dual feasible if and only if F t. Taking the difference F t+ F t we get F t+ F t = V t+ V t + qβδ t+ ψ t+ βδ t βδ t+ t k= ψ k where V t+ V t = δ t+ ux t+ +δ t+ MU,x t+ δ t MU,x t If F t = and ψ t+ >, then it must be the case that F t+ =, so F t+ F t =. This requires that qβδ t+ ψ t+ = V t+ V t + βδ t βδ t+ t ψ k > 4 We want to derive a difference equation for ψ t = β t ψ t such that F t = for all t +. From equation 4 we have that if F t+ =, then ψ t+ is given by qβδ t+ ψ t+ = V t+ V t + βδ t βδ t+ t ψ k 5 Considering equation at time t and multiplying both sides by βδ we get qβδ t+ ψ t = βδv t V t + t βδ t βδ t+ ψ k 6 Taking the difference between. and 4, and using the definition ψ t = β t ψ t we get k= k= k= qδ t+ ψt+ = βδv t V t V t+ V t + qβδ δ t ψt Hence, for any t > +, ψ t satisfies the recursion ψ t+ = α ψ t h t 7 8

9 where α qβδ qδ h t q ux t+ βux t + βmu,ā. and α >, h t is increasing by Lemma 2. Solving recursively, we get that k ψ +k+ = α k ψ + α k n h +n+, k. 8 n= The final step is to specify conditions on the initial value ψ + that guarantee that the sequence {ψ t } t + in equation 8 is nonnegative. Lemma 4. Fix t then. If ux + βux + βmu,ā then ψ t for all t + if and only if ψ + β t + q qδ n+ ux +n+2 βux ++n+ βmu,ā n= qβδ 2. If ux + βux + βmu,ā < then ψ t for all t + if and only if ψ +. Proof. Consider an initial condition of the form ψ + = n= α n+h ++n + 9 for some to be determined. Replacing the initial condition in 9 in 8 above, we get ψ ++k = α k k α n+h ++n +α k α k n h ++n = n= α k n h ++n +α k n=k Hence, ψ ++n is nonnegative if = α k n=k α n+h ++n + n=k n= α n+h ++n, k. 9

10 Let s define where h t is increasing. H k n=k α n+h ++n, First, let s consider the case with ux + βux + βmu,ā. Because h ++n is increasing, we have in this case that h ++n is positive for all n, so we need to take =, which corresponds to the condition in the Lemma. Next, we consider the case with ux + βux + βmu,ā <. Let s define t = sup{t > + : h t }. By definition, H k is decreasing for k < t and increasing for k t. This means that H k has a minimum at k t, and so H k has a maximum at k. Thus, ψ ++k is nonnegative if and if Replacing in 9 we get that ψ + = α n+h ++n. n=k n= k n= α n+h ++n α n+h ++n. α n+h ++n n=k By definition of k, h ++n for all n k which means that k n= h α n+ ++n, so it is enough to consider ψ +.2 Verification of Optimal Policy.2. Case: < < t First, we consider a policy in which < < t. The following propositions characterizes sufficient conditions for this policy to be optimal. Proposition 2. Let t = max{t : βδ t q}. Let < t be such that: if = then βδ t V V βδ t βδq n= V + V βδ βδq n= p +n p V +n V p +n p V +n V,

11 and if > then V V βδ t +n V t βδ t βδq +n V n= ux βux + βmu,ā V + V βδ p +n V t βδq +n V. n= If this conditions are satisfied, then the optimal policy is { t } t. Proof. The first condition at time yields V V + βδ t ψ = η. Replacing η in the first order condition at time + yields Solving for ψ and η we get ψ = From here we get that V + V βδ t βδ t + ψ +βδ t + qψ + =. V + V βδ t βδ t + + βδ q βδ ψ + η = V V + +βδ t V + βδv βδ t βδ t + V βδ βδt qψ t βδ + 2 qψ +η = βδ q V + βδ t + q V βδ t βδ t + V + βδβδt q qψ t βδ + 3 Notice that min{t > : βδ t q}, which means that + βδ + q βδ > so the solution always involves choosing the smallest possible ψ +. Suppose that satisfies the conditions in the proposition and consider the multiplier η = V V + +βδ t V + βδv βδ t βδ t + ψ = V + V βδ t βδ t + + βδ q βδ ψ + ψ t =, < t ψ t = β t q V βδ βδt qψ t βδ + qδ n+ ux t+n+ βux t+n+ βmu,ā, t + n= qβδ By Lemma 4, the multipliers {ψ t } t are nonnegative. By weak duality, to verify the optimality of

12 {p t } t, it is enough to verify that the proposed multipliers are dual feasible and that qψ +η equals the expected payoff of { t } t. Step : First, we verify that qψ +η = t= p t V t V. Replacing the expressions for η and ψ we get βδ q V t qψ +η = + βδ t + q V V βδ t βδ t + + βδt q q n+ δ t +n+2 ux t βδ βδ t qβδ +n+2 βux +n++ βmu,ā which can be written as n= 4 βδ q V t qψ +η = + βδ t + q V V βδ t βδ t + βδ t q q n+ + V δβ t δβ t + t qβδ +n+2 +βδv +n+ +βδv +n 5 n= Using telescopic sums, we have that q n+ V +n+2 +βδv +n+ +βδv +n = [ q qβδ n= [ q βδ qβδ n= n+ q V +n+2 qβδ n+ q V +n+ qβδ βδv V + + q βδ qβδ βδ q βδ qβδ n= qβδ [ q n V +n+] n+ q qβδ n ] V +n+ qβδ n= [ n q n+ ] q n V +n] +βδ V t qβδ qβδ +n = n= q n V t qβδ +n+ βδ q βδ q n V t qβδ qβδ +n = n= n= βδ q qβδ q n V t qβδ +n+ = n= βδ q qβδ V V + + q βδ2 q n qβδ 2 V t qβδ +n+ V V + q βδ qβδ n= 2

13 Replacing in equation 5 we get βδ q V t qψ +η = + βδ t + q V V βδ t βδ t + βδ t q βδ q + βδ t βδ t + qβδ V V + + q βδ2 q n qβδ 2 V t qβδ +n+ = n= q βδ βδ t + qβδ βδ t βδ t + V V βδ t q q βδ 2 q n + βδ t βδ t + qβδ 2 V t qβδ +n+. So, after some manipulations and replacing t we get that qψ +η = q βδ qβδ βδ t + βδ t βδ t + V V + = q βδt + βδ t qβδ t + V V + = V V + = n= n= + +n V +n V = βδ t q n= q βδ 2 βδ t βδ t + qβδ 2 βδ t q βδ t qβδ t + q n V t qβδ +n+ p t V t V t= q βδ qβδ q n= q n= qβδ qβδ n V +n+ n V +n+ Step 2: The only step left is to verify that η and ψ +. We can write ψ = + V + V βδ t βδ t + βδ t βδ t + q n+ δ t +n+2 ux +n+2 βux +n++ βmu,ā n= = βδ t βδ t + which means that qβδ βδ qβδ q qψ = βδ t βδ t + V + βδ V + qβδ q βδ 2 βδ t βδ t + qβδ 2 q q n= q βδ 2 βδ t βδ t + qβδ 2 qβδ q n= n V +n+, qβδ n V +n+ 3

14 Using the definition for t we get so q βδ t βδ t + q βδ t βδ t + βδ qβδ q βδ 2 qβδ 2 + = q qψ = q βδ t + pt V + q βδ t q q q βδ t + = q βδ t q, Hence, replacing in qψ +η = t= p tv t V Step we find that η = qβδt + q βδ t + pt V V = V V βδ t βδ t qβδ t + n= In the case of ψ +, notice that if and only if β t + q n= βδ t βδ t qβδ t + βδ t qβδ t + βδ t q +n V t +n V ++n V +n+ n= ++nv +n+ qδ n+ ux +n+2 βux ++n+ βmu,ā n= qβδ q n+ δ t +n+2 ux +n+2 βux ++n+ βmu,ā n= qβ = βδ q qβδ V V + + q βδ2 qβδ 2 V V + + βδ qβδ n= q n= qβδ p +n V +n V n V +n+ Step 3: Feasibility for t <. If >, then we We need to verify that If we replace η and ψ we get V t V η + βδ t ψ, t <. V t V + βδt βδ t βδ t βδ t + V + V + q βδ t+ βδ t + ψ t βδ + 6 4

15 Evaluating 6 at time and simplifying we get βδv V +V + V + βδ q βδ t βδ t + βδ ψ +. If we replace βδv V +V + V = δ t ++n ux +n+ βux +n+ βmu,ā and ψ + we get qδ n ux +n+ βux +n+ βmu,ā n= qβδ Feasibility for t < follows from the following lemma Lemma 5. If ux βux + βmu,ā then V t V η + βδ t ψ, t <. Proof. We prove the statement by induction. Let F t = V t V η + βδ t ψ an consider periods t+, t and t, for t. Then, we get βδf t F t = βδv t V t + βδ t+ βδ t ψ F t+ F t = βδv t+ V t + βδ t+ βδ t ψ, so taking difference we get βδf t F t F t+ F t = βδv t V t V t+ V t = ux t+ βux t + βmu,ā Given that ux t+ βux t + βmu,ā, we get that βδf t F t F t+ F t which means that F t F t. The results follows from the fact that F = and that ux t+ βux t + βmu,ā is increasing. 5

16 Notice that ψ + = Suppose that β t + q qδ n+ ux +n+2 βux ++n+ βmu,ā n= qβδ qδ n+ ux +n+ βux +n+ βmu,ā n= which is equivalent to qβδ qδ n ux +n+ βux +n+ βmu,ā. n= qβδ If this conditions are satisfied we get that qδ n+ ux +n+ βux +n+ βmu,ā n= qβδ qδ n ux +n+ βux +n+ βmu,ā, n= qβδ which is equivalent to ux + βux + βmu,ā. By Lemma 2, the right hand side is monotonic, and this means that ux βux + βmu,ā. Moreover, following similar computations as the ones we did to compute qψ +η we get that qδ n+ ux +n+ βux +n+ βmu,ā n= qβδ is equivalent to V V βδ βδq n= p +n V t +n V.2.2 Case: p > Next, we consider the case where p >. In this case, t p t < so η =. 6

17 Proposition 3. Suppose that then the optimal policy is {p t } t V V βδ βδ q p t p V t V Proof. Because p >, the constraint t p t is slack and η =. Consider the multiplier at t = ψ = = βδ q t= qδ n+ δ n+2 ux n+2 βux n++ βmu,ā n= qβδ qβδ V βδ q V + q βδ2 qβδ 2 βδ q and from the first order condition we have that From here, we get that ψ = V V βδ +ψ βδ q. βδ q n= qβδ qψ +η = qv V βδ + βδ q 2 βδ qβδ V βδ q V + q βδ2 qβδ 2 βδ q = q βδq V + q βδ q n+ V n+ qβδ qβδ n= = p tv t V t= n V n+, q n= qβδ n V n+ which verifies that the value of the dual problem equals the value of the primal. We only need to verify that ψ so the constructed multipliers are dual feasible. ψ = = qβδ V βδ q V + βδ βδ q 2 [ βδq which means that ψ if and only if βδ βδ q V V + βδ βδ q p t t= p t V t t= ] p V t V, p t p V t V V V t= 7

18 .2.3 Case: p t + p t+ = Finally, we need to consider the case in which for all < t qδ n+ ux +n+2 βux +n++ βmu,ā 7 n= qβδ which means that theconditions inpropositions 2 and3are not satisfied. In thiscase, we consider a policy such that the incentive compatibility constraint constraint is binding at time zero, p t +p t+ = and the incentive compatibility constraint is slack at t, t+, which yields ψ = V t+ V t βδ t βδ t+ 8 βδ t η = V t V βδ t βδ t+ V t+ V t 9 This means that the probability of monitoring at time t is p t = q βδ t+ βδ t βδ Because the inequality 7 is satisfied, we can take ψ t = for all < t < t and satisfy all the complementary constraints at < t < t. We can pin down the multipliers ψ and η using the first order conditions at time t and t+. Replacing in the first order conditions at time t+2, we get that the complementary slackness condition is satisfied for ψ t+2 = if and only if +βδv t+ V t+2 βδv t = ux t+2 βux t++ βmu,ā >, which is necessarily the case if qδ n+ ux t+n+2 βux t+n++ βmu,ā. n= Finally, we verify that qβδ V t+ V t βδ t qψ +η = q βδ t βδ t+ +V t V βδ t βδ t+ V t+ V t βδ t q = βδ t βδ t+ V t+ V t+v t V = p tv t +p t+v t+ V. 8

19 References Anderson, E. J. and P. Nash 987. Linear Programming in Infinite Dimensional Spaces: Theory and Applications. New York: Wiley. 9