Optimal Dynamic Mechanism Design and the Virtual Pivot Mechanism

Transcription

1 Optmal Dynamc Mechansm Desgn and the Vrtual Pvot Mechansm Sham M. Kakade Ilan Lobel Hamd Nazerzadeh March 25, 2011 Abstract We consder the problem of desgnng optmal mechansms for settngs where agents have dynamc prvate nformaton. We present the Vrtual-Pvot Mechansm, that s optmal n a large class of envronments that satsfy a separablty condton. The mechansm satsfes a rather strong equlbrum noton (t s perodc ex-post ncentve compatble and ndvdually ratonal). We provde both necessary and suffcent condtons for mmedate ncentve compatblty for mechansms that satsfy perodc ex-post ncentve compatblty n future perods. The result also yelds a strkngly smple mechansm for sellng a sequence of tems to a sngle buyer. We also show the allocaton rule of the Vrtual-Pvot Mechansm has a very smple structure (a Vrtual Index) n mult-armed bandt settngs. Fnally, we show through examples that the relaxaton technque we use does not produce optmal dynamc mechansms n general nonseparable envronments. 1 Introducton We study the problem of desgnng optmal mechansms for envronments wth dynamc prvate nformaton and propose a mechansm that s proft-maxmzng n a class of envronments that we call separable. In a separable envronment, the valuaton functon of an agent can be decomposed as the product (or sum) of a functon of the agent s frst sgnal and another functon of the agent s future sgnals. As an example, consder a manufacturer that sells to several retalers. Each retaler has two peces of prvate nformaton: her proft margn (whch she knows a pror) and the demand she faces (whch she learns over tme and could potentally be tme-varyng). Another compellng applcaton s that of onlne advertsement auctons where a publsher sells the space on her webste to advertsers. The advertsers usually know a-pror ther proft margns on Department of Statstcs, Wharton School, Unversty of Pennsylvana skakade@wharton.upenn.edu. Work performed n part at Mcrosoft Research. Stern School of Busness, New York Unversty lobel@stern.nyu.edu. Work performed n part at Mcrosoft Research. Mcrosoft Research, New England Lab hamdnz@mcrosoft.com We would lke to thank Maher Sad, Gregory Lews, Daron Acemoglu, Susan Athey, Markus Mobus, Mallesh Pa, Andrzej Skrzypacz, and Rakesh Vohra for many nsghtful suggestons and helpful comments. 1

2 each sale, but they estmate over tme the converson rates (fracton of ads that turn nto sales). Ths s also a separable envronment. The optmal mechansm we propose, the Vrtual-Pvot Mechansm, s qute ntutve t combnes deas based on the vrtual value formulaton of Myerson 1981 for statc revenue-optmal mechansm desgn and the dynamc pvot mechansm proposed by Bergemann and Välmäk 2010 for maxmzng socal welfare. The mechansm essentally maxmzes an affne transformaton of the socal welfare whch corresponds to a certan vrtual surplus. Furthermore, the mechansm satsfes strong (perodc ex-post) notons of ncentve compatblty and ndvdual ratonalty. One notable specal case of our results s the settng wth only one buyer. Namely, consder a settng where the mechansm at each perod has one tem to sell to a sngle buyer. The mechansm has a fxed producton cost γ for the tem. Under separablty assumptons, the optmal mechansm n ths settng has a surprsng smple form (wth a smple ndrect mplementaton whch we present later) the mechansm offers the agent a menu of contracts, of the form (p, M(p)) to the agent. If an agent chooses a contract, she wll be charged an upfront payment of M(p) and afterwards the mechansm posts a prce of p > γ at each tme step the agent has the opton to pay more upfront for cheaper prces n the future. Note that even f the agent s valuaton s ncreasng (or decreasng) over tme and the seller s fully aware of ths fact, the optmal mechansm nvolves offerng the tem at all perods at a constant prce p. In the general soluton wth multple buyers, the Vrtual-Pvot Mechansm stll retans ths flavor. Roughly speakng, each agent, based on her ntal type, s assgned a certan weght functon n an affne transformaton of the socal welfare that s maxmzed by the mechansm, see Secton 5.1. The more the agent pays up front the hgher her mportance wll be n the socal welfare functon (leadng to more allocatons to her n the future). Our settng consders a mechansm whch allows agents to report ther type every round. In partcular, ths mples that they are able to re-report all of ther hstorcal prvate nformaton that has bearng on the current and future values. Allowng re-reportng of prvate sgnals s a crucal step n obtanng perodc ex-post ncentve guarantees. Once we obtan perodc ex-post ncentve compatblty for all future perods, we are able to provde necessary and suffcent condtons for ncentve compatblty at the frst perod. We drectly show these condtons are satsfed for our optmal mechansm. Fnally, we provde examples of how the standard relaxaton approach to dynamc mechansm desgn wll not succeed wthout certan addtonal assumptons. 1.1 Related Work Two natural objectves n the dynamc mechansm desgn are maxmzng the long term socal welfare of all buyers (effcency) and maxmzng the long term revenue or proft of a seller (optmalty). Wth regards to maxmzng the long term socal welfare, there are elegant extensons of the effcent (VCG) mechansm to qute general dynamc settngs, ncludng the dynamc pvot mechansm of Bergemann and Välmäk 2010 and the dynamc team mechansm of Athey and Segal 2007 (see also Cavallo et al. 2007, Bapna and Weber 2008, Nazerzadeh et al. 2008). The lterature on dynamc revenue-optmal mechansm has been prmarly focused on settngs 2

3 where the agents arrve and depart dynamcally over tme, but ther prvate nformaton remans fxed, see Vulcano et al. 2002, Pa and Vohra 2008, Sad 2009, Gershkov and Moldovanu 2009, Skrzypacz and Board In these settng, the mechansm desgner faces a dynamc problem, but the ncentve constrants of each agents are essentally statc because agents do not obtan any new prvate nformaton over the course of the mechansm. For surveys on dynamc mechansm desgn see Bergemann and Sad 2011, Parkes We consder a settng where the prvate nformaton of the agents changes over tme, a lne of research that was poneered by Baron and Besanko Courty and L 2000 provde an optmal mechansm for an envronment where agents have prvate nformaton about the future dstrbuton of ther valuatons. Battagln 2005 studes a settng wth a sngle agent whose prvate nformaton s gven by a 2-state Markov Chan and shows that the optmal allocaton converges over tme to the effcent allocaton. In contrast to the results n Battagln 2005, n the settngs we consder, the allocaton dstorton generated by the agents ntal prvate nformaton does not dsappear wth tme. For a more detaled dscusson, see Subsecton 5.1. A closely related work to ours s that of Ëso and Szentes 2007 who study a two-perod model where each agent receves a sgnal at the frst perod and the seller can also allow each agent to receve an addtonal prvate sgnal at the second perod. Under certan concavty and monotoncty condtons on the sgnals, they show that the optmal mechansm allows the agents to receve ther second sgnals; however, agents do not obtan any rents from the fact second perod sgnal s prvate. They also propose a handcap aucton for the case where the agents valuatons are gven by the sum of the frst and second perod sgnals. We use smlar deas and show that for a broad class of envronments, the seller s able to extract the nformaton rent assocated wth all sgnals except the ntal one, even f the seller does not control the agents ablty to obtan further prvate sgnals. However, as we show n Secton 7, there exst dynamc settngs where the seller cannot extract the entre nformaton rent from future sgnals. We also note the work n Deb 2008, whch provdes an optmal mechansm n a settng wth only one buyer where the value s Markovan n the prevous value, among other techncal condtons. Another closely related work to ours s by Pavan et al. 2008, 2009, whch presents a comprehensve characterzaton of necessary condtons for ncentve compatblty n both fnte horzons (Pavan et al. 2008) and nfnte horzons (Pavan et al. 2009). Fndng suffcent condtons for ncentve compatblty turns out to be a dffcult challenge, as Pavan et al acknowledge. 1 A settng where they show ncentve compatblty (Proposton 12) s one where types evolve accordng to an AR(k) auto-regressve stochastc process wth non-negatve coeffcents, the set of feasble actons of the mechansm s a lattce (therefore excludng problems such as allocatng prvate goods between two or more agents) and actons do not affect the evoluton of types. 2 See Subsecton 3.3 for further dscusson on how our methodology relates to pror work. 1 As for ncentve compatblty n perod one, we were only able to check t applcaton-by-applcaton, but we have been able to verfy t n a few specal settngs. (Pavan et al. 2008, page 4) 2 See Assumptons SCP and DNOT, both used n provng Proposton 12 n Pavan et al

4 1.2 Organzaton We organze our paper as follows. In Secton 2, we formalze our model, defne concepts such as ncentve compatblty and optmalty of mechansms. In Secton 3, we dscuss our approach for desgnng optmal mechansms. In partcular, both necessary and suffcent constrants for ncentve compatblty are provded here. We ntroduce the noton of separablty n Secton 4 and provde an upper bound on the revenue of any mechansm n a separable envronment. In Secton 5, we propose our mechansm and states our man optmalty result. Specal cases (ncludng the settng wth only one buyer) are consdered n Secton 6. Secton 7 provdes smple examples showng how the usual ncentve constrants from statc mechansm are nsuffcent for the dynamc case. It also shows that wthout our separablty assumptons, the partcular relaxaton approach we take s nsuffcent. The Appendx contans all the proofs. 2 Prelmnares In ths secton, we formalze our model and defne concepts such as ncentve compatblty and optmalty of mechansms. 2.1 The Dynamc Envronment We consder a dscrete-tme, δ-dscounted nfnte-horzon (t = 0, 1, 2,...) model that conssts of one seller and n agents (buyers). The seller decdes upon an acton a t at each perod t among the feasble set of actons A t, at a cost of c t (a t ) to the seller, where a t = (a 0, a 1,, a t ) represents all the actons taken by the mechansm up to tme t. At every perod, each agent {1,..., n} receves a prvate sgnal s,t S,t. In partcular, we make the followng assumpton about the frst sgnal s,0 throughout the paper: Assumpton 2.1. For each agent, s,0 0, 1 s real valued and dstrbuted accordng to F. Furthermore, assume that F s strctly ncreasng and has a densty, whch we denote by f. Ths frst sgnal summarzes all the ntal prvate nformaton of the agent (whch has bearng on her entre stream of valuatons). Furthermore, for all t 1, each agent also receve a prvate sgnal s,t S,t here we not concerned wth whether or not these future sgnals are real or not (the set S,t s arbtrary for t 1). The type of agent at tme t s the sequence of sgnals of the buyer up to (and ncludng) tme t, whch s denoted by s t = (s,0,..., s,t ). The type provdes a summary of all the agent s prvate nformaton whch has bearng on all her current and future valuatons. For notatonal convenence, we let vector s t = {s t } n denote the (jont) types of all agents at tme t. At each perod t, agent obtans value v,t (a t, s t ), whch s a functon of her type and the seller s past and current actons. We assume quas-lnear utltes and denote the payment of agent at tme t by p,t, so that the (nstantaneous) utlty of agent at tme t s gven by u,t = v,t (a t, s t ) p,t. We also assume throughout the followng regularty condton. 4

5 Assumpton 2.2. The partal dervatve v,t(a t,s,0,...,s,t ) s,0 bounded by V <. exsts for all, t, a t, and s t, and t s We now specfy the stochastc process over the sgnals. The sgnal s,t that agent receves at tme t may be correlated to her prevous sgnals s,0,..., s,t 1 and the past actons of the seller a 0,..., a t 1, but t s ndependent (condtonally on the seller s actons) of all sgnals of the other agents. Formally, the stochastc sgnal s,t s determned by the stochastc kernel K,t (s,t a t 1, s t 1 ). Wthout loss of generalty, we make the assumpton that the frst sgnal s ndependent of the future sgnals: Assumpton 2.3. For each agent, the dstrbuton of the ntal sgnal s,0 s ndependent of the future sgnals s,t for t 1. In fact, wthout loss of generalty, one can assume that all the sgnals are ndependent (as noted by Pavan et al ). Even under ths assumpton, mportantly, note that the value of agent at any future perod (t 1) may stll be correlated wth the sgnal s,0. Here, we only explctly assume s,0 to be ndependent of the future arbtrary dependences among future sgnals are permtted. Whle ths assumpton s wthout loss of generalty, an example n Secton 7 suggests that there are techncal reasons for whch ths s formulaton natural (n partcular, ths example shows how the relaxaton approach we take may fal n a dfferent representaton). We also assume the mechansm has the ablty to exclude agents from the system at tme t = 0. That s, t can select a subset of the agents that wll obtan no value (and wll not make payments) at any perod t 0. The excluson of an agent from the system does not mpact the value obtaned by the other agents f the mechansm stll takes the same sequence of actons a 1,..., a t. Assumpton 2.4. The set of feasble actons A 0 at tme t = 0 s equal to 2 {1,,n}, that s, the set of all subsets of {1,.., n}. If / a 0, then agent s excluded from the system,.e., p,t = 0 and v,t (a t, s t ) = 0 for all t, at, and s t. No agent obtans mmedate value from the choce of a 0,.e., v,0 (a 0, s,0 ) = 0 rrespectve of whether a 0 or not. Also, the value obtaned by each agent does not depend on the excluson of other agents. In addton, the cost ncurred by the mechansm only depends on the actons not on the excluded agents. The assumpton mples that for any par of actons a 0, a 0 n A 0 such that a 0 and a 0, the value v,t (a 0, a 1,, a t, s t ) = v,t(a 0, a 1,, a t, s t ) for all t, a 1,..., a t, and s t. Also, c t(a 0, a 1,..., a t ) = c t (a 0, a 1,..., a t ) for all t of course, excluson of an agent may change the choce of the actons taken by the mechansm. The assumpton that the agents do not obtan value at t = 0 s made wthout loss of generalty and for smplcty of presentaton. Nevertheless, the mechansm may charge the agents p,t 0 at that tme. The above assumpton smplfes satsfyng the partcpaton constrants. For example, f an agent only obtans negatve values from the actons, she would be excluded from the mechansm. Observe that f the actons taken by the mechansm correspond to allocatons of tems to agent, ths assumpton can be smply satsfed. 3 The argument essentally follows from ths observaton: any stochastc process {X t} t N may be smulated by a functon f such that x t+1 = f(x 1,..., x t, z t) where the z t s are..d. and unform on 0, 1 (here f s constructed based on the dstrbuton Pr(X t+1 X 1,... X t)); see Pavan et al

6 At each perod t 0, 1. Each agent receves her prvate sgnal s,t K,t ( a t 1, s t 1 ). 2. Each agent provdes a report, ŝ t, of her current type, st = (s,0,..., s,t ), as determned by her prvate hstory h,t. In partcular, ŝ t = R (h,t ). 3. As a functon of the publc hstory, h t, and the current reports, ŝ t, the mechansm determnes the acton a t A t and the payments p,t for each agent. In partcular, a t = q(h t, ŝ t ) and the jont prces are {p,t } n = p(h t, ŝ t ). Fgure 1: A generc mechansm For our theorems to hold, we make further restrctons on the functonal form of the values. In partcular, we wll place a separablty assumpton on the envronment, whch we precsely state n Secton 4. Throughout the paper, suppose Assumptons 2.1, 2.2, 2.3 and 2.4 hold. 2.2 Mechansms, Incentve Constrants, and Optmalty A mechansm M(q, p) s defned by a par of an allocaton rule q( ) and a payment rule p( ). We let Q denote the set of all allocaton rules. By the Revelaton Prncple (cf. Myerson 1986), wthout loss of generalty, we focus on (dynamc) drect mechansms. 4 At each perod t, each agent, makes a report, denoted by ŝ t, of her type st. Usng our standard shorthand notaton, we denote the jont reports of all agents by ŝ t = {ŝ t } n. Note that snce s t = (s,0,..., s,t ) ncludes the set of all sgnals that each agent has receved, each agent re-reports all of ther prevous sgnals at every perod. The report of an agent can be condtoned on the hstory, whch we now specfy. The publc hstory at tme t, denoted by h t, s the sequence of reports and actons of the mechansm untl perod t 1; namely, h t = (ŝ 0, a 0, ŝ 1, a 1,, ŝ t 1, a t 1 ). The prvate hstory of agent at tme t, denoted by h,t, ncludes the publc hstory and her current type (sequence of sgnals she receved up to, and ncludng, tme t),.e., h,t = (s,0, ŝ 0, a 0, s,1, ŝ 1, a 1,, s,t 1, ŝ t 1, a t 1, s,t ). The allocaton and payment rules are functons of the publc hstory at tme t, h t, and the reports of all agents at tme t, ŝ t. The allocaton rule determnes the acton taken by the mechansm and the payment rule determnes the payment of each agent. The reportng strategy of agent, denoted by R, s a mappng from her prvate hstory h,t to a report of her current type ŝ t. Mechansm M and the reportng strategy profle R = {R } n determne a stochastc process whch s descrbed n Fgure 1. We now defne the ncentve constrants of the mechansm. Denote the expected (dscounted) future 4 The Revelaton Prncple mples that an equlbrum outcome n any ndrect mechansm can also be nduced as an equlbrum outcome of an (ncentve compatble) drect mechansm. 6

7 value of agent under the (jont) reportng strategy R n mechansm M by: = E δ t v,t (a t, s t ) V M,R and the expected (dscounted) future utlty (of under R n M) as: = E δ t( v,t (a t, s t ) ) p,t U M,R t=0 t=0 where the expectaton s wth respect to the stochastc process nduced by the reportng strategy and the mechansm. Smlarly, for the expected value and utlty of agent, condtoned on a prvate hstory h,t and type of the other agents s t, we have: V M,R (h,t, s t ) = E δ τ v,τ (a τ, s τ ) h,t, s t τ=t U M,R,t (h,t, s t ) = E δ τ (v,τ (a τ, s τ ) p,τ ) h,t, s t τ=t Note that ths expectaton s well defned (even on prvate hstores whch have probablty 0 under R), snce the reportng strateges are mappngs from all possble prvate hstores of agent (and we have condtoned on the publc hstory and current jont type). Roughly speakng, the noton of ncentve compatble s one n whch no agent wants to devate from the truthful strategy, as long as all other agents are truthful. Ths nvolves a somewhat delcate quantfcaton wth regards to the hstory. Our (weaker and stronger) notons of ncentve compatblty are dentcal to those n Bergemann and Välmäk Defnton 2.1. (Incentve Compatblty) Let T denote the (jont) truthful reportng strategy. Dynamc mechansm M s (Bayesan) ncentve compatble (IC) f, for each agent, truthfulness s a best response to the truthful strategy of other agents precsely, f for each and R, U M,T U M,(R,T ) Dynamc mechansm M s perodc ex-post ncentve compatble f, for each agent and at any tme t, truthfulness s a best response to the truthful strategy of other agents precsely, f for each and tme t, reportng strategy R, prvate hstory h,t, and current type of the other agents s t : U M,T,t (h,t, s t ) U M,(R,T ),t (h,t, s t ) (1) Note that the (weaker) Bayesan noton of IC mples that the truthful reportng strategy s a best response from a prvate hstory that s generated under T wth probablty 1. In contrast, the (stronger) perodc ex-post noton demands that the truthful strategy s a best response on any 7

8 prvate hstory, even those whch have probablty 0 under T (e.g. those hstores where agents ms-reported n the past). See Bergemann and Välmäk 2010 for further dscusson. The noton of ndvdual ratonalty s one, where at the equlbrum, the agents choose to partcpate (as t demands that the agents utltes be non-negatve). Precsely, Defnton 2.2. (Indvdual Ratonalty) Let T denote the (jont) truthful reportng strategy. Mechansm M s (Bayesan) ndvdually ratonal (IR) f, for each agent, the expected future utlty under the truthful strategy s non-negatve,.e., U M,T 0. Mechansm M s perodc ex-post ndvdually ratonal f the expected future utlty s nonnegatve for each agent and tme t, prvate hstory, h,t, and jont type of the other agents s t M,T,.e., U,t (h,t, s t ) 0. The expected proft of a mechansm M s the dscounted sum of all payments of the agents mnus the cost of the actons ) n Proft M = E δ ( c t t (a t ) + p,t (2) t=0 under the (jont) truthful reportng strategy T. The objectve of the seller s to maxmze ths expected proft, subject to both the ncentve compatblty and ndvdual ratonalty constrants. Precsely, Defnton 2.3. (Optmalty) A Bayesan ndvdually ratonal and Bayesan ncentve compatble mechansm s optmal f t maxmzes the expected proft among all Bayesan ndvdually ratonal and Bayesan ncentve compatble mechansms. Note the optmal mechansm s only requred to satsfes the weaker Bayesan ncentve constrants. Ths defnton of optmalty guarantees that the mechansm obtans an expected proft hgher than (or, at least, equal to) any other mechansm that s ncentve compatble and ndvdually ratonal. Ideally, we mght hope for an optmal mechansm whch also satsfes the stronger (perodc ex-post) ncentve constrants, whch ensures truthfulness s a best response even f agents have devated n the past. As we show, the mechansm we propose, the Vrtual-Pvot Mechansm, enjoys these stronger guarantees. =1 3 A Relaxaton Approach We now provde a methodology for optmal dynamc mechansm desgn. The relaxaton approach we take s the standard one also used n Ëso and Szentes 2007, Pavan et al. 2008, Deb The dffculty s n un-relaxng,.e., showng that a canddate for the optmal polcy satsfes the more strngent dynamc IC constrants. Here, we are able to provde both necessary and suffcent condtons for dynamc IC. In partcular, the use of the perodc ex-post noton of ncentve compatblty s crtcal n ths characterzaton. 8

9 3.1 Relaxng In ths secton, we consder a smpler, yet closely related, problem where we can utlze known statc mechansm desgn technques to desgn an optmal mechansm these technques are also used n Ëso and Szentes 2007, Pavan et al. 2008, Deb The dea s to relax the optmzaton problem (of fndng the optmal mechansm) by only mposng certan ncentve constrants that arse n a smpler verson of the problem. Roughly speakng, we attempt to solve a (smpler) less-constraned optmzaton problem. The crtcal ssue s n showng that the soluton to ths less-constraned problem s also the optmal soluton for the orgnal problem. Defnton 3.1. (Relaxed Envronment) Consder an envronment where only the ntal type s,0 s prvate to each agent, whle all her future sgnals are observed by the mechansm. We defne ths to be the relaxed envronment and refer to our orgnal envronment as the dynamc envronment. Whle the mechansm n the relaxed envronment has full nformaton wth regards to the agents sgnals from t 1, note that s,0 may affect all the future values of the agent. Observe that any drect mechansm n the dynamc envronment nduces a mechansm n the relaxed envronment n a natural way: for t 1, smply use the agents actual sgnals s,1,..., s,t as well as the reported ntal sgnal ŝ,0 as the reported type {ŝ t } (as the nput to the allocaton and payment rules of the mechansm). The followng lemma s a rather straghtforward observaton. Lemma 3.1. Let E be a dynamc envronment and E relaxed be the correspondng relaxed envronment. We have that: If M s an ncentve compatble and ndvdually ratonal mechansm n E, then t s an ncentve compatble and ndvdually ratonal mechansm n E relaxed. Let R be the optmal revenue n E relaxed. Suppose a (Bayesan) ncentve compatble and ndvdually ratonal mechansm M n E has revenue R, then M s optmal for both E and E relaxed. Ths lemma suggest a natural optmal mechansm desgn approach: frst, fnd an allocaton rule q of an optmal mechansm n the relaxed envronment E relaxed ; then determne f there exsts a prcng rule for p such that: 1) the mechansm (q, p ) s IC and IR n the dynamc envronment E; 2) the expected revenue t acheves s R. If such a prcng s possble, then (q, p ) s optmal n E. In our separable envronments, we show ths approach s applcable. Furthermore, n Secton 7, we dscuss the lmtatons of ths approach, where we provde certan non-separable envronments for whch the optmal revenue n E s strctly less than the optmal revenue n E relaxed. Envelope and Revenue Lemmas Snce n the relaxed envronment the only pece of prvate nformaton for each agent s s,0, usng the standard approach from statc mechansm desgn (see Myerson 1981, Mlgrom and Segal 2002), we provde the followng lemma. 9

10 Lemma 3.2. (Envelope Condton) Suppose that the mechansm M s IC n the relaxed envronment. Then for all, s,0 and s,0, s,0 U (s,0, s,0 ) U (s,0, s,0 ) = E δ t v,t (a t, s,0, s,1,..., s,t ) s s s,0 =z,0 t=0,0 s,0 = z, s,0 dz. (3) where U (s,0, s,0 ) s utlty of agent under the truthful strategy n M, wth the ntal types are s,0 for and s,0 for the other agents. Agan usng standard technques from statc mechansm desgn, we can use the envelope condton above to establsh the proft of any IC mechansm n the relaxed envronment. Lemma 3.3. (Expected Proft) Suppose that the mechansm M s IC n the relaxed envronment. Then, the expected proft obtaned by the mechansm, Proft M, s equal to: ( n ( E δ t v,t (a t, s t ) 1 F (s,0 ) v,t (a t ) ), s,0, s,1,..., s,t ) n c t (a t ) U M (0, s,0 ) (4) f =1 (s,0 ) s,0 where the expectaton s taken over s,0 and s,0. Ths lemma can be used to derve a canddate for the optmal allocaton rule: f we pck an allocaton rule that maxmzes the equaton above and pck a payment rule that makes t both IC and IR, then we wll have an optmal mechansm. =1 3.2 Un-Relaxng From the relaxed envronment, we can fnd a canddate for an optmal allocaton rule. The man challenge here s how to fnd a payment rule and show that such a mechansm satsfes dynamc IC constrants. It turns out that t s natural to break ths nto two stages. The frst step s understandng how to ensure IC for t 1. Here, there seems to be no general methodology n the lterature (note that we are not assumng any structure on the stochastc process for the sgnals s t, for t 1). Our approach nvolves gong one step further and tryng to nsure perodc ex-post IC for perods t 1. Recent work by Bergemann and Välmäk 2010 show how to guarantee perodc ex-post IC n the context of maxmzng socal welfare. Our results make use of ths, but to do so, a crtcal conceptual step s to allow agents to re-report ther entre type at every perod. Ths way, we are able to obtan perodc ex-post IC for t 1. For t = 0, where s,0 s real valued, we explctly characterze the necessary and suffcent condtons for dynamc IC based on the fact that we have a perodc ex-post IC mechansm for perods t 1. Ths s a key techncal step n our proof. Re-Reportng and Perodc Ex-Post IC Recall that each agent reports her entre type s t = (s,0,..., s,t ) at each perod t, not just her most recent prvate sgnal s,t. At the frst glance, t may seem that ths re-reportng of past prvate 10

11 sgnals s redundant. However, there are a few of reasons why ths approach s qute natural, both conceptually and techncally. Re-reportng smplfes the task of obtanng perodc ex-post IC guarantees. It gves an opportunty for agents that have reported untruthfully n the past to correct ther past ms-reports and, n ths way, return to truthful reportng course. In fact, t s unclear how to obtan such a guarantee for a mechansm whch does not allow re-reportng such a mechansm must ncentvze an agent who has ms-reported n the past to report truthfully n the future, wthout the ablty to query the agents about ther prevous sgnals. For such mechansms (whch do not allow re-reportng), often the best that the mechansm can do s offer the agent an opportunty to msreport agan n order to course-correct. In essence, these are the technques used n Ëso and Szentes 2007, Pavan et al. 2008, Deb 2008 to obtan IC (not perodc ex-post IC) by restrctng the value functons or the stochastc process. Necessary and Suffcent Condtons for IC In the prevous subsecton, we argued that re-reportng smplfes the task of constructng a perodc ex-post IC mechansm. We postpone the dscusson of how we can use re-reportng to actually construct a perodc ex-post IC mechansm untl Secton 5. For now, assume that a mechansm M s perodc ex-post IC for all perods t 1. That s, for any perod t 1, any agent, prvate hstory h,t, types of other agents s t, and reportng strategy R, Eq. (1) s satsfed. We now provde necessary and suffcent condtons for such a mechansm to be IC (at perod t = 0). Consder a subset of an agent s reportng strateges that we denote by x x. Defne x x as the reportng strategy n whch the agent reports x as her frst type s,0 (at t = 0), and subsequently (re-)reports t as x n all future perods (t 1). Furthermore, under the strategy x x, all other sgnals s,t (for t 1) are truthfully reported. In others words, at t = 0, she ntally reports sˆ,0 = x, and, for t 1, she reports ŝ t = (x, s,1, s,2,..., s,t ). In x x, we also allow x and x to be functons of s,0. For example, the truthful strategy T can be represented as s,0 s,0. The expected utlty of agent under mechansm M and reportng strategy x x gven her ntal type s,0 s U M,(x x,t ) (s,0 ). For notatonal convenence, we drop the explct dependence on the mechansm and the other agents playng the truthful strategy and denote ths by: U x x (s,0 ) = U M,(x x,t ) (s,0 ). (5) Smlarly, we defne the expected value of agent under strategy x x, assumng other agents are truthful by: V x x (s,0 ) = V M,(x x,t ) (s,0 ). (6) We also use the notaton U x x (s,0, s,0 ) and V x x (s,0, s,0 ), when we condton on the ntal types of the other agents s,0. Suppose the mechansm M s one whch s perodc ex-post IC for perods t 1. Under such a mechansm, f agent devates at perod t = 0, whle all other agents are truthful, agent s best response strategy at all future perods t 1 s to reveal her true type. Therefore, f her true frst 11

12 type s s,0, then to verfy f truthfulness s a best response, we only need to verfy that the truthful polcy provdes more utlty then all msreportng strateges of the form s,0 s,0. Therefore, f mechansm M s perodc ex-post IC for perods t 1, then t s also IC at t = 0 f, and only f, for any true type x and tme 0 report x, U x x (x) U x x (x). Subtractng U x x (x ) from both sdes, we get the followng characterzaton: the mechansm M s IC f, and only f, for all x and x, U x x (x) U x x (x ) U x x (x) U x x (x ). (7) Furthermore M s perodc ex-post IC f the above holds where we condton on the other types s,0. That s, the mechansm s perodc ex-post IC f for all x, x and s,0, U x x (x, s,0 ) U x x (x, s,0 ) U x x (x, s,0 ) U x x (x, s,0 ). (8) These observatons are useful n that t we can use envelope condtons to precsely characterze ncentve compatblty n terms of the expected values of the agents. Frst, we obtan that perodc ex-post IC for t 1 mples the followng lemma. Lemma 3.4. (Perodc Ex-Post IC) Suppose that mechansm M satsfes the perodc ex-post IC condtons for all t 1. Then, for all x and x n 0, 1, we have U x x (x) U x x (x ) = x x (s) dz (9) s s=z V x z It s straghtforward to show that the partal dervatve exsts and, for any x, y and z, s gven by: V x y (s) = E δ t v,t (a t, s,0, s,1,..., s,t ) s s=z s s,0 =s,0 s,0 = z (10) t=0 where the expectaton s under jont strategy (x y, T ) n M (see Lemma A.1 n the Appendx). The followng lemma uses the characterzaton above to obtan both necessary and suffcent condtons for ncentve compatblty (at t = 0). Lemma 3.5. (Necessary and Suffcent Condtons for IC) Suppose that the mechansm M satsfes the perodc ex-post IC condtons for all t 1. Then, M s IC for all t 0 f, and only f, both condtons below are satsfed: (Envelope Condton) For all x and x, U x x (x) U x x (x ) = (Interval Domnance) For all x and x, x x (s) dz s s=z V z z x x x x (s) dz. (11) s s=z V z z (s) dz. (12) s s=z V x z 12

13 Furthermore, M s ex-post perodc IC f and only f the prevous two condtons are satsfed when we condton on every possble other ntal types ŝ,0. The result above s analogous to the characterzaton of ncentve compatblty n standard sngleparameter settngs, where an envelope condton and monotoncty are used to characterze IC (see Myerson 1981). The envelope condton above s a standard one, but nterval domnance replaces monotoncty n a dynamc settng. It compares the utlty obtaned by the truthful strategy (lefthand sde) wth other strateges of the form x s,0 (rght-hand sde), as these are the only plausble canddate strateges when the mechansm s ex-post IC for perods t On Our Methodology Although prevous papers n the lterature (see Ëso and Szentes 2007, Pavan et al. 2008) also provde optmal mechansms usng the relaxaton approach, we emphasze that our constructon and results do not mmedately follow from them. The key challenge we address n our paper s showng that the allocaton rule generated by the relaxaton has an assocated payment rule that makes the mechansm IC and IR n the dynamc settng. Our soluton requres a combnaton of usng the re-reportng technque, wth constructng payments based on Bergemann and Välmäk 2010 to obtan perodc ex-post IC for perods t 1, as well as provng IC (at t = 0) by usng our characterzaton of IC under the assumpton of perodc ex-post IC for t 1. Furthermore, we show n Secton 7 that the relaxaton approach does not work n every settng. In fact, the second example provdes a smple dynamc envronment n whch the usual notons of monotoncty hold for the optmal allocaton n the relaxed envronment, and yet, ths same allocaton rule s not optmal n the dynamc envronment (clearly showng how statc notons of monotoncty are nsuffcent). Although we are not able to address the challengng problem of explctly characterzng the necessary and suffcent propertes of an envronment for whch ths relaxaton approach wll succeed, we do provde envronments n whch both: the relaxaton approach fals and varous assumptons of our separable envronment are volated. Roughly speakng, these show that at least some varant of our assumptons are requred for the relaxaton approach to be successful. 4 Separablty We now defne a class of envronments where the optmal allocaton n the relaxed envronment s closely related to an (affnely transformed) effcent allocaton. In the next secton, we provde an optmal mechansm for ths class. To be able to construct an optmal dynamc mechansm, we need to assume some structure on how the agents values relate to ther sgnals. The next property specfes two natural relatonshps between the sgnals and the values. Property 4.1. (Functonal Separaton) An envronment satsfes functonal separaton f the value functon of each agent s ether multplcatvely or addtvely separable: 13

14 The value functon of agent s multplcatvely-separable f there exsts functons A and B,t such that: v,t (a t, s t ) = A (s,0 )B,t (a t, s,1,..., s,t ) (13) The value functon of agent s addtvely-separable f there exsts A, B,t, C,t such that: v,t (a t, s t ) = A (s,0 )C,t (a t ) + B,t (a t, s,1,..., s,t ) (14) Defnton 4.1. We call an envronment separable f Assumpton 2.3 and Property 4.1 hold. 5 Note that an envronment may not be separable at the frst glance, but there mght exst a transformaton of the sgnals and value functons whch makes the envronment separable. In the followng secton, unless otherwse stated, we assume the envronment s separable. 4.1 Examples of Separable Envronments Sponsored Search A promnent example of multplcatvely separable value functons arses n the settng of onlne advertsng. Consder a sponsored search aucton for a keyword that corresponds to a certan product. Suppose agent s an onlne retaler of such a product who partcpates n the correspondng aucton. Every tme a user types n the keyword, the ad spaces are allocated to the retalers. Every tme a user purchases the product from them, the retaler obtans a proft of s,0 (and 0 otherwse). The type of each agent (besdes s,0 ) would represent the Bayesan belef about the probablty of a purchase occurrng gven the retaler s ad s shown. Therefore, v,t (s t ) = s,0 Prpurchase s,1,..., s,t. After each tme the ad of retaler s shown to a user, the retaler updates her belef about probablty of a purchase. Supply Chan A wdget manufacturer supples one or more retalers who sells these products to consumers. The wdgets have some assocated producton cost c t ( ) that are borne by the manufacturer. The retalers have two peces of prvate nformaton: ther proft margn on the wdgets (whch they know upfront and s captured by s,0 ) and the demand they face for the wdgets (whch they learn over tme and s captured by s,1,..., s,t ). Each retaler s value s then gven by v,t (s t ) = s,0 Demand(s,1,..., s,t ). AR(k) For an example of addtvely separable value functon, consder auto-regressve (AR) processes. One example of an AR(k) model for the evoluton of the valuaton of each agent s as follows: the ntal value of agent s gven by v,1 = s,0, and every tme the tem s allocated to agent her valuaton s updated accordng to v,t = k τ=1 γ,t,τ v,t τ + η,t (a t, s,1,..., s,t ), where γ,t,τ are constants and η s a nose process. It s straghtforward to use functons A (s,0 ), C,t (a t ), and B,t (a t, s,1,..., s,t ) to model ths process as an addtvely separable value functon. 5 We do assume that Assumpton 2.3 holds throughout the paper, but we state the defnton above as a combnaton of Property 4.1 and Assumpton 2.3 n order to clearly state that for an envronment to be separable, the value functon of each agent must satsfy both a functonal and a statstcal (ndependence of frst sgnal) separaton. 14

15 4.2 The Relaxed Envronment and the Vrtual Welfare In the relaxed envronment, we can use the standard technques of statc mechansm desgn (Myerson 1981, Mlgrom and Segal 2002) to establsh an upper bound on the proft of the optmal mechansm. The next lemma establshes that the proft of any IC mechansm s an affne transformaton of the socal welfare of the agents. The affne factors are gven by the functons α and β n the lemma. Note that they are only depend on the ntal sgnals (and the actons of the mechansm) and do not explctly depend on the sgnals from t 1. Ths observaton underles our constructon of the optmal mechansm. Lemma 4.1. Consder the relaxed envronment and an ncentve compatble mechansm M. Suppose the envronment s separable (as n Defnton 4.1), and A, B,t and C t, are unformly bounded. Then, under the stochastc process nduced by M and the truthful reportng strategy, the expected dscounted sum of payments by each agent s equal to E δ t p,t = E δ t( α (s,0 )v,t (a t, s t ) + β,t (a t, s,0 ) ) E U M,T (s,0 = 0, s,0 ) t=0 t=0 (15) where the functons α and β,t are gven by: For multplcatvely-separable values, α (s,0 ) = 1 1 F (s,0 ) A (s,0) f (s,0 ) A (s,0 ) β,t (a t, s,0 ) = 0 For addtvely-separable values, α (s,0 ) = 1 β,t (a t, s,0 ) = 1 F (s,0 ) A f (s,0 ) (s,0 )C,t (a t ) The lemma above yelds a bound on the proft of the optmal mechansm for the relaxed envronment. Recall that Lemma 3.1 establshed that the proft for the dynamc envronment s bounded by the proft from the relaxed one. Combnng these two lemmas and the fact that an IR mechansm must satsfy U M,T (s,0 = 0) 0, we obtan the followng proft bound. Corollary 4.1. Under the assumptons n Lemma 4.1, for both the relaxed and the dynamc envronments, the Proft M of any ncentve compatble and ndvdually ratonal mechansm M s bounded as follows: Proft M max q Q E δ t ( n =1 where Q s the set of all allocaton rules. ) ( ) α (s,0 )v,t (a t, s t ) + β,t (a t, s,0 ) c t (a t ), (16) The bound above determnes an upper bound on the proft of any optmal dynamc mechansm. In the next secton, we provde a dynamc mechansm that satsfes IC and IR and acheves ths upper bound. 15

16 5 The Vrtual-Pvot Mechansm We now present the Vrtual-Pvot mechansm an optmal dynamc mechansm n separable envronments. The key nsght from Secton 4 s that the proft of a dynamc mechansm s bounded by an affne transformaton of the socal welfare of the agents, where the affne parameters are gven by the functons α and β,t n Lemma 4.1. We defne an affne weght functon through a par of vectors (ˆα, ˆβ), such that ˆα = (ˆα 1,, ˆα n ) R n and ˆβ = ( ˆβ 1,, ˆβ n ) ( A R ) n, where A ncludes all possble acton vectors a t for any t. In partcular, ˆβ s allowed to depend acton a t, so that ˆβ(a t ) = ( ˆβ 1 (a t ),, ˆβ n (a t )) R n. For any (ˆα, ˆβ), tme t, and vectors of actons a t and types s t, the weghted socal welfare wth respect to (ˆα, ˆβ) s defned as W (ˆα, ˆβ) (a t 1, s t ) max q Q E ( n δ τ τ=t =1 ( ˆα v,τ (a τ, s τ ) + ˆβ ) (a τ ) c τ (a )) τ st, a t 1, (17) where the max s over all the possble allocaton rules. Usng a standard dynamc programmng argument, the weghted socal welfare satsfes the followng (Bellman) equatons: n ( W (ˆα, ˆβ) (a t 1, s t ) = max E ˆα v,t (a t, s t ) ˆβ ) (a t ) c t (a t ) + δw (ˆα, ˆβ) (a t, s t+1 ) a t A t st, a t 1 (18) =1 where s t+1 s the next (random) type when condtoned on s t and a t. Note, however, that the affne parameters (ˆα, ˆβ) we need to use to acheve the bound from Corollary 4.1 are not numbers (or, n the case of β, functons of the sequence of actons), but functons of the frst sgnal s,0 of each agent. An mportant challenge n mplementng an IC mechansm s elctng s,0 n an ncentve compatble way n order to obtan the desred (ˆα, ˆβ). An mportant desgn choce n the Vrtual-Pvot Mechansm s to use the frst report of s,0 to determne the affne parameters (ˆα, ˆβ) and mantan those affne parameters fxed for all perods, rrespectve of future re-reports of s,0. The Vrtual-Pvot mechansm s presented n Fgure 2. The mechansm conssts of two stages: (Subscrpton Phase) At tme 0, each agent, reports her ntal type, ŝ,0. Then, the mechansm assgns affne parameters (ˆα = α (ŝ,0 ), ˆβ ( ) = β,t (, ŝ,0 ) ) to each agent, where the functons α and β are gven n Lemma 4.1. Then, the mechansm excludes the agents whose expected dscounted payments would be negatve (or zero). If p (ŝ 0) 0 ( see defnton n Eq. (24) ), then / a 0. Otherwse, agent a 0 and pays p,0 (ŝ 0 ) ( see defnton n Eq. (25) ). (Allocaton Phase) For t 1, the Vrtual-Pvot mechansm s equvalent to an affne dynamc pvot mechansm. The affne parameters are fxed and the mechansm solcts reports from the agents n order to choose actons that maxmze the affnely transformed socal welfare W (ˆα, ˆβ). 16

17 The Vrtual-Pvot Mechansm: (Subscrpton Phase) At tme t = 0, for each agent, She reports ŝ,0. Let ˆα α (ŝ,0 ), ˆβ (a τ ) β,τ (a τ, ŝ,0 ) for all τ 1 and a τ A τ. If p (ŝ 0) 0 ( see Eq. (24) ), then / a 0 (agent s excluded). If p (ŝ 0) > 0, then let a 0 and charge her p,0 (ŝ 0 ), see Eq. (25). (Allocaton Phase) At each tme t = 1, 2,... Each agent reports ŝ t. Let a t be an acton that maxmzes W (ˆα, ˆβ) (a t, ŝ t ), see Eq. (19). Let m,t be the flow margnal contrbuton of agent, see Eq. (21). The payment of each agent s equal to p,t (ŝ t ) v,t (a t, ŝ t ) m,t ˆα. Fgure 2: The Vrtual-Pvot Mechansm To gan some ntuton, let us consder the multplcatve-separable case. Roughly speakng, an agent wth a hgher ntal sgnal s,0, would be assgned a larger ˆα. A larger ˆα ncreases the weght of the agent n the affne transformaton and hence ncreases the value obtaned by the agent. We dscuss the allocaton and payment rules n more detals n Secton 5.2. Before that, we present our man results. 5.1 Optmalty We make the followng assumptons. Assumpton 5.1. (Monotone Hazard Rate) Assume that Assumpton 5.2. Assume that: f (s,0 ) 1 F (s,0 ) s strctly ncreasng. (Multplcatve Case) If the value functon of agent s multplcatvely separable, then A (s,0 ) s strctly ncreasng, twce dfferentable, and concave n s,0. (Addtve Case) If the value functon of agent s addtvely separable, then, for all a t A, A (a t ) s strctly ncreasng, twce dfferentable and concave n s,0. Also, C,t (a t ) s postve. 17

18 The functon A (s,0 ) = s,0 s an example of a functon that satsfes Assumpton 5.2. These assumptons mply that α s strctly ncreasng for multplcatvely separable value functons and that β,t s dfferentable and strctly ncreasng for addtvely separable value functons (see Lemma A.2). Theorem 5.1. (Optmalty) Suppose that the envronment s separable and that Assumptons 5.1, and 5.2 hold. Then, the Vrtual-Pvot mechansm s optmal n both the relaxed and the dynamc envronments. In addton, the Vrtual-Pvot mechansm s perodc ex-post ndvdually ratonal and perodc ex-post ncentve compatble. The proof of ths theorem s presented n Subsecton 5.3. The assumptons above allow us to satsfy the dynamc IC condton from Lemma 3.5. For optmalty of the mechansm n the relaxed envronment, a weaker set of assumptons could potentally be suffcent. The Vrtual-Pvot Mechansm s optmal for both the relaxed and dynamc envronments and the proft obtaned by the mechansm, as well as the utlty obtaned by the agents, are dentcal n both envronments. Therefore, the agents obtan no nformaton rent for perods t 1. That s, the agents are not able to obtan any beneft from the fact that sgnals s,1,..., s,t are prvate. Ths no-nformaton-rent property was noted n a two-perod model by Ëso and Szentes 2007 where the mechansm s able to control whether or not agents obtan a second prvate sgnal. Theorem 5.1 mples that the no-nformaton-rent property holds even n nfnte-horzon problems where the sellers has partal control (or even no control) over what prvate sgnals agents obtan over tme (sgnals evolve accordng to a stochastc kernel K,t (s,t a t 1, s t 1 )), as long as the envronment s separable. We show n Secton 7 that ths property does not extend to general non-separable settngs. Snce there s no nformaton rent for perods t 1, there s no allocaton dstorton assocated wth sgnals s,t for t 1. The ntal sgnal s,0, however, causes dstorton from the effcent allocaton at every perod as f the mechansm desgn problem was a statc one. To see ths easly, consder a settng where each agent has a multplcatvely separable valuaton and A (s,0 ) = s,0,.e., the value functon of agent s v,t = s,0 B,t (a t, s,1,..., s,t ). The Vrtual-Pvot Mechansm allocates n order to maxmze the vrtual valuatons of ( s,0 1 F (s,0 ) f (s,0 ) ) B,t (a t, s,1,..., s,t ). That s, the frst sgnal s,0 s replaced at every perod by the vrtual value s,0 1 F (s,0 ) f (s,0 ) of statc mechansm desgn (see Myerson 1981). Our results contrast to the ones of Battagln 2005, Zhang 2011, where the mpact of the frst sgnal s,0 on the value v,t s transent (t dsappears as t grows) and, therefore, the allocaton dstorton s also transent. 5.2 The Allocaton and Payment Rules We frst dscuss the allocaton rule of the mechansm. At each tme t, the mechansm chooses allocaton a t that maxmzes W (ˆα, ˆβ) (a t 1, ŝ t ) where a t 1 = (a 0,, a t 1 ) represents the past 18

19 actons of the mechansm. From Eq. (18), we have a t argmax {at A t} { n =1 ( ˆα v,t ((a t 1, a t ), ŝ t ) + ˆβ ( a t 1, a t ) ) c t (a t 1, a t ) (19) +δe W (ˆα, ˆβ) ( (a t 1, a t ), s t+1 ) } s t = ŝ t Note that only reports from two tme perods (0 and t) are used to determne a t. That s, ŝ 0 s used to determne the affne parameters and ŝ t s used to determne the agents types at perod t. At tme t, the mechansm does not use the agents reports between tmes 1 to tme t 1 (for the allocaton or payments). We now show how the payments are determned. We start from the payments p,t for t 1 and then use those to construct p,0. To make the mechansm ncentve compatble, p,t s determned such that the (nstantaneous) utlty of agent at tme t s proportonal to her flow margnal contrbuton to the affnely transformed socal welfare, denoted by m,t. where W (a,b) and a,t m,t = W (ˆα, ˆβ) (a t 1, ŝ t ) δe W (ˆα, ˆβ) W (ˆα, ˆβ) (a t, s t+1 ) s t = ŝ t, a t (a t 1, ŝ t (ˆα, ˆβ) ) + δe W (a t, s t+1 ) s t = ŝ t, a t, a,t s the affnely transformed socal welfare obtaned n the absence of agent (ˆα, ˆβ) W (a t 1, s t ) max q Q E τ=t δ τ j: j ( ˆα j v j,τ (a τ, s τ j ) + ˆβ ) j (a τ ) c τ (a τ ) s the acton that maxmzes W (ˆα, ˆβ) (a t 1, s t ) at tme t. Equvalently, we have m,t = j=1 st, a t 1 (20) n ( ˆα j v j (a t, ŝ j,t ) + ˆβ ) j (a t ) c t (a t ) (21) W (ˆα, ˆβ) (a t 1, ŝ t ) + δe The payment by agent at tme t s then gven by (ˆα, ˆβ) W (a t, s t+1 ) s t = ŝ t, a t, a,t p,t (ŝ t ) = v,t (a t, ŝ t ) m,t ˆα. (22) In Bergemann and Välmäk 2010, the dea of such a payment based on flow margnal contrbutons was ntroduced and shown to establsh ncentve compatblty for the welfare maxmzng allocaton rule. Smlarly, the payments that we use (whch are scaled versons of the flow margnal contrbutons) establsh ncentve compatblty for the affnely transformed welfare maxmzng allocaton rule. 19

20 We now construct the payment at tme 0. Consder the allocaton rule q that maxmzes the weghted socal welfare condtoned on the reports at tme 0,.e. ( n q argmax q Q E δ t (ˆα v,t (q t, s t ) + ˆβ (q t )) ) ) c t (q t ) s 0 = ŝ 0 (23) =1 where q t = q(h t, s t ) and q t = (q 0,, q t ). We drop the (explct) dependence of q t on h t and s t to smplfy the presentaton. Note that f the agents are truthful, then q and a correspond to the same allocaton rule. Defne p (ŝ 0) as follows: ŝ,0 p (ŝ 0 ) = V (ŝ 0 ) 0 V z z (s,0, ŝ,0 ) dz (24) s,0 s,0 =z where: V z z (s,0, ŝ,0 ) s,0 = E δ t v,t(q t, s,0, s,1,... s,t ) =z s,0 s,0 =z s,0 = z, s,0 = ŝ,0. s,0 The value p (ŝ 0) s the payment of agent n the relaxed envronment, gven by the envelope condton. If p (ŝ 0) 0, then the mechansm excludes agent (that s, / a 0 ). The total expected dscounted sum of payments n the relaxed and dynamc envronments must match n order to acheve our optmalty bound. Therefore, p (ŝ 0) must be equal to expected dscounted sum of payments from agent. Hence, the payment of agent at tme 0 equals p,0 (ŝ 0 ) = p (ŝ 0 ) E δ t p,t (s t ) s 0 = ŝ 0. (25) 5.3 Un-Relaxng: Proof of Theorem 5.1 In ths subsecton, we present the three steps of the proof of Theorem 5.1. followng lemmas are gven n the appendx. The proofs of the The frst step s to show that the mechansm, f ncentve compatble, does ndeed yeld the proft from the upper bound n Corollary 4.1. The argument used to prove ths lemma s a standard one from Myerson We also show that the Vrtual-Pvot Mechansm s perodc ex-post ndvdually ratonal. Lemma 5.1. If the Vrtual-Pvot mechansm s ncentve compatble, then t s optmal. Moreover, t s perodc ex-post ndvdually ratonal at t = 0. The lemma below guarantees that, under the Vrtual-Pvot mechansm, t s always a best response for agents to report ther types truthfully regardless of the hstory, at any tme t 1 (assumng that other agents wll be truthful n the future but not necessarly n the past). Ths lemma follows the technque of Bergemann and Välmäk 2010, except that t maxmzes an affne transformaton of the socal welfare, nstead of the socal welfare tself. 20