Univrsiy of Pnnsylvania ScholarlyCommons Opraions, Informaion and Dcisions Paprs Wharon Faculy Rsarch 6-2005 Dcing Rgim Shifs: Th Causs of Undr- And Ovrracion Cad Massy Univrsiy of Pnnsylvania Gorg Wu Follow his and addiional works a: hp://rposiory.upnn.du/oid_paprs Par of h Marking Commons, and h Probabiliy Commons Rcommndd Ciaion Massy, C., & Wu, G. (2005). Dcing Rgim Shifs: Th Causs of Undr- And Ovrracion. Managmn Scinc, 51 (6), 932-947. hp://dx.doi.org/10.1287/mnsc.1050.0386 This papr is posd a ScholarlyCommons. hp://rposiory.upnn.du/oid_paprs/201 For mor informaion, plas conac rposiory@pobox.upnn.du.
Dcing Rgim Shifs: Th Causs of Undr- And Ovrracion Absrac Many dcision makrs opra in dynamic nvironmns in which marks, compiors, and chnology chang rgularly. Th abiliy o dc and rspond o hs rgim shifs is criical for conomic succss. W conduc hr xprimns o s how ffciv individuals ar a dcing such rgim shifs. Spcifically, w invsiga whn individuals ar mos likly o undrrac o chang and whn hy ar mos likly o ovrrac o i. W dvlop a sysm-nglc hypohsis: Individuals rac primarily o h signals hy obsrv and scondarily o h nvironmnal sysm ha producd h signal. Th xprimns, wo involving probabiliy simaion and on involving prdicion, rval a bhavioral parn consisn wih our sysmnglc hypohsis: Undrracion is mos common in unsabl nvironmns wih prcis signals, and ovrracion is mos common in sabl nvironmns wih noisy signals. W s his parn formally in a saisical comparison of h Baysian modl wih a paramric spcificaion of h sysm-nglc modl. Kywords rgim shif, blif rvision, subjciv probabiliy, chang poins, undrracion, ovrracion Disciplins Marking Probabiliy This journal aricl is availabl a ScholarlyCommons: hp://rposiory.upnn.du/oid_paprs/201
Dcing Rgim Shifs: Th Causs of Undr- And Ovr-Racion* Cad Massy Gorg Wu Fuqua School of Businss Gradua School of Businss Duk Univrsiy Univrsiy of Chicago Durham, NC 27708 Chicago, IL 60637 cad.massy@duk.du gorg.wu@gsb.uchicago.du Jun 4, 2004 ABSTRACT Many dcision makrs opra in dynamic nvironmns, in which marks, compiors, and chnology chang rgularly. Th abiliy o dc and rspond o hs rgim shifs is criical for conomic succss. W conduc hr xprimns o s how ffciv individuals ar a dcing such rgim shifs. Spcifically, w invsiga whn individuals ar mos likly o undr-rac o chang and whn hy ar mos likly o ovr-rac o i. W dvlop a sysmnglc hypohsis: individuals rac primarily o h signals hy obsrv and scondarily o h nvironmnal sysm ha producd h signal. Thr xprimns, wo involving probabiliy simaion and on involving prdicion, rval a bhavioral parn consisn wih our sysmnglc hypohsis: undr-racion is mos common in unsabl nvironmns wih prcis signals and ovr-racion is mos common in sabl nvironmns wih noisy signals. W s his parn formally in a saisical comparison of h Baysian modl wih a paramric spcificaion of h sysm-nglc modl. * W hank paricipans a svral confrncs and numrous workshops for hir commns and suggsions. W paricularly hank J.B. Haon for inroducing us o his ara, and Bill Goldsin, Chip Hah, Rick Larrick, Jack Soll, Yuval Ronsrich, hr anonymous rviwrs, and h Associa Edior for hir valuabl commns. W also hank Kailyn Hwang, David Malony, John Mors, Amanda Snow, and Carolin Ts for hlp in collcing h daa.
1. INTRODUCTION Dcision makrs mus ofn mak dcisions in unsabl nvironmns. An invsor mus lc whhr o kp hr invsmns in quiy. A Cnral Bank mus dcid whhr a wakning conomy mris an inrs ra cu. A convnional railr mus dcid whhr o mbrac an Inrn sragy. In ach of hs xampls a dcision makr rcivs a signal (.g, a sris of poor arnings announcmns, an unxpcdly high unmploymn figur, an onslaugh of ailrs ) and mus judg whhr ha signal augurs a nw rgim (.g., h ons of a bar mark, an conomy hadd oward rcssion, a shif o an onlin conomy), or is jus an xrm oucom from h incumbn rgim. Th abiliy of dcision makrs o corrcly idnify h ons of a nw rgim can man h diffrnc bwn succss and failur. Indd, h difficuly of sparaing signal from nois has ld o wll-publicizd insancs of ovr-racion (i.., bliving a rgim shif has occurrd bfor i acually has), as wll as undr-racion (i.., bliving a rgim shif has no occurrd whn in fac i has). Considr wo xampls. In h la-1980 s, Xrox obsrvd Japans smiconducor manufacurrs shifing rsourcs o a nw, X-ray-basd manufacuring mhod. Xrox, bliving his indicad a fundamnal shif in smiconducor chnology, rallocad significan company rsourcs o his nw approach. Yars lar i was clar ha such a shif did no occur and, in fac, would no for h forsabl fuur (Grov, 1999). Convrsly, in h la-1970 s, Schwinn was slow o rspond o h advn of h mounain bik. Managmn blivd ha h surg in mounain bik populariy was jus anohr fad. Schwinn s rlucanc o inroduc a mounain bik cos h company is dominanc in h Amrican bicycl mark and conribud o is vnual bankrupcy (Crown & Colman, 1996). For obvious rasons, hs xampls mus b inrprd wih cauion. In ordr o undrsand h facors ha impac h abiliy of individuals o dc rgim shifs mor prcisly, w invsiga undr- and ovr-racion xprimnally. W conduc hr sudis and find ha individuals xhibi sysmaic biass in hir abiliy o dc rgim shifs. Subjcs in our sudis obsrv signals, and basd on hs signals indica whhr hr has bn a shif from on rgim o a scond rgim. 1
W mploy an xprimnal paradigm in which h firs rgim is rprsnd by an urn ha conains mor rd han blu balls (a rd urn ), and h scond rgim is rprsnd by an urn ha conains mor blu han rd balls (a blu urn ). A any im, h xprimn may swich from drawing balls from h rd urn o drawing balls from h blu urn. In his sup, hr valus ar ndd for drmining h appropria (Baysian) racion: (i) h signal: h squnc of rd and blu balls obsrvd; (ii) signal diagnosiciy: h dgr o which h rd urn diffrs from h blu urn; and (iii) ransiion probabiliy: h chanc ha h sysm will rmain wih h rd urn, or swich o h blu urn. This simpl xprimnal paradigm is inndd o capur h ssnial faurs of many ralworld siuaions. For an invsor dciding whhr o kp hr invsmns in quiy, h wo sas migh b a bull mark and bar mark. An arnings announcmn is an informaiv bu imprcis signal, wih h informaivnss of his signal varying across marks and ovr im. Finally, h mark may vacilla bwn hs wo sas, wih h hisorical frquncy of chang capurd by h ransiion probabiliy. W posi and find srong vidnc for a sysm-nglc hypohsis: individuals pay inordina anion o h signal, and nglc diagnosiciy and ransiion probabiliy, h aspcs of h sysm ha gnrad h signal. Morovr, w suggs ha sysm nglc lads o a prdicabl parn of undr- and ovr-racion: individuals ar mos pron o undr-racion in unsabl nvironmns wih prcis signals and o ovr-racion in sabl nvironmns wih noisy signals. Our hr sudis, wo judgmn asks and a choic ask, show ha subjcs do xhibi subsanial sysm nglc and ha such sysm nglc dos indd rsul in h posid parn of undr- and ovr-racion. Th papr procds as follows. In Scion 2, w rviw prvious mpirical rsarch on h dcion of rgim shifs, as wll as rlvan rsarch on judgmn in saionary nvironmns. W xnd hs idas and dvlop h sysm-nglc hypohsis. W s his hypohsis in hr sudis. In Scion 3, w prsn a judgmn sudy in which subjcs judg h posrior probabiliy ha h procss has swichd rgims (from h rd urn o h blu urn). In Scion 4, w prsn a scond judgmn sudy ha ruls ou rror as an xplanaion for our rsuls. In Scion 5, w prsn a choic sudy in which subjcs prdic h nx obsrvaion. In all hr sudis w find srong 2
vidnc of sysm nglc, a lack of snsiiviy o h sysm characrisics, diagnosiciy and sabiliy, and h prdicd parn of ovr- and undr-racion. W provid formal suppor for his insnsiiviy by simaing a family of quasi-baysian modls. W conclud in Scion 6 wih a discussion of h implicaions of h rsarch and an oulin of fuur work. 2. BACKGROUND Prvious Rsarch W brifly oulin som of h major mpirical findings in h sudy of rgim-shif dcion. Th sudis hav varid considrably in mhodology. In som sudis, h squncs wr prsnd squnially (Robinson, 1964; Chinnis & Prson, 1968, 1970; Barry & Piz, 1979), whil in ohr sudis, a squnc was providd in is niry (Thios al., 1971). Som subjcs simad whn h procss changd from on rgim o anohr (Barry & Piz, 1979; Thios al., 1971), whras ohr subjcs simad h probabiliy ha h daa wr drawn from on rgim or anohr (Chinnis & Prson, 1968, 1970). Sudis hav also varid nvironmnal paramrs such as diagnosiciy (Robinson, 1964; Chinnis & Prson, 1968, 1970), h payoff srucur (Barry & Piz, 1979), and h ra a which subjcs rciv simuli (Robinson, 1964). Ths sudis hav shown ha individuals gnrally rspond o h possibiliy of chang and do so in h righ dircion. Byond ha, hough, rsuls ar mixd. Across hs sudis, som bhavior is approximaly opimal (Chinnis & Prson, 1968), bu mos is no (Barry & Piz, 1979; Thios al., 1971). Sudis find boh undr-racion (Barry & Piz, 1979; Chinnis & Prson, 1968) and ovr-racion (Brown & Ban, 1975; Chinnis & Prson, 1970; Ess, 1984). On of h fw hms conncing his liraur is h ida ha individuals rspond parially o changing nvironmnal condiions. Chinnis & Prson (1968) sa: ()h subjcs, whil snsiiv o h diffrnc in diagnosic valu of h daa in h wo condiions, wr no adqualy snsiiv (p. 625). For xampl, Barry & Piz (1979) sysmaically vary payoffs and find individual bhavior largly unchangd. This hm rsonas wih h sysm-nglc hypohsis w dvlop in h nx scion. Rapopor al. (1979) provid a mor comprhnsiv ramn of chang dcion, formally 3
modling h dcision nvironmn and valuaing opimal policis. Thy dvlop and s hr dscripiv modls in which individuals ac only whn h signal provids vidnc xcding som hrshold. Criically, hs modls incorpora xrm vrsions of sysm nglc ha do no rflc any snsiiviy o nvironmnal condiions. For xampl, on modl rcognizs a chang whn h signal xcds som fixd probabiliy hrshold. 1 W suggs ha h hrshold modl is oo xrm. Insad, w hypohsiz ha individuals rspond o changs in nvironmnal condiions, bu insufficinly so. In ohr words, hir bhavior is dicad mor by signals han by h sysm gnraing h signals. W call his h sysm-nglc hypohsis. In h nx scion, w show how sysm nglc lads o undr-racion in som condiions and ovr-racion in ohrs. Rsarch Hypohsis Our sysm-nglc hypohsis draws on judgmn and dcision making rsarch in saic nvironmns. Our spcific poin of dparur is Griffin & Tvrsky (1992), who rconcild wo wllsablishd, bu smingly conradicory, findings: consrvaism and rprsnaivnss. Consrvaism posis ha individuals upda hir blifs oo slowly in h fac of nw vidnc (Edwards, 1968; Grhr, 1980). Rprsnaivnss, on h ohr hand, suggss ha individuals xrapola oo radily from small sampls, lading o blif rvisions ha ar oo dramaic (Kahnman & Tvrsky, 1973). Griffin & Tvrsky showd ha sampl siz can accoun for his apparn conradicion. Mos consrvaism sudis involv larg sampls, whil mos rprsnaivnss sudis involv smallr sampls. Misundrsanding h impac of sampl siz on h posrior probabiliy lads subjcs o mak consrvaiv rvisions wih larg sampls and radical rvisions wih small sampls. Mor gnrally, Griffin & Tvrsky disinguishd bwn h srngh and wigh of vidnc. Informally, h srngh of vidnc is is magniud, whil h wigh of h vidnc is is rliabiliy. Imagin you ar askd which way a coin is biasd, 70% hads or 70% ails. Hr, h proporion of hads rprsns h srngh of vidnc, whil h sampl siz rprsns h wigh 1 This modling approach is similar o ha akn by Barry & Piz (1979), Brown & Ban (1975), Ess 4
of vidnc. Thus, 4 hads ou of 5 flips has high srngh bu low wigh, whil 32 hads ou of 60 flips has low srngh bu high wigh. Griffin & Tvrsky showd ha individuals sysmaically ovrwigh h srngh of vidnc or how wll ha vidnc machs h hypohsis in qusion, and undrwigh h wigh of vidnc or h diagnosiciy of h signal. As a rsul, 4 hads ou of 5 is sn o b mor complling han 32 hads of ou of 60, conrary o Bays Rul (sinc h diffrnc bwn h numbr of hads and ails is a sufficin saisic). W apply his ida o h problm of rgim-shif dcion. In such problms, an individual rcivs signals ha ar gnrad from on of wo rgims. As in saic nvironmns, a dcision makr mus considr h diagnosiciy of h signal, i.., how srongly corrlad h signal is wih h rgim ha i favors. In a dynamic nvironmn, a dcision makr mus also ak ino accoun h ransiion probabiliy, i.., how likly i is o chang from on rgim o anohr. Alhough a judgmn abou h liklihood of chang should dpnd on boh diagnosiciy and ransiion probabiliy, w suggs ha individuals will no incorpora hs wo sysm paramrs opimally. Insad, our sysm-nglc hypohsis posis ha individuals rspond primarily o h signal and scondarily o h sysm ha gnrad h signal. In h parlanc of Griffin & Tvrsky (1992), h signals provid h srngh of vidnc and h sysm paramrs provid h wigh of vidnc. A major rason ha h signal is ovrwighd is ha i is mor salin: h signal is usually in h forground whil h sysm paramrs ar in h background. In many siuaions, h sysm paramrs ar no known and prhaps unknowabl. In his sns, h sysm-nglc hypohsis is akin o h corrspondnc bias or fundamnal aribuion rror in social psychology (Jons & Harris, 1967): Individuals nd o ovrsima h xn o which bhavior is du o disposiion or prsonaliy and undrwigh h xn o which bhavior is causd by h undrlying siuaion. Th sysm-nglc hypohsis yilds an imporan prdicion: ovrmphasizing h srngh of vidnc a h xpns of is wigh mans ha individuals ar mos likly o undr-rac o an indicaion of chang whn h wigh of ha vidnc is high and ovr-rac whn h wigh is low. Th wigh of vidnc is highs whn diagnosiciy is high and h sysm is unsabl (i.., high ransiion probabiliy), and lows whn diagnosiciy is low and h sysm is sabl (i.., low (1984), and Robinson (1964). 5
ransiion probabiliy). Thus, individuals should b mos rsponsiv o indicaions of chang in prcis (high diagnosiciy) and unsabl (high ransiion probabiliy) nvironmns and las rsponsiv in noisy (low diagnosiciy) and sabl (low ransiion probabiliy) nvironmns. If individuals bhav similarly across sysms, w should s a parn wih rlaivly mor undrracion in prcis/unsabl nvironmns and rlaivly mor ovr-racion in noisy/sabl nvironmns. Griffin & Tvrsky (1992) apply a similar logic and show ha a srngh and wigh accoun prdics ovrconfidnc whn vidnc has high srngh and low wigh (.g., low diagnosiciy or small sampl siz), and undrconfidnc whn vidnc has low srngh and high wigh (.g., high diagnosiciy or larg sampl siz). Transiion Probabiliy Low (Sabl) High (Unsabl) Low (Noisy) Signal Diagnosiciy Ovr- Racion High (Prcis) Undr- Racion Figur 1: Th sysm-nglc hypohsis: Prdicd parn of ovr- and undr-racion as funcion of sysm paramrs Figur 1 dpics hs prdicions succincly. I is imporan ha our prdicion is a rlaiv on: hr should b rlaivly mor ovr-racion in h norhws cll, and rlaivly mor undrracion in h souhas cll. Th hypohsis is siln abou absolu lvls. Indd, h sysmnglc hypohsis is consisn wih ovr-racion vrywhr, undr-racion vrywhr, or a mixd parn, providd ha h gradin slops in h prdicd dircion. No also ha h prdicions of Figur 1 apply o indicaions-of-chang. Racions o indicaions-of-no-chang (or businss as usual ) should no follow his parn. Rahr, undr-racion o indicaions-of-nochang should b srongs in unsabl sysms wih imprcis signals, sinc h bas-ra of chang is highs and h signal las informaiv. 6
W s h sysm-nglc hypohsis in hr sudis by manipulaing diagnosiciy and ransiion probabiliy and varying h ask paricipans fac. 3. STUDY 1: JUDGMENT TASK Our xprimnal sing allows us o compar individual prformanc agains a normaiv sandard and hus s our bhavioral prdicions. Th dcision makrs in our xprimnal paradigm hav all h informaion ncssary o calcula Baysian rsponss and hnc provid opimal judgmns. This allows us o focus our analysis vry spcifically on h mannr in which individuals rvis probabiliy judgmns, and how hs judgmns dvia from Baysian updaing. Saisical Procss W bgin by dscribing h saisical procss usd in our xprimn. Th saisical procss is dpicd in Figur 2. Subjcs obsrv signals gnrad by on of wo rgims. Th ask is o dc if and whn h sysm shifs from on rgim o h ohr. Th wo possibl rgims ar rd and blu. L R ( B ) indica ha h procss is in h rd (blu) rgim in priod. Each of h wo rgims producs wo possibl signals: a rd ball or a blu ball. Whil boh rgims can produc ihr color, h rd rgim favors rd balls (producing hm wih probabiliy p R > 0.5 ) and h blu rgim favors blu balls (producing rd balls wih probabiliy p B < 0.5 ). In our sudis, hs probabiliis ar always symmric, i.., p = 1 p. Thus, p / p is a masur of h diagnosiciy (d) of h signal, wih a highr valu indicaing a mor diagnosic signal. R B Th ransiion bwn h wo rgims is dfind by hr characrisics. Firs, in priod 0, bfor h firs signal is drawn, h procss is in h rd rgim ( Pr( R 0) = 1). Scond, bfor ach signal is producd, including h firs on, h procss migh swich o h blu rgim. Th probabiliy of such a swich is h ransiion probabiliy, q = Pr( B+ 1 R). (This is calld a swich probabiliy in our xprimnal simuli.) Third, h blu rgim is an absorbing sa, i.., if h procss swichs o h blu rgim i rmains hr unil h nd of h rial ( Pr( B+ 1 B) = 1). Again rcall h invsor xampl from our inroducion. Diagnosiciy capurs h informaivnss of a signal such as an arnings announcmn, whil h ransiion probabiliy R B 7
capurs h sabiliy of a bull or bar mark,.g., h liklihood ha a bull mark urns ino a bar mark. Alhough h absorbing sa is a simplifying assumpion in our modl, for many dcision horizons hr will b a mos on chang. For xampl, mos quiy marks cycl bwn bull and bar marks a a sufficinly low frquncy ha for a givn dcision on can (and probably should) hink of h nw rgim as an absorbing sa. Of cours, hr ar many ohr siuaions in which hr is ruly an absorbing sa (.g., an quipmn failur, an obsol chnology, c.). Sysm Rliabiliy (Diagnosiciy) Mhodology Insabiliy (Transiion Probabiliy) Rd Bin 60% Rd Balls 40% Blu Balls Rgims 2% Transiion Probabiliy Blu Bin 40% Rd Balls 60% Blu Balls Figur 2: Saisical Procss Signal Sudy 1 was conducd on compur using a spcially dsignd Visual Basic program. W rcruid 40 Univrsiy of Chicago subjcs, advrising h ask as a probabiliy simaion ask in ordr o yild subjcs comforabl wih probabiliy. Th mdian numbr of undrgradua and gradua mahmaics and saisics classs akn by ach subjc was hr. Th compur program bgan by inroducing h saisical procss usd in h xprimn, and illusrad h procss using 4 dmonsraion rials and 2 pracic rials. Daild insrucions and scrn shos of h program ar found in h Elcronic Companion Papr (hncforh, ECP). Following h inroducion, ach subjc compld 18 rials consising of 10 signals (priods). Each rial was govrnd by a diffrn s of paramrs (s blow). Subjcs wr firs shown h paramrs, p R, p B, and q, govrning ha rial. Ths paramrs wr displayd coninuously 8
hroughou ach rial, and subjcs wr old ha h paramrs would chang across rials. Thy wr hn shown a squnc of rd or blu balls drawn randomly basd on h s of paramrs. Afr sing ach signal, subjcs indicad h probabiliy ha h las ball was drawn from h blu rgim (i.., h probabiliy ha h saisical procss had changd). Subjcs wr no allowd o chang a probabiliy onc i was nrd. W usd 3 diffrn diagnosiciy lvls and 4 diffrn ransiion-probabiliy lvls, yilding 12 xprimnal condiions. Diagnosiciy lvls ( d = p / p ) wr 1.5, 3, and 9, corrsponding o ( p, p ) valus of (.6,.4), (.75,.25), and (.9,.1). Th ransiion probabiliis wr.02,.05,.10, and R B.20. W chos h diagnosiciy lvls and ransiion probabiliis o span a rasonabl rang of h paramr spac. For xampl, 3 of 15 squncs show a rgim shif for q =.02, whil 12 of 15 squncs do so for q =.20, so moving h ransiion probabiliy in ihr dircion would yild a procss ha ihr always or nvr producd rgim chang during our sris. W randomly gnrad 5 uniqu squncs for ach condiion, using a saisical procss wih h ru paramr valus, yilding 60 squncs in oal (h acual squncs ar found in ECP). Each of h subjcs rcivd 18 of h 60 squncs in a randomizd ordr, and a las on squnc from ach of h 12 condiions. Each of h 60 squncs was judgd by a oal of 12 subjcs. W paid subjcs according o a quadraic scoring sysm ha paid $0.10 maximum (.g., if a subjc indicad wih crainy ha h procss was in h blu rgim, and h procss was in fac in h blu rgim) and -$0.10 minimum (.g., if a subjc indicad wih crainy ha h procss was in h blu rgim, and h procss was in fac in h rd rgim). Such a schm is propr, and hus ruh-rvaling for risk-nural subjcs (Brir, 1950). Subjcs wr givn fdback a h nd of ach rial abou if and whn h procss shifd from h rd rgim o h blu rgim. Thy wr also informd how much mony hy mad or los on ha paricular rial. Normaiv Modl Our dpndn masurs ar h probabiliy judgmns providd by subjcs. W valua hs probabiliis by comparing hm o a Baysian sandard. To do so, w driv h Baysian R B 9
soluion for our xprimnal framwork (s Appndix). L r dno h -h signal, whr r = 1 ( r = 0 ) if a rd (blu) ball is drawn in priod, and H = ( r1,..., r) b h squnc of signals hrough priod. Th Baysian posrior odds of a chang o h blu rgim afr obsrving hisory b xprssd as H can j 1 + 1 j 2 r k k= j d j= 1 b p Pr( B H ) 1 (1 ) (1 ) q q q = = b, (3.1) 1 p Pr( R H ) (1 q) 1 (1 q) whr b p dnos h Baysian probabiliy ha h procss has swichd o h blu rgim by. Th xprssion can b dcomposd ino svral componns. Th firs componn, ( q ) 1 (1 ) /(1 q), is a funcion only of h ransiion probabiliy q and h numbr of signals and provids a bas ra for h liklihood of a chang, i.., h liklihood of chang in h absnc of any daa. Th las componn, d + 1 j 2 r k k= j, is simply h diagnosiciy raisd o an xponn rflcing h diffrnc bwn numbr of blu balls and rd balls drawn ovr h par of h hisory from priod j o priod. (This las componn drmins h posrior probabiliy in a saionary nvironmn [cf. Edwards, 1968].) Th middl componn provids mahmaical wighs accouning for h various pahs hrough which h procss can chang from h rd rgim o h blu rgim. Exprimnal Rsuls W prsn boh aggrga and individual-lvl rsuls. Rcall ha our xprimn rquird ha ach of our 40 subjcs provid subjciv probabiliis ha h procss had swichd o h blu rgim for ach of 10 signals in 18 rials. Considr firs an ovrviw of hs judgmns. L p b b h mpirical judgmn and p p b h absolu diffrnc bwn h subjc s judgmn and h Baysian probabiliy for signal. Th man absolu diffrnc was.17 (mdian=.08, sd=.21). Subjcs paymns wr basd on h diffrnc bwn hir subjciv probabiliy of a chang o h blu rgim and h acual rgim (1 if blu, 0 if rd). Th man of his absolu diffrnc was.25 (mdian=.05, sd=.34), gnraing an avrag paymn of $11.62 (mdian=$11.99, rang of $6.60 10
o $14.67). Th avrag paymn o a Baysian agn would hav bn $14.23 (mdian=$14.22, rang of $12.72 o $15.38). Our primary inrs, howvr, is blif rvision how probabiliy judgmns rspond o nw signals. Thus, w considr changs in probabiliis rahr han absolu lvls and compar mpirical changs in probabiliy judgmns wih normaiv changs. Dfining mpirical chang is sraighforward: p = p p 1. Consrucing a propr normaiv masur of chang rquirs a bi mor car. In calculaing h normaiv rspons o a signal, w ak a subjc s prvious b probabiliy judgmn p 1 as h prior rahr han h prvious Baysian probabiliy, p 1. If w us Bays Rul wih p 1 as h prior, w g 1 2r 1 2r b 1 1 R R = b p p 1 q + p B q p B p p p q p 1 1 1 1, (3.2) whr b p is h Baysian rspons o signal r aking p 1 as h prior. This rdfiniion allows our normaiv chang masur o rflc a subjc s blif prior o rciving a signal. Thus, rgardlss of h accuracy of a subjc s prior blif, our procdur auomaically adjuss o ha prior in ordr o valua h subjc s blif rvision. If, for xampl, a subjc s prvious judgmn was low rlaiv o h Baysian probabiliy, sh should b allowd o mak a mor dramaic rvision in ligh of h nw signal (indd, sh is xpcd o). Convrsly, if a subjc s prvious judgmn was oo high rlaiv o Bays Rul, hr is lss room for hr o subsqunly incras hr judgmn and hrfor sh should mak a mor consrvaiv rvision. Sinc h objciv is o focus on h rvision islf, w ak as h normaiv chang masur, p = p p, and compar his masur o h mpirical chang masur, p = p p 1. b b 1 W rsric our anion o mdian judgmns o minimiz h rol asymmric rror may play in gnraing sysm nglc (Erv, Wallsn, & Budscu, 1994). Erv al. offr an rror accoun in which rsponss ar unbiasd ru judgmns prurbd by rror. This accoun rquirs ha h mdian rspons b unbiasd. In Sudy 2, w conrol for rror mor sysmaically. W also 11
rpor man judgmns in ECP. 2 W firs show ha subjcs ar aniv o diffrncs in signals. As xpcd, h mdian mpirical chang, p, is considrably diffrn for blu signals han rd signals,.084 vrsus.001. Th mdian normaiv chang masurs ar.158 and.028 for blu and rd, rspcivly. Thus, h mdian diffrnc bwn h mpirical chang and normaiv chang, p p, our summary b masur of ovr- and undr-racion, is slighly ngaiv for blu signals (.074), indicaing a small ovrall ndncy o undr-rac o indicaions of chang. To s for sysm nglc, w considr how our masur of undr-racion, p p, varis b across condiions. Rcall ha h prdicd gradin shown in Figur 1 applis only o indicaions of chang, hus w rsric our anion o blu signals. Th sysm-nglc hypohsis implis ha undr-racion is mor common whn diagnosiciy and ransiion probabiliy ar high han whn diagnosiciy and ransiion probabiliy ar low. Indd, h mdian normaiv chang, b p, is.231 in h high ransiion-probabiliy/high diagnosiciy condiion, and.056 in h low ransiionprobabiliy/low diagnosiciy condiion. In conras, h mdian mpirical chang, wo condiions is much lss diffrniad,.116 and.022, rspcivly. p, across h Figur 3 dpics h diffrnc bwn h mpirical and normaiv chang masurs in ach of our 12 xprimnal condiions. No firs ha w obsrv undr-racion in 11 of h 12 condiions and ovr-racion in 1 of h 12 condiions. By islf his parn dos no provid vidnc for or agains h sysm-nglc hypohsis. Wha mars is h gradin bwn h souhas cll (high diagnosiciy and ransiion probabiliy) and h norhws cll (low diagnosiciy and ransiion probabiliy). Consisn wih sysm nglc, h gras undr-racion occurs in h souhas-mos cll, whil h scond las undr-racion occurs in h norhws-mos cll. In h majoriy of h pairwis comparisons (37 of 48), h parn of undr-racion is monoonic as 2 W calcula h mdian chang for ach of h 600 obsrvaions (60 rials and 10 obsrvaions pr rial) and hn ak h avrag across h appropria cagory. For xampl, o calcula h mdian mpirical chang for a rd ball, w ak h mdian mpirical chang for h 329 (of 600) obsrvaions in which h signal is a rd ball. W hn ak h avrag of hs 329 mdians. This mhod rflcs h lumpinss wih which subjcs us h rspons scal. Taking h avrag ovr h mdians provids us wih a smoohr masur. 12
ransiion probabiliy and diagnosiciy incras, as prdicd by sysm nglc. Th lack of prfc monooniciy parially rflcs simuli which ar randomly gnrad and hnc no ncssarily rprsnaiv of h undrlying procss. Ovr-Racion 2% Mdian Chang in Esimas: Empirical - Baysian 0% -2% -4% -6% -8% 1.5-10% Undr-Racion 2% 5% 10% Transiion Probabiliy 20% 9 3 Diagnosiciy Figur 3: Ovr- and undr-racion, by condiion, as masurd by h mdian diffrnc bwn chang in mpirical probabiliy judgmns and chang in Baysian probabiliis, b p p (Sudy 1, Judgmn Task). Esimaion Th parn in Figur 3 provids suppor for h sysm-nglc hypohsis. I is possibl bu unlikly ha h hypohsizd parn rflcs arifacs in h randomly gnrad squncs. Thus, w provid a formal s of h sysm-nglc hypohsis by gnralizing (3.1). W call hs gnralizaions quasi-baysian modls in h spiri of Edwards (1968), who xamind signal snsiiviy in his sudy of consrvaism (s also Chinnis & Prson, 1968, 1970). Th modls ar Baysian in srucur, bu includ addiional paramrs o accord mor closly wih h mpirical obsrvaions. In ordr o modl snsiiviy o h wo criical dimnsions, ransiion probabiliy and diagnosiciy, w add wo paramrs, α and β, o our Baysian xprssion (3.1). Adding hs paramrs yilds 13
1 j 1 β + 1 j 2 r k p Pr( B H ) 1 (1 ) (1 ) αq q q k= j = = d, (3.3) p Pr( R H ) (1 ) j 1 1 (1 ) i αq = q whr p is h subjc s probabiliy judgmn ha h procss has swichd o h blu rgim by. In (3.3), α capurs snsiiviy o ransiion probabiliy, whil β capurs snsiiviy o diagnosiciy. This spcificaion has svral usful propris. Th xprssion is Baysian whn α= 1 and β = 1. As α shrinks, h ransiion probabiliy plays an incrasingly small rol (as α 0, p 0). As β shrinks, h signal has an incrasingly small impac (as β 0, p /(1 p ) (1 (1 αq) ) /(1 α q), i.., h bas ra odds of chang in h absnc of informaion). W rm h modl in (3.3) h powr modl. Th formulaion in (3.3) is wll-suid for valuaing h dgr of consrvaism in blif rvision (cf. Edwards, 1968). Howvr, h powr modl is inadqua for valuaing sysm nglc, bcaus i xplicily assums ha paramr valus ar consan across diffrn sysms. Sysm nglc, on h ohr hand, suggss ha hs paramr valus will vary sysmaically, rflcing insnsiiviy o sysm changs. To clarify h limiaions of h powr modl, considr a mor gnral formulaion in which β dpnds on h diagnosiciy condiion n, n d β n. For xampl, in n h Baysian modl β n = 1 for all n, implying ha d β n incrass linarly as diagnosiciy incrass. In h powr modl, β may b lss han 1 (as in consrvaism) or grar han 1 bu mus b consan across all condiions. Thus, h powr modl is acually qui snsiiv o changs in nvironmnal condiions. Indd, h rsponsivnss in h powr modl incrass proporionally wih h undrlying paramr, wih h proporion drmind by h dgr of consrvaism. No also ha his modl rsrics bhavior o b xclusivly consrvaiv (if β n < 1) or xclusivly radical (if β > 1 ), and hus prmis only undr-racion or ovr-racion, rspcivly. n Th sysm-nglc hypohsis prdics ha paramr valus diffr sysmaically in ach condiion. To illusra, considr h xrm cas of compl sysm nglc, in which individuals rspond o signals idnically rgardlss of h sysm producing h signals. For β, compl sysm 14
3 nglc rquirs ha d β1 2 1 = d β 2 = d β 3. In our xprimnal dsign, d 1 = 1.5, d 2 = 3.0, and d 3 = 9.0. Thus, h qualiy rquird by compl sysm nglc implis ha β 1 = k, β 2 =.37k, and β 3 =.18k, whr k is any posiiv consan. Of cours, his illusras only h xrm vrsion of sysm nglc. Mor gnrally, sysm nglc rquirs ha for q m < q n and d m < d n, paramr simas b ordrd so ha α m >α n and β m >β n. To valua his prdicion, w gnraliz (3.3) as follows. L α=α 1Q1+α 2Q2+α 3Q3+α 4Q4 and β =β 1D1+β 2D2+β 3D3, (3.4) whr Q m is a dummy variabl corrsponding o ransiion probabiliy q m, and D n is a dummy variabl corrsponding o diagnosiciy condiion d n. This approach xplicily allows α m and β n o vary across nvironmnal condiions, hus providing a formal s of h sysm-nglc hypohsis. Th powr modl is a spcial cas of his modl in which α 1 =α 2 =α 3 =α 4 and β 1 =β 2 =β 3, and h Baysian modl is a spcial cas of h powr modl in which α =β = 1 for all mn., No ha whil h sysm-nglc hypohsis dos no mak any prdicions abou h ovrall lvl of consrvaism in a paricular nvironmn, i dos mak xplici prdicions, via h m n monoonic ordring of α m and β n, abou h parn of ovr- and undr-racion. Th sysmnglc modl xplicily allows for mixd parns of ovr-racion ( α, β > 1 for som mn), and undr-racion ( α, β < 1 for som mn)., m n m n W fi p, using h modl in (3.3) and manipulaing h odds form. W us nonlinar rgrssion wih h usual assumpion ha rrors ar normally disribud around zro. 3 W sima his modl for ach of h 600 obsrvaions using h mdian judgmn for ach obsrvaion. W also sima h modl for ach of h 40 individual subjcs. W sima boh h powr modl and h sysm-nglc modl (3.4). Th simas for h modls ar shown in Figur 4. Rcall ha sysm nglc prdics α m >α n and β m >β n for m > n. Ths ordrings provid 9 pairwis comparisons as mpirical ss of h sysm-nglc 3 Rsidual analysis indicas his is a good assumpion. 15
hypohss, 6 for h α-cofficins and 3 for h β-cofficins. All 9 pairs of simas ar in h dircion prdicd by sysm nglc, and all ar significanly diffrn (all comparisons, p <.01). 4 No also ha paramr valus ar boh grar han and lss han on. This implis h coxisnc of consrvaiv and radical blif-rvision, an imporan fac concald by h powr modl. Thr ar hr infrncs o draw from Figur 4. Firs, comparing h lvl of h sysm nglc plo wih h Baysian plo rvals h ovrall lvl of consrvaism: lowr lvls imply grar consrvaism. Scond, comparing h slop of h sysm-nglc modl wih h Baysian modl and compl-nglc modls rvals h dgr of sysm nglc: h spr h slop, h mor sysm nglc. Th paramr simas ar clos bu disinguishabl from compl nglc. Finally, comparing h sysm-nglc modl wih h powr modl rvals h bnfi of using h sysm-nglc modl: grar diffrncs imply grar valu. Figur 4 shows sysm nglc for boh paramrs. 5 W also simad h sysm-nglc modl for individual subjcs. Ovrall, his modl fis h individual-lvl daa rasonably wll, wih mos individual-lvl paramr simas ordrd as prdicd. Rcall ha h sysm-nglc hypohsis suggss ha as sysm paramrs incras, modl simas will dcras, suggsing 9 pairwis comparisons (6 for h α-cofficins and 3 for h β-cofficins). Across all subjcs, 79% of h α-cofficins and 77% of h β-cofficins ar ordrd in h prdicd dircion. Finally, w valuad prformanc a diffrn poins in im o considr h ffc of xprinc. Rcall ha subjcs xprinc 18 rials ovr h cours of h xprimn (in addiion o wo pracic rials) and hus migh larn o avoid sysm nglc. W divid h sssion ino four quarrs comprising of 4, 5, 5, and 4 rials, simaing sysm-nglc modls for ach quarr sparaly. Bcaus bhavior in h final 3 quarrs appars similar and larning appars limid o h firs quarr, w aggrga hos 3 quarrs and compar hm agains h firs quarr. Whil w 4 2 2 Th fis of h modls ar as follows: powr modl ( R =.86 ), sysm-nglc modl ( R =.95 ), and 2 Baysian modl ( R =.84 ). Sinc h purpos of h simaion is o s our psychological hypohsis, w do no mphasiz h goodnss of fi. 16
find significan sysm-nglc in boh priods, i is mor pronouncd in h firs quarr. Spcifically, w obsrv much grar snsiiviy o low ransiion-probabiliis in h firs quarr han in h subsqun quarrs. Nvrhlss, significan sysm-nglc prsiss hroughou h xprimn. Paramr valus for ach quarr ar rpord in ECP. Paramr Valus 2.0 1.5 1.0 0.5 0.71 (.03) 1.69 (.14) 1.16 (.08) 0.71 (.04) Sysm Nglc Compl Nglc Powr Modl Baysian 0.50 (.02) Paramr Valus 2.0 1.5 1.0 0.5 0.99 (.04) 1.78 (.11) 0.95 (.05) Sysm Nglc Compl Nglc Powr Modl Baysian 0.78 (.04) 0.0 α1 α2 α3 α4 Transiion Probabiliy Paramrs 0.0 β1 β2 β3 Diagnosiciy Paramrs Figur 4: Paramr simas for modls fiing Sudy 1 mdian daa using nonlinar rgrssion. Th lf panl dpics h α paramrs (ransiion probabiliy), and h righ panl dpics h β paramrs (diagnosiciy). Sandard rrors for h sysm nglc and powr modls ar in parnhss and indicad by h vrical bars. A vrsion of h compl nglc modl is givn as a horical baslin. No ha h compl-nglc modl is no uniqu and includs all paralll ranslaions. Discussion Sudy 1 rflcs sysm nglc a wo lvls. Firs, subjcs show h hypohsizd gradin of undr- and ovr-racion: undr-racion is mos prvaln in high diagnosiciy/high ransiion probabiliy sysms, and ovr-racion occurs mos in low diagnosiciy/low ransiion probabiliy sysms. Scond, his parn is rflcd in our formal s of sysm nglc. Esimaion of our quasi-baysian modl producd diagnosiciy and ransiion-probabiliy paramrs ha wr all ordrd consisnly wih h sysm-nglc hypohsis. Ths simas indica ha individuals ar snsiiv o changs in normaivly rlvan nvironmnal paramrs, bu insufficinly so. Indd, h paramr simas ar much closr o compl nglc han Baysian updaing. Ths findings cho h obsrvaions of prvious rsarchrs in non-saionary nvironmns (Chinnis & 5 W also simad a dcay modl in which mor disan obsrvaions rcivd lss wigh. Alhough his modl rvald rliabl dcay, h magniud was sligh and incorporaing his addiional paramr did no 17
Prson, 1968; Barry & Piz, 1979). Individual-lvl analyss provid similar suppor. Though hr is significan hrogniy, h vas majoriy of subjcs xhibi sysm nglc. On h whol, alhough w s slighly mor undr-racion han ovr-racion, w rira ha h sysm-nglc hypohsis is agnosic abou h ovrall lvl of undr- and ovr-racion. Rahr, sysm nglc prdics a rlaiv ffc, no an absolu on. 4. STUDY 2: JUDGMENT AND ERROR I is possibl ha h bhavior w obsrv in Sudy 1 is a saisical arifac gnrad by rgrssion ffcs or asymmric rror (Budscu, Erv, & Wallsn, 1997). For xampl, if h Baysian sandard calls for an sima of 1, h only possibl rror is undr-simaion. Erv al. show how an rror modl can xplain h co-xisnc of undr- and ovr-confidnc. Such rror modls ar incrasingly popular in boh psychology and conomics, and hav bn usd o xplain non-xpcd uiliy bhavior (Hy & Orm, 1994; Hy, 1995; Ballingr & Wilcox, 1997), as wll as ovrconfidnc (Erv, Wallsn, & Budscu, 1994; Brnnr, 2000). Whil hr is lil doub ha asymmric rror conribus o undr- and ovr-racion in boh xprimnal and ral-world sings, w bliv ha rror is no ncssary for producing h parns obsrvd in Sudy 1. To invsiga his, w conducd a scond judgmn sudy o s spcifically for h ncssiy of h rror xplanaion. In Sudy 2 w machd squncs o sysms o produc h sam Baysian posrior. In som cass h sysm was srongly suggsiv of chang (.g., an unsabl and prcis sysm) whil h signal was no, whil in ohr cass h signal was srongly suggsiv of chang whil h sysm was no (.g., a sabl and noisy sysm). This dsign allows us o diffrnia h sysmnglc hypohsis from an rror xplanaion. An rror modl of h yp suggsd by Erv al. would prdic no diffrncs in racion across h sysms sinc w hold h Baysian posrior consan. On h ohr hand, sysm nglc prdics a gradin idnical o h on shown in Sudy 1, in which suggsiv signals ar givn much mor wigh han suggsiv sysms. significanly improv h modl. 18
Mhodology Th mhodology usd is idnical o ha usd in Sudy 1 wih a fw noabl xcpions. Th xprimn consisd of 2 pracic rials followd by 30 rials, ach consising of 6 priods. Our sudy usd a 3 3 3 dsign, crossing hr diffrn ransiion probabiliy lvls (.05,.10, and.15), hr diffrn diagnosiciy lvls ( d = p / p of 1.5, 2.33, and 4), and hr Baysian posrior probabiliis (.4,.5, and.6). W consrucd 3 squncs for ach of h 9 sysm clls o corrspond o ach Baysian posrior probabiliy lvl. Thus, squncs wr machd o sysms o produc narly idnical posrior probabiliis. For xampl, squncs of R B ( r 1 = 1, r 2 = 1, r 3 = 1, r 4 = 0, r 5 = 0, r 6 = 0 ) and ( r 1 = 1, r 2 = 1, r 3 = 0, r 4 = 1, r 5 = 0, r 6 = 1) yildd posriors of.413 and.415 for ( d = 1.5, q =.05 ) and ( d = 4.0, q =.15 ), rspcivly. Th machs wr approxima, ranging from posriors of.381 o.420,.470 o.520, and.586 o.619. Th avrag across all 3 machs rangd from.491 o.511 across h 9 clls. W also includd 3 addiional fillr squncs o yild a oal of 30 squncs. Each of h subjcs rcivd all 30 squncs in a randomizd ordr. Th acual squncs ar found in ECP. Th nd o mach Baysian posriors across diffrn sysms influncd our dcision o us squncs of 6 obsrvaions (as opposd o 10), as wll as mor modra paramr valus (.g., maximum diagnosiciy of 4) and Baysian posriors (40-60%). Sinc h squncs wr no drawn randomly, w also machd h frquncy of h acual rgims o h Baysian posriors. For xampl, of h 15 insancs in which h Baysian posrior was bwn.45 and.55, w l 8 of 15 (53%) b gnrad by h blu bin and 7 of 15 (47%) b gnrad by h rd bin. W rcruid 32 Univrsiy of Chicago undrgraduas as subjcs. Th scoring schm was similar o ha usd in Sudy 1. W usd a quadraic scoring sysm ha paid $0.08 maximum (.g., if a subjc indicad wih crainy ha h procss was in h blu rgim, and h procss was in fac in h blu rgim) and -$0.08 minimum (.g., if a subjc indicad wih crainy ha h procss was in h blu rgim, and h procss was in fac in h rd rgim). 19
Subjcs wr givn fdback a h nd of ach rial as o if and whn h procss shifd from h rd rgim o h blu rgim. Thy wr also informd how much mony hy mad (or los) on ha paricular rial. Rsuls W prsn boh aggrga and individual-lvl rsuls. Rcall ha our xprimn rquird ha ach of our 32 subjcs provid subjciv probabiliis ha h procss had swichd o h blu rgim for ach of 6 signals in 30 rials. L b p b h mpirical judgmn and p p b h absolu diffrnc bwn h subjc s judgmns and h Baysian probabiliy for signal. Paymns wr basd on h diffrnc bwn hir subjciv probabiliy of a chang o h blu rgim and h acual rgim (1 if blu, 0 if rd). Th man of his absolu diffrnc was.33 (mdian=.20, sd=.35), gnraing an avrag paymn of $7.70 (rang of $4.80 o $9.35). Th avrag paymn o a Baysian agn would hav bn $10.16. b To s for sysm nglc, w considr how our masur of undr-racion, p6 p6, changs across h 9 xprimnal clls. W considr only h 6 h obsrvaion sinc posriors ar machd only on his obsrvaion. W show his masur poold across h 3 posrior probabiliy lvls in Figur 5. W s h prdicd gradin: ovr-racion nds o b srongs whn diagnosiciy and ransiion probabiliy ar low, and undr-racion nds o b srongs whn diagnosiciy and ransiion probabiliy ar high. In pairwis comparisons of clls, h parn of undr-racion is gnrally as prdicd, wih pairwis comparisons corrcly ordrd in 23 of 27 comparisons. W also prformd a similar individual-lvl analysis of h daa. For ach subjc, w b considrd h parn of undr- and ovr-racion, avraging h masur of racion, p6 p6, ovr ach of h 3 Baysian posrior lvls. W hn cound how many of h 27 pairwis comparisons wr ordrd as rquird by sysm nglc. 29 of 32 subjcs had a majoriy of comparisons in h prdicd dircion (14 or highr of 27). Th numbr of pairwis comparisons in h prdicd dircion rangd from 5 o 27 (mdian=19, man=18.6). 20
Esimaion W fi h sysm-nglc modl o h Sudy 2 daa using h sam basic procdur oulind arlir. As bfor, w fi p, using nonlinar rgrssion and h modl in (3.3). This modl was simad for h 162 obsrvaions using h mdian judgmn for ach obsrvaion. Th simas ar found in Figur 6. Th parn of paramr simas looks vry similar o ha found wih Sudy 1. Th slop of h simas is consisn wih sysm nglc, and h paramr simas ar clos o wha would b obaind by a compl-nglc modl. 20% Ovr-Racion Mdian Dviaion: Empirical - Baysian 10% 0% -10% -20% 1.5-30% 5% 10% Transiion Probabiliy Undr-Racion 15% 4 2.33 Diagnosiciy Figur 5: Ovr- and undr-racion, by condiion, for ach of h 3 lvls of posrior probabiliy and aggrgad ovr h 3 lvls, as masurd by h diffrnc bwn mdian mpirical b probabiliy judgmns and h mdian Baysian probabiliis, p p (Sudy 2) Discussion 6 6 Sudy 2 invsigad h possibl rol of random rror in h parn found in Sudy 1. Whras asymmric rror almos crainly conribus o sysm nglc, Sudy 2 indicas hr is mor o sysm nglc han jus rror. W find h sam srong parn of undr- and ovr-racion prdicd by h sysm-nglc hypohsis vn afr nuralizing h rol of rror. This sudy also dmonsras h xn o which signals ar privilgd ovr sysms. By dsign, h gradin w obsrv in his sudy rvals h rlaiv mphasis of signals ovr sysms. 21
Unlik Sudy 1, h squncs usd in Sudy 2 wr no randomly gnrad. W did so o cra a vry sark s bwn h rror accoun and sysm nglc. In hory, his approach could bias judgmns in favor of h sysm-nglc hypohsis, as subjcs may discoun sysm paramrs afr obsrving a squnc ha is no rprsnaiv of a paricular sysm. W rgard his possibiliy as highly unlikly sinc i would rquir ha subjcs ar snsiiv o boh h undrlying sysms and h rprsnaivnss of squncs drawn from hm. Ovrall, w bliv h approach in his sudy complmns Sudy 1, in which w us randomly gnrad squncs. Paramr Valus 1.5 1.0 0.5 1.16 (.06) 0.80 (.04) Sysm Nglc Compl Nglc Baysian 0.51 (.03) Paramr Valus 2.0 1.5 1.0 0.5 1.69 (.11) 1.13 (.08) Sysm Nglc Compl Nglc Baysian 0.68 (.06) 0.0 0.0 α1 α2 α3 β1 β2 β3 Transiion Probabiliy Paramrs Diagnosiciy Paramrs Figur 6: Paramr simas for modls fiing Sudy 2 mdian daa using nonlinar rgrssion. Th lf panl dpics h α paramrs (ransiion probabiliy), and h righ panl dpics h β paramrs (diagnosiciy). Sandard rrors for h sysm-nglc and powr modls ar in parnhss and indicad by h vrical bars. A vrsion of h compl-nglc modl is givn as a horical baslin. No ha h compl-nglc modl is no uniqu and includs all paralll ranslaions. 5. STUDY 3: CHOICE TASK Th xprimnal dsign of Sudy 3 was similar o ha of Sudis 1 and 2. Th major diffrnc was ha subjcs wr askd o prdic h color of h nx ball rahr han o provid a probabiliy sima. Exprimnal Dsign Th program, wrin in Visual Basic, includd svral scrns of inroducion, 2 pracic rials, and 18 rials of 10 priods ach. For ach rial, subjcs wr shown h rlvan paramrs, p R, p B, and q, govrning ha rial. Thy wr hn shown a squnc of rd or blu balls drawn 22
randomly basd on h s of paramrs. Bfor sing ach signal, subjcs wr askd o prdic h color of h nx ball. As in Sudy 1, w varid diagnosiciy and ransiion probabiliy. Sudy 3 usd almos h sam paramrs as Sudy 1: ( p, p ) = (.6,.4), (.75,.25), and (.9,.1) and q =.025,.05,.10, and.20 R B ( q =.02 was usd in Sudy 1). Thus our 3 4 dsign yilds 12 xprimnal condiions. W randomly gnrad 3 uniqu squncs for ach of hs 12 condiions, craing 36 oal squncs. Each subjc xprincd half of h squncs, and ihr 1 or 2 of h 3 squncs from ach condiion. Squncs wr randomizd for ach subjc. W rcruid 50 Univrsiy of Chicago sudns o paricipa in his sudy. Th mdian numbr of undrgradua and gradua mahmaics and saisics classs akn by our subjcs was again 3. Subjcs did no rciv fdback rgarding if or whn a chang acually occurrd, nor wr hy shown h opimal (Baysian) rsponss. Thy did, howvr, find ou immdialy whhr hir prdicion was corrc. W paid subjcs 9 cns for ach corrc prdicion (ou of 180). Normaiv Modl As in Sudy 1, w us h Baysian rspons as h sandard for valuaing subjc bhavior. Rcall ha (3.1) givs us h Baysian probabiliy ha h procss has swichd o h blu rgim b a 1: p 1 = Pr( B 1 H 1). Th probabiliy, hn, ha a ball is drawn from h blu rgim a, Pr( ) p + (1 p ) q, and h probabiliy ha a blu ball is obsrvd, Pr( r = 0 1), is b b B H 1, is 1 1 ( p + (1 p ) q)(1 p ) + (1 p )(1 q)(1 p ), which is grar han.5 if and only if b b b 1 1 B 1 R Pr( B H ) = p + (1 p ) q >.5. Normaivly, hn, a subjc should prdic a blu ball if sh b b 1 1 1 blivs h procss is in h blu rgim, and prdic a rd ball ohrwis (assuming h rwards for corrc prdicions ar symmric, as is ru in our sudy). Exprimnal Rsuls W bgin by comparing h accuracy of h mpirical prdicions o h accuracy of h Baysian prdicions. Th Baysian prdicions (69%) wr br han h mpirical prdicions (64%). Howvr, hr was considrabl hrogniy in accuracy across subjcs, wih accuracy H 23
ras ranging from 59% o 73%. Thus, paymns rangd from $9.63 o $11.79 (man=$10.62). A Baysian agn would mak $11.20 on avrag. As in Sudy 1, our primary inrs is blif rvision. In h prsn conx, w obsrv blif rvisions whn subjcs chang hir prdicions. L c ( c b ) b h subjc s prdicion (Baysian prdicion) of h color of h -h ball ( c = 0 if a blu ball is prdicd). Thus, an mpirical blif-rvision corrsponds o c c 1, whras a Baysian blif-rvision corrsponds o c b b c 1. Fwr mpirical han Baysian blif-rvisions implis undr-racion, whil h opposi implis ovr-racion. Across all condiions, w obsrv blif rvisions a h ra of 16.1% (mpirical) and 11.3% (Baysian). Thus, on avrag, subjcs rvis hir prdicions 42% mor ofn han h normaiv prdicion. By his masur, hr is an ovrall ndncy o ovr-rac. W nx urn o how his masur varis across xprimnal condiions. Rcall ha sysm nglc implis ovr-racion will b mos xrm in sabl/noisy nvironmns and las xrm in unsabl/prcis nvironmns. Figur 7 dpics h diffrnc bwn h prcnag of mpirical and Baysian blif-rvisions across all 12 xprimnal condiions. Posiiv valus indica ha mpirical changs xcd Baysian changs, and, hus, ovr-racion. W obsrv h prdicd gradin bwn h souhas cll (unsabl/prcis) and h norhws cll (sabl/noisy). Undrracion is concnrad in h souhas cornr and ovr-racion in h norhws cornr, as sysm nglc suggss. As an alrnaiv masur of ovr- and undr-racion, w considr h amoun of vidnc rquird o induc a blu-ball prdicion. Normaivly, his hrshold should no vary across condiions: as soon as h daa suggs ha h blu rgim is mor likly han h rd rgim, subjcs should prdic a blu ball. Th sysm-nglc hypohsis, on h ohr hand, suggss ha individuals will rquir subsanially mor vidnc, objcivly masurd, in unsabl/prcis nvironmns han in sabl/noisy nvironmns. To s his prdicion, w analyzd bhavior in h following way. For vry rial ha a subjc compld, w idnifid h priod in which sh firs prdicd a blu ball. This priod indicas h poin a which h subjc blivs a rgim shif has occurrd. W hn calcula h 24
normaiv probabiliy undrlying ha prdicion, i.., h Baysian prior ha a blu ball will b drawn in ha priod. This is h vidnc, objcivly masurd, on which h subjc bass hr prdicion. Finally, w ak h man of hs probabiliis for ach xprimnal condiion, pooling all rials and subjcs wihin ach condiion. Figur 8 compars hs mans across all 12 xprimnal condiions. 25% Ovr- / Undr-Racion 20% 15% 10% 5% 0% Ovr-Racion -5% 1.5-10% 2.5% 5.0% Transiion Probabiliy Undr-Racion 10.0% 20.0% 9 3 Diagnosiciy Figur 7: Ovr- and undr-racion as masurd by blif rvisions (Sudy 3, Choic Task). Plod ar h diffrncs in h frquncy of Empirical and Baysian blif-rvisions for ach condiion. As prdicd, hrsholds ar considrably highr in h souhas cornr han in h norhws. On avrag, subjcs in h mos unsabl/prcis nvironmn rquird vidnc indicaing a.90 chanc ha h sysm has shifd o h blu rgim in ordr o mak hir firs blu prdicion, i.., hy undr-racd. Convrsly, whn in h mos sabl/noisy nvironmn hy rquird only a.11 chanc ha h sysm has shifd o h blu rgim in ordr o mak hir firs blu prdicion, i.., hy ovr-racd. Esimaion To formally s h sysm-nglc hypohsis for our choic ask, w adap our simaion procdur o accoun for h binary dpndn variabl. W ak a singl-agn sochasic choic modl approach (.g., Camrr & Ho, 1994; Wu & Gonzalz, 1996), modling a rprsnaiv 25
agn wih nois (s also, McFaddn, 1981). Thr ar wo sps o his procss. In h firs sp, w obain a subjciv probabiliy of drawing h nx ball from h blu rgim. W assum subjcs bas hir prdicion on his probabiliy. In h scond sp, w ransform his subjciv probabiliy ino a sochasic prdicion abou h nx obsrvaion, i.., h probabiliy ha subjcs will prdic a blu ball. Th objciv of his modl is o mach his probabiliy wih h obsrvd prcnag of subjcs prdicing a blu ball. 90% 80% 70% Thrshold 60% 50% 40% 30% 20% 10% 0% 3 1.5 Diagnosiciy 2.5% 5.0% 10.0% Transiion Probabiliy 20.0% 9 Figur 8: Man Baysian probabiliy of having changd o h blu rgim a h im of h firs blu prdicion (Sudy 3, Choic Task). Baysian probabiliis ar calculad a h im of h firs blu-ball prdicion by ach subjc in ach rial. This summary pools subjcs and rials wihin ach condiion. Rcall ha p 1 dnos h subjciv probabiliy ha h 1-h ball was drawn from h blu rgim. Sinc our subjcs prdic h -h ball, w nd o advanc p 1 on priod as w did wih h normaiv modl. Thrfor, w ak π ( 1 (1 = p + p 1) q)(1 pb) + (1 p )(1 q)(1 p ) o b h subjciv probabiliy ha a blu ball will b drawn from h blu 1 R b rgim a. No ha his is simply h normaiv xprssion wih p 1 subsiud for p 1. Nx w fi his subjciv probabiliy using (3.3) o h obsrvd prdicions. W dno h prcnag of subjcs who prdicd a blu ball for ball, % B. In h normaiv modl, subjcs 26
would prdic a blu ball if π >.5 (hus, % B i would always b ihr 0 or 1). To fi mpirical obsrvaions, w assum a sochasic choic funcional which allows h rprsnaiv agn o b imprfcly snsiiv o π, ( + η ψπ ) 1/ 1 xp, (5.1) whr π is simad, as daild in h prvious paragraph, using ihr h powr modl or h sysm-nglc modl. No ha h rspons funcion capurs how discriminaing h parn of prdicions is o changs in π wih discriminabiliy incrasing in η, and ha (5.1) is symmric if η=.5ψ. W sima η and ψ (as wll as h paramrs ndd o fi π ) using nonlinar lassquars. W ak h rror o b h diffrnc bwn (5.1) and % B, h prcnag of subjcs who prdicd a blu ball, and choos h paramrs ha minimiz h sum of squard rrors. Paramr Valus 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 3.35 (.23) 0.71 (.10) 2.15 (.15) Sysm Nglc Compl Nglc Powr Modl Baysian 1.15 (.07) α1 α2 α3 α4 Transiion Probabiliy Paramrs 0.69 (.05) Paramr Valus 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 2.97 (.31) 1.41 (.09) 0.91 (.07) β1 β2 β3 Diagnosiciy Paramrs Sysm Nglc Compl Nglc Powr Modl Baysian Figur 9: Paramr simas for modls fiing Sudy 3 using h sochasic choic funcional. Th lf panl dpics h α paramrs (ransiion probabiliy), and h righ panl dpics h β paramrs (diagnosiciy). Sandard rrors for h sysm-nglc and powr modls ar in parnhss and indicad by h vrical bars. A vrsion of h compl-nglc modl is givn as a horical baslin. No ha h compl-nglc modl is no uniqu and includs all paralll ranslaions. Th simas for h powr modl and h sysm-nglc modl ar found in Figur 9. Mos significanly, modl simas for boh paramrs ar ordrd in h mannr prdicd by h sysm-nglc hypohsis. As in Sudy 1, w ak his ordring as h formal s of h sysmnglc hypohsis. Rcall ha sysm nglc prdics ha paramr simas will dcras as 0.50 (.05) 27