Strategies for E stim ating B ehaviou ral F requency in Survey Interview s

Transcription

1 Ó M E M O R Y, , 6 (4 ), 33 9 ±3 66 Strategies for E stim ating B ehaviou ral F requency in Survey Interview s F re derick G. C onrad U S B ureau of Labor Statistics N orm an R. B row n U niversity of Alberta, C anada E rin R. C ashm an U niversity of Illinois, U SA W hen people answ er survey questions of the form ``D uring the past m onth, how m any tim es d id yo u...? their respo n se s p ro v ide v aluable data for rese archers and policy m akers. Y et the w a y respondents produ ce their answ ers to these ``b eh aviou ral f req u e n c y q u estio n s is n o t w e ll u n d ers to o d. T h is a rtic le dem onstrates that su rvey resp ondents can use an array of distinct estim ation strateg ies, depen d ing on w h at inform ation is available in their m em o ries. T h e k ind of event inform ation that people use is related to factors such as the regularity of occurrence, sim ilarity of one episode to the next, and frequency. In a study conducted as a telephone su rvey, resp ondents verbal reports and resp onse-tim e patterns indicate that they usually answ er behavioural frequency questions by either retrieving and co unting episodes, re trieving or estim ating rates o f occurrence, or converting a general im pression of frequency into a num erical quantity. T he third strategy should be of particular concern to survey researchers be cause respo ndents provide a qu antitative estim ate w ithout any releva nt num erical know ledge. T he set of strategies and the factors that influence their use are integrated into a statistical m odel that could help su rvey practitioners to im prove data quality and m em ory researchers to broaden their perspective. R equests for reprin ts should be sent to F rederick C onrad, B ureau of Labor Statistics, 2 M a ssa ch u se tts A v e nu e N E, S u ite 49 15, W as h in gton, D C , U S A. E m a il: C o nra d b ls.g o v W e are grateful to B rian H arris -K ojetin, Steve M iller, and C lyde T ucker for their invaluab le statistical advice. W e thank Jim K ennedy, Paul M ullin, Stanley P resser, D an W rig ht, and an anonym ous review er fo r discussing earlier versions of the m anuscript. W e also thank D avid C antor, Jam es D oree, M arj F lem ister, C lyde T ucker, and Jon Tru sw ell fo r logistical and m ethodological assistance. The view s expressed here are those of the authors alone an d do not necessarily reflect the vie w s of the ir in stitutio ns. T h is re se a rc h w a s sup p orted in p a rt by an N S E R C o pe ra ting g ra n t a w a rd e d to th e se c o nd a uth o r Psych ology P ress L td

2 340 CONRAD, BROWN, CASHMAN INTRODUCTION People estim ate various frequencies on an everyday basis. For exam ple, w hen people seek m edical treatm ent, they are typically asked to describe the freq u enc y o f th eir sy m p to m s. W he n th ey co n sid er m o ving to a ne w neighbourhood, they m ight assess how frequently they have heard about crim e in that area. W hen they purchase a new product, one consideration m ight be how often they believe they w ill use the product. T his article concerns the w ay people estim ate the frequency of autobiographical events w hen asked about their behaviour on survey questionnaires. W e report the results of a study conducted as a telepho ne survey in w hich respondents w ere tim ed as they answ ered questions about the frequency of their ow n behaviour. A fter answ ering each question, they described how they arrived at their answ er and then rated the regularity and sim ilarity of the target events. So-called behavioural frequency questions are usually of the form ``D uring the last m onth how m any tim es did you pay to see a m ovie? w here both the tim e fram e and event category can vary. T he responses to these questions have w idesp read political and econom ic im plications, so there is social value in understanding how they are form ulated. For exam ple, the m onthly unem ploym ent rates reported by the U S governm ent are based, in part, on responses to behavioural frequency questions on federal surveys. In addition, there are good m ethodological reasons for studying perform ance on this task. It is a relatively fam iliar task for m ost people and it is carried out in naturalistic settings, w ith stim uli (autobiographical events) that have personal m eaning, yet answ ering behavioural frequency questions can be studied w ith som e experim ental control. T here are relatively distinct bodies of literature on how people answ er behavioural frequency questions in survey situations and how they estim ate frequency in laboratory tasks. Survey m ethods researchers (e.g. B lair & B urton, 1987; B urton & B lair, 1991; M eans & L oftus, 1991; M enon, 1993) have focused their studies on the use of tw o response strategies: episode enum eration (counting all retrieved instances of the event category w ithin the specified tim e period) and rate-based inferences (recalling that the behaviour happens once a w eek and reasoning that it m ust, therefore, have occurred four tim es in the last m onth). C ertain conditions prom ote the use of one strategy over another and these have been m anipulated experim entally (e.g. B lair & B urton, 1987). T his is essentially, a dual-strategy view in w hich responses are based on either counts of retrieved episodes (enum eration) or retrieved rates of occurrence. In this article w e present evidence that people use additional strategies for answ ering these questions but the im portant point is that survey respondents are assum ed to be capable of using m ultiple strategies. In contrast, individual laboratory studies have tended to focus on the use of individual strategies. For exam ple, T versky and K ahnem an (1973) dem onstrated that people use the ease of retrieving instances as a key to frequencyð the

3 STRATEGIES FOR ESTIMATING FREQUENCY 341 availability heuristic. H intzm an (1988) m odelled frequency judgem ents as a com parison of a target item s sim ilarity to exem plars in m em oryð a sim ilarity principle. B arsalou and R oss (1986) provided evidence that subjects retrieve and count category instances w hen reporting category frequencyð an enum eration strategy. A lthough m any m em ory researchers have noted that subjects m ay estim ate frequency in m ore than one w ay (B egg, M axw ell, M itterer, & H arris, 1986; B ruce, H ockley, & C raik, 1991; H intzm an, 1976; H ow ell, 1973; Johnson, R aye, W ang, & T aylor, 1979; Jonides & Jones, 1992; Jonides & N aveh- B enjam in, 1987; V oss, V ereb, & B isanz, 1975), the data from laboratory studies point prim arily to the use of individual strategies. In a series of laboratory studies that are in m any w ays analogous to the current survey-form at study, B row n (1995) illustrated that individual subjects used m ultiple strategies to assess the frequency of item s that they had recently studied. H e used a co m bination of verbal protocols and resp onse tim es to differentiate the strategies. In the current article, w e present a study of how respondents answ er behavioural frequency questions w hich is intended to integrate and extend findings from these tw o bodies of literature. W e identify a range of strategies that differ in their conditions of use as w ell as in their underlying processes, and w e suggest that respondents use w hichever strategy is best served by the available inform ation. THE MULTIPLE STRATEGY PERSPECTIVE O ur central idea is that people estim ate frequency based on the contents of their m em ory or, in the absence of relevant content, the state of their m em ory. In order to count episodes, they m ust be able to retrieve episodes; in order to infer frequency based on typical rate of occurrence they m ust be able to retrieve rate inform ation. W hen they cannot retrieve this type of inform ation, they m ay be able to use qualitative im pressions of frequency, such as ``that happened a lot, converting the im pression into a num erical estim ate. W hen none of this inform ation is directly available, people can evaluate the state of their m em ory to derive frequency inform ation. W e term this m em ory assessm ent. T he availability heuristic (Tversky & K ahnem an, 1973) is a good exam ple: T he m ore difficult it is to retrieve instances, the less frequent they are assum ed to be. In principle, one can alm ost alw ays assess the state of one s m em ory, regardless of w hether episodes, rates, or other relevant inform ation can be retrieved. C onsequently, w e view m em ory assessm ent strategies as default procedures. T his view im plies that if one could know the value of certain variables, one could at least rule out the use of certain strategies. For exam ple, in order to enum erate episodes, one needs to be able to distinguish one rem em bered episode from the next. R em em bered episodes w ill be relatively distinguishable if they occur on an irregular schedule and do not have sim ilar characteristics. If one can

4 342 CONRAD, BROWN, CASHMAN determ ine just how distinctive one episode is from another, it w ould be possible to determ ine w hether an enum eration strategy is feasible. W e discuss certain variables that have been associated w ith event distinctiveness later. From a practical view point, this m eans survey researchers m ay be able to determ ine w hat strategies respondents are capable of using and to rule out others because the necessary inform ation is lacking. T his m ay be useful, for exam ple, in evaluating response accuracy. A n enum eration strategy is m ore likely to lead to underreporting than overreporting because people are m ore likely to forget individual episodes than to recall episodes that either did not happen or occurred outside the relevant tim e interval (Sudm an & B radburn, 1974). A m ethodological intervention that increases reported frequency can be assum ed to be increasing response accuracy if one can be sure respondents are enum erating. H ow ever, if the episodes about w hich they are questioned are not distinctive, then enum eration is unlikely and the assum ptions about accuracy m ay need to be questioned. (N ote, that w e do not address response accuracy in this article but concentrate instead on the processes that underlie frequency estim ates.) A lthough the m ultiple strategy view leads to clear predictions about w hen certain strategies w ill not be used, it is less decisive about w hen a particular strategy w ill be used. T his is because m ore than one kind of relevant inform ation m ay be available. For instance, a film enthusiast m ight know that he or she goes to the m ovies tw ice a w eek (rate inform ation) and m ight also be able to recall the nam es and plots of the last several m ovies attended (episode inform ation); either source of inform ation could be sufficient for an estim ate, so the issue is w hich source respondents w ill use. Fortunately, there are a num ber of factors know n to affect strategy choice and the w ay these factors tend to com bine m akes one strategy m ore likely to be used than another. L ater in the article, w e report a statistical analysis in w hich w e exam ined the com bined effect of several variables on the use of particular strategies. Factors That Affect Strategy Use Several factors are know n to affect strategy use. B lair and B urton (1987) and B urton and B lair (1991) point to the im portance of actual frequency in predicting strategy use. T hey found that episodes tend to be enum erated w hen frequency is low ; that is, w hen there are relatively few episodes to recallð rate inform ation tends to be used w hen frequency is higher. Frequency w as m anipulated by eliciting judgem ents for either one-m onth or six-m onth reference periods, the idea being that frequency is greater for longer tim e periods. T hey also found that the tim e allocated to respond affects strategy selection: m ore tim e led to m ore episode enum eration. Finally, question w ording had a w eak effect on response strategy: ``H ow often...? produced

5 STRATEGIES FOR ESTIMATING FREQUENCY 343 m ore responses expressed as rates than ``H ow m any tim es...?. In their discussion, the authors speculate that the regularity and distinctiveness of events m ay affect strategy as w ell (B lair & B urton, 1987, p.2 87; B urton & B lair, 1991, pp.6 4±65). T he regularity w ith w hich events occur has subsequently been show n to affect how respondents produce behavioural frequencies. M eans and L o ftus (1991) found that activities that presum ably occur irregularly, like visiting the doctor for an injury, led to frequency judgem ents based on recalled incidents m ore than did activities like visiting the allergist, w hich presum ably occur on a regular schedule. T hey also show ed that sim ilar episodes (m inor m edical conditions) w ere recalled less often than distinctive on es (serious m edical conditions), although they did not show that such activities led respondents to use rate know ledge instead of enum erating episodes. T hey did not m easure event regularity or distinctiveness, and did not look at both variables together. M enon (1993) has system atically explored the effects of event regularity and sim ilarity on strategy choice. In a series of studies, she found that events w ere prim arily enum erated w hen event instances occurred on an irregular basis and w hen subjects considered them to be distinctive (low in rated sim ilarity). O ther levels of these event characteristics led to increased use of rate know ledge. M enon s perspective differs from our ow n in that she adopts a strict du al stra teg y vie w Ð resp on den ts eith er en um era te or use ra te inform ation, but there is no role for non-num erical strategies such as those that involve general im pressions or m em ory assessm ent. H ow ever, like M enon (1993), w e give considerable w eight to the role of regularity and sim ilarity in strategy choice. Predictions T he m ultiple strategy perspective leads to several predictions. First, the regularity w ith w hich instances of an event occur w ill be related to the use of rate-based strategies and (indirectly) to the use of enum eration-based strategies. W e assum e that people are likely to acquire rate inform ation w hen events occur on a relatively regular schedule. If som eone perform s a certain activity every Thursday from 2:00 to 3:30, it is likely the person w ould becom e aw are of this. In contrast, it is unlikely som eone w ould have w eekly rate inform ation available for an event that m ay occur several tim es in one w eek, and then not again for several w eeks. W e predict, therefore, that strategies based on rate inform ation w ill be used m ore often for regularly occurring than for irregularly occurring events. Second, regularity should reduce the accessibility of event instances. T his is because a regularly occurring event has identical tem poral characteristics each tim e it takes place, so people m ay poorly encode the separate episodes. In other w ords, all else being equal, instances of regularly occurring events should be

6 344 CONRAD, BROWN, CASHMAN m ore sim ilar to one another than instances of irregularly occurring events. In turn, this im plies that instances of regularly occurring events should be m ore difficult to retrieve than instances of irregularly occurring events, and hence less likely to evoke enum eration. T hus, there are tw o reasons for predicting that ratebased responses w ill be m ore com m on for regularly occurring events: such events prom ote the encoding of rate inform ation and the decontextualisation of individual events. A num ber of autobiographical m em ory studies (e.g. B arclay & W ellm an, 1986; B rew er, 1988; L inton, 1982; N eisser, 1986; W agenaar, 1986) have dem onstrated that events judged to be sim ilar to m any others are hard to recall and recognise. H ighly sim ilar episodes lead people to represent com m on features of those episodes, but not features that distinguish one from the next (N eisser, 1981; Strube, 19 87). T herefore, our seco nd prediction is that enum eration should be rare w hen instances of a target activity closely resem ble one another, and should be used m ore often as the episodes becom e m ore distinctive and hence are easier to retrieve. O ur third prediction concerns the tim es to answ er a frequency question. In the current study, response tim es are used as a m eans of corroborating the subjects retrospective strategy reports. W e assum e that it takes a fixed am ount of tim e to retrieve one episode, so that the m ore episodes retrieved the longer the response tim e (B ousfield, Sedgew ick, & C ohen, 1954; B row n, 1995). In contrast, w e assum e that people associate rate know ledge w ith particular event categories and (as a result) it is directly retrievable. It should take no m ore tim e to retrieve a rate of once a day than of once a m onth, all else being equal. O ur third prediction, therefore, is that response tim es should increase w ith estim ated frequency w hen respondents enum erate episodes, but there should be no relation betw een response tim e and estim ated frequency w hen they use a rate. Sim ilarly, there is no reason, a priori, w hy response tim es sho uld be related to estim ated frequ enc y w hen respo nde nts use heuristics like availability. In sum m ary, w e are predicting that enum eration w ill be relatively com m on w hen event instances are distinctive and occur at irregular intervals, and that rate-based resp onses w ill be prevalent w hen the events occur at regular intervals. W e are also predicting that response tim es w ill increase w ith reported frequency w hen people enum erate, and that they w ill not w hen people rely on rate-based strategies. N ote that, although w e assum e that respondents have m ultiple strategies available to them, w e have focused our predictions on enum eration and rate-based strategies. T here is little em pirical evidence about w hat other strategies respondents m ight use to answ er frequency questions on surveys. In light of this, our study has an exploratory character to it as w ell. W e designed the study to identify these additional strategies and to exam ine w hich factors (am ong sim ilarity, regularity, reported frequency, and response tim e) are related to their use.

7 STRATEGIES FOR ESTIMATING FREQUENCY 345 METHOD Design The study w as conducted as a telephone survey. Three experienced interview ers asked 106 respondents to estim ate the frequency of 10 autobiographical events and to rate the regularity and sim ilarity of those events. 1 T he respondents w ere contacted from a sam ple of 250 telephone num bers. T hese num bers w ere random ly draw n from a nationally representative sam ple, based on rural and m etropolitan areas of the U nited States and stratified by C ensus region and population. If potential respondents declined to participate, interview ers did not attem pt to persuade them to take part. E ach interview w as audiotaped w ith the respondent s consent. Procedure The respondents w ere presented w ith four tasks. First they w ere asked to answ er 10 behavioural frequency questions (see T able 1). T hey w ere told to think silently before answ ering, and to take as m uch tim e as they needed, but to respond as soon as they had m ade up their m ind. Second, after answ ering each question, the respondents w ere asked to explain how they cam e up w ith the answ er; that is, they w ere asked for a retrospective protocol (E ricsson & Sim on, 1993). 2 The interview ers occasionally probed if they felt the protocol w as not inform ative. In the third task, the interview ers presented the 10 events in the T able 1 questions and asked respondents to rate the regularity of each on a 4-point scale. R espondents w ere told that if they engaged in an event on a routine schedule it w as V ery R egular (4); if they never engaged in it on a routine schedule it w as V ery Irregular (1). The interm ediate response options w ere Som ew hat R egular (3) and Som ew hat Irregular (2). T he respondents w ere explicitly instructed not to confuse (in)frequent events w ith (ir)regular events. T he fourth task required respondents to judge the sim ilarity of the events on a 4-point scale. T hey w ere told that if their experiences w ere virtually the sam e each tim e they engaged in a behaviour then they should rate it V ery Sim ilar (4); if each encounter w ith the event seem ed like a totally unique experience, then it should receive a V ery D ifferent (1) rating. For sim ilarity ratings betw een these extrem e values respondents w ere given the option of Som ew hat Sim ilar (3) and Som ew hat 1 T he e ve n t c a te go rie s w e re c h ose n to va ry in the sim ilari ty a n d re g ula rity o f th e ir in dividu a l instances on intuitive grounds; how ever, this w as not system atically contro lled or evaluated, and is not discussed fu rther. 2 W e c olle c te d ret ro sp e c tive a s o pp ose d to c o nc u rre nt p ro toc o ls in o rd er to o bta in stra te g y rep o rts th a t did n ot a ffe c t th e tim e of th e re sp o nse in th e firs t ta sk. A lth ou gh re tro sp e c tiv e p ro to co ls ca n be a le ss d ire c t m e a sure o f m e nta l p ro c e ss, the y do n ot in te rfe re w ith the prim ar y ta sk a s c on c urr en t think - aloud m ethods can (Schooler, O hlsson, & B rooks, 1993 ).

8 346 CONRAD, BROWN, CASHMAN TABLE 1 Stimulus Questions D ur ing th e la st m o nth, h ow m a ny tim e s d id yo u conduct a transaction w ith an A utom ated T eller M achine? 2. perform a transaction w ith a teller in a bank? 3. sho p in a g ro c e ry store? 4. shop in a departm ent store? 5. purchase gas fo r your car? 6. pa y to ha v e yo u r ca r re p a ire d? 7. receive subscrip tion m agazines by m ail? 8. receive catalogues by m ail? 9. eat ice cream? 10. eat spicy fo od? D ifferent (2). W e settled on the num ber of points and the particular verbal labels for both the regularity and sim ilarity scales after pilot testing the rating tasks. T his approach to assigning sim ilarity and regularity ratings to stim ulus events differs from others in the literature. In particular, M enon (1993) collected ratings from independent judges and established a set of stim uli that w ere rated either high on both regularity and sim ilarity, low on both, or high on one and low on the other. A particular event w as then considered fixed in these characteristics for all respondents. T his w as effective under M enon s approach in part because her stim ulus events w ere relatively fine-grained (e.g. snacking, w ashing one s hair) and her subjects w ere of relatively uniform dem ographics (undergraduate stu den ts). O ur stim ulu s e vents w ere so m ew hat m o re com p licated (e.g. perform ing a transaction w ith a teller in a bank), and our respondents m ore diverse dem ographically (a stratified, national sam ple). T his m eans that in our sam ple, tw o different respondents m ight experience events quite differently. C onsider a social security recipient w ho deposits her cheque at the bank each m onth w hen it arrives and interacts w ith the sam e teller each tim e. T his event w ould be quite regular and sim ilar. In contrast, a busy professional w hose salary is directly deposited and w hose transactions are prim arily w ith an autom ated teller m achine, w ould be likely to interact w ith a hum an teller irregularly (although each interaction m ay be sim ilar to the next). U nder our approach, an event s regularity and sim ilarity w ere based on w ithin-subject judgem ents. This m eant that one respondent could judge an event to be V ery R egular and another respondent could rate the sam e event V ery Irregular, and subsequent analyses could take into account such individual differences. For this reason, average regularity and sim ilarity ratings are not particularly im portant in our analyses. T hree different random orders of the events w ere generated for each respondent, one for eliciting the frequency reports and retrospective protocols, one for the regularity rating task, and one for the sim ilarity rating task. A single

9 STRATEGIES FOR ESTIMATING FREQUENCY 347 reference period of one m onth w as used for the frequency questions. O ur pilot study indicated that respondents found it disruptive to shift reference periods betw een questions and that zero responses w ere relative infrequent across the events for this tim e fram e. R esponse C oding. The authors listened to a subset of taped interview s and developed a prelim inary taxonom y for the response strategies that w ere evident in the protocols. T w o groups of tw o coders then classified the com plete set of protocols, reaching each coding decision by consensus w ithin a group, and expanding the coding taxonom y as needed. B ecause the groups consistently disagreed on how to code one event (receiving subscription m agazines) it w as elim inated from further analysis. In a first pass at coding the protocols, the intergroup reliability for the nine rem aining item s w as.7 5. A ll discrepancies w ere then resolved through discussion betw een the groups so that subsequent analyses w ere based on com plete agreem ent. T he final set of strategies is presented in Table 2. R esponse Tim es. T he duration of each taped response w as m easured from com pletion of the question by the interview er to initiation of the spoken frequency by the respondent. R e sponse tim es included the durations of extraneous speech prior to the actual frequency, for exam ple, qualifiers such as ``I d say about... and verbalisations of respondents thinking, such as ``W ell I do it tw ice a w eek, so.... T he durations w ere m easured by both groups of coders. A s their m easurem ents w ere highly correlated (r =.9 7, P <.0 1, n = 950), only one set of tim es w as further analysed. A bout 9% of these TABLE 2 Major Response Categories and Percentage of Observations R esponse Strategy Percentage of O bservations N on-zero R esponses 1. Episode E num eration R ate R etrieval R ate Estim ation R a te an d A dju stm e n t 9 5. G eneral Im pression U ncodable 18 n = 6 60 Z er o R e sp on se s 7. A ttem pted Enum era tion R ate R etrieval U ncodable 5 n = 2 65

10 348 CONRAD, BROWN, CASHMAN observations w ere excluded because respondents asked for clarification or w ere disrupted before responding. RESULTS AND DISCUSSION T he overall response rate w as 42% w hich is som ew hat low for m ost surveys. H ow ever, the research procedure could have been at least partially responsible: 14% of those contacted w ere w illing to be interview ed but not w illing to be tape recorded. 3 It is possible that som e of those people w ho w ere contacted but declined to participate could have been persuaded to take part; how ever, to reduce interview tim e and associated cost, the interview ers w ere instructed not to engage in such ``conversion activities. Response Strategies R esponse strategies w ere classified separately for frequency reports of zero (26% ) and all other values (74% ). L ooking first at the non-zero responses, 18% of the protocols w ere uncodable. A total of 96% of the rem aining data w as assigned to one of five response strategy categories: episode enum eration, rate retrieval, rate estim ation, rate and adjustm ent, general im pression. T o a certain extent, all but episode enum eration rely on inform ation about the general event category, rather than individual episodes, and so m ight be reduced to a single rate strategyð and this is w hat is typically done (e.g. B urton & B lair, 1991; H ubbard, 1992). H ow ever, the protocols suggested that the underlying processes w ere different, so the five distinctions w ere m aintained. T he rem aining 4% of the codable protocols w ere assigned to the category of item enum eration (in particular, listing the catalogues typically received in answ ering the catalogue question). These are not consid ered further because they w ere rare and diverged from the phenom enon of interest in that respondents w ere not concerned w ith events but rather w ith item s. B ased on reports in the literature, w e expected evidence in the protocols of both episode enum eration and of som e kind of rate strategies, and both w ere observed: 27% of the protocols w ere coded as episode enum eration and 36% w ere coded as som e type of rate-based strategy. A n exam ple of an episode enum eration protocol, in response to the question about gas purchases, is ``T hree. Today and... about ten days ago on a job... and one other tim e w hen I w as practically em pty. W e subdivided the rate-based responses into rate retrieval (15% ), rate estim ation (12% ), and rate and adjustm ent (9% ). B y rate 3 T h e k in d o f re sp on se-t im e m e th o do log y p ro m o te d by B a ssili ( ; B a ssili & F le tc h e r, ) w ould be of som e help under these circum stances because it tim es telephone responses ``live as opposed to m easuring recorded responses. H ow ever, our pro cedure involved coding retro spective stra te g y re p orts w hic h re qu ires a p e rm a ne n t re c ord o f th e re po rt. F or th is re a son w e ta p e re co rd e d th e interview s.

11 STRATEGIES FOR ESTIMATING FREQUENCY 349 retrieval w e are referring to rate inform ation stored som e tim e prior to the interview, e.g. ``Four. I go grocery shopping on Fridays. R ate estim ation refers to those observations for w hich respondents com pute rate inform ationð as opposed to retrieving itð w hile answ ering the question. A rate estim ation protocol referred to the resp ondent s know ledge that the event occurred regularly but w ith an uncertain frequency, that frequency w as som ehow constructed in the response process. A n exam ple, in response to the autom ated teller m achine question, is ``E ight... L ast w eek I w ent tw ice so... in the last m onth that w ould be eight. R espondents w ere considered to use the rate and adjustm ent strategy w hen they relied on rate inform ation (w e did not distinguish betw een retrieved and estim ated rates for this strategy) and adjusted up or dow n to account for exceptions to the rate. A n exam ple from the q uestion about grocery shopping is: ``E ight... O nce a w eek, regularly, and four m ore tim es for... spontaneous item s. In addition to episode enum eration and the several rate-based strategies, w e observed evidence in 18% of the protocols that respondents relied on general im pressions. T he key characteristic of general im pression reports is that respondents use a coarse sense of m agnitude to infer a specific frequency. This w as usually indicated by so-called vague quantifiers (W right, G askell, & O M uircheartaigh, 1994)Ð m agnitude term s that have quantitative im plications but are not explicitly num erical, for exam p le ``all the tim e, or ``a lot. C onsider the follow ing report in response to the question about bank teller transactions. ``E ight... W e re in the process of buying a house so I ve been going there a lot. T his strategy is fundam entally different from enum eration (and its variants) and rate-based responding (and its variants) in that respondents w ho lack any num erical frequency inform ation can provide a num erical response by converting their im pressions of frequency to a num ber. T urning now to the zero responses, 26% of all reported frequencies w ere zero. O f these, 5% w ere uncodable. T he protocols for the rem aining 21% of the responses indicated that respondents use tw o strategies to determ ine that that they have not engaged in an activity over a given tim e fram e. In particular, the protocols contained evidence of attem pted enum eration and rate retrieval. Protocols w ere coded for attem pted enum eration if there w as evidence that a m em ory search failed to locate any episodes w ithin the tim e period. For exam ple, a protocol provided in response to the ice cream question w as, ``Z ero... T rying to rem em ber if m y w ife had bought any this m onth. She hasn t. T hese w ere apparent for 15% of zero responses. For 80% of zero responses, respondents indicated that they never engage in the activity or sim ply knew they had not done so in the previous m onth. T hese protocols w ere coded for rate retrieval. For exam ple, in answ ering the question about A T M activity, one respondent reported ``N one... I don t use them.

12 350 CONRAD, BROWN, CASHMAN Regularity and Similarity Ratings C onsistent w ith findings reported by M enon (1993), regularity and sim ilarity had clear effects on the choice of response strategy. M ean regularity and sim ilarity ratings for the m ajor response strategies are presented in T able 3. L ooking first at non-zero responses, regularity varied w ith the five response strategies, F (4,4 24) = 17.41, P <.01, M SE = W hen respondents enum erated episod es, they judged events to be less regular than w hen they retrieved rates, F(1,4 24) = , P <.0 1, M SE = , confirm ing our first prediction. M oreover, regularity ratings on trials exhibiting episode enum eration w ere low er than for all other strategies, F(1,4 24) = , P <.01, M SE = , by a Scheffe test. T his last effect underscores the idea that events are enum erated prim arily w hen they occur on an irregular basis. Finally, frequency reports relied on general im pressions w hen events w ere less regular than for the three strategies involving rates, F (1,4 24) = 13.54, P <.01, by a ScheffeÂtest. It seem s, therefore, that like enum eration, general im pressions are used in the absence of rate inform ation. H ow ever, as is described in the next section, the frequency conditions under w hich enum eration and general im pressions are used are different. Sim ilarity show ed m uch the sam e pattern as regularity. O verall, sim ilarity varied w ith response strategy, F(4,4 23) = 4.3 9, P <.0 1, M SE = In particular, sim ilarity ratings for episode enum eration w ere low er than for rate retrieval, TABLE 3 Mean Regularity and Similarity Ratings for the Major Response Strategies R esponse Strategy R egularity* Sim ilarity* * N on-zero R esponses 1. Episode Enum eration R ate R etrieval R ate Estim ation R ate and A djustm ent G en eral Im pression Z e ro R es p on se s 6. A ttem pted Enum eration R ate R etrieval * 1 = very irreg ular, 4 = very regular * * 1 = very different, 4 = very sim ilar 4 A ll A N O V A re su lts re p orted h e re a re ba se d o n m od e ls tha t in c lu de a sub je c t te rm a nd a n ite m term as classified variables. The subject term is included to account for repeated m easures effe cts and th e ite m term is in c lud e d to ac c o un t fo r ite m e ffe c ts. A ll c o ntra sts re po rte d o n th e b a sis o f A N O V A are plan ned unless otherw ise noted.

13 STRATEGIES FOR ESTIMATING FREQUENCY 351 F (1,4 23) = , P <.0 1, M SE = 8.7 0, con firm ing the secon d prediction. Sim ilarity ratings w ere low er for episode enum eration than for all of the other strategies, F(1,4 23) = , P <.0 1, M SE = , by a ScheffeÂtest, consistent w ith the idea that enum eration is used prim arily w hen episodes are quite dissim ilar and, consequently, differentiated in episodic m em ory. B oth event characteristics varied w ith the strategies observed for zero responses. E vents w ere judged less regular for rate retrieval than for attem pted enum eration, F(1,1 44) = 5.2 5, P <.0 5, M SE = 2.4 8, though both m ean ratings are very low (1.40 and 1.8 5, respectively). K now ing that the event of interest has not occurred in the reference period, respondents m ay w ell be m apping their belief that the event could happen in a given m onth onto the regularity rating scale. B y this view, the m ore plausible it is that an event m ight occur, the m ore regularly it is judged to occur. Sim ilarity ratings are low in general, but low er for rate retrieval (2.16) than for attem pted enum eration (2.46), F(1,1 41) = 4.3 9, P <.0 5, M SE = O ne explanation is that respondents treat the low end of the sim ilarity scale as a ``uniqueness scale w here low er values are m ore unique: events that are know n never to occur w ould be extrem ely unique (low sim ilarity) if they w ere som ehow to occur, and events that can plausibly occur w ould share som e characteristics w hen they do in fact take place, m aking them less unique (higher sim ilarity). R egularity and sim ilarity w ere correlated across the entire data set; that is for both non-zero and zero responses (r =.4 8, P <.0 1, n = 936). A lthough one can im agine scenarios in w hich regularity and sim ilarity are independent, their correlation is consistent w ith our earlier contention that events that occur on a regular schedule are experienced m uch the sam e from one episode to the next: even if they differ in all other characteristics, each occurrence of a regularly occurring event is tem porally sim ilar to the next. Frequency A nother factor that has been show n to affect w hich estim ation strategies respondents choose is the approxim ate level of an event s frequency. In particular, subjects are m ore likely to enum erate low -frequency than highfrequency events in arriving at frequency reports, presu m ably because the effort of retrieving individual episodes increases w ith the num ber of episodes (B lair & B urton, 1987; B urton & B lair, 1991; M eans & L oftus, 1991). This is replicated in the current study. M ean frequencies and tim es for five strategies are presented in T able 4. O verall, frequencies vary w ith strategy, F (4,4 25) = , P <.0 1, M SE = In addition, frequency for episode enum eration is low er than for rate retrieval, F(1,4 25 ) = , P <.0 1, M SE = and for all other strategies, F(1,4 25) = , P <.0 1, M SE = , by a ScheffeÂtest. T he observation that respondents prim arily enum erate low -frequency events (2.3 occurrences per m onth on average) m ay w ell be related to the presence of

14 352 CONRAD, BROWN, CASHMAN TABLE 4 Mean Frequency and Response Times for the Major Response Strategies R esponse Strategy Frequency Tim e (in seco nds) N on-zero R esponses 1. E pisode E num eration R ate R etrieval R ate E stim ation R ate and A djustm ent G eneral Im pression Zero R esponses 6. A ttem pted Enum eration R ate R etrieval contextual inform ation for these events. In order to retrieve events, there m ust be som e inform ation to distinguish one event from another. C ontext can play exactly such a distinguishing role. B ecause of their repetition, high-frequency events m ay becom e decontextualised (B ahrick & K aris, 1982; L inton, 1982; Strube, 1987) and so cannot easily be retrieved. R espondents used general im pressions to report the highest average estim ates (12.3 occurrences per m onth). T his is consistent w ith a finding by B ruce (B ruce & R eed, 1988; B ruce & V an Pelt, 1989) concerning estim ates of event frequency on a bicycle trip. T he relationship betw een general im pressions and high frequency estim a tes can be explained if w e assum e that repeated occurrences allow respondents to directly, albeit im precisely, encode the fact that particular episodes are frequent. For exam ple, the frequency stored for a repeated event m ight be ``a lot. Infrequently occurring events w ould be less likely to lead to the im pression that the behaviour is rare, because there is sim ply less opportunity to form this im pression. B ecause respondents do not have precise rate inform ation available w hen they use a general im pression strategy (these events receive the second low est regularity ratings), all they can do is convert their im pression that frequency is high to a relatively large num ber. Figure 1 presents the percentages of different frequency responses for episode enum eration, rate retrieval, rate estim ation, and general im pression strategies. The prim ary thing to notice is the peaks: w hen respondents enum erated episodes (Fig. 1a) their estim ates w ere usually sm all w ith the num ber of larger estim ates dropping off quickly. T he peaks for rate retrieval (Fig. 1b) and rate estim ation (Fig. 1d) strategies prim arily reflect w eekly activity: T he peaks at four, eight, and tw elve (over the tw o strategies) suggest that respondents had access to w eekly rate inform ation and then m ultiplied these w eekly rates by four (the num ber of w eeks in a m onth). A lthough rate retrieval and rate estim ation both

15 FIG. 1. Frequency of reported values for four strategies: (a ) episode enum eration, (b) rate retrieval, (c ) general im pression, and (d ) rate estim ation. 353

16 354 CONRAD, BROWN, CASHMAN sho w e v id e nc e o f co n v ertin g w e ek ly in form atio n to m on thly figu res, respondents verbal protocols suggest that the underlying rates have different origins for these tw o strategies. R etrieved rates are so nam ed because they appear to be stored; estim ated rates appear to be the result of partial enum eration. T his distinction is further supported by the reaction tim e data presented in the next section. W hen respondents answ ered on the basis of their general im pressions (Fig. 1c) their estim ates w ere often divisible by five. T his has the appearance of rounding (H uttenlocher, H edges, & B radburn, 1990). It w ould m ake sense for respondents using the general im pression strategy to produce rounded estim ates because they need to convert their im pressions, w hich are qualitative, into num bers; one w ay to do this is to report a prototypical quantity for the available verbal description, for exam ple, quantifying an im pression of ``pretty often as ``10. In this dom ain (as in m any others) such prototypical values tend to be divisible by five. O ne reason that a respondent w ould m ap an im pression to such a value is that he or she is probably m ore likely to think of a prototypical than a non-prototypical quantity. M oreover to provide a non-prototypical quantity, say 11, w ould im ply a level of precision that a prototypical value w ould not necessarily convey; because the conversion from an im pression to a num ber is inherently im precise, respondents m ay be m ore inclined to report the rounded value. Response Times N ot only do frequency reports vary w ith response strategies, but they are related to response tim es in different w ays for the various strategies. R esponse tim es have been exam ined in behavioural frequency studies in relation to accuracy (B urton & B lair, 1991) and as an indicator of effort (M enon, 1993). In the current study, response tim es provide detail about how the different strategies are executed. In particular, w e predicted that as frequency reports based on enum eration increase, the latencies for those reports should increase, but frequency and tim e should not be associated for rate-based reports. A s predicted, the tim e to report a frequency using this strategy increases w ith the size of the frequency (r 2 =.2 5, P <.0 1). A lso as predicted, there is no relationship betw een response tim e and frequency w hen respondents answ ered by retrieving a rate. R egression equations, in w hich response tim e is a function of estim ated rate, are presented for five strategies in T ab le 5. T he differences betw een the functions for these tw o strategies indicate that, for enum eration, a noticeable am ount of tim e is associated w ith the retrieval of each additional episode (about.8 4 seconds, using the slope of the regression equation for enum eration as an estim ate), but for rate retrieval, tim e does not change w ith the size of rates; the invariance of tim e w ith frequency for rate-based responses suggests that access to rate inform ation requires a fixed num ber of m ental

17 STRATEGIES FOR ESTIMATING FREQUENCY 355 TABLE 5 Regression Equations for Response Time as a Function of Estimated Frequency R e sp o nse Stra te g y E qu a tio n r 2 P E p iso de E n u m era tion tim e = (re po rte d fre q ue n c y).2 5 < R a te R e trie va l tim e = (re po rte d fre q ue n c y).0 0 n.s. G e ne ra l Im p re ssion tim e = (re po rte d fre q ue n c y).0 0 n.s. R a te E stim a tio n tim e = (re po rte d fre q ue n c y).1 4 < R a te a nd A djustm e n t tim e = (re po rte d fre q ue n c y).0 4 n.s. Tim e is m easured in seconds. operationsð presum ably a single retrieval. T here w as also no relationship betw een response tim e and the m agnitude of the reported frequencies w hen respondents used their general im pressions. C learly, general im pressions are not based on en um eration. T he pattern for this strategy could indicate that im pressions are retrieved fully form ed. A lternatively, it could suggest that the result of assessing one s m em ory is a general im pression. W e cannot distinguish betw een these interpretations on the basis of the current data. W hen respondents based their estim ates on rate estim ation, their response tim e increased w ith the size of the estim ates (r 2 =.1 4, P <.0 1) although the relationship does not appear to be as strong as it is for episode enum eration (the slope of the regression equation is.1 5). M edian response tim es (for non-zero responses) are plotted against reported frequency for episode enum eration and rate estim ation in Fig. 2. If there w ere less than five observations for a particular frequency, that value w as om itted from the plot. B oth curves show a linear relationship betw een tim e and frequency, although the curve for rate estim ation is clearly shallow er than the curve for enum eration. O ne explanation for a slope of interm ediate steepness is that rates are estim ated by enum erating episodes for a portion of the reference period, for exam ple a w eek; the respondent then generalises to the com plete tim e period m uch as w ould be the case w ith rate retrieval. It is the extrapolation part of the strategy that w ould w eaken the relationship betw een frequency and tim e relative to that observed for pure enum eration, because it is possible for a single m ental operation (presum ably m ultiplication) to produce a relatively large frequency report. N o relationship betw een frequency and tim e is evident for rate and adjustm ent. T herefore, rate and adjustm ent m ay resem ble rate retrieval m ore than rate estim ation (w here there w as a positive relationship betw een tim e and frequency). In both rate and adjustm ent and rate retrieval, the inform ation on w hich respondents base their estim ates is know n to them a priori and is not estim ated through partial enum eration. H ow ever, because responses based on rate and adjustm ent w ere generally slow er than those based on rate retrieval, adjusting the rate seem s to require extra tim e. T his suggestion is strengthened by

18 356 CONRAD, BROWN, CASHMAN FIG. 2. M edian response tim e as a fu nction of reported fre quency and response strategy. the observation that regularity and sim ilarity judgem ents for the tw o strategies are a bo u t the sa m e, in dic atin g tha t the y are u sed un d er co m p a ra ble circum stances. A hom ogeneity of slopes test confirm s that the relationship betw een tim e and frequency varies w ith response strategy, F(4,3 68) = 7.7 4, P <.0 1, M SE = In particular, the regression line for episode enum eration w as steeper than that fo r ra te r e tr ie v a l a n d g e n e r a l im p r e s si o n, F (1,3 6 8 ) = , P <.0 1, M SE = T he line for rate estim ation w as also steeper than that for rate

19 STRATEGIES FOR ESTIMATING FREQUENCY 357 retrieval, F(1,3 68) = 6.2 0, P <.0 5, M SE = , by a ScheffeÂtest, bu t not as steep as the episode enum eration line, F (1,3 68) = , P <.01, M SE = , by a ScheffeÂtest, supporting the idea that the rate estim ation strategy is indeed a hybrid of episode enum eration and rate retrieval strategies. T hese results suggest that respond ents m ay set a threshold for tim e spent enum erating and once this is exceeded, they extrapolate. R egardless of the reported frequency, rate retrieval is faster than episode enum eration, F(1,3 68) = , P <.0 1. T his essentially replicates a pattern found by M enon (1993) in w hich frequency responses for regular item s w ere faster than those for irregular item s. A nd it is also consistent w ith the finding by B urton and B lair (1991) that inducing respondents to use m ore tim e in responding increases the incidence of enum eration. B ut a lesson of the current study is that overall tim e tells only part of the story. In arriving at a strategy, respondents are probably influenced by the num ber of discrete, tim e-consum ing steps they w ill need to execute in order to respond. Finally, the tw o strategies observed for zero frequencies have different response tim es. R ate retrieval responses w ere faster than attem pted enum eration, F (1,1 0 3) = , P <.0 1, M S E = A sim p le m o d el fo r a tte m p te d enum eration is that respondents search for a reportable episode for a reasonable am ount of tim e and abandon the search (report ``zero ) after that. B ased on the regression equation for (non-zero) episode enum eration, tim e to enum erate one episode is about 3.5 secondsð in the sam e ballpark as the m ean response tim e of 3.9 seconds for attem pted enum eration. O ne reading is that subjects threshold is the tim e required to enum erate one item. R ate retrieval responses w ere particularly fast for zero reports; faster, in fact, that non-zero reports based on the sam e strategy. T his difference could arise if non-zero reports dem and extra tim e w hen a rate expressed in one tem poral unit, say w eeks, is applied to a reference period expressed in other units, say a m onth; in contrast, no such extrapolation is required in order to know therefore, report zero. that an event never occurs and, A Multivariate Approach to Strategy Choice E ach of the variables w e have exam ined in the previous sections (regularity, sim ilarity, reported frequency, and response tim e) is related to respondents choice of estim ation strategy. O ne problem w ith looking at these relationships individually is that it is hard to see how they jointly lead a respondent to use a particular strategy on a particular occasion. For exam ple, low regularity leads respondents to enum erate w hen frequency is low ; how ever, low regularity leads respondents to use their general im pressions w hen frequency is high. T his relatively sim ple exam ple involves tw o variables and tw o strategies. T he patterns becom e harder to describe w hen they involve three or four variables and as m any strategies. In order to see how particular patterns of several variables