Deterministic vs probabilistic strategies in detection - Springer Link

Procedure There were 4 practice sessions and 27 experimental sessions in which different procedures were employed. Each session lasted about 2 h. This paper deals with the re suits of one procedure that was employed during 5 experimental sessions: MICHAEL KUBOVY, The Hebrew University of Jerusalem, Jerusalem, Israel AMNON RAPOPORT, The University of North Carolina, Chapel Hill, North Carolina 17514 Nos. 5, 8,12,17, and 21. The Ss were told that the distributions, a and and b, represented heights of women and AMOS TVERSKY, The Hebrew University of Jerusalem, Jerusalem, Israel men, respectively, that they need not A binary detection task, free from sensory components, is investigated. A deterministic represent any adult population they were model prescribing a fixed cutoff point is confirmed; a probabilistic model, which familiar with, that they were unimodal, symmetric, and partially overlapping, and generalizes Lee's micromatching model for externally distributed stimuli, is rejected. that men and women had equal chances of being sampled on each trial. S was The present study is concerned with a results of the latter two studies presented on each trial with a single decision task in which an individual is corresponded more closely to the stimulus drawn randomly and with equal required to decide from which of two micromatching model. Since in these probability from either a or b. He was distributions a particular observation has studies Ss were not given prior information instructed to decide on each trial whether been drawn. It is assumed that the about the exact shape of the two the stimulus was drawn from a or from b, individual has had adequate exposure to distributions, and since the number of and to indicate his decision by writing the distributions and that he knows the trials (either 400 or 500) was probably too either "A" or "B" on a response sheet. payoffs for correct and incorrect decisions. small to ensure adequate learning, the effects of learning might have obscured the Feedback was provided at the end of each The study attempts to discover whether Ss trial. use a deterministic strategy prescribing a decision strategy employed by Ss. In an The Ss were run in a group. In each fixed cutoff point or a probabilistic attempt to obtain more conclusive session, consisting of 400 trials, an equal strategy to which responses are to be evidence on the nature of this strategy, we number of stimuli from the two mixed according to the conditional have (I) freed the decision process from sensory components by using as stimuli distributions were projected on a screen. In probabilities of the two distributions. the four practice sessions Ss were paid IS£7 This problem is of central importance to numbers drawn from either of two (approximately $2.00) per session, theories of decision making and in distributions; (2) ensured adequate regardless of their performance. Different particular to signal detection theory (Green knowledge of the distributions by payoff conditions were used in these & Swets, 1966), yet it is difficult to providing Ss with several thousand learning sessions, and Ss were asked to compute the investigate within the context of sensory trials; and (3) motivated Ss to behave tasks, since the noise and the signal are not optimally by using monetary payoffs number of points they had gained after each block of 100 trials. They were specified exactly on a given trial. In such contingent upon their performance. encouraged to try different response tasks, therefore, departures from optimal METHOD strategies in the practice sessions and to performance are attributable to an maximize the number of points gained. improper selection of the cutoff point, to Stimuli The stimuli were two sets (denoted by a The five experimental sessions were the use of a probabilistic strategy, to imperfect sensitivity, or to all of the above. and b) of 500 four-digit numbers conducted like the practice sessions except that wages were proportional to the In a series of studies, Lee (1963), Lee constructed so as to provide the best number of points gained by each S. The and Janke (1964, 1965), and Lee and approximation to two normal distributions five payoff conditions used in these Zentall (1966) have attempted to answer with means rna = 1630, mb = 1797, and sessions-one in each session-are presented this question by using externally standard deviations sa =Sb = 167 (d' = I). in Fig. I. The mean earnings of the group distributed stimuli. Three different sensory was fixed at approximately IS£7.50 per continua were employed in these studies: Subjects The Ss were seven volunteers-three session. Ss were paid half their earnings at position of a dot in a plane, grayness of a the end of each session; the balance was patch of paper, and two-digit numbers. In females and four males-all first-year paid upon completion of the experiment. each case, stimuli were drawn from either psychology majors at the Hebrew one of two normal distributions. Using the University of Jerusalem. The data of one RESULTS exact specification of the stimuli, Lee and male S were discarded after it was To compare the deterministic and his associates compared the deterministic established that he had seriously probabilistic models, critical points were response model with a probabilistic misunderstood parts of the instructions. response model, named the micromatching Table 1 hypothesis, derivable from statistical Percentaae of Trial. on Which the Cutoff Point Model II Violated for Each Sand Eac:b Payoff Condition learning theory. In general, the results of the first two studies fell abou t midway Payoff Condition between the two models, whereas the

Deterministic vs probabilistic strategies in detection *

"This research was supported by a grant to the first two authors from the Faculty of Social Sciences of the Hebrew University of Jerusalem. The costs of preparing the report were partially supported by Grant M·I 0006 from the National Institute of Mental Health to the Unversity of North Carolina. TIle authors wish to thank Mr. E. Zvulun for assistance in data collection.

Perception & Psychophysics, 1971, Vol. 9 (5)

s

1

2

3

4

0

Mean

1

1.00 4.75 7.00 3.00 3,20 17.20 6.13

0.20 1.00 0.75 5.50 8.25 5.75 3.58

0.25 5.75 1.00 18.50 2.00 9.75 6.21

3.50 4.00 8.00 6.75 16.25 9.00 7.92

2.75 4.20 4.25 1.IiO 12.20 8.00 5.50

1.55 3.95 4.20 7.15 8.40 9.95 5.87

2 3 4 5

6 Mean

Copyright 1971, Psychonomic Journals. Inc.. Austin, Texas

427

alM

b~ (1 )

A

B

a~ bl~

A

B

A

aEEJ

alili]

b~

b~ (3)

(2 )

A

B

B

aillOl

V(B,b) =[3, whence PCB I x) PCb I x)[3 P(A \ x) = Pea I x)a .

(Note that the generalized micromatching model, denoted GM, is also applicable to cases where the prior probabilities are unequal.) If A(c) =B(c) for any critical point, c, it follows that peA I c) =PCB I c) = .5, and hence, by the above equation, PCb I c) =~ P(alc) [3

b~

(I)

(2)

(4)

A Chi-square test of the hypothesis that A(c) = B(c) over all payoff conditions was Fig. 1. Payoff matrices for the five payoff conditions. A, B denote responses; a, b nonsignificant (p > .10). denote distributions. To evaluate the adequacy of the two models, the frequency of correct responses P(b I x)U(B,b) + Pea I x)V(B,a) computed for each S in each payoff expected under each of them was condition. A critical point is a stimulus = PCb I x)V(A,b) + P(a I x)V(A,a) compared to the observed frequency of value for which the number of violations of correct responses. For the deterministic a deterministic model is minimal. where, e.g., E [V(B,x)1 is the expected model, the expected frequency was Formally, let A(x) be the total number of utility of choosing B after observing x, and computed by treating the critical points as trials on which S responded A to stimuli V(B,b) is the utility of choosing B when hypothetical cutoff points and applying that were larger than x, and let B(x) be the the observation was drawn from b. Since in them to the actual stimuli presented to the total number of trials on which S the present study V(A,b) = V(B,a), we can Ss. For the GM model, Eq. 2 was employed responded B to stimuli that were smaller let V(A,b) = V(B,a) = 0, V(A,a) =a, and to obtain the value of a/[3 for each S and than x. A critical point is a stimulus-value x for which A(x) + B(x) is minimal. When a critical point was not unique, the median critical point was selected for further analysis. To test the deterministic model, we treated the critical points as 1.0 hypothetical cutoff points, thereby obtaining an index of departure from the model. The percentages of trials on which a 0.9 deterministic model was violated for each S and each payoff condition are presented in 0.8 Table I. The proportions of B responses as a function of the stimulus value (the Proport ion of OJ psychometric function) for a typical S 2 3 8 Responses (53) in all five payoff conditions are P(BI~) 0.6 displayed in Fig. 2. The presence of violations in Table I reflects a certain degree of mixing, showing 0.5 that the deterministic model is not perfectly satisfied. Nevertheless, the 0.4 percentage of violations is not substantial, and an adequate statistical procedure is 0.3 required to test and compare the two models. The micromatching model (Lee, 1963) 0.2 asserts that for any stimulus, x, P(B I x) _ P(b I x) p(A I x) - p(a I x) This model, however, does not take payoffs into account. We have, therefore, generalized the model by assuming that

p(B I x) P(A I x)

428

E[V(B,x)] E[V(A,x) ]

0.1 • 0

SUBJECT 3

C1

Stimulus Value Fig. 2. Proportion of B responses in blocks of 20 as a function of mean stin1Ul.u~ value (on a log likelihood-ratio scale, i.e., log [P(x I b){P(x I a)]) for ea~h pa~~ff con~lllOn for S 3. Successive curves are displaced upward by 0.05. Respective critical pointsIc. • . • . . , Cs ) are marked along the abscissa. Perception & Psychophysics, 1971, Vol. 9 (5)

Table 2 Chi-Square Values for the Comparison of Observed Frequency of Correct Responses With Frequencies Expected Under the Two Response Models for Each 8 Across Payoff Conditions

S

Cutoff Point Model

Micromatching Model

1 2 3 4 5 6

0.33 2.49 5.13 2.40 3.52 6.81

22.89(iii) 40.32(ili) 13.06(i) 15.82(ii) 26.78(iii) 43.06(ili)

20.68

159.90(iii)

Overall

p(BI.2)

(iJp < .05, (iiJp < .01. (iiiJp < .001. Note-Each individual comparison is based on 6 at: the overall comparison is based on 30 at. Table 3 Chi-Square Values for the Comparison of Fig. 3. ROC plots of the data with the curves predicted by the cutoff point model Observed Frequency of Correct Responses (solid line) and the micromatching model (dashed line) for all Ss, With Frequencies Expected Under the Two Response Models for Each Payoff Table 4 Condition Across Ss Parameter Estimates of the ROC Curve Predicted by the Cutoff Point Model Cutoff MicroS3 S4 S6 Sl S2 85 Payoff Point matching Condition Model Model .8335(i) Intercept 1.0206 .9186 1.0170 .8700 .9693 Slope .9965 1.0252 1.0248 .9175 .9767 1.0145 1 8.71 2.17 39.83(iii) 2 3.36 (iJp < .05 63. 22(iii) 3 1.86 45.03(iii) 4 4.52 model (assuming the validity of the circumstances, the locations of the cutoff 5 2.23 2.65

PCB I.!)

deterministic model) is considerably lower in these latter conditions than in (iii)p < .001. Conditions 2. 3, and 4. The overall Note-Each individual comparison is based discrepancy (across all Ss and payoff on 5 at: the overall comparison is based conditions) is highly significant for the GM on 30 at. model and nonsignificant for the payoff condition separately. These values deterministic modeL Figure 3 displays the ROC plots of the were substituted into Eq. I to obtain the expected frequency of A and B responses data and the theoretical curves predicted from which the expected frequencies of by the two models for each of the Ss. The correct responses for each stimulus were intercept and slope of the linear regression calculated. The overall expected frequency line (employing double probability for a given S on a given session was coordinates) fitted to the data of each S in obtained by adding the expected the five payoff conditions by the frequencies of correct responses over all maximum likelihood method (Dorfman & stimuli presented in this session. The Alf, 1968) according to the deterministic chi-square values for the comparisons model, are presented in Table 4. The 95% between the observed and expected confidence intervals for II out of the 12 frequencies of correct responses under the estimates include 1,0, which is the value two models are presented in Table 2 for predicted by the signal detection theory. each S and in Table 3 for each payoff assuming normal distributions with equal variances and d' =1, condition. In summary. although some degree of The results show that the discrepancy between the observed and expected mixing is observed. the GM model is clearly frequencies is nonsignificant under the inadequate, contrary to the conclusion of deterministic model and highly significant Lee and his colleagues, The difference under the GM model for each of the Ss between the findings may be due to a over all payoff conditions. Table 3 shows difference in procedure. Whereas the Ss of that the same result holds for the three the present study received extensive nonextreme payoff conditions (2.3. and 4) practice with the two distributions (2,000 over all Ss. In the extreme payoff trials before the first experimental session, conditions (I and 5). the expected and several thousand more thereafter), proportion of correct responses predicted Lee's Ss learned the distribu tions in the by both models approaches .5. Hence the course of a relatively short experiment statistical power of the test of the GM (400 or 500 trials). Under these Overall

20.68

159.90(ili)

Perception & Psychophysics, 1971, VoL 9 (5)

points probably shifted during the study. It is not surprising, therefore, that Lee's Ss produced data that are closer to the probabilistic model than the present data. Strictly speaking, the deterministic model is also violated, for it is intolerant of any error, Nevertheless, it yields a very good account of Ss' behavior when evaluated in terms of the frequency of correct responses. And this, after all, is the basis on which both distributions and strategies are learned.

REFERENCES DORFMAN, D. D. & ALF, E., JR. Maximum likelihood estimation of parameters of signal detection theory-a direct solution.

Psychornetrika, 1968, 33, 117-124. & SWETS, J. A. Signal detection theory and psychophysics. New York: Wiley, 1966,

GREEN, D. M.,

LEE. W. Choosing among confusably distributed stimuli with specified likelihood ratios. Perceptual & Motor Skills, 1963,16,445467. LEE, W., & JANKE. M. Categorizing externally distributed stimulus samples from three continua. Journal of Experimental Psychology, 1964,68,376-382.

LEE, W., & JANKE, M. Categorizing externally distributed stimulus samples from unequal molar probabilities. Psychological Reports, 1965, 17, 79-90.

LEE, W., & ZENTALL, T. R. factorial effects in the categorization of externally distributed stimulus samples. Perception & Psychophysics. 1966, 1. 120-124.

[Acceptedfor publication August 31,1970.)

429