Computer program for quasi-random stimulus sequences with equal

Behavior Research Methods, Instruments, & Computers 1995,27 (1),88-98

Computer program for quasi-random stimulus sequences with equal transition frequencies PHILUP L. EMERSON Cleveland State University, Cleveland, Ohio and RANDALL D. TOBIAS SAS Institute, Cary, North Carolina C language routines are presented for the generation of randomized stimulus sequences constructed from multiple presentations of m stimuli satisfying sequential constraints with respect to the frequencies of the occurrence ofn-gram subsequences. Applications are suggestedfor sequential experiments in which main effects for the present stimulus and the stimuli in the preceding (n-l)-length substring can be tested, as well as the interactions among stimuli at the various positions in the substrings.: Experiments in behavioral and cognitive research often involve runs. Each run consists of a sequence of trials, ~nd on each trial a stimulus is presented and a response IS observed. Often, there are more trials than distinct stimuli, so that data are obtained from several trials for each stimulus, in a single run. A familiar way to provide some statistical control for sequential order effects is to randomize the order of presentation of trials independently for each subject, run, or both. Thus, sequential effects are introduced randomly in the replications so that observed mean differences among the effects of the different stimuli are not confounded systematically with sequence effects and are tested against error terms that would be expected to contain, on the average, the residual random sequential effects that also affect the means. Sometimes it is desirable to perform randomizations ofthis nature under constraints. It is common to force each stimulus in a finite set to be presented exactly equally often. In that case, the full set of trials can be defined beforehand in any order, and then random permutations can be generated by some method resembling the shuffling of a deck of cards (see Brysbaert, 1991; Castellan, 1992). In the terminology to be used here, this is a first-order sequential constraint. It is possible to go further and specify constraints on the frequencies with which each stimulus on trial k follows each of the other stimuli and itself on trial k- I. That process introduces second-order se~ quential constraints, and it is possible to go even further and impose third and higher order constraints.

The authors addresses are P. L. Emerson, Department of Psychology, Cleveland State University, Cleveland, OH 44115 (e-mail: r0264@ vmcms.csuohio.edu); R. D. Tobias, SAS Institute, SAS Campus Dr., Cary, NC 27512-8000 (e-mail: [email protected]).

Copyright 1995 Psychonomic Society, Inc.

88

. !here can be several different objectives in the imposition of such constraints in randomized stimulus seq.uences. ~he main suggestion here is for balancing analySIS of vanance (ANOYA) designs with respect to the sequential dependencies of various orders to be tested and the applications most likely would be in repeated measures or mixed ANOYA designs. To keep the first example reasonably simple, but nontrivial, assume that three stimuli, 0, 1, and 2, are to be tested equally frequently and that each is to be preceded in the sequence by itself and each of the others, exactly twice, with one observation of a dependent variable recorded on each of 18 trials. Notice that 19 trials will actually be required, because the first of only 18 trials would not satisfy these sequential constraints. More generally, a preamble of n - 1 trials is needed, where n is the highest order of constraint to be balanced (n = 2 in this example). The responses from these preamble trials are not to be analyzed in the ANOYA, as their function is merely to provide the prefixes for the first few stimuli in the main sequence. At the same time, of course, they can serve as the last few trials in a set of warm-up trials. A second concern is the existence of such 19-trial sequences. Some do exist, as is shown by the example (the preamble is separated from the main sequence by the vertical stroke); 01021220110122110020. By a method discussed in the Appendix, it is calculated that there are 729,000 different sequences in this case. Consider an ANOYA that could be applied to the responses on trials in an experiment designed according to the above example, Assume 10 subjects, and assume that the usual ANOYA assumptions are met. There are three possible stimuli, and there are three possible predecessors for each stimulus, leading to 9 possible transition patterns. The first two columns of the summary table for a possible ANOYA are shown in Table 1. Thus, in addition to testing for differences among the means ofthe responses to the stimuli used on the trials from which responses are obtained, it is also possible to test for any effects of the stimuli on the preceding trial, and even more interesting, perhaps, for the interaction of present stimulus by predecessor. Though data from 18 trials are available for each of the 10 subjects, the responses for the two occurrences of each digram are assumed to be averaged to obtain a single datum in the analysis outlined in Table 1. Alternatively, the analysis of Table 1 could be augmented by the introduction of another within-subjects variable, "ordinal position" with two levels, in which it is taken into ~ccount that each of the nine transition patterns occurs m each of two ordinal positions (Position 1 or 2). Then ordinal position would appear as a source of variation in the table with 1 df, and its interaction with every other source would also appear. The total df then would be 179 rather than 89. Such an expanded analysis

RANDOMIZED BALANCED SEQUENCES

Table 1 A Possible ANOVA Scheme in a Case of 10 Subjects, m = 3, n = 2, andf= 2 Source (Grand mean) Subjects ~~~

Stimulus X subjects Predecessor Predecessor X subjects Stimulus X predecessor Stimulus X predecessor X subjects Total

df (I) 9

2 18 2 18 4 36 89

might be used in a rough investigation of global sequential effects-primacy, recency, or both-in addition to the local sequential effects addressed by the analysis suggested in Table 1. However, ordinal Positions 1 and 2 in the random sequences can be separated by any number of trials from 1 up to the number of trials minus 1. On the average, they might be expected to be separated by about half the sequence length. It is not apparent that there are any special dangers or considerations concerning the ANOVA assumptions that arise for this special kind of design, other than those that apply to any repeated measures design. The usual precautions and tests would seem to be called for, such as the sphericity test with repeated measures. Designs such as that suggested above assume that all the runs of trial sequences proceed exactly according to the prescribed randomized orders and that a usable response is observed on each trial. Thus, no allowance is made for situations that arise sometimes in simple reaction time experiments-stimulus anticipations (response after the warning signal, during the foreperiod, before the trigger signal) occur, or there are failures to respond in a reasonable time. If situations such as these occur, the balance in the above type of design and analysis is lost, but it could still be desirable to have the stimulus sequence balanced, perhaps to avoid the accidental development of expectations based on apparent regularities in parts of the sequence, or wide excursions of'stimulus adaptation level if the stimuli are different intensities of a sensory stimulus. Likewise, with multi choice reaction time experiments, there are various classes of responses, such as correctincorrect. In the analysis of such data, the sequential interactions of the type suggested above would not be directly testable because of the unequal-n nature of the data divided according to response type. With choice reaction time, often there are response sequence effects in addition to stimulus sequence effects. For example, reaction times tend to be relatively long on trials immediately following error trials, and the balancing of stimulus sequences does not necessarily impose balance in the response sequence effects. However, it could still be a desirable control to have the stimulus transitions balanced in the overall presentation sequence, though not within the response classes.

89

Also, under some circumstances it could be of interest to analyze the frequencies in the various response categories as a function of the stimulus transition patterns. In a letter identification task, for example, it might well be of interest to determine whether a particular response to a particular stimulus is more or less likely, depending on what the preceding stimulus was. With confusion matrices for the 26 letters ofthe English alphabet in mind, it has been verified that the methods presented here can generate sequences with m = 26 stimuli, n = 2 for digrams, and f = 1 for a single occurrence of each of the 676 digrams. In experiments yielding data from long sequences of trials, sequence effects often are reported (Hyman, 1953; Kornblum, 1969, 1973; Lockhead & Hinson, 1986; Luce, Nosofsky, Green, & Smith, 1982; Nosofsky, 1983; Purks, Callahan, Braida, & Durlach, 1980; Ward & Lockhead, 1970), and it is likely that they might actually occur more often than reported, because they are not the focus of interest in some research. These randomized balanced sequences could provide quite a neat way to test for sequence effects, including the interactions mentioned earlier, in experiments in which a usable response is obtained on each trial. They could also be used in other cases to control for possible carryover effects when approximate uniformity of the frequency of recent prior experiences is desired. It is shown in the Appendix that any sequence balanced to the nth order in this sense is also balanced to the kth order for 0 < k < n. Also, if stimuli are quantitative and designated with distinct real numbers, as would be the case with stimulus intensities, the autocovariances of the stimuli in a sequence of trials are identically zero for lags of 1 up to n - 1. Ward and Lockhead (1970) investigated sequential effects in an absolute judgment task in which subjects were to identify auditory tone stimuli randomly selected from a set of 10 tones graduated in loudness steps of 1 dB. They found significant effects on the response of the present trial, from stimuli presented as many as six trials earlier, which is a rather extreme case of carryover effects. In their discussion, they suggested certain kinds of interactions that might account for some ofthe details of their results, but if they had used sequences of the present kind, they might have been able to test some of those interactions directly. The method of sequence construction can be adapted to quite a variety of designs, by choices of the values of three parameters. Method The parameters are m, the number of distinct stimuli; n, the length of the n-grams for which balancing is to be achieved; and f, the number of occurrences of each of the m" possible n-grams. Each n-gram consists of a prefix of length n - 1, followed by an individual stimulus. The problem of construction is somewhat similar to that of large Latin squares. Even though the most closely related cases (Cochran & Cox, 1957; Cox, 1958; Mann, 1949; Nair, 1967; Patterson, 1952; Sampford, 1957; Wagenaar, 1969; E. 1. Williams, 1949, 1950) do not carry

90

EMERSON AND TOBIAS

over directly to the present situation, something is learned from the suggestion of R. M. Williams (1952) and the theorem of Hutchinson and Wilf (1975), based on the connection between balanced sequences and Euler paths in certain connected graphs. As they pointed out, sequences of the present kind, and of some related kinds, are equivalent to Euler paths in Euler graphs, so known methods of constructing Euler paths can be used to generate sequences. There are several efficient deterministic methods ofconstructing such paths, and any could be used directly to generate a few sequences for noncritical applications, but modifications are needed in order to sample with approximately equal probabilities from all possible sequences in a given case. The inclusive tracings method (Tutte, 1966), due to Euler himself, seemed as simple as others, so the present method is based on it. The graph within which the path is to be constructed consists of M vertices (M = m n - 1) and some interconnecting edges. Each vertex represents a unique member of the set of all possible distinct stimulus subsequences of length n - 1. An edge is drawn between any pair of vertices if the subsequence represented by one can follow the subsequence represented by the other, with an overlap of n - 2. The cases of n = 2 are covered, but do not fully illustrate the principle, so consider the example of(m,n,j) = (2,3,1), with Stimuli A and B. Then the vertices represent the subsequences AA, AB, BA, and BB. The edges from AA go to AA (a loop) and AB; those from AB go to BA and BB; those from BA go to AA and AB; those from BB go to BA and BB (another loop), and that completes the graph. It is an Euler graph in that it is connected, and the number of inward edges is equal to the number of outward edges for each vertex. The more complex case of(m,n,j) = (2,4,1) is illustrated in Figure 1. These two cases correspond to examples used by Good (1947) in a similar application of Euler graphs. An Euler path in such a graph is one that traverses each edge exactly once, finally arriving back at

o

~AAA~ BAA

!

eAAB

!

return(-l) /* failure return for(I-' ; I I, is also balanced with parameters (m,n-I,fm), with the proviso that the first position of the original preamble now be ignored so that the effective new preamble is oflength n-2. I It is useful to consider the stimulus on any given trial in the

PROOF OF THEOREM

main sequence in association with its prefix, which is the string

96

EMERSON AND TOBIAS

of preceding stimuli of length n - I. The prefix ofthe first stimulus in the main sequence, thus, is the complete preamble. By the definition of the constraints, there are m distinct stimuli, mr :' distinct prefixes, and m» distinct ordered pairs of prefix and stimulus. A single prefix-stimulus pair is of length n, and each of the m stimuli occurs f times in each of the n positions in the full sequence oifm: occurrences of prefix- stimulus pairs. Consider any sequence satisfying the assumed constraints, with n > I, and consider the structure of the same sequence with n decreased by I-that is, with the prefixes shortened from n-I to n-2. Now, a single prefix-stimulus pair is of length n - I, but each of the m distinct stimuli occurs mftimes in each ofthe n-I prefix-stimulus positions, for a total oifm» occurrences ofprefix-stimulus pairs. With the first ofthe original preamble trials now disregarded, the resulting sequence is balanced with parameters (m,n-I,fm).

Q.E.D. Comment on Theorem 1 The principle clearly can be iterated all the way down to n - I = I, so sequences balanced for nth order constraints are also balanced for constraints of all lower orders. This is of interest in ANOVA applications, because the data analysis and ensuing inferences depend strongly on whether or not the effects to be tested are balanced in the design. THEOREM 2

The stimulus pattern in the last n - I positions in the main sequence is identical to that of the preamble. PROOF OF THEOREM 2

Any sequence that is balanced to order n is equivalent to an Euler path in an Euler graph whose vertices are all the mr:' distinct subsequences of length n-I, and whose edges connect pairs of vertices that overlap sequential1y by n - 2 stimuli. An Euler path in an Euler graph traverses each edge exactly once and must end at the same vertex from which it started (Tutte, 1966). Hence, the pattern of the last n - I stimuli in the sequence must be the same as that of the first.

Q.E.D. THEOREM 3, FROM HurCHINSON AND WILF (1975)

Let Vi (i = I, ... , m) be given positive integers, and let vij (i,j = I, ... , m) be given nonnegative integers. How many words can be madefrom an alphabet ofm letters, in such a way that letter i appears exactly Vi times in the word (i = I, ... , m), and exactly v ij times letter i is followed by the letter j (j = I, ... , m)? That number, N, is given by

in which M = m. This is Hutchinson and Wilf's (1975) Equation 1. Necessary and sufficient conditions that N> 0 for the case of the last letter in the word being the same as the first are M

M

(a) ~>ik = ~>ki(i =1,2, ... , M), k~l

k~1

M

(b)

Vi

=

~>ik + Dii1(i = 1, ... , M), k~!

(A2)

and

(c) the value of the determinant in Equation Al be a positive integer.

Equations A2 are the conditions of Hutchinson and Wilf's (1975) Equations 9, except that (c) was implied by the context and not stated explicitly. In (b), i 1 is the first letter of the word, and N of Equation Al is the number ofwords beginning with the letter i I. Hutchinson and Wilf did not use two different symbols, M and m, but we have done so here because of a generalized interpretation to be discussed below. Comment on Theorem 3 The class of sequences called "words" by Hutchinson and Wilf (1975) overlap only those in which n = 2 among those considered here, in which vij = f (constant). They are more general in allowing unequal frequencies of occurrence of the m letters (stimuli, in the present context) and of the m 2 possible transition patterns. Also, Hutchinson and Wilf stated necessary and sufficient conditions that N> 0 for the case complementary to that of Equations A2, in which the last letter is different from the first, but that case too is outside the class of sequences considered here. Though only cases of n = 2 are covered directly, it is easy to extend Equations Al and A2 for any integer n ~ 2. Extension of Equations Al and A2 to n ~ 2 The trick in this extension is to replace "letters" and "words" in the description of Hutchinson and Wilf(1975), with "symbols" and "chains," where a symbol is defined as a string of letters of length n- I, and a chain is a word, but is structured in terms of symbols rather than letters. Successive symbols overlap by n-2letters and, because of this overlap, only some symbols may follow others, for n > 2. According to the constraints imposed by the definition of the (m,n,f) sequences considered here, each symbol must occurfm times in the chain, except that the first in the chain occurs one extra time in the role of the preamble. The constraints on the frequencies of transitions from symbol to symbol are formulated by assigning values to the elements of the vij matrix. Every nonzero element under the present assumptions is equal to f The structure of the vij matrix is discussed most readily by adopting a particular convention concerning the designation of the letters and symbols. Let the m letters be 0, I, ... , m -I; the symbols then are (n-I)-digit m-ary numbers. There are mr:' symbols, and they can be designated as positive integers to specify rows and columns in the matrix. The size of the vij matrix, with this extension of Equation AI, becomes mn" by m n - ! rather than m by m, and so M in Equations Al and A2 is now taken to be mr:', The pattern of the elements of the v ij matrix in this scheme is quite regular and simple, because for n > 2, the last n-2Ietters of the kth symbol in the sequence must be the same as the first n-2 of the (k+ I)th. Each row of the matrix consists of all zeros, except for a string ofm adjacentfvalues. In the first row, these nonzero values occupy the first m columns. Any other row, i, is obtained by rotating Row 1 (i-I)m positions to the right, with wrapping of elements shifted out to the right, back in at the left. The nonzero elements form m cycles of a descending staircase pattern, with stairsteps of width m. The length of each of those cyclic staircases is mr:' matrix columns, and so there are mn - 2 steps per cycle and m cycles. The specifics of using Equations Al and A2 with this modification are to take M= mn-I,

97

RANDOMIZED BALANCED SEQUENCES

Vi =

mj+ Du!

for i = I, ... , M, (A3)

and V .. ij

=

{j,

m- modM(i-I)< j S m'modM(i-I)+m

0, otherwise for i.j = I, ..., M,

in which modMO is the remainder obtained in the integer division of its argument by M. In principle, this completes the extension, but a simple expression for the determinant in Equation AI, under the present simplifying constraints, would be convenient. Without proof, but based on numerical evaluations in cases up to m + n < 9, the value is asserted here to be (A4) The proof of M-I for the exponent onjis easy, as is the proof of M-2 for the other exponent in the special case of n = 2. M- n for the other exponent for n > 2 is a conjecture from numerical evaluations in the cases mentioned. Therewith, the number of sequences beginning with the symbol i] is found to be Nz, given by (AS) and the total number of sequences is N 3 = MN2 , given by (A6) which is known to be correct for m ally, and is conjectured otherwise.

Table At Normal Approximations to Chi-Square for Test of Uniformity of Probabilities of Selection From Sample Space of Size N 3 for Inclusive Tracings (z;) and First Tracings (Zj) N3

Bins

m

n

f

e

Zj

zf

4 16 36 216 256 400 1,296 4,900 63,504 65,536 160,000 331,776 729,000 853,776 1,679,616

4 16 36 216 256 400 1,296 4,900 7,938 8,192 8,000 8,660 8,100 8,712 7,776

2 2 2 3 2 2 2 2 2 2 2 4 3 2 2

2 3 2 2 4 2 3 2 2 5 3 2 2 2 4

I I

100 100 100 100 100 100 100 100 10 10 10 10 10 10 10

-2.04 -0.48 -1.81 -1.25 6.39 -0.70 0.69 -1.33 1.27 14.13 -0.13 7.11 3.78 -1.03 9.58

-0.97 -1.39 -0.92 -1.81 -1.32 -1.09 -0.84 .1.00 -0.12 1.30 1.34 -1.98 0.03 -0.92 -0.39

2 I I 3 2 4 5 I 3 I 2 6 2

TableA2 The Four Reduced Sequences Representing the 256 in the Sample Space for the Case of (m, n,f) = (2,4,1)

Sequence

Frequency

0000100110101111000 0000100111101011000 0000101100111101000 0000101101001111000

6,805 6,925 5,962 5,908

+ n < 9 or n = 2, gener-

A Test of Randomness For the IS smallest values of N 3, it was feasible to test whether or not the two methods, inclusive and first tracings, sample all the N 3 sequences with equal probabilities. Table Al shows the obtained results. For N 3 up to 4,900, a complete set of all N 3 ofthe sequences was generated as a target set. Then 100N3 others were generated, and the frequencies with which those of the target set were matched were tabulated. As is indicated in the e column, the expected frequency for each of the N 3 bins, then, is 100. Due to memory limitations, only a fraction of the N 3 different sequences could be stored as the target set, for N 3 > 4,900. The numbers of sequences in those fractional target sets are shown in the "Bins" column ofTable A 1. These sets contained the specified numbers of different sequences, so they represent the definite fraction, Bins/N 3 , of the sample space. By the time these fractional sets were generated, it had already become clear that the first-tracings method was more likely to generate completely random sequences, so the fractional target sets were all constructed by the first-tracings method and then were used to test both the inclusive and first-tracings methods. Also, for N 3 > 4,900, only ION3 test sequences were generated, so the expected frequency per bin in those cases is only 10. In the chi-square tests, the expected frequencies were taken to be equal to the theoretical, rather than based on the observed total, so the djfor the tests are given by Bins, rather than by Bins-I. The Zi (for inclusive tracings) and zf (for first tracings) are based on the Wilson-Hilferty normal approximation to chi-square, which is more accurate than the familiar one due to Fisher (Kendall & Stuart, 1963, p. 371) and is especially so for small df Five ofthese 15 Z values were significantly larger than zero for inclusive tracings, but none were for first tracings. Repe-

titions of some of the tests suggested that the indicated nonrandomness for inclusive tracings is restricted to certain cases, such as (m,n,f) = (2,4,1), for N 3 up to 4,900. Moreover, in the case of (2,4,1), the nonrandomness is not as severe as it may seem. In that case, the 256 different sequences can be reduced to just four minimal sequences, which are shown in Table A2, by rotations, changes of direction, and interchanging the two characters 0 and I. The four corresponding frequencies are also shown in Table A2, and though these depart quite significantly from the expected value of 6,400, the deviations are not so large as to make the method of inclusive tracings useless for randomization in practical experiments where only a few sequences are needed. These four reduced sequences fall into two groups with estimated probabilities of .536 and .464, respectively, each representing half the sample space of size 256. Within groups, the frequencies do not depart significantly from uniformity. Extended Sequential Characteristics The constraints force exact sequential balance for stimuli with lags up to n-I steps. What are the consequences for sequential dependencies at lags of n and larger? This question was investigated mainly with a lag of n steps, because greater lags involve more intervening random steps and would tend to weaken any propagation of the effects of the constraints. For each of 40 cases, 1,000 sequences were generated by each of the two methods, and the m-by-m matrix of frequencies was tabulated for stimulus i on trial k, and stimulusj on trial k+n. A systematic effect was found, and it seems to ensue inherently from the nature of the constraints and to depend very little, if at all, on the method of generation, inclusive or first tracings. Let p(j Ii) be the probability that stimulus j occurs on step k+n, given that stimulus i occurred on step k. In a purely ran-

98

EMERSON AND TOBIAS

2

IS 0

"'4 X

3

....~3 .J ....

4

~2

IS

~1 0

1

2

3

F

4

!5

Figure AI. Probability estimates, for n = 2, that stimulus i at step k+n followsstimulus i at step k; in balanced sequences with parameters (m,n,f) (plus signs) and in comparable random permutations of fm n - I occurrences of each of m stimuli ('-dledcircles) for m = 2,3,4,5 andf= 1,2,3,4,5. The parameter on the right is m, and the horizontal lines are at probabilities 11m.

5

2

0

"'4 X

>

3

.J

4 5

t:;3

....

~2

s

~1 0

1

2

3 F

4

5

Figure A2. Probability estimates, for n = 3, that stimulus i at step k+n followsstimulus i at step k; with the same plotting corwentions as those in Figure AI.

dom sequence of independent multinomial trials, p(i I i) = 11m, but that surely cannot be true for finite sequences in which there is a fixed number of occurrences of each stimulus, because of the principle of sampling without replacement. This applies even to simple random permutations off occurrences of each of m stimuli, which is a degenerate case of the present class of sequences with n = 1. Independent multinomial trials, therefore, is not an appropriate null model for comparison, and the degenerate case of n = I is more nearly comparable. For comparison to cases of (m,n,f) with n > 1, the n = I sequences were constructed with (m,l,fm n - i ) , so that they would be nearly equal in length and would contain nearly the

same frequencies of each of the m stimuli as in the case of (m,n,f). The method of randomization for the sequences with

n = 1 was that of Brysbaert's (1991) Algorithm 10 and Castellan's (1992) Algorithm 2. Figures Al and A2 show empirical estimates of p(i I i) at a lag of n, for n = 2 and 3, respectively, withf= 1,2,3,4,5 and m = 2,3,4,5 as the curves plotted with plus signs, and the corresponding n = I comparison curves plotted with filled circles. The horizontal lines are at p = 11m, as would be expected with independent multinomial trials. Clearly, all curves begin at the lowest point at f = I and increase gradually toward the 11m asymptotes with increasing f For the simulated cases of n = I, there exe fmv :' occurrences ofa given stimulus andfm n occurrences of all stimuli, so the theoretical values of p(i I i) at a lag of n can be calculated on the principle of sampling without replacement, as (fm n - 1-1)/(fm n-l). The empirical data, shown as filled circles in Figures I and AI, are very near the corresponding theoretical values. A similar theoretical basis for predictions of the cases of n > 1 is not known, but the exact probabilities were calculated by counting in the complete sets of sequences for the eight cases of N 3 up to 4,900 (see TableAI). For the seven of those cases included in Figures 1 and AI, the empirical estimates from the randomized sequences closely match the exact values. Separate plots are not shown for inclusive and first tracings, because they were almost indistinguishable on the scale ofthese graphs and did not appear to differ systematically. It is apparent that the principle of sampling without replacement affects all cases, but has stronger effects for n > 1 than in the simulated cases of n = 1. However, its effects decrease with increases in m, n, andf, as a simple consequence of longer sequences. Unequal!

Though it might seem hard to imagine applications of unequal f, at least one case has been used in the design of published research. Kornblum (1969) measured choice reaction times ina design with n = 2 and m = 4, but with unequalfvalues for the different digram patterns. He was interested in varying the ratio of "repetition" trials with transition patterns 00, 11, 22, and 33 to "nonrepetition" trials with patterns in which the members ofa digram are different. He varied the relative frequency of nonrepetitions at eight levels from .39 to 1.00. Hick (1952) and Hyman (1953) associated an information measure (related to that of Shannon & Weaver, 1949) with certain observed characteristics ofreaction time data, but Kornblum argued that they had not successfully unconfounded those effects from those of repetitions and nonrepetitions, and he proceeded to do so himself. Most of the f"mdings reported in this appendix do not apply to cases ofunequalf, but Equation Al applies to Kornblum's situation, with Vii = a (constant) being the frequency of each kind of repetition, and vij = b (constant) for i = jbeing the frequency of each kind of nonrepetition. Equations A2 indicate that such sequences exist for any integers a ~ = 0 and b > O. The subroutines of Listing I can generate Kornblum-type sequences, if used with a main program that sets up the array of specified frequencies, and in other cases ofunequalfas well. (Manuscript received April 8, 1992; revision accepted for publication April I, 1994.)