Author's personal copy c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 3 ( 2 0 0 9 ) 104–108
journal homepage: www.intl.elsevierhealth.com/journals/cmpb
An adaptive staircase procedure for the E-Prime programming environment W. David Hairston a,b,∗ , Joseph A. Maldjian b a b
Human Research and Engineering Directorate, US Army Research Laboratory, USA Department of Radiology, Wake Forest University School of Medicine, USA
a r t i c l e
i n f o
a b s t r a c t
Article history:
Many studies need to determine a subject’s threshold for a given task. This can be achieved
Received 20 May 2008
efficiently using an adaptive staircase procedure. While the logic and algorithms for stair-
Received in revised form
cases have been well established, the few pre-programmed routines currently available
11 August 2008
to researchers require at least moderate programming experience to integrate into new
Accepted 18 August 2008
paradigms and experimental settings. Here, we describe a freely distributed routine developed for the E-Prime programming environment that can be easily integrated into any
Keywords:
experimental protocol with only a basic understanding of E-Prime. An example experiment
E-Prime
(visual temporal-order-judgment task) where subjects report the order of occurrence of two
Psychophysics
circles illustrates the behavior and consistency of the routine. © 2008 Elsevier Ireland Ltd. All rights reserved.
Methods Staircase Adaptive
1.
Introduction
Many studies in psychology and physiology require the use of stimuli or tasks that are near a threshold level of discriminability. Because the absolute threshold value for any stimulus can vary across scenarios and subjects, it is often necessary to determine a subject’s threshold in conjunction with the study procedure. One common way for determining the threshold is to present several levels of the stimulus and determine the desired value from a derived psychometric function. While this method typically provides good results, it requires many repetitions of each stimulus level, which can be very time consuming and is impractical for many situations. An alternative method is an adaptive “staircase” procedure, an algorithm that actively adjusts the stimuli on-line in response to the subject’s performance [1]. The procedure adjusts according to pre-set rules and terminates when the
stimuli approximate the subject’s threshold, making it a much more efficient method at the cost of lower data availability [2]. Due to this efficiency, staircase procedures are ideally suited for experiments targeting low perceptibility stimuli, and the algorithm logic has been described in depth previously [3,4] and several variants have been developed (e.g. [5,6]; for reviews see [7] or [8]). Currently, a limited number of pre-programmed procedures are freely available to researchers online (see Appendix A for some examples) for integrating a staircase procedure into new experimental paradigms. However, all currently available versions are implemented in fairly complex programming environments such as Matlab (The Mathworks Inc., Sherborn, MA) or C++, requiring some degree of programming proficiency to use. In contrast, E-Prime (Psychology Software Tools Inc., Pittsburgh, PA) is a programming suite specifically targeted for designing behavioral experiments. Because of its
∗ Corresponding author at: Auditory Research Team, VAP-HRED, B 520, Army Research Lab, Aberdeen Proving Ground, MD 21005, USA. Tel.: +1 410 278 5925; fax: +1 336 716 0798. E-mail addresses:
[email protected],
[email protected] (W.D. Hairston). 0169-2607/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.cmpb.2008.08.003
Author's personal copy c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 3 ( 2 0 0 9 ) 104–108
105
ease of use, low cost, and requirement for only very minimal programming skills, E-Prime is targeted for small psychology laboratories and researchers without a programming background or related facilities. Additionally, it is used by an active community with a large number of freely available pre-made experimental protocols (e.g., the STEP database [9]). Here, we describe a staircase routine developed for use in the E-Prime environment that can be easily added or modified to fit most scenarios with only minimal understanding of script-level programming. We describe an experiment using the routine to acquire the threshold for a visual temporal-order-judgment (TOJ) task that illustrates its performance.
staircases run independently, with the result averaged at the end of the procedure. In the variant described here, the target variable is modified according to a 2-down, 1-up rule, where the value increases when an incorrect response is given, and decreases only after two consecutive correct responses. Analogous routines are provided for other rules (e.g. 3 down, 1 up, etc.). The value of each staircase begins at a user-defined value, and changes according to a user-defined step size. After a defined number of stimulus reversals (e.g. change in the direction of stimulus trend), the step size decreases, and will do so again after yet another number of response reversals. The procedure terminates after a user-defined (default = 16) number of reversals.
2.
2.1.2.
Methods
2.1.
Software
2.1.1.
Overview
The procedure and associated experiment was developed in E-Prime 1.1.4.4. The staircase procedure adjusts the value of some target variable defined by the user. The only requirement is that the variable (typically a particular property of an Object, such as the duration) to be adjusted must be accessible by using Inline code. Multiple (default = 6, but can be any number)
Components
The routine is comprised of two primary components; each is an Inline script that can simply be copied into any experiment in E-Studio, or a new experiment can be built around the existing framework. The first component is an Inline script object containing the parameters of the staircase, all of which can be user-defined and is easily edited without the need for a detailed understanding of programming. The user can define the number of staircases to occur (default = 6) simultaneously, the initial values of each, the initial step size, and the ratio between up and down steps. Additionally, the user can define
Fig. 1 – Screenshot of the Inline script containing the staircase parameters. Every feature of the staircase procedure (e.g., step size, number or reversals, etc.) can be easily set by the user by changing the values within this script.
Author's personal copy 106
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 3 ( 2 0 0 9 ) 104–108
the number of reversals before changing the step size, and the new step size once this value is reached (up to 4 possible step sizes). All other necessary variables, such as the maximum number of reversals before termination and the number of values to include in the final reported average are user-defined within this script, according to documentary labels for each. Fig. 1 shows a screenshot of a sample portion of this script. The user can implement this component by simply copypasting the text from the sample provided into his/her own experiment, and modifying each variable as appropriate for their own paradigm. The second component is an inline script that contains the script for the staircase manipulation. It needs no modifications from the user and can simply be copied and pasted into a new experiment as-is. This script prints the status of each trial (including accuracy, current number of reversals, etc.) to the E-Studio Debugger window, enhancing on-line tracking of the subject’s progress. Accuracy, in this case, refers to whether the subject responded correctly for the stimulus present, providing the basis for the consequent stimulus adjustment. Additionally, all relevant data is written to a tabdelineated text file named according to the subject number. This Inline script is embedded in a List object so that it will repeat the requisite number of times to complete the procedure. In addition to these two Inline script objects, a third Inline contains the declaration of the target variable to be adjusted (a single line of code edited by the user). Users can implement the staircase by simply copying and pasting these three Inline scripts into an experiment, along with the “user script” contained in the example; this provides the appropriate variable and class definitions for the routine to operate correctly. Examples containing the staircase procedure, along with limited documentation are freely available from the corresponding author, and our laboratory website (www.fmri.wfubmc.edu).
2.1.3.
Logic and implementation
The actual implementation of the routine mimics the logic of the staircase algorithm as discussed in detail previously [1,3,4,8,10]. In short, the “staircase” Inline script maintains the current status and appropriate changes to the target variable, and contain three primary components. First, an evaluation is made as to whether a reversal in stimulus presentation (changing from lowering to raising, or vice versa) has been made. If it has, a variable is incriminated to keep track of the total number of reversals made thus far. This is necessary so that the procedure will auto-terminate when the pre-defined maximum has been reached. Second, the current step size is evaluated and, if necessary, is decreased, based on pre-determined values (set in the Parameters script). Third, the target variable is adjusted according to the amount of the step size (the value of which was just determined) according to a rule (e.g., 2-down, 1-up) depending upon the script chosen. Specifically, if the response was incorrect, meaning that the subject did not give the intended report, the value is increased by a step. In contrast, it will be instead decreased only if both the current and previous responses were correct (2 down, 1 up rule). Otherwise, no change is made. Additional embedded functions track the
total number of trials, ensure the algorithm does not exceed a stimulus max or minimum, write a data file, and compute an average reversal value.
2.2.
Example experiment
The procedure has been used and partially described in previous studies [11–14]. Subjects sat in a sound-attenuated booth and performed a visual TOJ task in which they reported which of two circles appeared first on a computer screen (LG 915FT+, 200 Hz vertical scan). Following instructions, a fixation cross appeared, which subjects were instructed to keep their eyes focused on. On each trial, the first of two circles appeared on the screen after a delay of 1500 ms. The circle could be either 5 cm above or below the fixation cross. Following a variable stimulus onset asynchrony (SOA), a second circle appeared at the location opposite the illuminated first circle. Subjects then responded using a button press with the right hand. Following the response, both circles disappeared, and a new trial began. Participants ran approximately 15 practice trials before starting the series. The staircase algorithm was used to vary the duration of the SOA between visual onsets (in E-Prime, the duration property of a Wait object). Three staircases ran independently, starting at SOAs of 15 ms, 55 ms, and 85 ms. For each, the initial step size was 20 ms and decreased to 10 ms after five reversals, and then to 5 ms after the next four reversals. The SOA increased one step after each incorrect response, and decreased one step after 2 consecutive correct responses. Each staircase terminated after 16 reversals, with an average calculated from the last 5 reversal values. This process converged on perceptual thresholds with average performance rates of ∼70–75% accuracy. In order to verify that the SOA derived from the staircase procedure was near threshold for each individual, subjects performed a second block, with SOA values set at, 10 ms above, and 10 ms below their previously determined thresholds. Each of these three SOAs was repeated 20 times in random order.
3.
Results
For the group (N = 15), the mean threshold SOA derived by the staircase procedure was 45.3 ms (±2.7 ms S.E.). Data from the follow-up block (verification) showed a mean accuracy of 75.3% (±2.5% S.E.) for the final values derived with the algorithm. On average, the complete procedure required 110.4 (±2.96 S.E.) total trials for all three staircases to terminate and exit the procedure (due to reaching the maximum number of allowable response reversals for each). The shortest case was only 90 trials, and the longest required 141 trials for all three to terminate. In general, the final averages created from each of a subject’s three staircases were within 10–15 ms (2–3 minimum steps of minimum size) of one another, showing reasonable consistency in the acquisition of the target threshold within a subject. In Fig. 2, panels A and B show staircases for two representative subjects. Each line represents an independently adjusted procedure; each trial randomly selects between the procedures. Note the sawtooth-like profile of the trend that is typical for this type of adjustment, with each of the three
Author's personal copy c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 3 ( 2 0 0 9 ) 104–108
107
resulting in non-representative final values. An example of this is shown in Fig. 2C. This subject made several incorrect consecutive responses within one procedure (SC 1, see dashed circle) which led the target value to be increased to a point where it was well beyond the threshold range. This agrees with the well-documented susceptibility of staircase procedures to unreliable results for subjects who are inconsistent or show unusually high false alarm rates. For most studies, results from these aberrant procedures would be thrown out prior to compiling the total mean score per subject with new data acquired in their place.
4.
Fig. 2 – Example staircase behavior for three subjects (A, B, and C). A and B are typical subjects. All three staircases (solid, dashed, and dotted lines; SC = staircase number) approximate similar values by the termination of each procedure so that the mean for each (averaged across the last five reversal values) are within less than 15 ms (three steps) of one another. In C, a series of consecutive incorrect responses (dotted circle) has led the algorithm to increase the SOA beyond the normal threshold range, resulting in an inaccurate assessment for that procedure.
trends approximating similar values by the end of the procedure. In a few cases, the range between staircase results within a subject was greater (six subjects). In these instances, two of the three staircase results were relatively close to one another, with the remaining results well outside this range and skewing the total mean. Visual inspection of the data showed that during one of the staircase procedures a series of inappropriate responses had led the algorithm off track,
Discussion
The results described above confirm that the implemented procedure for E-Prime operates in a manner typical for this type of algorithm. Overall, the multiple staircases per subject converged on similar end points, supporting the general efficacy of the algorithm. Although a handful of subjects did show one staircase with differing results from others collected at the same time, this is not an uncommon occurrence with staircases and may be related to the relatively small maximum reversal count used in this paradigm, which can increase variability [10]. Running multiple staircases per subject and averaging across the results generally alleviates the potentially negative influence of aberrant results from one routine; a number of studies within our own lab have used this routine with success [11–14]. While the algorithm used in this example is not new and has been in use for several decades (e.g. [1,4]), this is the first procedure of its kind developed to be easily built into numerous kinds of experiments. The current version is structured so that it can be easily copied and integrated into any new or existing E-Prime protocol with the use of only rudimentary script-based programming skills. This, in conjunction with the general simplicity of the E-Prime interface, provides the opportunity for researchers to derive perceptual thresholds without having the need for extensive programming resources. The staircase protocol described here for the E-Prime programming environment provides a useful tool for the scientific community. In its current version, the procedure is structured to be easily customizable to the needs of the user. For example, although the TOJ experiment described here used a variable step-size technique, a fixed-step-size could instead be implemented without any change to the code structure. The step size, number of critical reversals, or any other global variable can similarly be changed. Here, we used the procedure to modify the duration of a pause between images, functionally adjusting the SOA between onsets of the stimuli. However, any other property or object could be adjusted, such as the filename of a media file or the location of an object on the screen.
5.
Conclusions
We have described a software routine for performing an adaptive staircase procedure within the E-Prime programming
Author's personal copy 108
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 3 ( 2 0 0 9 ) 104–108
Table A.1 – Example routines currently available for performing staircase procedures Environment Matlab Matlab Matlab Matlab C++ Python
Website
Notes
http://www.edvul.com/fast.php http://psy.ucsd.edu/∼rdbeer/ http://psychtoolbox.org/ http://arthur.is.verweg.com/projects/staircase/ http://virtualreality.physiol.ox.ac.uk/SCASE/main.html http://www.psychopy.org/home.php/Docs/Tutorial2
environment. The general results of the procedure are typical, as verified using a TOJ task in a sample experiment. This implementation, which is freely available to the scientific community, is ideally suited for laboratories with minimal programming resources that plan to use this software suite for developing experimental paradigms.
Conflict of Interest None declared.
Acknowledgements Funding for the study was provided in part from the National Institutes of Health NIBIB grant EB004673 in conjunction with the Human Brain Project.
Appendix A See Table A.1.
references
[1] T.N. Cornsweet, The staircase method in psychophysics, Am. J. Psychol. 75 (1962) 485–491. [2] C.A. Johnson, B.C. Chauhan, L.R. Shapiro, Properties of staircase procedures for estimating thresholds in automated perimetry, Invest. Ophthalmol. Vis. Sci. 33 (1992) 2966–2974. [3] G.B. Wetherill, H. Levitt, Sequential estimation of points on a psychometric function, Br. J. Math. Stat. Psychol. 18 (1965) 1–10.
“FAST” toolbox Basic staircase procedure Psychophysics toolbox Based on original Levitt algorithm Based on Johnston et al. [15] Description of staircase in PsychoPy
[4] H. Levitt, Transformed up-down methods in psychoacoustics, J. Acoust. Soc. Am. 49 (Suppl. 2) (1971), 467+. [5] M.A. Garcia-Perez, Properties of some variants of adaptive staircases with fixed step sizes, Spatial Vision 15 (2002) 303–321. [6] R. Alcala-Quintana, M.A. Garcia-Perez, A comparison of fixed-step-size and Bayesian staircases for sensory threshold estimation, Spatial Vision 20 (2007) 197–218. [7] B. Treutwein, Adaptive psychophysical procedures, Vision Res. 35 (1995) 2503–2522. [8] M.R. Leek, Adaptive procedures in psychophysical research, Percept. Psychophys. 63 (2001) 1279–1292. [9] B. MacWhinney, J. St James, C. Schunn, P. Li, W. Schneider, STEP—a System for Teaching Experimental Psychology using E-Prime, Behav. Res. Methods Instrum. Comput. 33 (2001) 287–296. [10] M.A. Garcia-Perez, Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties, Vision Res. 38 (1998) 1861–1881. [11] W.D. Hairston, J.H. Burdette, D.L. Flowers, F.B. Wood, M.T. Wallace, Altered temporal profile of visual-auditory multisensory interactions in dyslexia, Exp. Brain Res. 166 (2005) 474–480. [12] D.A. Hodges, W.D. Hairston, J.H. Burdette, Aspects of multisensory perception: the integration of visual and auditory information in musical experiences, Ann. N.Y. Acad. Sci. 1060 (2005) 175–185. [13] W.D. Hairston, D.A. Hodges, J.H. Burdette, M.T. Wallace, Auditory enhancement of visual temporal order judgment, Neuroreport 17 (2006) 791–795. [14] W.D. Hairston, D.A. Hodges, R. Casanova, S. Hayasaka, R. Kraft, J.A. Maldjian, J.H. Burdette, Closing the mind’s eye: deactivation of visual cortex related to auditory task difficulty, Neuroreport 19 (2008) 151–154. [15] E.B. Johnston, B.G. Cumming, A.J. Parker, Integration of depth modules: stereopsis and texture, Vision Res. 33 (1993) 813–826.
