Soc Psychiatry Psychiatr Epidemiol (1999) 34: 360±366
Ó Steinkop Verlag 1999
ORIGINAL PAPER
N. Schmitz á J. Kruse á C. Heckrath L. Alberti á W. Tress
Diagnosing mental disorders in primary care: the General Health Questionnaire (GHQ) and the Symptom Check List (SCL-90-R) as screening instruments* Accepted: 16 February 1999
Abstract Background: The treatment of mental disorders in Germany is mainly done by primary care physicians. Several studies have shown that primary care physicians have diculty in diagnosing these disorders. Recently, several self-report questionnaires have been developed that can be used as screening instruments to identify psychopathology in primary care settings and in the community. The aim of this paper was to investigate the screening properties of the General Health Questionnaire (GHQ-12) and the Symptom Check-List (SCL90-R) in a primary care setting in Germany. Method: A randomly selected sample (n = 408) of adult outpatients from 18 primary care oces in DuÈsseldorf was screened using the German versions of the GHQ-12 and the SCL90-R. A structured diagnostic interview (SCID) and an impairment rating (IS) were used as a gold standard to which both questionnaires were compared. Test performance was evaluated by receiver operating characteristic (ROC) analysis. Results: We found no dierence in the performance of the general scores of the two questionnaires. Both instruments were able to detect cases. Complex scoring methods oered no advantages over simpler ones for the GHQ-12. ROC analysis con®rmed that the SCL-90-R subscales ``anxiety'' and ``depression'' showed acceptable concurrent validity for the diagnostic groups anxiety and depression (according to DSM-III-R). Conclusions: GHQ-12 and SCL-90-R appeared to be useful tools for identifying mental disorders
N. Schmitz (&) á J. Kruse á C. Heckrath á L. Alberti á W. Tress Clinic for Psychosomatic Medicine and Psychotherapy, Heinrich-Heine University, Bergische Landstrasse 2, H19, D-40605 DuÈsseldorf, Germany e-mail:
[email protected], Tel.: +49-211-9224723, Fax: +49-211-9224709 * Grant support for this investigation was given by the German Federal Ministry for Research and Technology (BMFT).
in primary care practice and research. The use of GHQ12 or SCL-90-R, employed as a ®rst step, supplemented by a second-stage interview, may enhance the detection rate of mental disorder in primary care settings.
Introduction The problem of identifying mental disorders is increasingly recognised as an important health care issue. Mental disorders result in substantial patient suering and health care cost and are present in primary care in at least 20±36% of primary care outpatients (Spitzer et al. 1994; Tress et al. 1997). In fact, more patients with mental disorders are cared for in the primary care sector than in the mental health sector (e.g. Manderscheid et al. 1993). However, several studies have shown that primary care physicians have diculty in diagnosing these disorders in the majority of patients, who usually present with somatic symptoms suggestive of a medical condition, while volunteering few psychological complaints. As Spitzer et al. (1994) pointed out, major obstacles to the recognition of mental disorders by primary care physicians include inadequate knowledge of the diagnostic criteria, uncertainty about the best question to ask for evaluating whether those criteria are met, and time limitations inherent in a busy oce setting. Katzelnick et al. (1997) have shown that identi®cation and treatment of mental disorders in primary care can reduce disability and health care utilisation and improve quality of life. Recently, several self-report questionnaires have been developed that can be used as screening instruments to identify psychopathology in primary care settings and in the community. Examples are the General Health Questionnaire (GHQ, Goldberg 1972) and the Symptom Check-List (SCL-90-R, Derogatis 1977; Franke 1995). Both SCL-90-R and GHQ are well-researched instruments and are frequently used for case identi®cation (e.g. Gureje and Obikoye 1990; Koeter 1992; Witnitzer et al. 1992; Lykouras et al. 1996). Once an appropriate
361
cut-o point has been chosen, the questionnaires can be used as screening devices for the detection of psychological distress. A valid comparison between the German version of the GHQ-12 and the SCL-90-R cannot be made because to date the two instruments have not been examined simultaneously using the same case criterion and the same population. The aim of the present paper was to compare the criterion validity of SCL-90-R (90 items) with the criterion validity of a short form of GHQ (GHQ-12, 12 items) in a primary care setting, while the external criterion is the presence or absence of psychopathology as indicated by an experienced psychotherapist. Receiver operating characteristic (ROC) analysis was used as an evaluation technique for these two tests, using data from an epidemiological investigation in primary care (Tress et al. 1997).
Subjects and methods Instruments The GHQ and SCL-90-R are self-report questionnaires for the detection and measurement of psychopathology. An overall index based on all items is generally considered to be a measure of psychological (dys)function. On the basis of speci®c cut-o points, respondents can be assigned to a group with or a group without substantial psychopathology. Schmitz and Davies-Osterkamp (1997) and Davies-Osterkamp et al. (1996) discussed several cut-o points for the SCL in clinical practice. GHQ and SCL have been used as screening instruments in primary care in several studies (e.g. Mari and Williams 1985; Witznitzer et al. 1992; Joukamaa et al. 1995; Linden et al. 1996; Tiemens et al. 1996).
General Health Questionnaire The GHQ-12 is a self-report instrument for the detection of mental disorders in the community and among primary care patients. Accordingly, the GHQ has a four-point response scale, which is usually scored in a bimodal fashion ± symptom present: ``not at all'' (0); ``same as usual'' (0); ``rather more than usual'' (1); and ``much more than usual'' (1). The response scale is simply ``yes'' or ``no''. For psychometric analysis it is possible to use a simple Likert scale (0-1-2-3), although there are only small dierences from the point of view of case identi®cation between results produced by the different scoring methods. The total score, obtained by summing up the scores of the individual items, is a measure for severity of illness, as described by Goldberg and Williams (1991).
Symptom Check List SCL-90-R The SCL-90-R is a 90-item self-report symptom inventory designed to screen for a broad range of psychological problems. Each of the 90 items is rated on a ®ve-point Likert scale of distress, ranging from ``not at all'' (0) to ``extremely'' (4). Subsequently the answers are combined in nine primary symptom dimensions: Somatization, Obsessive-Compulsive, Interpersonal Sensitivity, Hostility, Depression, Anxiety, Paranoid Ideation, Phobic Anxiety and Psychoticism. In addition, three global indices provide measures of overall psychological distress: the Global Severity Index (GSI), the Positive Symptom Total (PST) and the Positive Symptom Distress Index (PSDI).
Impairment Score The Impairment Score (IS, German: BSS, Schepank 1995; Franz et al. 1997) is a standardised instrument which allows trained and clinically experienced interviewers to assess the severity ± ranging from ``not at all'' (0) to ``extremely'' (4) ± of clinically present psychological impairment on three subscales: physical, psychic, and socio-communicative (behaviour) impairment due only to mental disorders (not caused by somatic reasons). The total range of the sum-score is from 0 (no impairment) up to the maximum value 12 (extreme impairment); patients with a sum-score of 4, 5 or 6 can be described as medium symptomatic while patients with a score above 7 can be described as severely symptomatic. Structured Clinical Interview The Structured Clinical Interview (SCID, Wittchen et al. 1990) for the Diagnostic and Statistical Manual of Mental Disorders (DSMIII-R) is a diagnostic instrument that is widely used by researchers and clinicians in order to guide the diagnostic evaluation process. SCID ascertains the presence and severity of psychological signs and symptoms in the 4 weeks prior to the interview. Settings and population sample The study was an epidemiological observational study. Prevalence of mental disorders in primary care was assessed in a sample of 572 German adult outpatients from 18 randomly selected primary care clinics in DuÈsseldorf (Germany). Data collection took place from January 1995 to December 1995. In each practice, patient recruitment was carried out over a 2-week period during oce hours. Consecutive attenders of the clinics who were aged between 18 and 70 were asked to participate within a 2-week period. Subjects were informed about the general purposes of the study and asked to give their informed consent. After the general practitioner's consultation, the patients ®lled in symptom checklists (SCL-90-R, GHQ-12) and were examined and diagnosed by a mental health professional (among others, IS, DSM-III-R classi®cation). This interview was conducted blind (i.e. without knowledge of the questionnaire results), using the SCID (Wittchen et al. 1990). Patients who ful®lled the following two criteria were de®ned as cases of mental disorders: (a) a total score of 5 or more on the IS (i.e. considerable psychological impairment) and (b) any speci®c DSM-III-R diagnosis made according to the SCID. Therefore, non-cases were patients who were physically ill and had no or mild psychological impairment. The reasons for consultation were classi®ed using the Reason for Visit Classi®cation (RVC, Schneider et al. 1979). The most frequently mentioned reasons for visit were respiratory symptoms (22.2%), symptoms of the musculoskeletal system and connective tissue (14.0%), and symptoms of the digestive system (9.7%), while only 3.5% mentioned psychological problems. These results are consistent with a large epidemiological survey of primary care providers in Germany, where RVC was evaluated in a sample of 12,000 subjects (Schach et al. 1989). Detailed study methodology is reported by Tress et al. (1997). The sample comprised 179 men (31.3%) and 393 women (68.7%), mainly aged between 23 and 65 (mean = 42.7, SD = 15.7). The main diagnoses are summarised in Table 1. Of the 572 persons in the sample, 408 ®lled in SCL-90-R and GHQ-12 completely. Receiver operating characteristic curve The performance of screening questionnaires is usually expressed in terms of sensitivity, speci®city, and positive and negative predictive power. These parameters are quantitative expressions of the relations of questionnaire scores to some external case criterion. The SCID interview and the IS served as the gold standard against which both SCL-90-R and GHQ were compared.
362 Table 1 Frequencies of DSMIII-R diagnoses for cases and non-cases
Adjustment disordersa Alcohol dependencya Anxiety disordersa Depressive disordersa Eating disordersa Psychological factors aecting physical conditiona Somatoform disordersa Other disordersa Multiple disorders a
Cases % of total sample (n = 572)
Non-cases % of total sample (n = 572)
Total
Female (n = 393)
Total
11.3 4.0 5.2 5.2 3.8 6.6
12.4 1.3 5.6 3.4 4.2 6.9
8.8 9.9 5.1 6.1 2.9 5.8
4.9 2.9 3.3 1.5 2.0 9.1
5.6 0.6 3.6 0.5 1.9 8.4
3.5 8.2 2.8 2.0 2.4 10.6
17.1 2.2 16.9
19.3 2.1 17.4
12.3 2.3 15.8
13.6 0.3 7.2
13.3 0.0 6.8
13.4 1.2 7.9
Female (n = 393)
Male (n = 179)
Some of the individuals are listed in category Multiple disorders
Sensitivity is de®ned as the number of true cases of a disorder detected by the test (true-positives) divided by the number of all diseased subjects. Conversely speci®city is de®ned as the number of true-negatives (nondiseased subjects who were considered negative by the test), divided by the number of all nondiseased subjects. Positive predictive value is de®ned as the proportion of true cases who are correctly diagnosed, while negative predictive value is de®ned as the proportion of nondiseased subjects who are correctly diagnosed. The predictive values are clinically useful but depend very strongly on prevalence. With the introduction of receiver operating characteristic (ROC) analysis, an innovative method has become available for the graphic description of the relationship between sensitivity and speci®city and their relationship to dierent cut-o points. A ROC curve is obtained by combining sensitivity and speci®city data over all cut-o points. At each cut-o point, sensitivity is plotted as a function of 1 ) speci®city (or false positive rate). The points can be connected by a smooth curve. For a perfectly accurate test (sensitivity = 1, speci®city = 1), the ROC curve is a horizontal line connecting the points (0, 1) and (1, 1). For a random test whose discriminatory ability is no better than chance, the ROC curve will be a diagonal line connecting the points (0, 0) and (1, 1). This has been referred to as the ``line of no information''. Most actual tests will produce curves lying between these two extremes. The higher the sensitivity and speci®city at various cut-o points, the more closely the curve approaches the upper left corner of the graph. A measure used for the overall performance of an instrument is the area under the curve (AUC). Parametric and nonparametric methods exist that allow the calculation of the AUC and the comparison of tests. In this study, we used a computer program (ROCFIT, Metz et al. 1993), which ®ts a curve by a maximum likelihood technique.
Results Preliminary analysis showed that there was a linear relationship between the GHQ-12 and the SCL-90-R. The Table 2 The ®ve most prevalent items of the SCL-90-R and the GHQ-12
Male (n = 179)
correlation was 0.64 for the total scores, suggesting that a general factor for the two instruments may be present. Figure 1 shows the distribution of the Global Severity Index (GSI) for the cases and non-cases. As expected, there are large overlapping parts of the empirical distributions, although the distributions are dierent for the ®rst and second moments. As shown in Fig. 2, similar results can be found for the GHS General Score. Dierences in the total scores can be found for the cases regarding gender: women are characterised by higher scores for the global indices (SCL-90-R: female mean = 0.94 (SD = 0.65), male mean = 0.71 (SD = 0.44), t = 2.01, df = 140, P = 0.037; GHQ: female mean = 5.07 (SD = 4.12), male mean = 3.68 (SD = 3.42) t = 1.91, df = 140, P = 0.059), while there are no signi®cant dierences for the population of non-cases. The most prevalent items of the SCL-90-R and the GHQ are shown in Table 2. To make the prevalence of the items comparable, the items of the SCL-90-R were dichotomised according to the scaling of the GHQ. Somatic symptoms seemed to be common in this sample, but were not more prevalent than psychological symptoms. Similar results were found by Araya et al. (1992) in a primary care sample in Chile. Figure 3 presents the performance of the SCL-90-R and GHQ-12 total scores as screening instruments for mental disorders in primary care. The curves were obtained by plotting sensitivity against false-positive rate for all cut-o points of the two screening tests and ®tting a smooth likelihood curve. Both scores show acceptable concurrent validity (i.e. ROC curves well above the di-
SCL-90-R
Prevalence % (n = 142)
GHQ-12
Prevalence % (n = 142)
3 ± unpleasant thoughts 31 ± worry
52.5 50.0
52.8 50.0
27 ± lower back pains 2 ± nervousness 66 ± restless sleep
50.0 47.5 46.5
2 ± constantly under strain 8 ± not reasonably happy, all things considered 10 ± unhappy and depressed 1 ± lost sleep over worry 9 ± not able to enjoy day-to-day activity
47.9 45.8 45.1
363 Fig. 1 Histogram of the Global Severity Index (GSI) of the SCL-90-R for cases and noncases
agonal line of no information). Moreover, the ROC curve for the SCL-90-R total score (GSI) is nearly identical to that of the GHQ total score. A numerical presentation for sensitivity, speci®city and predictive values is provided in Table 3 and Table 4. The dierences between SCL-90-R and GHQ were small in this German primary care sample. A two-sample Z-test (twotailed) was applied to test the dierence for statistical signi®cance (Erdreich and Lee 1981). There was no signiFig. 2 Histogram of the GHQ General Score for diseased and nondiseased individuals
®cant dierence between the areas under the curves [SCL90-R: mean AUC = 0.75 (SD = 0.026); GHQ: mean AUC = 0.73 (SD = 0.028); Z = 0.73, P = 0.464]. No signi®cant dierence was found between ROC curves for GHQ bimodal and Likert scaling procedure. Dierences between the performance of the two questionnaires were, therefore, probably due to chance. In a second step we examined validity of the SCL-90R anxiety and depression subscales. Validity was es-
364
tablished with the DSM-III-R diagnosis for anxiety (300.00±02; 300.21±23; 300.29±30; 309.89) and depression (296.20±23; 296.29; 300.40; 311.00) disorders. Both scales show acceptable concurrent validity for the two diagnostic groups (i.e. ROC curves well above the main diagonal). Results are shown in Table 5. In comparison, the GHQ-12 was not designed to screen for dierent diagnostic groups. Although several studies have found a two-factor structure of the GHQ-12 (e.g. Gureje 1991), it is more appropriate to use the GHQ-12 as a global screening instrument. Alternatively, another version of the GHQ (GHQ-28; 28 items with anxiety and depression subscales) can be used to screen for depression and anxiety.
Discussion This is the ®rst study reporting on the comparative performance of the GHQ-12 and the SCL-90-R in a German primary care setting, using identical external criteria. The major ®nding is that the GHQ-12 and the SCL90-R general scores performed equally well in detecting Fig. 3 Receiver Operating Characteristic (ROC) curve for cases and non-cases using SCL90-R GSI and GHQ-12 General Score
mental disorders in a sample of 18 randomly selected primary care clinics in DuÈsseldorf, Germany. The small dierence between the questionnaires in their ability to detect cases of psychological morbidity was not statistically signi®cant. The two screening instruments were found to be easy to administer, although there was a dierence in the time spent by patients to complete them (roughly 2±5 min for the GHQ-12 and 10±20 min for the SCL-90-R). The questions in both questionnaires were well understood by the respondents. In our opinion, the SCL-90-R compared to the GHQ12 has some de®nite advantages. The SCL-90-R covers a broad range of psychological problems and symptoms of psychopathology. A global index as a measure for general distress and nine primary symptom dimensions can be computed. The analysis of the anxiety and depression subgroups indicated that the subscales can be used as screening instruments, too. In contrast, the GHQ-12 contains only 12 items. As a consequence, only a general distress factor can be computed from the items. On the other hand, there is often a time limitation in primary care clinics. Administration and evaluation of the SCL-90-R needs much more time than the application of the GHQ-12.
365
If a primary care physician is only interested in general psychological distress, the GHQ-12 questionnaire can be used as a screening instrument in the primary care sector. However, if there is a need for more detailed information concerning depression and anxiety, the GHQ-28 may be used. A more detailed diagnosis is attained by using the SCL-90-R. Nevertheless, some limitations of the screening instruments must be recognised. GHQ-12 and SCL-90-R focus on breaks in normal function, rather than upon lifelong traits. Additionally, both questionnaires are not able to detect personality and adjustment disorders (e.g. Goldberg and Williams 1991). As a result, misclass®cation may occur. Figures 1 and 2 show great overlaps of the symptom scores between cases and non-cases. There are several cases with low scores on GHQ-12 and SCL-90-R, indicating low levels of psychological distress. Further analysis of these subjects indicated that more than half of the cases with diagnoses of ``adjustment disorders'' and ``psychological factors aecting physical condition'' have GHQ sum scores of less than 2 and GSI scores of less than 0.5. In fact, these diagnostic groups are not well identi®ed by the two screening instruments. On the other hand, all subjects in this study were physically ill, which may result in a general distress factor (high scores on both scales) for the non-cases, too. Further sources of mis-
classi®cation may occur due to inaccuracy of diagnoses and the denying of symptoms in questionnaires. Some restrictions must be kept in mind regarding the measurement of mental disorders with the GHQ-12 and the SCL-90-R in the present study. First, the present study was not designed to assess validity of the two instruments in a primary care setting. No further selfreport measures (e.g. Inventory of Interpersonal Problems, Beck Depression Inventory, Spielberg Anxiety Inventory, etc.) were used to study construct validity of the instruments. Second, interviewing, diagnosing and rating was done by one mental health professional. Although this mental health professional was supervised by a team of researchers (physicians and psychologists), the single interviewer design may lack reliability and validity of the diagnostic data (e.g. Horowitz et al. 1979). Third, the study was conducted at 18 primary care clinics located in DuÈsseldorf, Germany. Although the primary care physicians were selected randomly, the present study does not comprise a nationally representative sample. Dierences across cities and countries may occur due to dierences in sociodemographics. The high prevalence rate of mental disorder (36.8%) needs to be replicated in further studies. Fourth, there was a small sample size in the diagnostic groups anxiety and depression. Comorbidity appeared in both groups. However, our ®ndings are consistent with other studies. Sandanger et al. (1998) used the Hopkins Symptom Checklist-25 (HSCL-25) in a Norwegian population survey as a screening instrument for mental disorders. They found a better performance for the subscales depression and anxiety than for the general score. Despite these limitations, the goals for the ®eld testing of the German version of the SCL-90-R and the GHQ-12 in a primary care setting were achieved. The instruments showed acceptable qualities for diagnosing mental health disorders in the primary care sector. The use of GHQ-12 or SCL-90-R, employed as ®rst step, supplemented by a second-stage interview, may enhance the detection rate of mental disorder in primary care settings. There is a need for future research in this ®eld. Due to time limitation in primary care clinics, a computerised administration of the self-report measures could be used in primary care.
Table 5 Relationship between DSM-III-R diagnostic group and SCL-90-R subscales
DSM-III-R diagnostic group ``Depression''
DSM-III-R diagnostic group ``Anxiety''
Screening tool: SCL-90-R Subscale ``Depression''
Screening tool: SCL-90-R Subscale ``Phobic anxiety''
Cases (according to DSM-III-R)
n = 33 Mean = 1.21 SD = 0.87
n = 36 Mean = 0.94 SD = 0.90
Non-cases (according to DSM-III-R)
n = 128 Mean = 0.37 SD = 0.39
n = 128 Mean = 0.16 SD = 0.22
Area under the curve
AUC = 0.81 SD = 0.044
AUC = 0.86 SD = 0.038
Table 3 Validity coecients for the SCL-90-R at dierent thresholds for all screened individuals Threshold
0.4
0.5
0.6
0.7
0.8
0.9
Sensitivity Speci®city Positive predictive value Negative predictive value
0.75 0.59 0.51 0.81
0.64 0.74 0.58 0.77
0.57 0.78 0.60 0.76
0.52 0.82 0.62 0.75
0.46 0.85 0.63 0.73
0.39 0.95 0.77 0.70
Table 4 Validity coecients for the GHQ at dierent thresholds for all screened individuals Threshold
1.5
2.5
3.5
4.5
5.5
6.5
Sensitivity Speci®city Positive predictive value Negative predictive value
0.68 0.65 0.53 0.78
0.60 0.74 0.57 0.76
0.51 0.81 0.60 0.74
0.46 0.85 0.64 0.73
0.39 0.89 0.67 0.72
0.32 0.93 0.72 0.71
366
References Araya R, Wynn R, Lewis G (1992) Comparison of two self administered psychiatric questionnaires (GHQ-12 and SRQ-20) in primary care in Chile. Soc Psychiatry Psychiatr Epidemiol 27: 168±173 Davies-Osterkamp S, Strauss BM, Schmitz N (1996) Interpersonal problems as predictors of symptom related treatment outcome in long-term psychotherapy. Psychother Res 6: 164±176 Derogatis LR (1977) SCL-90-R, administration, scoring, and procedures manual for the R(evised) version. Johns Hopkins University, School of Medicine, Baltimore Erdreich LS, Lee ET (1981) Use of Relative Operating Characteristic analysis in epidemiology. Am J Epidemiol 114: 649±662 Franke GH (1995) SCL-90-R: Die Symptom-Check-Liste von Derogatis ± Deutsche Version. Beltz Test Gesellschaft, GoÈttingen Franz M, Schmitz N, Tress W (1997) MoÈglichkeiten zur empirischen Erfassung des Schweregrades psychogener Erkrankungen. Prax Klin Verhaltensmed Rehabil 37: 7±12 Goldberg D (1972) Detecting psychiatric illness by questionnaire. Oxford University Press, Oxford Goldberg D, Williams P (1991) A user's guide to the General Health Questionnaire. NFER-Nelson, Windsor Gureje O, Obikoye B (1990) The GHQ as a screening tool in a primary care setting. Soc Psychiatry Psychiatr Epidemiol 25: 276±280 Gureje O (1991) Reliability and the factor structure of the Yoruba version of the 12-item General Health Questionnaire. Acta Psychiatr Scand 84: 125±129 Horowitz LM, Inouye D, Seigelman EY (1979) On averaging judges' rating to increase their correlation with an external criterion. J Consult Clin Psychol 47: 453±458 Joukamaa M, Lethinen V, Karlsson H (1995) The ability of general practitioners to detect mental disorders in primary health care. Acta Psychiatr Scand 91: 52±56 Katzelnick DJ, Kobak KA, Greist JH, Jeerson JW, Henk HJ (1997) Depression among high utilization patients: eect of identi®cation and primary care treatment on service utilization, disability, and quality of life. Psychiatr Serv 48: 59±64 Koeter MWJ (1992) Validity of the GHQ and SCL anxiety and depression scales: a comparative study. J Aect Disord 24: 271±280 Linden M, Maier W, Achberger M, Herr R, Helmchen H, Benkert O (1996) Psychological disorders and their treatment in general praxis in Germany. Results of a World Health Organization (WHO) study. Nervenarzt 67: 205±215 Lykouras L, Adrachta D, Kalfakis N, et al (1996) GHQ-28 as an aid to detect mental disorders in neurological inpatients. Acta Psychiatr Scand 93: 212±216
Manderscheid RW, Rae DS, Narrow WE, Locke BZ, Regier DA (1993) Congruence of service utilization estimates from the epidemiologic catchment area project and other sources. Arch Gen Psychiatry 50: 108±114 Mari JJ, Williams P (1985) A comparison of the validity of two psychiatric screening questionnaires (GHQ-12 and SRQ-20) in Brazil, using Relative Operating Characteristic (ROC) analysis. Psychol Med 15: 651±659 Metz CE, Wang PL, Kronman HB (1993) ROCFIT. Department of Radiology and the Franklin McLean Memorial Research Institute, University of Chicago Sandanger I, Moum T, Ingebrigsten G, Dalgard OS, Sorensen T, Bruusgaard D (1998) Concordance between symptom screening and diagnostic procedure: the Hopkins Symptom Checklist-25 and the Composite International Diagnostic Interview I. Soc Psychiatry Psychiatr Epidemiol 33: 345±354 Schach E, Schwartz FW, Kerek-Bodden HE (1989) EVaS-StudieEine Erhebung uÈber die ambulante medizinische Versorgung in der Bundesrepublik Deutschland. Deutscher AÈrzte Verlag, Cologne Schepank H (1995) Der BeeintraÈchtigungsschwerescore (BSS). Ein Instrument zur Bestimmung der Schwere einer psychogenen Erkrankung. Beltz Test Gesellschaft, GoÈttingen Schmitz N, Davies-Osterkamp S (1997) Klinische und Statistische Signi®kanz ± diskutiert am Beispiel der Symtom Check Liste (SCL-90-R). Diagnostica 43: 80±96 Schneider D, Appleton L, Mdemore T (1979) A Reason for Visit classi®cation for ambulatory care. Data evaluation and Methods Research 2 (78), National Center for Health Statistics, Public Health Service. US Government Printing Oce, Washington D.C. Spitzer RL, Williams JBW, Kroenke K, et al (1994) Utility of a new procedure for diagnosing mental disorders in primary care. JAMA 272: 1749±1756 Tiemens BG, Ormel J, Simon GE (1996) Occurence, recognition, and outcome of psychological disorders in primary care. Am J Psychiatry 153: 636±644 Tress W, Kruse J, Heckrath C, Schmitz N, Alberti L (1997) Psychogene Erkrankungen in hausaÈrztlichen Praxen. Z Psychosom Med 43: 211±232 Wittchen HU, Schramm E, Zaudig M, Spengler P, Rummler R, Mombour W (1990) SKID-Strukturiertes Klinisches Interview fuÈr DSM-III-R. Beltz, Weinheim Witznitzer M, Verhulst FC, van den Brink W, et al (1992) Detecting psychopathology in young adults: the Young Adult Self Report, the General Health Questionnaire and the Symptom Checklist as screening instruments. Acta Psychiatr Scand 86: 32±37
