International Journal for Quality in Health Care 2000; Volume 12, Number 1: pp. 19–24
Patient questionnaires in primary health care. Validation of items used in asthma care ¨ M3 AND MARGARETA MO ¨ LLER4 MALOU LINDBERG1, JOHAN AHLNER2, TOMMY EKSTRO 1
Division of General Practice and Primary Care, 2Division of Clinical Pharmacology, 3Department of Pulmonary Medicine and Department of Neuroscience and Locomotion, Department of Medicine and Care, Faculty of Health Sciences, Linko¨pings University, Sweden
4
Abstract Objective. To evaluate each item in a patient questionnaire for the purpose of investigating whether the validity of each item is acceptable. Design. The questionnaire was completed by the patients at an ordinary follow-up visit for their asthma, and within 1 week a nurse interviewed them by telephone with the aim of analysing the validity of each item through the use of predetermined criteria. Settings. Patients with asthma in primary health care settings in Sweden. Study participants. Fifty-one patients were consecutively included from three different primary health care units. Results. Nine of 13 items had an acceptable validity. The four items that were not found to have acceptable validity were developed further. Conclusion. Evaluating each item in a questionnaire by means of interviews with the specific patient population is a useful method of assuring that the intention of the patient questionnaire has been met. Keywords: asthma, interview, quality improvement, questionnaire, validation
The symptoms of asthma include recurrent episodes of wheezing, chest tightness, shortness of breath and coughing [1]. In Sweden, the prevalence of asthma is estimated at 5–8% and the disease affects at least 450 000 people [2–4]. Various international guidelines are available regarding the management of asthma [5]. These guidelines describe what is required to attain good quality in asthma care [6,7]. It is commonly the case that management/treatment is not optimal with respect to the guidelines, and a multidimensional approach in evaluating the quality of asthma care would be of great value [8]. ¨ stergo¨tland, Sweden, a previous study In the county of O was carried out to investigate the quality of health care received by asthmatics at each primary health care unit in the county [9]. In that study a patient questionnaire was used to survey the quality of care by means of combining the patient questionnaires with administrative data from each primary health care unit (Figure 1). Comparisons among the
different participating primary health care units were also carried out. The questionnaire was designed through teamwork by representatives of the different professions involved in asthma care in the county. An expert panel then discussed whether the questionnaire appeared to assess the desired qualities. The goal was for each item in the questionnaire to include important aspects in accordance with international guidelines [6,7]. The questionnaire was to reflect important factors in asthma management, such as the monitoring of peak expiratory flow and symptoms, regular asthma check-ups by health care professionals, and the patient’s own knowledge about their disease, their medications and prevention. Several questionnaires which assess the patient’s own experiences of asthma evaluate quality of life [10,11]. These quality of life questionnaires can also be used indirectly in quality assurance in asthma care, but some of them are extensive and take a long time to answer, and are therefore
Address reprint requests to M. Lindberg, Department of Research and Development in Primary Health Care, SE- 595 30 Mjo¨lby, Sweden. E-mail:
[email protected]
2000 International Society for Quality in Health Care and Oxford University Press
19
M. Lindberg et al.
Figure 1 Patient questionnaire and agreement level results. not always applicable in the clinical situation. The purpose of the questionnaire described above was to create a simple questionnaire that could easily be incorporated as part of recurrent follow-ups of the quality of asthma care in health care services. The aim was for the questionnaire to conform to the standards of international goals for asthma care [6,7]. The aim of the present study was to evaluate the validity of each individual item in the questionnaire to ascertain whether answers to the different items in the questionnaire correspond to what was intended by the professionals.
Method Asthmatics from three different primary health care units were consecutively included in the study. Patients with the diagnosis asthma, diagnosis code 493 (ICD 9), >17, who were at an ordinary follow-up visit, were included. Patients receiving care for acute asthma exacerbations were not included in the study.
20
Patients gave their informed consent to participate in the study. After the consultation with the physician they answered the questionnaire, ‘For the patient with asthma’ (Figure 1). After the questionnaire had been filled in a member of the staff forwarded it to the Department of Research and Development in Primary Health Care and reported the patient’s name and telephone number by telephone to an asthma nurse. The asthma nurse then telephoned the patient within 1 week to conduct a telephone interview. Each interview took about 30 minutes and had both open and closed alternative answers. One nurse, who was trained in this interview method, performed all of the interviews (Table 1). The intention was to follow up each item in the questionnaire in the interview form. The interviewer regarded only episodes that had occurred during the week prior to the visit. Before start of the study the interview form was tested in a pilot study. Criteria for having understood each item in the questionnaire in a correct way were stipulated in advance by the professionals. Guided by the answers given in the telephone
Patient questionnaires in primary health care
Table 1 The questionnaire, the telephone interview questions and the predetermined criteria Item Question Predetermined criteria ............................................................................................................................................................................................................................. Knowledge/technique Item 1 Do you have your own PEF-instrument? See footnote Have you been instructed in how to make a PEFKnows what a PEF diagram is diagram? What do you measure with a PEF-instrument? Lung function. Peak Expiratory Flow How often do you measure PEF? Twice a week or when I get worse Do you know your normal PEF-value? States the PEF-value in litres/minute Do you know your alarm PEF-value? States the PEF-value in litres/minute Item 3 How long have you taken medication via inhalation? See footnote Have you received instructions on how to inhale? Item 5 What do you know about asthma prevention? Mentions actions such as taking prophylactic medication, cleaning up the environment, etc. What should you avoid being exposed to? Mentions allergens or irritants Compliance Item 2 How often do you take your asthma medication? A ‘yes’ answer in the questionnaire must be confirmed by stating that medication is taken daily Care and treatment Item 4 Can you describe your written plan of action? Written orders on a special form or a medication list Item 7 What is your doctor’s name? States the name of the doctor Item 8 How often do you go for an asthma check-up? Do you automatically get follow-up appointments? See footnote Symptoms Item 9 Have you visited a health care facility directly, See footnote without an appointment, in the last 6 months? Item 11 Can you describe your typical asthma attack? Did you use a B2 agonist last week because of See footnote shortness of breath? Item 12 Last week were you: A ‘yes’ answer in the questionnaire must be • awakened due to shortness of breath? confirmed by a ‘yes’ for one of the four • awakened due to coughing? symptoms • awakened due to wheezing? Do you suffer from insomnia because of your asthma? Item 13 Did you have to limit your physical activity last week See footnote due to your asthma? Could you exercise the way you wanted last week See footnote without shortness of breath? Life style Item 10 Are you a current smoker? See footnote If so, how much do you smoke each day? Information/knowledge Item 6 Have you received information about the disease? See footnote What happens in your airways when you get shortness Airway constriction and inflammation of breath? 1
To satisfy the predetermined criteria, a ‘yes’ answer to an item in the questionnaire had to be confirmed in the interview, as did additional information as specified. PEF, Peak expiratory flow.
interviews, the asthma nurse made a subjective judgement as to whether the answers in the questionnaires were in accordance with the predetermined criteria (Table 1). For each question, the proportion of answers that were similar on both the questionnaire and the interview was
calculated. This proportion will henceforth be referred to as the agreement level. The agreement level for each item in the questionnaire was calculated for the whole study group. Prior to its start the study was approved by the ethics committee.
21
M. Lindberg et al.
Statistical analysis Our interest focused on finding items with a poor agreement level. For each item, we tested the null hypothesis p =0.85 versus the alternative hypothesis p < 0.85, where p is the population agreement level, and 0.85 was chosen arbitrarily. Exact binomial probabilities were calculated, and P-values < 0.05 were considered significant. With this sample size, this means that H0 will be rejected for items with an observed agreement level Ζ 75%. No Bonferroni adjustment was calculated because in this case it is more important to keep a low probability of a type-I error than to reduce the probability of a type-II error. The sample size was determined in order to get 80% power if p =0.70.
action plan as being the prescription itself or the information included on the packaging of the medication or inside the packaging. (ii) Have you received information about asthma prevention? Agreement level of 66% P< 0.001 Fifteen patients who answered ‘yes’ in the questionnaire stated in the telephone interview that they did not know anything about asthma prevention. Another two patients did not reach the predetermined criteria concerning asthma prevention. (iii) Did you have more than two asthma attacks last week? Agreement level of 56% P< 0.001
Results Sample All 51 patients invited to take part in the study agreed to participate, and filled in the questionnaire. The study group consisted of 32 females and 19 males, mean age 48 years (range, 18–86 years). Fifty patients who completed the study were on regular inhalation therapy, with a mean usage time of 11 years (range, 1–35 years). Three of the patients were using medication that did not include steroids. The remaining 47 patients had an inhaled steroid prescription with a mean diurnal-dose of 750 lg (range, 200–1600 lg), and three of the patients also had daily oral steroid prescribed. Another three patients had oral steroid for use during periods of exacerbation. Forty-nine patients had a prescription for shortacting inhaled B2-agonist, and 11 patients had a prescription for long-acting inhaled B2-agonist. One-fifth of the patients reported that they did not take their medication as prescribed, and one-third reported that they had had an acute consultation with a doctor during the last 6 months because of their asthma. Two-thirds reported asthma-related symptoms during the last week and two-thirds had used a mean of 11 doses (range, 1–30 doses) of short-acting inhaled B2-agonist during the last week because of dyspnoea. There was one drop-out from the telephone interview part of the study because it was not possible to contact the person within the 7-day limit. The telephone interview was carried out a mean of 2.7 days (range, 0–7 days) after the questionnaire had been answered. Of a total of 13 items, nine showed an acceptable agreement level of >75%, and the highest agreement level was 98%. See figure 1. In the comparison between the questionnaire and the interview, four items showed a low agreement level of Ζ 75%. The following items had a low agreement level: (i) Do you have a written plan of action? Agreement level of 64% P< 0.001 Nine patients who had answered ‘yes’ in the questionnaire said in the phone interview that they did not have any written plan of action. Another nine patients said that they had understood the written
22
Twenty-two patients who had answered ‘no’ in the questionnaire said in the phone interview that they had used 3–30 doses of a short-acting inhaled B2agonist during the last week due to dyspnoea. (iv) Were you bothered by symptoms during the night last week? Agreement level of 74% P< 0.05 Twelve patients who had answered ‘no’ in the questionnaire stated in the phone interview that: • they had awakened with shortness of breath due to asthma (n=3) • they had been bothered by coughing due to asthma (n=10) • they had been bothered by wheezing due to asthma (n=2) • they had been bothered by interrupted sleep due to asthma (n=4). (In some cases the same patient had more than one symptom.)
Discussion This study focuses on the difficulty in constructing simple questions aimed at a specific disease population. A question that is addressed to a specific patient group and that seems to be expressed clearly is not always understood as intended by the professionals who formulated it. It is important to evaluate the items in a questionnaire in order to ensure that the questions measure what they were intended to measure. Despite the fact that an expert panel had discussed both the face validity and the content validity of the questionnaire before its use in an earlier study [9], our study shows a low agreement level in four of 13 items in the questionnaire. This points out the necessity of continuing the evaluation procedure in order to improve the validity of a method. When the expert panel discussed the content validity before using the questionnaire, their aim was to determine whether there were any irrelevant items or whether any important areas had been missed. That is considered a minimum prerequisite before starting to administer a questionnaire [12].
Patient questionnaires in primary health care
In the present study the patients in each unit were included consecutively by using the diagnosis code for asthma (ICD 9). Other studies have shown that patients with cardiopulmonary or asthma-like conditions such as chronic obstructive pulmonary disease or functional breathing disorder have received the diagnosis of asthma (ICD 9) [13,14]. Taking that into consideration, some of the patients might not be ‘true’ asthma patients. However, the primary care units that took part in this study had their own spirometers and a special asthma practice, and they also had access to and knowledge about the local asthma consensus report, which is in accord with international consensus. This could indicate that the patients in this study received the correct diagnosis [15–17]. The items with low agreement levels were those that concerned symptoms due to the disease and items that reflected the patient’s own knowledge of the disease and treatment. Symptom items such as an item in which a patient with a long history of asthma is asked if he/she has been bothered by nocturnal symptoms are not sensitive enough. An asthmatic patient has probably adjusted to the symptoms and they have become a part of their life situation. The answers to the questions consequently showed an underestimation of the symptoms. The need for more specific symptom questions becomes apparent through the interviews. Based on this information, the question about nocturnal symptoms has been expanded to include the following four questions: (i) Were you awakened last week due to shortness of breath? (ii) Were you awakened last week due to coughing? (iii) Were you awakened last week due to wheezing? (iv) Do you suffer from insomnia because of your asthma? Nocturnal worsening of asthma is a significant problem in the management of asthma. It has been shown to be an important symptom in relation to the severity of the disease, and it is also reflected in mortality statistics [18]. It is therefore of great importance to pinpoint those patients who suffer from nocturnal symptoms for therapeutic consideration [19]. The item in the questionnaire that was intended to register asthma attacks also had a low agreement level, as the patients under-rated their asthma attacks. The interview made it clear that to the patient, an asthma attack meant a more serious, acute situation, resulting in a visit to the emergency ward, than the professionals intended when they constructed the item. For this reason, another question should follow this one namely, ‘Did you use a B2-agonist last week due to shortness of breath?’ The aim was to be able to assess the frequency of the patient’s asthma attacks. There were patients who denied having had asthma attacks. Nevertheless, they had used 3–30 doses of a B2-agonist during the last week due to dyspnoea. None of the patients who reported asthma attacks seemed to exaggerate. The other two items that showed a low agreement level dealt with the patient’s own knowledge of the disease. These questions were, ‘Do you have a written plan of action?’ and
‘Have you received information about asthma prevention?’ To these questions 18 and 17 patients, respectively, answered in the affirmative, but in spite of this their answers in the interviews did not meet the stipulated criteria. Additional questions that would lead to more precise answers are, ‘Can you describe your written plan of action?’ and ‘What do you know about asthma prevention?’ Development of the above four questions with a low agreement level was done with the help of knowledge obtained from the interviews. Our intention with the evaluation procedure was to create a valid, uncomplicated patient questionnaire aimed at reflecting a part of the quality of care for an asthma population in primary health care. Several other patient questionnaires have been constructed for the purpose of improving the management of asthma. Some of them focus on asthma knowledge, quality of life, or are used in epidemiological studies of asthma [10,20,21]; however, none has been designed for the purpose of studying the quality of asthma care in a simple way, and their usefulness in clinical practice can be questioned. A short, validated questionnaire would make it possible for the general practitioner to evaluate the asthma care he/ she provides in comparison with the quality of the care given by others, in the same or in other primary health care units. Another application would be to evaluate interventions. This patient questionnaire could be used as a pre- and postmeasure method to study whether an intervention affects the quality outcomes in asthma care. Used together with administrative data, it also would be a valuable method for evaluating whether the intentions stipulated in international guidelines for asthma management are met in a specific asthma population. The possibility of conducting multidimensional quality assurance in asthma care will also give us basic data for making decisions about how asthma care should be organized in order to receive priority. This would confirm our ambition to use evidence-based management in asthma care, which is important as public health care has limited resources.
Conclusion There is an obvious risk that questions in patient questionnaires will not be understood as intended by the professionals who formulated them. Therefore it is important that the validity of a patient questionnaire is determined not only by the judgement of an expert panel. Evaluating each item in a questionnaire by means of interviews with the specific patient population is one method of assuring that the intention of the patient questionnaire has been met.
Acknowledgements We thank Olle Eriksson, Department of Mathematics, University of Linko¨ping, Sweden, for statistical advice.
23
M. Lindberg et al.
References 1. Dantzker D, Pingleton S, Pierce J et al. Standards for the Diagnosis and Care of Patients with Chronic Obstructive Pulmonary Disease (COPD) and Asthma. Am Thorac Soc, 1986; 225–244. ˚ berg N, Lindberg U. Allergic diseases in Swedish school 2. A children. Acta paediatr Scand 1988; 77: 821–825. ˚ berg N. Asthma and allergic rhinitis in Swedish conscripts. 3. A Clin Allergy 1989; 19: 59–63. 4. Norrman E. High prevalence of asthma and related symptoms in teenagers in Northern Sweden. Eur Respir J 1993; 6: 834–839. 5. Richard A. National and international guidelines for the diagnosis and treatment of asthma. Curr Opin Pulmon Med 1997; 3: 51–55. 6. International consensus report on diagnosis and treatment of asthma. Bethesda: National Heart, Lung, and Blood Institute, National Institute of Health. Publication no. 92–3091, March 1992. Eur Respir J 1992; 5: 601–641. 7. Guidelines for the Diagnosis and Management of Asthma. Bethesda: National Asthma Education and Prevention Program. National Institutes of Health publication, 1997. 8. Liza C, Reynold A. Assessing Quality of Care in Asthma Management. Drug Benefit Trends 1998; 10(8): 35–40. 9. Lindberg M, Ekstro¨m T, Mo¨ller M. Quality Assurance of the care of Bronchial Asthma patients treated at the Primary Health ¨ stergo¨tland (in Swedish). Report from the Care Level in O Department of Research and Development in Primary Health Care 1996: p. 1. 10. Hyland ME. The Living with Asthma Questionnaire. Respiratory Med 1991; 85: 13–16. 11. Marks GB, Dunn SM, Woolcock AJ. A scale for the measurement of quality of life in adults with asthma. J Clin Epidemiol 1992; 45: 461–472.
24
12. Streiner DL, Norman GR. Health Measurements Scales. A Practical Guide to Their Development and Use, 2nd edn. Oxford: Oxford Medical Publications, 1998. 13. Martinez FJ. Diagnosing chronic obstructive pulmonary disease. The importance of differentiating asthma, emphysema, and chronic bronchitis. Postgrad Med 1998; 103(4): 112–117, 121–122, 125. 14. Ringsberg K. Patients with asthma-like symptoms but negative asthma test and patients with bronchial asthma. Physiological, psychological and social characteristics. Linko¨ping University, Medical Dissertations. No.522, 1997. 15. Adelroth E, Hargreave FE, Ramsdale EH. Do physicians need objective measurements to diagnose asthma. Am Rev Respir Dis 1986; 134: 704–707. 16. Joyce D, Chapman KR, Kesten S. Prior diagnosis and treatment of patients with normal methacholine challenge and unexplained respiratory symptoms. Chest 1996; 109: 697–701. 17. Ringsberg K, Lo¨whagen O, Sivik T. Psychological differences between asthmatics and patients suffering from an asthma-like condition, functional breathing disorder: a comparison between the two groups concerning personality, psychosocial and somatic parameters. Integr Physiol Behav Sci 1993; 28: 358–367. 18. Robertson CE, Rubinfeld AR, Bowej G. Deaths from asthma in Viktoria: a 12-month study. Med J Aust 1990; 152: 511–517. 19. Martin RJ, Kraft M. Nocturnal asthma: therapeutic considerations. Clin Immunother 1996; 6: 443–453. 20. Fitzclarence CAB, Henry RL. Validation of an asthma knowledge questionnaire. J Paedistr Child Health 1990; 26: 200–204. 21. Asher MI, Keil U, Anderson HR et al. International study of asthma and allergies in childhood (ISAAC): rationale and methods. Eur Respir J 1995; 8: 483–491.
Accepted for publication 2 November 1999
