Seven principles for the development of multiple-choice questions ...

Seven principles for the development of multiple-choice questions - Short instructions for authors in Breathe This short instruction provides an overview regarding important aspects, which should be considered when writing multiple- choice (MC) items of the well-proven types (type A or type Kprime).

A) General aspects regarding type A questions MC questions of the ‘one best answer type’ (type A) consist of introductory sentences (=stem) followed by a list of five suggested responses (=options): The correct option (=key) on the one hand and the incorrect options (=distractors) on the other hand. Sample question 1 (type A questions: One best answer / single choice) A 25-year-old woman has had nearly continuous daytime sleepiness for 6 years. She either falls asleep or «blacks out» involuntarily several times a day, especially in business meetings, and has to be awakened by colleagues. She has frequent nocturnal awakenings, sometimes associated with nightmares. She has been told that she snores, although she now lives alone. Review of symptoms is remarkable for a several-year history of almost daily «collapsing» spells, lasting 20 to 30 seconds, during which she feels her knees buckle, requiring her to sit for a few minutes. She is 152 cm tall, weighs 70 kg, has a neck circumference of 43 cm, a blood pressure of 100/72 mmHg, and an otherwise normal physical examination. What is the most appropriate next step? (A) polysomnography (B) polysomnography followed by a Multiple Sleep Latency Test (MSLT) (C) overnight screening pulse oximetry (D) periodic office follow-up and advice to avoid or eliminate obesity, sleep deprivation, and bedtime alcohol or sedative intake (E) diagnostic psychiatric evaluation [correct answer: B]

Sample question 2 A 52-year-old smoker (25 PY) suffers from chronic cough, especially in the morning and slight dyspnoea while climbing the stairs to his flat on the third floor. Physical examination is normal, as is his chest x-ray. Spirometry reveals a fixed bronchoobstruction with a FEV1 of 65 % (of predicted) with an FEV1 / FVC ratio of 0.58. You diagnose a chronic obstructive pulmonary disease. Which of the following measures slows the progression of the disease? (A) smoking cessation (B) inhalation of anticholinergics (C) long-term inhalation of glucocorticoids (D) inhalation of long acting Beta2-agonists in combination with glucocorticoids (E) systemic corticoids [correct answer: A]

The following principles are essential to the preparation of MC questions, which are apt to contribute to valid and reliable examinations.

1. Relevant content Does your question relate to content that a) is frequently used in medical practice, b) represents a common source of mistakes or c) relates to mistakes that may have serious consequences? Please do not relate to rarities, subtleties, ... or your personal hobbies. The blueprint is the consensus-based European syllabus published in Breathe 2006; 3:59-70.

2. Topic, which needs application of knowledge Does your question require application of knowledge? Please do not only test recall of isolated facts but formulate questions, which require the candidates to understand given information, analyse a situation, form a synthesis, draw a conclusion...

3. Focussed question with homogeneous options Focus on one relevant aspect of the topic; all the options have to belong to exclusively one content dimension such as diagnoses, localisations, causes, management decisions, … Each option has to be plausible so that a candidate who lacks knowledge of the correct answer cannot view it as absurd.

4. Clear, unambiguous best answer Avoid 'correct' answers with existing controversial doctrines. Best and second-best answer should be clearly separated.

5. Proper difficulty (50 - 90 % correct answers) Questions should be of medium difficulty, so that 50-90 % of the candidates are expected to answer them correctly. If questions are too difficult, even good candidates tend to guess; if they are too easy, good candidates get confused, but weak candidates get a 'present'.

6. Unambiguous, concise, simple phrasing No trick questions; no unnecessary information; no double negatives (negative stem, negative option); use only very common abbreviations; paraphrase rather than using words lifted directly from a textbook; avoid imprecise quantifications like frequently, usually, often, etc…

7. Avoid cues Make all options grammatically consistent with the stem Keep the wrong options in about the same length and specificity as the correct answer Don’t use the opposite of the correct answer as a wrong option Don’t reuse elements of the correct answer in wrong options (convergence cue) Don’t use absolute terms (always, never, all...) as a means of making options incorrect List the options in a logical order if there is one; don’t place the correct answer predominantly in the middle (as option C or D).

B) General aspects regarding type Kprime questions MC questions of the ‘quadruple correct/incorrect decision type (type Kprime, K') consist of a question or incomplete statement (=stem) followed by a list of four answers or statement completions. Candidates have to decide for each of the four answers whether they are correct or incorrect and thus have to mark them appropriately with a (+) or (-). One, two, three, four or none of the answers may be correct. Sample question 3 (type Kprime, K': Quadruple correct/incorrect decision type) Which of the following statements concerning benign asbestos pleural effusion is/are true? (A) The presence of hemorrhagic fluid almost always indicates malignancy. (B) The latency period between exposure to asbestos and onset of disease is the same for mesothelioma and benign pleural effusion. (C) It rarely produces diffuse pleural thickening. (D) The predominant cell on cytologic examination of pleural fluid is the lymphocyte. [correct answer: - - + +]

Important points for the phrasing of Kprime items: • • • • • •

Phrase the stem always positive and the options positive whenever possible (to avoid double negatives) Phrase the stem neutral with regard to the number of correct options (to avoid cuing) Each option unambiguously has to be correct or false Put only one statement in every option (to avoid uncertainty about what has to be judged) Keep every option independent from each other Avoid having predominantly items with 3 or 4 correct options. Asse ss me nt a nd E v a lua t ion Unit, Inst i tu te of Me di c a l E duc a t ion IM L , (René K re bs & Anja Rogausc h), F a c u l t y o f Me di c ine , Uni v e rs i t y of Be rn, Konsums t ra sse 13, 3010 Bern, Sw i t z e r l a nd.

How to prepare good MC-Questions

Short Instructions for Authors in Breathe