representation and utilization of information during ... - McGill University

REPRESENTATION AND UTILIZATION OF INFORMATION DURING THE CLINICAL INTERVIEW IN MEDICINE

DY

DAVID R. KAUFMAN

A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES AND RESEARCH IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS

DEPARTMENT OF EDUCATIONAL PSYCHOLOGY AND COUNSELLING McGILL UNIVERSITY, MONTREAL AUGUST 1987 © David R. Kaufman, 1987

(

ACKNOWLEDGEMENTS

1 would like to thank aIl the endocrinologists, residents and students, who so graciously volunteered their time and effort to participate in this study. 1 wish to express my deepest gratitude to my supervisor, Dr. Vimla Patel for her continuous support, patience. and encouragement at every stage of this project. 1 wou Id like to thank Dr.

y ogesh Patel for recommending the patient who participated in our study and for providing invaluable assistance with the medical aspects of the study. 1 would also like to acknowledge Dr Jeffrey Wiseman who provided medical consultation in the analysis of protocols. 1 am deeply indebted to all my colleagues and friends at the Centre for Medical Education who provided generous assistance in the analysis of data and in the preparation and the editing of this manuscript. In particular, 1 would like to thank José Arocha, Aldo Braccio, Stephen Chase, Anoop Chawla, Anne Cruess, and Zaida Léon. Trish Fung-ALing and Myléne Trottier were involved with this study from the onset and deserve special thanks. 1 would like to thank Alain Breuleux for translating the abstract. 1 wish to express my deepest appreciation to my family for their continuous support and encouragement throughout my academic career and indeed throughout my life. 1 would particularly like to thank my brother Kenny, who so generously allowed me to borrow his Macintosh computer for the better part of a year so that 1 could complete this thesis.

(~ i

ABSTRACT

....

This study evaluated the ability of subjects at 3 levels of expertise, expert physicians, residents and medical students, in the acquisition, representation, and utilization of patient information in the context of solving a complex medical problem. Each subject interviewed a volunteer medical outpatient and was subsequently requested to provide a differential diagnosis. The doctor-patient dialogue was analyzed using cognitive methods of discourse analysis. These methods were used to characterize differences in the content and nature of the history-taking process and in the development of problem representations. The study characterized differences at two levels of representation, observations and findings. Observations are the minimal semantic units of the doctor patient discourse. Findings are higher order units that derive meaning in specifie medical contexts. Differences were found between groups of subjects in the accuracy of diagnoses and in the qualitative nature of representations. These differences were manifested most clearly in tenns of a series of efficiency measures designed to characterize the ability of subjects to generate findings. In general, the expert physicians were more selective in the elicitation and processing of critical and relevant findings. An attempt is made to characterize these differences in terms of the strategies used to acquire and represent patient information.

ii

(

RESUME Cette étude évalue, dans le contexte de la résolution d'un problème médical complexe, les habiletés d'acquisition, de représentation et d'utilisation des informations concernant un patient chez des sujets de trois niveaux d'expertise médicale: des médecins experts, des résidents et des étudiants en médecine. Chaque sujet a été appelé à questionner un patient volontaire en clinique externe et, par la suite, à produire un diagnostic. Le dialogue médecin-patient a été analysé à l'aide de méthodes cognitives d'analyse du discours. Ces méthodes ont servi à caractériser des différences dans le contenu et la nature du processus au cours duquel l'histoire de cas est établie ainsi que dans l'élaboration d'une représentation du problème. Ces différences sont caractérisées à deux niveaux de représentation: les observations et les conclusions (findings). Les observations constituent les unités sémantiques minimales du discours médecin-patient. Les conclusions sont des unités de niveau supérieur qui prennent un sens dans des contextes médicaux spécifiques. Des différences ont été trouvées entre les groupes de sujets pour ce qui est de l'acuité des diagnostics et de certains aspects qualitatifs des représentations. Ces différences apparaissent plus clairement à travers une série de mesures d'efficacité conçues pour caractérise-la capacité des sujets à générer des conclusions (findings). En général, les médecins experts ont été plus sélectifs en suscitant et en traitant les conclusions critiques et pertinentes. Une tentative est faite pour caractériser ces différences en terme de stratégies employées pour obtenir et représenter les informations concernant le patient.

( iii

-

PRESENT ATIONS AND PUBLICATIONS

The work presented in tbis thesis have been reported in the following:

Presen tations: Kaufman, D. R. (1985). The clinical interview: A cognitive perspectiv~. In a Symposium on "Cognitive approaches to comprehension and problem solving in medicine" (V. L. PateI, Chair) at the I8th Reunion du Club de pédagogie Medical du Québec, Montréal. Kaufman, D. R. & PateI, V. L. (1986). Problem representation and the clinical interview: Expert-Novice Differences. Paper presented at the American Educational Research Association Meeting, San Francisco, CA. Kaufman, D. R. & PateI, V. L. (1987). Causal reasoning in medical explanation: Expertintennediate-novice differences. Paper presented at the American Educational Research Association Meeting, Washington, DC.

Publications: PateI, V. L., Evans, D. A. & Kaufman, D. R. (in press). Cognitive Framework For Doctor-Patient Communication. In Evans, D. A. & PateI, V. L.(Eds.) Cognitive Science in Medicine. Boston: MIT Press.

iv

(

T ABLE OF CONTENTS

Acknowledgments .................................................................................. i Abstract. ............................................................................................. Ü Resumé .................................................... " ....................................... .iii Presentations and Publications ................................................................... iv List of Tables ....................................................................................... vi . .. LIst 0 f F'19ures ..................................................................................... vu . f A d' ... LIst 0 ppen Ices ................................................................................ VUI

Chapter One: Review of Literature Introduction ................................................................................. 1 Problem Solving in review ............................................................... 3 Medical Problem-Solving ................................................................. 9 Chapter Two: The Clinieal Interview and the Task of Diagnostic Reasoning The Clinical Interview ............................................................... 28 The Diagnostic Reasoning Task. ................................................... 31 Chapter Three: Experimental Methods Subjects .................................................................................... 38 Patient ...................................................................................... 38 Procedure .................................................................................. 38 Clinical Case .............................................................................. 39 Methods of analysis ...................................................................... 40 The coding of Doctor-Patient Dialogue............................................ 41 Analysis Of Clinical Observations ................................................ .45 Analysis Of Clinical Findings ...................................................... 47 Chapter Four: ResuUs and Discussion Diagnostic Accuracy ...................................................................... 52 Clinical Observations ..................................................................... 54 Clinical Findings .......................................................................... 61 Information in Focus ..................................................................... 71 Chapter Five:

General Discussion ...................................................... 80

References ....................................................................................... 87

v

LIST OF TABLES

Table 1 Accuracy of diagnosis by level of expertise .................................................. 53 Table 2 Summary of observations generated by experts ............................................. 56 Table 3 Summary of observations generated by residents .......................................... .57 Table 4 Summary of observations generated by medical students .................................. 58 Table 5 Summary of findings generated by experts .................................................. 65 Table 6 Summary of findings generated by residents ................................................ 66 Table 7 Summary of findings generated by medical students ....................................... 67 Table 8 Doctor-patient dialogue Medical Student 5................................................74.1 Tabel9 Doctor-patient dialogue Resident 3 ......................................................... 76.1 TabellO Doctor-patient dialogue Expert 2........................................................... 77.1 Tabelll Doctor-patient dialogue Expert 5........................................................... 78.1

.... vi

LIST OF FIGURES

Figure 1 Percentage of observations by level of expertise..... ......................... ............. 55 Figure 2 Percentage of findings by level of expertise................................................. 63 Figure 3 Percentage of critical and relevant findings by level of expertise......................... 69

( vii

LIST OF APPENDICES

Appendix A

Experimental Tasks Appendix B

Oinical Text Appendix C

Differential Diagnoses by Level of Expertise Appendix D

Experts' Observations by Category Appendix E

Residents' Observation by Category Appendix F

Medical Students' Observations Categorized Appendix G

Positive Findings

viii

CHAPTER ONE

REVIEW OF LITERATURE

Introduction The doctor ;'ltient interview is an interactive, goal-clirected process, diagnosis being one of the primary goals. The diagnostic process can be characterized as having two major objectives (Morgan & Engels, 1969): one is to characterize the nature of the pathophysiological process with a degree of precision. The other is to evaluate the consequences of this process in tenns of the individu al patient. Competency in the clinical interview is a multifaceted construct involving abilities and attributes such as communication/interpersonal skills, medical knowledge and diagnostic reasoning skills (Neufeld, 1985). The clinical intetview is perhaps the most important source of medical data that a physician has access to (Feinstein, 1967). Yet, it is one of the most neglected aspects of medical education (DiMatteo, 1979). In recent years there has been an expression of increasing dissatisfaction with CUITent measures of clinical competence (Gale & Marsden, 1983). These measures have been found to be lacking in test-retest reliability, and in most measures of validity beyond face validity (Norman, Tugwell, Feightner, Muzzin, &

J acoby, 1985). In essence, the se traditional measures are inadequate predictors of who will become a "good physician". It has been suggested that the most serious difficulty is the lack of a precise description of the many dimensions of clinical competency (Neufeld, 1985). There is no theoretical framework from which to construct indices of competence. This thesis presents a framework in which the clinical interview can be viewed as a problem-solving situation. This study focuses on the representation and utilization of patient information in the course of the clinical interview. This investigation uses a

(

contrastive approach, whereby clinicians are compared at three levels of expertise in taking 1

a patient's history and diagnosing the clinical problem. The literature review is divided into two sections. In the first section, general

concepts of problem-solving and cognitive psychology are discussed. In addition, selected studies of expert-novice problem-solving are reviewed. The second section focuses on research into medical problem sol" ing. This section aIso discusses in sorne depth the current theoretical models and perspectives that have evolved in rnedical problem-solving. The second chapter reviews the nature of the clinical interview and the task of diagnostic reasoning. This section provides a theoretical framework and ration aIe for the experiment.

2

(

Problem Solving in Review

Recent research in problem-solving has shifted from the investigation of performance in artificiaI and closed environments to investigating performance in "real world" complex domains (Chi, Glaser, & Rees, 1982). Problem-solving in these artificial tasks (e.g., the tower of Hanoi problem) was not depenaent on subjects' vast stores of domain knowledge. The shift in research to "knowledge rich" domains brought about a corresponding emphasis on the nature and structure "f knowledge (Glaser, 1985). Problem representation has been one of the primary concerns of these studies. Problem-solving research is typically conducted within the framework of the general information-processing model (Newell & SimcIl, 1972). The concept of problem space is central in this theoretical approach. The problem space is a cognitive structure consisting of a set of knowledge states and operators that transfonn the se states (Newell, 1980). The problem consists of a set of initial states, set of goal states, and a set of path constraints. The constraints de termine the permissible operators and the possible knowledge states that can be invoked in the course of problem-solving. The problemsolving process is characterized by a search in the problem space, working out from a current state by applying operators, elaborating the constraints, and moving toward the goal state (Newell, 1980). The problem representation is constructed while working within this problem space (Voss, Greene, Post, & Penner, 1982). A problem representation can be defined as a cognitive structure constructed by the problem-solver on the basis of his or her domainrelated knowledge and ilS internai organization (Greeno & Simon, 1985). This includes the problem solver's model of the problem, its elements, relationships among these elements, and inferences about the problem derived from prior knowledge. In much of the problem-

(

solving literature the terms problem representation and problem space are not clearly

3

......

differentiated. The concept of a problem space represents a general work space and the concept of problem representation refers to a closer approximation of the configuration of elements and states in the problem that suggest a solution. The problem representation refers to a subject's understanding or interpretation of the problem (Voss et al .• 1982). We can assume that there is a many-to-one mapping from objects in the problem space to objects in the representation. Within a given problem space, there exist numerous possible problem representaticns. The study of expertise is one of the principal paradigms in problem-solving research. Comparing experts to novices provides us the opportunity to explore the aspects ofperfonnance that undergo change and result in increased competency (Lesgold, 1984) .

.

. This has contributed to an increasing interest in a developmental psychology of perfonnance changes that occur as different knowledge structures emerge and cognitive strategies are acquired (Glaser, Lesgold. & Lajoie, 1986). A consistent theme across studies of the development of expertise has been the evolution of knowledge structures and the corresponding impact these structures have in facilitating the recognition of significant objects within a problem. de Groot's (1965) pioneering research in chess represents one of the earliest characterizations of expert-novice differences. In one of his experiments, subjects were allowed to view a chess board for 510 seconds and were then required to reproduce the position of the chess pieces from memOI)'. The grandmaster chess players were able to reconstruct the mid-game positions with better than 90% accuracy, while novice chess players could only reproduce approximately 20% of the correct position. When the chess pie ces were placed on the board in a random configuration, not encountered in the course of a normal chess match, expert chess masters' recognition ability feIl to that of novices. This result suggests that superior recognition ability is not a function of superior memory, but is a result of an enhanced ability to recognize typical situations (Chase &

4

--------------------------------------(

-

Simon, 1973). This phenomena is accounted for by a process known as "chunking". A chunk is any stimulus or patterns of stimuli that has become familiar from repeated exposure and is subsequently stored in memory as a single unit (Larkin, McDermott, Simon, & Simon, 1980). It has been proposed that a chess master has approximately 50,000 chunks of different chess patterns stored in long term memory (Simon & Gilmartin, 1973). Chase and Simon (1973) attempted to uncoverdifferences in chunking ability that correspond to levels of expertise. The results indicated that players at each level of expertise were able to retrieve pieces in chunks. The principal difference was the size of the chunks. Experts' chunks were larger, containing three to six pieces. The experts' chunks tended to define meaningful game relations among the pieces. These chunks are

"1

not static knowledge structures but provide a foundation from which a mas ter can plan a sequence of moves (Chi, Glaser & Rees, 1982). The phenomena of superior performance resulting from chunking has been demonstrated in various skill games such as, Go (Reitman, 1976) and bridge (Charness, 1979), as weIl as in more complex domains. Egan and Schwarz (1979) replicated the studies in chess, by comparing skilled electronic technicians with novice subjects in reca1ling symbolic circuit drawings. The results indicated a superior performance in recall by the experts when the drawings corresponded to a normal circuit configuration. This

r.

,-1

advantage did not hold for randornly arranged structures. In addition, the resuIts indicated

r:

that expert technicians reconstruct circuit diagrams according to the functioning nature of the elements in the circuit, while novices are more likely to chunk according to the spatial proximity of the elements. This again demonstrates the pragmatic utility of chunks. The organization of an expert's knowledge not only enhances performance, but facilita tes the selective processing and acquisition of information. Chiesi, Spilich, and

(

Voss (1979) compared high domain knowledge subjects (HK) with low domain

'-

5

,

knowledge subjects (LK) in a series of tasks. They proposed that the game of baseball has a hierarchical goal structure, with winning the game being at the top of the structure. Understanding the activity of the game consists of relating sequences of actions and state changes to the goal structure. The results indicated that HK subjects were more readily able to recognize changes in basebaU description than LK individu aIs. This difference increased as a function of the importance of change in relation to the goal structure. HK individuals demonstrated a greater capacity to recall event sequences. The authors attribute this to the fact that HK individuals are better able to keep track of the states of goal-related variables than LK individuals. Knowledge-based differences impact on the problem representation and determine the strategies a subject uses to solve a problem. Simon and Simon (1978) compared a novice subject with an expert subject solving textbook physics problems. The results indicate that the expert solved the problem in one quarter of the time required by the novice with fewer errors. The novices solved most of the problems by working backward from the unknown problem solution to the givens of the problem statement. The expert worked forward from the givens to solve the necessary equations and deterrnine the particular quantities they are asked to solve for (Simon and Simon, 1978). According to Larkin and colleagues (Larkin, Mcderrnott, Simon & Simon, 1980) a forward-reasoning strategy is only possible when working on problems where one has had considerable experience.

The forward-reasoning strategy requires that a subject's

representation of the problem leads to the generation of a chain of inferences toward the solution state. This requires sufficient knowledge of the consequent changes in states that each inference produces (Lesgold, 1984). Novices are incapable of making these chains of inferences and require goals and subgoals to direct their search. The management of goals

.......

and subgoals may occupy considerable time and deplete short-term memory of necessary resources (Larkin et al., 1980).

6

(

Chi, Feltovich and Glaser (1981) evaluated the abilities of subjects to sort a set of physics problems into categories that reflect the common properties of the problems. The novice subjects who had one semester of mechanics tended to group problems according to similarity of surface structure, such as "spring" or "inclined plane" problems. In contrast expert physicists categorized the problems by virtue of their underlying principles or funda~entallaws,

such as Newton's Second Law. It is proposed that these categories

refleet knowledge schemata (Chi, et a!., 1982). In the course of learning, there is a graduaI shift in organization of knowledge, from one centering on the surface features to one where these physical features suggest abstracted principles. A general consensus eoncerning the nature of expertise is now emerging aeross domains. Experts have a greater capacity to recognize significant elements in a problem, elaborate the constraints and formulate an effective problem representation. It is in the quality, coherency and completeness of the initial problem representation that allows an individu al to pursue high-yield strategies (such as forward-chaining) and efficiently arrive at a successful solution. It is generally agreed that an expert's problem-solving ability is a result of years of domain related experience, in which he or she builds up a rich, highly interconnected network of infonnation units, commonly referred to as schemata. This network serves as an index to rapidly guide experts to relevant parts of their knowledge store (Larkin et al., 1980) Schemata, as a construct, differ from chunks in that they contain both declarative and procedural knowledge (Anderson,1985). The declarative knowledge contained in the schema gene,'ates potential problem configurations and specifies the conditions of applicability, which are then tested against the information in the problem (Chi et al., 1982). The procedural knowledge generates potential solution strategies. This section provides a general discussion of selected studies in expert-novice problem solving across domains. The intention is to introduce important theoretical

7

constructs and highlight certain experimental results that characterize expert-novice differences. This discussion serves the purpose of providing a framework in which studies of Medical problem-solving can be viewed.

8

...

------------~-~--~-_ .. _._----_.-

.

Medical Problem-Solving

The nature of diagnostic expertise has been an issue of formal enquiry for more than twenty-five years (e.g., Rimoldi, 1961). Early work on clinical problem-solving focused on probabilistic models of diagnoses (cf. Eistein, Shulman & Sprafka, 1978). These models were generally variants of Bayes Theorem, which is a powerful formula for calculating the degree of change associated with a new item of information. Accurate decision-making is contingent on having access to the prior probability of diagnostic entities, the pertinent patient data, and the probability of their joint occurrence. Ledley and Lusted (1959) presented one of the early influential mathematical models of clinical reasoning. The theory proposed that the diagnosis is a two stage process. In the fIfst stage physicians evaluate patients' complaints by perfonning a logical analysis, eliminating from consideration disease categories that are not related to the

symptom complex presented. Then the most likely diagnosis is determined by (implicitly) calculating the conditional probability that a patient presenting with these symptoms has each of the possible disease complexes under consideration. These authors also provide a set of mathematical equations designed to make the decision-making process more rigorous. Though there has been considerable subsequent research on probabilistic models of diagnoses and medical decision-making, this mooel is discussed in tbis review because it was proposed as a realistic psychological model of medical problem-solving. While statistical models may yield very accurate results (Elstein, 1984), it has become evident that they do not characterize the mental processes of a physician (Johnson, 1982). Tversky and Kahneman (1974) demonstrated that human probability estimates are biased by the availability from memory of similar instances and their representativeness of those particular class of instances. Leaper, Horrocks, Staniland and de Dombal (1972) found that physician's subjective probability judgements are seriously flawed. In addition,

9

_....

they laek the ability to manipulate probabilistie data and derive the necessary judgements. Feinstein (1973a) points out that the necessary quantitative data are seldomly available and that statistical procedures ignore the internal path of c1inical reasoning as a logical process. Feinstein (1973a,b) proposes an elaborate theory of clinical reasoning as a logical process. Diagnostic reasoning is described as a process of passing through a series of "explanatory stations" during which the input data of a patient's manifestations are converted to the output, a diagnosis of a particular disease. The sequence begins with the detennination that the patient has a manifestation for which an explanation is to he sought. The manifestation is then referred to a domain. A clinical do main is a portion of the body that is the structural or functional source of the manifestation. A domain may refer to an organ, region, channel or physiological system of the body. The next step is to further renne the symptomatological description upon which a disorder can be identified. A disorder is defined as a gross abnormality in structure or function. Once a disorder is identified the search continues with the physician seeking conrmnatory evidence and exploring further the exact etiology and underlying pathophysiology of the diseuse process. Feinstein describes the sequential process in great detail delineating a very powerful problem-solving approach.

However the description purports that the process is

fundarnentally logical rather than psychological. Little consideration is devoted to issues of memory limitation, gaps in prior knowledge and the nature of representations. Bashook (1976) suggests that this represents an idealized perspective of how a c1inician should behave rather than a realistic description of clinical problem-solving. In a series of studies, Rimoldi (1955, 1961) compared medicul students at differem levels of training with physicians. The principal goal of these studies was to validate a psychometrie instrument known as "The Test of Diagnostic Skills". In the ex periment,

-

~bjects

were presented with preliminary patient data and were required to request

addition al information. This information was presented to them on eue cards. The nature

10

and the chronological sequence of the questions asked were evaluated. The studies found that students tended to focus on redundant and irrelevant infonnation. The physicians were more discriminating and hypothesis-directed in their lines of questioning. These studies are important because They introduced the contrastive approach to medical problem-solving (Expert-Novice) and for recognizing the importance of context in the clinical interview. That is to say, a question derives meaning in a particular context and, accordingly, assessment procedures need to take this into consideration. Infonnation processing psychology and decision theory had a significant impact on early Theories ofmedical problem solving. Wortman (1972) proposed a theory suggesting that medical diagnosis is a search through a hierarchically organized memory composed of disease categories associated with a parallel hierarchy containing heuristic decision rules for evaluating these categories. This theory influenced developments in medical artificial intellegence as weIl as subsequent studies in medical problem-solving. He aIso developed one of the [rrst clinical computer simulations in medicine based on heuristic strategies extracted from the "think aloud" protocol of an expert neurologist. This simulation demonstrated the utility of computer modeling in developing and refining a theory of medical reasoning.

Simulations have the advantage of explicitly representing the

relationship between clinical data, disease knowledge and the problem solving procedures necessary to connect them (Johnson, 1983). This allows an experimenter to interpret a subject's problem-solving protocol within the context of constructs posited in the simulation mode!. Kleinmuntz (1968) compared neurologists at different levels of expertise in acquiring and reasoning with patient infonnation. The experiment used a variant of the "20 Questions" garne. Each subject was presented with a brief problem statement and was required first to ask questions concerning the presence of symptoms and then state

(

diagnostic hypotheses. The questions were represented as nodes in a decision tree and the

11

branching pathways were evaluated. The study found that experienced neurologists asked fewer questions and correctly diagnosed the cases. These subjects asked about the signs and symptoms that yielded the greatest amount of information. These questions greatly reduced the size of the problem space and generated constraints that narrowed the range of diagnostic possibilities. Less experienced neurologists were less adept at using this strategy and tended to search extensively down incorrect tree branches. Barrows and Bennett (1972) investigated the problem-solving behavior of neurologists and students while interviewing "simulated patients". A simulated patient is an actor who has been trained to portray a patient and is required to report the same case history, display similar affect, and mimic the physical and sensory deficits exhibited by these patients. After the ~nterview subjects were asked to review their respective interviews and report their thoughts. The findings inc1uded the early generation of hypotheses by aIl subjects. Experienced neurologists asked more inquiry-oriented questions directed at testing hypotheses. Expert clinicians tended to keep their hypotheses broad and vague and allowed them to he shaped by patient data. Less experienced subjects tended to he more precise with their hypotheses and this sometimes resulted in premature c1osure. In gene:al, the hypotheses were multifaceted in that they tended to deal with alterations in anatomical structure, pathophysiology and clinical manifestations simultaneously. In the early 1970s two independent research groups, The Medical Inquiry Project at Michigan State University and the McMaster Medical School Project, conducted a series of studies intended to develop a model of clinical competency. The Michigan group's studies are reported in Eistein, Shulman and Sprafka (1978) and the McMaster investigations are described in Barrows, Feightner, Neufeld and Nonnan (1978). These studies were seminal in that they were the frrst to incorpora te experimental methods and theories of

-

cognitive psychology to formally investigate the nature of clinical competency. Their work was also influential because they were the frrst researchers to construct a psychological

12

(

model of clinical reasoning 011 the basis of a large body of empirical data. The principal goal of the se studies was to characterize the general aspects of medical expertise in tenns of cognitive processes used to acquire and manipulate patient data. The emphasis was on process rather than content or related knowledge. These investigations were influenced by then current models of general problem-so!ving ability (summarized in Newell & Simon, 1972). These models were principally based on investigations of "knowledge lean experimental tasks such as cryptarithmetic. Il

Elstein and colleagues (1978) attempted to discriminate between the performance of physicians judged by peers to he criterial and other noncriterial physicians in inteIViewing simulated patients. There were three primary sources of data; concurrent "think aloud" protocols during the interview, retrospective "think aloud" protocols (following the inteIView) and the analysis of the doctor-patient dialogue via cue-hypothesis matrices. The cue-hypothesis matrices cc. .1sisted of cues fr\)m each case weighted by an expert physician against every hypothesis mentioned. The subjects' interpretations were contrasted with the expert weighting scheme. The study evaluated subjects across a number of process variables, such as, total number of hypotheses generated, number of cues acquired and the point of generation of the fust hypothesis. The findings suggested that clinical reasoning is characterized by a highly efficient hypothetico-deductive proeess. Their model identifies four stages in the diagnostic process; cue acquisition, hypothesis generation, eue interpretation and hypothesis evaluat~f)n.

This proeess begins with attention to initial eues which le ad to the rapid

generation of a few select hypotheses, on average five in a single workup. In the process of cue interpretation, each subsequent cue is designated as positive, negative or noneontributory with respect to the hypotheses under consideration. The auiliors proposed that the majority of diagnostic decisions can he accounted for

(

by one of two roIes: Select the hypothesis with the maximum number of positive cues or

13

.............------------------------V

select the hypothesis with the maximum difference of positive minus negative cues. Il is aIso proposed that accuracy in diagnostic decision making depends on the thoroughness of eue acquisition and the accuracy of cue interpretation. However. the se dimensions are uncorrelated. The findings did not discriminate between criterial and noncriterial physicians ( as judged by a peer review). The McMaster studies (Barrows et al., 1978), using a similar experimental approach, evaluated the nature of clinical expertise along a developmental continuum, from novice medical student to expert physician. The results were largely consonant with the Michigan study. However, this study was able ta characterize with greater precision several process variables. For example, hypotheses are usually activated within the first

thirty seconds of the clinical encounter and student and physician alike acquire, on average,

2/3 of the available data. No differences between student and clinician were found except for the quality of the hypoÛ1eses considered and the accuracy of diagnosis. From these two sets of studies emerged a psychological model of clinical reasoning. The model proposes that hypotheses are generated very early in the encounter and serve as "organizing rubrics" in memory (Elstein et al., 1978). These hypotheses are generated as probable tenninal points in which the problem-solver proceeds in the inquiry to test the appropriateness of the various routes towards these end states. Each piece of data is weighted sequentially against ea~h hypothesis. Hypotheses can he organized either in terms of multiple problem spaces, when the problem may have several possible origins, or hierarchically, whtn the problem demands fmewgrained discriminations between variants of related diseases (13arrows et al., 1978). The nature of the problemwsolving processes, including strategies used ta acquire and manipulate data, are the same for novice medical students as weIl as experienced clinicians. The basic assumptions of this model have increasingly come into question (Grcen & Patel, 1985; Berner, 1984).

The ass.;rtion that expert diagnostic reasoning is

14

(,

characterized by a hypothetico-deductive method is questionable in view of the fact that the research in other domains has demonsttated this to be a weak and in efficient method of problem solving, more characteristic of novice rather than expert performance (e.g., Simon & Simon, 1978). One of the fundamental flaws of these studies is the failure to carefully

delineate the difference between the processes of cue interpretation and the generation of a diagnostic hypothesis. A hypothesis is merely a statement that assigns a certain truth value to a proposition. In medical tenns this can vary from the most general description to a specific diagnostic conclusion. Gale and Marsden (1983) attempted to expand and revise the above mentioned model of medical problem-solving. They used a similar experimental procedure as in the previously discussed studies (Elstein et al.,1978, Barrows et al., 1978) with the exception that they relied exclusively on stimulated retrospective protocols and used only real patients. Their methods of analysis differed in that they augmented the quantitative analysis performed in the studies that were previously discussed, with a detailed qualitative description. In addition, they attempted to separate a prediagnostic interpretation from a diagnostic hypothesis. A prediagnostic interpretation is defmed as;

"Any term, phrase or statement which indicates that the subject has made sorne active interpretation of the clinical information available where the result of this activity is not sufficiently specific to constitute a diagnostic hypothesis" (p55). A diagnostic hypotheses is an interpretation where a pathophysiological process is indicated with a degree of specificity as to constitute an acceptable diagnosis. A prediagnostic interpretation cannot be considered as even a broad or general diagnostic hypothesis. It reflects a clinician's interim assignment of a symptom or cluster of symptoms to the location of a given problem (e.g., a respiratory problem). The results across process variables (e.g., number of diagnoses considered) were

15

1

.~_~(

comparable to the Michigan and McMaster studies in that no differences were found between experts and novices. However the qualitative descriptions and the subsequent fonnulations paint a very different picture of Medical problem-solving. They describe the clinical interviewas a process of continuously restructuring arrays of information and extrapolating them to the appropriate Medical context. Clinical information can be structured in multiple ways in memory. The same array of information can be cognitively manipulated 50 that the eues have different relationships and differing degrees of dominance within a particular context. As infonnation is elicited, it is arranged in an extended clinical context and previously acquired information can take on new meaning. Differences in performance and expertise were attributed to the ability to elaborate constraints, access an appropriate context and manipulate the context to accommodate new infonnation. These differences can he accounted for by differlng degrees of experience in a clinical setting where one has the opportunity to exercise his or her Medical knowledge, practice and develop clinical skills. Gale & Marsden (1983) propose a model of Medical problem-solving that is more dynamic, consonant with models of expertise in other domains and possesses greater psychologie al plausibility than the hypothetieo-deduetive model. However, in many respects the model is less specifie and in its present fonn can not be subjected to experimental validation. In recent years, there has been a shift in problem-solving research from investigating general problem-solving abilities in tasks that are not dependent on vast stores of knowledge Ce.g., eryptarithmetic) towards real-life tasks that involve the use of domain specifie knowledge (e.g., physics and Medicine). Reeent investigations in Medical problem-solving have similarly shifted from an emphasis on global aspects of elinieal reasoning to a foeus on the nature and content of Medical knowledge used to solve a

-...

problem (Feltovich & Patel, 1984). At the University of Minnesota, Johnson and colleagues conducted a series of

16

-------------------

-------

studies designed to evaluate differences in the knowledge base that impact on clinical reasoning (Johnson, Duran, Hassebrock. Moller, Prietula, Feltovich & Swanson 1981; Feltovich, Johnson, Moller & Swanson, 1984). The theoretical framework of these investigations was predicated on recent cognitive models of memory used to describe problem-solving. In general, human memory is described as a hierarchical structure containing a large number ofrecurrent patterns or prototypicaI combinations of information that are commonly referred to as schemata or frames. These units of knowledge are matched against external data so that a particular configuration of information can he recognized as an instance of a stored pattern (Johnson et a!., 1981). Medical knowledge includes a store of disease models each of which specifies for a particular disease, the set of clinical manifestations a patient with the disease should present with and the corresponding pathophysiology. Johnson and colleagues (1981) proposed that expert physicians have a knowledge base that includes a hierarchy of diseases that are weIl organized and extensively differentiated into a number of disease variants which present themselves differently due to contrasts in severity, patient related factors (e.g., age), and the underlying pathophysiology. They aIso proposed that novices have relatively sparse disease models that have an internai structure that is fairly imprecise. This is a result of their limited exposure to patients. They investigated expert-novice differences in pediatric cardiology by presenting each subject with a series of patIent eues in a fixed order and asking them to think aloud. The cues consisted of patient history eues, physicaI examination findings, laboratory and investigative procedure findings (e.g., EKG). An expert computer simulation, Diagnoser, was aIso presented with the sa me sequence of infonnation. This simulation was based on a theory of diagnostic reasoning and was designed to simulate expert behavior in the domain of congenital heart disease. Each protocol was scored for the

(

hypotheses generated at each data point.

17

The results indicated that subjects in each group and the computer simulation model generated similar types of hypotheses and in similar quantities at each data point. However, the expert subjects and Diagnoser generated significantly more pathophysiological hypotheses. Less expert physicians exhibited a greater data-driven dependence, continuously shifting hypotheses depending on the most recent strong disease eue in the data. The subjects' success at diagnosing the case was dependent on their ability to genera.te the appropriate hypotheses and integrate the patient eues in relation to the major competing hypotheses. Subjects' errors were frequently a result of a failure to interpret a small set of data eues that either served to generate the key hypotheses or provided confrrmatory evidence for these hypotheses. The specifie heuristics either were not in memory or were not activated by the data. Less expert subjects were more prone to a failure to correctly evaluate data with respect to hypotbeses that they had previously generated. This is a result of impreeise disease models in which expeetations are more closely associated with the incorreet alternative hypotheses. The simulation provided valu able infonnation conceming the sources of error and the predictions that followed. Peltovich attempted to further delineate the nature of knowledge-based differences between novice and ~xpert (Feltovich, 1981; Feltovich, Johnson, Moller & Swanson, 1984). He elaborated on the theoretical framework proposed by his colleagues at the University of Minnesota and suggested that disease frames can be organized at three levels (Feltovieh, 1981). At the most generallevel, disease frames were organized within frames representing general disease categories (e.g., autoimmune disorders). At the interrnediate level are the diseases most prototypical or characteristic of a particular category. At the most fme-grained level are a set of frames that specify subtle variations of diseases that can

-

be differentiated across a series of dimensions, such as severity, temporal