Source Inversion and Matching in the UMLS Metathesaurus*

Source Inversion and Matching in the UMLS Metathesaurus* N.E. Olson

D.D. Sherertz M.S. Tuttle

M.S. Erlbaum, MD

LTI UMLS Group Lexical Technology Incorporated: 1000 Atlantic Avenue Suite 106; Alameda, CA 94501-1366 svytems, namely sNozED[61, iCD-9-cM[7J, and CPr-4[8J. Abstract One of three knowledge sources being The general structure and content of the developed as part of the mUm's Ems Project is a Metathesaurus are described separately[9j. Here the bimedical thesaurus, caled the Metathesaurus. The focus will be on the methods used to "homogenize" Metathesaurus contains inter-term relaionshgps across the disparate information structures in the six sources, and on the matching engine used to exploit six bomedical nomenclaturs and classfiation systems, deivablefiom lexical mapping techniques. 7he this structure to build MErA-i. first public version of the Metathesaurus, calledmErA-1, 1.1. Previous Work was built In two stages --fuist, source inversion and Earlier research by the authors showed that second, source matching. During the Sprin of 1989, lexical techniques which map terms from structured "official" versions for the six sources were obtained text to Mew provided a powerful, general method for In machine-readableform. Source spefic techniques suggesting links between information resources[101. were derived empiicaly to analyze the information Subsequent work demonstrated that these techstructure and content of each source. The results of niques could be generlized to map concepts from each analysis wre used to guide the "Inversio of the other sources, used to build META-i, to MeS[1 1]. the corresponding source, resulting in a homogeneous 'Lexical techniques' employ word-based mapping representation for all sowrces. The core concepts of rules for detecting such things as inversions, permuMETA-1 come prmarilyfromMEDuN index terns (MesHO. tations, common stems, and coordinations in mediPrevious work on lexical mapping methodology cal terminology. A lexically-based approach was developed algorithmic methods to link cncepts in d(f central to the implementation of the first ferent sources. These methods were refined Itera- comprehensive biomedical thesaurus of concepts tively, and used to Iplement a META-1 "!matching from these six resources[121. Since this thesaurus engine". The Initial version of META-i was con- has concepts from, and relationships across, six structed with this engine, by matching the MFrA-1 core biomedical information resources, it is termed a concepts to the other sources. This version of mETA-i Metathesaurus. was edited and enhanced by domain experts, after 1.2. ObJective the Inclusion of supplemtay informai, to proExperience with the design, implementation duce thefirst publicly released version ofMErA-1. and evaluation of a prototype Metathesaurus, called Introduction META-O (described in [111), demonstrated the imporThe National Library of Medicine (Num) has tance of careful analysis of the information structure underway a multi-year development effort called the and content of each source used to build the Unified Medical language §ystem (uMs). A goal of Metathesaurus. Each source adds unique elements in the NLM'Bs uMS Project is to provide automated assis- its structure, and the usefulness of each source tance in establishing conceptual links from a useres depended on the successful exploitation of those statement of a question or problem to a query idiosyncracies. It suggested the use of a formal answerable by a machine-readable biomedical infor- database approach to manage the complexity mation resource. The current status of the project is inherent in the source inversions and matching.. described separately[l]. One aspect of making con- Thus, an objective in building the initial version of ceptual links is the transparent mapping among dif- MErA-i was to design a matching engine that could ferent biomedical terminologies from distinct infor- work with a "database of facts", where facts were the mation sources. These mappings will reside in a explicit or derived structural and relational informathesaurus of biomedicine, named the urLs tion extracted from each source. It was recognized Metathesaurus, the first version of which is called at the outset that MTA-i would not have all possible META-1. META-1 includes as its primary source all semantic inter-term relationships, but that the ones concepts from MeSH (IM's Medical Subject it did have would be accurate and meaningful. Headings[21, used to index citations in MEDuNE[31). Because of this objective, the NLm strategy for META-i To improve coverage of clinical terminology, concepts development included review of every concept and were added from DsM-wii-R[4J and the 400 mostconcept relationships by domain experts. The process of implementing this editing is described frequent "terms" (findings, diseases, procedures, etc.) from three different cosrAR sites[51. Ihis set of separately131. The resulting editing "facts" were then incorporated into the database of facts derived concepts formed the "core" concepts for rTA-i. These core concepts were linked, through lexical from source inversion and matching, and used to mapping rules to each other, and to terms in three compute the final version of META-i. The objective other biomedical nomenclatures and classification was to accomplish all this within an 18-month This work has been supported by mm Contracts NO1-LM-8-3512 and NO1-LM-9-3517. 141

0195-4210/90/0000/0141$01.00 © 1990 SCAMC, Inc.

period, using a limited amount of human and computer resources.

abbreviations" in cpr-4). In addition, every source except cosrAR provided an explicit hierarchical context for each concept. The names of the uppermost levels of these hierarchies were not concept names, as they did not have codes associated with them. However, these "Hierarchical Tiltle" names were included as part of each source, to allow the full representation of the context of every concept name. The goal was to most effectively leverage the source context and relationships to enhance the source concepts to mTA-i concepts. Each of the six sources used in building s.r-i presented unique challenges, and short descriptions of the techniques developed for each source are presented in the following sections.

META-1 Methods A central problem that the Metathesaurus should help solve is the "naming' problem. The naming problem can be simply stated: different sources to be included in META-i use different names for the same meaning. For example, the MeS name for a common psychiatric disorder is "Bipolar Disorder". In SNOmD there is a disorder named "Manic-depressive psychosis". In ICD-9-CM there are severl different codes, one with the same name as SNOmD, and others with the phrase "Bipolar affective disorder", followed by various modifiers. One of these Is named "Bipolar affective disorder, unspecffied". In DSM-III-R, the names used are "Bipolar disorder NOS" and "Bipolar disorders" along with three fonns of "Bipolar disorder", followed by a modifier. All the names shown here are variants of the same meaning. The challenge of building a useful Metathesurus is to devise algorithms which utilize inter-source relationships to include all of these names under the same mrA-i concept. While this process Is not expected to detect all instances of using different terminology to name the same meaning, in META-1 all of the above names are appropriately linked under a single concept.

2.1.1. 1990 mom The initial version of MTA-1 used the 1989 ver-

sion of meSh. In pAng the schedule for building mbETA-1, the NLm recognized that by the time META-1 was released, the 1990 version MeSH would be available. So, the methodology had to allow incremental changes and additions from the 1990 version. MeSH was the least anomalous source, probably because Its use in mEuNE dictates its continual maintenance. Nonetheless, a few inconsistencies had to be corrected for successful source inversion, with the assistance of the MeS section at the m. Ihe only cleansing of mew concepts was to remove the phrase (Non MeS) from the matching forn of Mew terms containingit Some of the meS definitions Included lists of altemate terms; these were exracted and suggested as additional "definitional" synonyms. mesH had the richest set of associated terms, including abbreviations, and entry terms (abeled previously by LTI and mm staff as lexical variants, synonyms, narower, broader, or related terms). Ihe final form ofMeSH (that is, the 1990 version of McsH) used in building META-i contained 50,307 unique strings, broken down as 16,564 main concepts, 31,865 entry terms, 1,863 definitional synonyms, and 15 hierarchical titles. Compared to the initial 1989 version of MeSH, this represented an increase of 2,661 unique strings, broken down as 416 new main concepts, 2,250 new entry terms, and 5 fewer definitional synonyms.

2.1. Source Inversion of NTA-1 Concepts A Metathesaurus concept can be defined as a term in the context provided by a source; that is, a concept is more than a mere collection of words in a phrase. The Metathesaurus concepts provide a focus for a constellation of synonyms, related terms, and associated terminology from different sources. In this sense, mappings from the six wrrA--i sources will be a many-to-one mapping, in that numerous terms from these sources may be clustered under one concept, as appropriate, based on the mapping rules. The process of "inversion" of source concepts developed algorithms and filters to convert the diverse contextual structure and lexdcal styles employed in each machine-readable source into a particular format that can be loaded into a database. TIhis fornat makes each term in a source the primary object, and groups intra-source relationships for that term in a uniform way to maximize intersource commonality. Part of converting source concepts to a standard fonnat involved "cleansing-. In this process concepts in the source had transformations systematically applied (removing certain punctuation, and selected "noise" words). A concept name (called a "Main Heading" or `"Subheading" in MeSH, and a "Preferred Term" in the other five sources) in a source was distinguished by its having a unique code or identifier in that source. For DSMIII-R this code was the ICD-9-CM code associated with the concept, while for cosrAR it was Just the ordinal number in the alphabetic list of concepts. Every source contained terns associated with the main concept. In some sources the relationship between the associated term and main concept was explicit (synonyms in MesH and sNomED), while in other sources It was inferred from empirical experiments (index terms in ICD-9-CM, and "extended

2.1.2. D6-m-R For DM-iii-R, the concept names were entered manually from the list of 313 in Appendix H of [41. Dr. Seth Powsner of the Yale umIS group then assisted in deciding on two appropriate cleansing rules. One rule was to split DSM-III-R term names that included alternate names in parentheses. For example, the DSM-III-R term "Dream anxiety disorder (Nightmare disorder)" was split into two terms, "Dream anxiety disorder" and "Nightmare disorder". In the 10 cases where this was done, the orignal DM-lI-R form of the term was left as a concept name, so 20 alternate concept names were added. The other rule was to not expand the concept name with Its final digit modifier, but to just leave it without modifiers (for example, in Appendix H of [41, there are three modified forms of the concept "Schizophrenia, residual"; but in METm-i there is Just this one concept). There were 13 instances of these 142

modified concept names, accounting for 60 additional names in Appendix H that are not explicitly in META-1. Ihe handling of modifiers in all the sources is a research topic for rA-2. The final form of DSM-Ii-R used in building META-1 contained 367 unique strings, broken down as 267 (313-60+20) preferred terms, and 100 hierarchical titles. DSM-IJI-R had no explicit associated terms for its concepts, and none were created.

avae of fly, E-4840). The ifiters and procedures developed recognized that the prefenred term for the

code E-4840 is Diptera, NOS, which has as synonyms the following: Fly, NOS; Fly larvae; Maggot. Since "Larvae of fly' did not match any of the names under E-4840, it was retained as an alternate name for Diptera, NOS. The cleansing procedures removed all these parenthesized expressions. The tenn NOS was cleansed, or removed, for the purposes of matching, but its presence was retained as a fag. The formal syntax for representing the sNomED terms allowed any term to be reconstructed as it appears in the original source, but the string used for matching had all of the extraneous information removed. It was remarkable how regular the syntax of sNomED was, and its richness as a source of alternative matching possibilities was impressive. sNomm is a multi-axial system of nomenclature, so individual preferred terms are more atomic than in the other sources. The flnal form of SNOMED used in building META-1 contained 44,307 unique strings, broken down as 30,410 preferred terms, 7,147 synonyms, 4,616 related terms, 70 alternate names for preferred terms, and 2,064 hierarchical titles. 2.1.5. i)-cm In addition to obtaining the Ic-9-cM code tapes (both the long and short form), a typeset tape of the index for the ict-a-c diseases was obtained. The Information on these tapes reflected revisions to IcD> 9-CM through October, 1988. Although it was missing the tabular information for neoplasms and AIDS, the index tape was a rich source of more atomic terms associated with the ICD-9-CM codes. ICD-9-CM is a classification system, and it tends to lump diseases into general classes; for eaample, the MeSH main heading "Sturge-Weber Syndrome" is classified in ICD-9-CM with the code 759.6, under "Other hamartoses, not elsewhere classified" along with "Peutz-Jeghers Syndrome" and "von Hlppel-Lindau Syndrome". In the ICD--CM index there are 135 entries that refer to 759.6; one of these is under the general index term "Syndrome" below the entry "Krabbe's" and is "cutaneocerebral angloma". After a number of experiments with the index, a procedure was developed to recursively generate over 470,000 index terms from this hierarchical index For example, from the above entry the phrase "Syndrome; Krabbe's; cutaneocerebral angloma" was generated; semicolons were used to distinguish each level of the index in a phrase, so it could be matched as generated, or uninverted. After matching experiments with this large set of terms, it was decided to select only the 1, 2, and 3-word index terms, as these had the best chance of matching in other sources. The hierarchical context for IcD-a-CM was manually entered, as it was not on any of the tapes. The final form of IC-a-cm used in building MErA-1 contained 164,405 unique strings, broken down as 18,307 preferred terms (the long form name), 15,847 abbreviations (the short fonn name; some of the abbreviations were duplicates of the preferred term), 130,071 index terms, and 180 hierarchical titles.

2.1.S. cosmA

Ihe cosTr^ concepts were selected from the 400 most-frequently used cosrTR terms at each of 3 different cosTAR sites (2 in Boston, and 1 in Nebraska) over the last three years. After some experiments in matching these 776 terms (the remaining 424 were duplicated at more than one of the sites) to meS, one of the authors (MSE) added. 119 synonymous terms and 17 related term to improve the matching to the other sources (for example, the cosrmi term "REFRACTION ERROR" had "REFRACIIVE ERROR" added as a synonym, and "ACCOMMODATION ERROR" added as a related term). The purpose of including these cosrAR terms was to insure that current clinical terminology would be adequately represented in mI.am-i. It was hoped that few of the cosrAR terms would not be linked to other sources, and that some of them would be linked as synonyms within cosTAi The final forn of cosrAR used in building MErA1 contained 912 unique strings, broken down as 776 preferred terms, 119 synonyms, and 17 related terms. 2.1.4. Smom Both the code tapes (publicly available) and the typeset tapes (the information used to print the SNOM books) for the November 1982 revision of the 2nd edition of SNOMED were obtained. After considerable experimentation with the coding tapes, it was decided to use the typeset tapes as a more accurate source of concepts and hierarchical context for SNOMED. The coding tapes had some of the 1982 revisions applied incompletely, whereas these revisions were accurately represented in the typeset tapes. Procedures were developed which parsed all but a handful of sNomED concepts into a consistent form. Many SNOmD concepts include coordinated terms in parentheses; these were cleansed for matching, but retained. Algorithms were developed to associate the SNOMED hierarchical terms with the range of codes under them. SNOMED contains explicit "synonyms" and "related" terms for many of its preferred terms. In addition, some preferred terms in the Disease, Morphology, and Procedure axes have "horizontal pointers" which are either coordinated etiologic modifications and/or topographic sites of the preferred term. For example, the preferred tenn for the code D-0726, "Myiasis, NOS", has the following set of related terms: Myiasis, dermal (Skin region, T-02...); Creeping mylasis Cr-02...); Congo floor maggot disease tI-02...); Mylasis, ophthalmic (Eye, NOS, T-XXOOO); Ophthalmomyiasis (r-XX000); Myiasis, genitourinary (Genitourinary tract, T70000); Mylasis, intestinal (ntestinal tract, T05100); Mylasis, rectal (Rectum, NOS, T-68000) 143

and cosrAR preferred term "Obsessive Compulsive Disorder", and the IcD-e-cM preferred term "Obsessive-Compulsive Disorders" were all matched as lexical variants of the MeSH main heading "Obsessive-Compulsive Disorder". The rules used to determine lexical variants appear to give few false positive matches. 2.2.2. Stemmed Matching The second, more aggessive matching step, was to go through the same process as just described for lexical variants, but to singularize the last word after making it plurl, and then to use a stemmer described in [111 to stem the endings of all words in the string. For example, the MeSH main heading "Accident Proneness" and the SNOMED tenn "accident-prone" (a synonym for the SNOMED preferred term Accident Prone Behavior) were matched through stemming (proneness and prone both were stemmed to pron). Even more agessive matching rules were tested in MErT-o but based upon the results in META-o, these were abandoned. 2.3. Building the MKTA-L Matching Engine All terms in every source were put into a single database structure, and then all the relationships between them were coded in a single table. An algorithm was then devised to use these tables to build classes of candidate synonymous terms from the tables of terms and relationships. Ihose synonym classes that included terms from one of the core sources were then formed into MEuTA-i concepts by distributing terms in these classes into their appropriate slots in the MTrA-1 template. The related terms were then brought in and fonned into related concepts of the MrA-1 concepts. Additional information for the MeSH concepts was added (definitions, annotations, etc.) and contexts for all hierarchical sources were added.

2.1.6. CPT4 For CPT-4, tapes with both the short and long forms of the 1989 CPT-4 codes were obtained. Again, to get the hierarchical titles for cPr-4, the typeset tape for the prlnted book was obtained. A combination of a novel flltering algorithm, with a minor amount of manual revision, produced an accurate set of hierarchical titles. Cvr-4 is intended to be a list of descriptive terms and codes for procedures, and as such, tends to have very long, heavily modified names. For example, the long form name for the cPT-4 code 008622 is "Anesthesia for extraperitoneal procedures in lower abdomen, including urinary tract renal procedures, including upper 1/3 of ureter, or donor nephrectomy" while its short form name is "ANESTH, KIDNEY, URETER SURG". he short form names never exceed 46 characters, and are intended to be used in billing systems as short descriptions for a code. After some experiments on these names, it was recognized that certain abbreviations were consistently used to shorten some words. So a program was written to "expand" the abbreviated portions, to Improve the opportunities for matching. With the above example, the expanded abbreviation was "anesthesia for kidney, ureter surgery". In many cases this expansion did not change the short form, but in 790 cases generated an additional abbreviation for the preferred term. The flnal form of CPr-4 used in building mTA-i contained 16,014 unique stings, broken down as 7,299 preferred terms (the long form name), 7,154 abbreviations (the short form name; the rest were duplicates of the preferred term), 790 expanded abbreviations, and 771 hierarchical titles. 2.2. rrA-L Matching Techniques In META-1, matching was done on the cleansed form of all the names of the core concepts. A match was considered exact if, except for punctuation and letter case, the words within the name were the same, and in the same word order. Exact matches among the core concepts resulted in multiple sources being listed for that concept. Modifiers within phrases prevented exact matches, and adjectives such as "acute" or "secondary" were not detected or handled in any special way. Two matching steps were taken beyond exact matches. In both of these, described in more detail in the next sections, word order and letter case were ignored. in these extended matching steps, strings from all sources were converted into a "canonical form" (a basic standard format) before matching. 2.2.1. Lcal Variants The first extended matching step was lxidcal variant detection. To be called a lexical variant, two strings had to match after uninverting them around any commas (or semicolons, which were used in the icD-a-cm index terms), and then plurlizng the last word in the string, except that the following list of "'oise" words were ignored in either string: of, and, with, for, NOS, to, ki, by, on, the. In addition, a canonical form was generated where dashes were removed, and, if different, a forn where dashes were converted to spaces. For example, the iCD-9-CM index term "Disorder; obsessive-compulsive", the SNOMED

Results Applying all the steps described above to the sources for mE'rA-i resulted in the following: 16819 core concepts (MeSH+DSM-III-R+COSTAR) + 15154 related concepts - 31973 synonym classes (-1.4 synonyms/class) + 12058 synonyms -

44031 LV classes (-1.4 LVs/class)

+ 18981 lexical variants (LVs)

63012 unique strings 9482 multiple source occurrences (-1.2) - 72494 source occurrence unique strings Distribution of the 72494 Meta-1 strings: MSH 50307 SNM 10341 23% ICD 10037 6% codes 16327 5711 19% 3158 17% MC 16564 PT 5690 19% PT 2775 15% 15 HT HT 513 25% HT 22 12% ET 31865 SY 2614 37% AB 982 6% SD 1863 RT 1491 32% IT 6258 5% RX 33 47% DSM 367 COS 912 CPT 530 3% codes 267 776 212 3% PT 267 PT 776 PT 166 2% HT 100 SY 119 HT 170 22% RT 17 AB 191 3% -

+

144

Computer Center gave timely and helpPathologists' ful advice on the SNOMED tapes. Mr. Robert Seeman, Margie Zemott, and Audrey Wilson of the Commission on Professional and Hospital Activities assisted in obtaining and explaining the source tapes for IC-

The numbers in the table above for SNOMED

(SNM), ICD-9-CM (ICD), and CPT-4 (CPT) represent the number of that type in that source that was brought into META-1 through some match, as described above. For the core concept sources McSH (MSH), DSM-III-R (DSM), and cosrAR (COS), the numbers represent the

9-CM. Kathleen Nimr of LTI entered additional hierarchical information for ICD-9-CM not available on the tapes. Celeste Kirschner and Donna Whinnery of the Merrill Corporation (typesetters for the AMA) helped obtain the appropriate tapes for cpr-4.

entire set of source strings, since by design all of these sources are included in META-I. The abbreviations in the table above are as follows: MC (Main Concept), PT (Preferred Term), HT (Hierarchical Title), SY (Synonymous Term), ET (MeSH Entry Term, may be a synonym, broader, narrower, or related), AB (Abbreviation), RT (Related Term), IT (ICD-9-CM Index Term), and RX (SNOMED Related Xtra Terms,

References [1] Lindberg, DAB. and B. Humphreys. "The ULS Knowledge Sources: Tools for Building Better User Interfaces", to appear in SCAMC, 1990. [2] Sewell, W. Medical Subject Headings in MEDLARS, Bull. Med. LUbr. Assoc. 52(1):164-170, 1964. [31 Adams, S., S. Taine. Searching the Medical Literture, JAMA 188(3):251-254, 1964. [41 Diagnostic and Statistical Manual of Mental Disorders, Third (III) - Revised, American Psychiatric Association, Washington, DC, 1987. [51 Barett, G.O. "The Application of Computer Based Medical Record Systems in Ambulatory Care", NEJM, 310(25): 1643-1650, 1984. [6] C6te, RA., ed. SNOMED, Systematized

alternate names for preferred terms). Conclusions The building of ETA-1 demonstrates the ability

to use lexical matching algorithms, and a database of facts to implement a thesaurus of biomedicine. The careful analysis of the structure and content of each information source that makes up META-I allowed tuning the techniques to best leverage the information and structure content of that source. Designing a matching engine proved to be invaluable in understanding and managing the complexity inherent in the META-I term relationships. The database engine allowed incorporation of both new facts from updated versions of MeSH, as well as facts from review of the computed links by domain experts. The database engine is efficient enough to allow incremental updates to the Metathesaurus as new sources are added, and as further use and review changes inter-term relationships. The engine's has allowed overnight rebuilding of META-I efficiency with updated facts. This efficiency will insure that the UMLS Metathesaurus will be an extensible and modifiable resource that can accommodate changes and enhancements as new and better information sources become available. The approach of using automated methods to suggest links among Metathesaurus concepts, followed by review and enhancement of those links by domain experts provides the most effective leverage of both computer and human resources [131. The synergy of computing META-1, and then incorporating and retaining facts provided by domain experts, provides a cost-effective solution to building and maintaining a large, biomedical knowledge source. As feedback and evaluation from users of META-1 is received, the approach described in this paper allows suggestions and improvements to be tested and added to future versions of the UMLS Metathesaurus.

NOmenclature

of AMEDcine, College of American Pathologists, Skokie, IL, 1986. [71 ICD-9-C, The International Classfication of Diseases, 9th Revision, Clinical Modfication Vols. 13, 2nd edition, DHHS Pub. No. (PHS) 80-1260, Washington, D.C., 1980. [81 crr-4, Physicians' Current Procedural Technology, 4th edition, American Medical Association, Chicago, IL, 1988. [9] Tuttle, et. al. "Using META-1 -- the 1st Version of the UMLS

Metathesaurus", to appear in SCAMC, 1990. Sherertz, D.D., M.S. Tuttle, M.S. Blois, M.S. Erl[10] baum. "Intervocabulary Mapping Within the UMLS: The Role of Lexical Matching", in Proceedings of the 12th Annual Symposium on Computer Applications in Medical Care, RA Greenes, ed., Washington, D.C.: IEEE Computer Society Press, 201-206, 1988. [111 Sherertz, D.D., M.S. Tuttle, N.E. Olson, M.S. Erlbaum, S.J. Nelson. "Lexical Mapping in the UMS Metathesaurus", in Proceedings of the 13th Annual Symposium on Computer Applications in Medical Care, L.C. Kingsland, III, ed., Washington, D.C.: IEEE Computer Society Press, 494-499, 1989. [12] Tuttle, M.S., D.D Sherertz, M.S. Erlbaum, N.E. Olson, S.J. Nelson. "Implementing META-: The First

Acknowledgements

Version of the UMLS Metathesaurus", in Proceedings of the 13th Annual Symposium on Computer Applications in Medical Care, L.C. Kingsland, III, ed., Washington, D.C.: IEEE Computer Society Press, 483487, 1989. [13] Sperzel, W.D., M.S. Erlbaum, L.F. Fuller, D.D.

The authors gratefully acknowledge the cooperation and assistance of the organizations responsible for the sources used in building TrA-1. In particular, Peri Schuyler and Doug Johnston of the NLM provided invaluable help in understanding the intricacies of MeSH. Dr. Octo Barnett of the UMLS group at Massachusetts General Hospital compiled and provided the terms included from the 3 COSTAR sites. Dr. Seth Powsner of the MLS group at Yale helped decide on source inversion rules for the DSMIIm-R codes. John Clouse of the College of American

Sherertz, N.E. Olson, P.L. Schuyler, W.T. Hole, AG. Savage, PA. Passarelli, M.S. Tuttle. "Editing the

UMLS Metathesaums: Review and Enhancement of a Computed Knowledge Source", to appear in SCAMC, 1990. 145