Vol. 38, No. 6
399
Residency Education
Evaluating Family Medicine Residency COPC Programs: Meeting the Challenge Alison Dobbie, MD; Patricia Kelly, PhD, MPH, RN, APRN; Eldonna Sylvia, ARNP; Joshua Freeman, MD Background and Objectives: We conducted a review of the evaluation literature and outcomes from community-oriented primary care (COPC) programs in US family medicine residencies since 1969. Methods: We used a Medline and ERIC search for “community-oriented primary care” in English from 1969–2005. Results: Twenty-two articles were found that concerned US family medicine residency COPC. Six surveys over 25 years reported stable rates of COPC teaching (approximately 40%). Eight descriptive and eight evaluative papers described 14 residency COPC programs. Teaching and learning methods included block and longitudinal rotations and COPC projects. Evaluation methodologies included one quasi-experimental control group study, pretests and posttests of knowledge and attitudes, focus groups, and semi-structured interviews. Reported outcomes included changes in residents’ knowledge, attitudes, and behaviors; effect on graduates’ career choice and future practice; and impact on patient care and community health. Conclusions: Few studies have evaluated residency COPC programs. Evaluation has been less than rigorous, with variable results, but at least one study indicates positive outcomes at each evaluation level. More residency programs must evaluate and disseminate outcomes from their COPC projects to determine the value of COPC to residents, colleagues, community partners, and funding agencies. (Fam Med 2006;38(6):399-407.)
Community medicine has been a Residency Review Committee (RRC) requirement in family medicine since 1969,1 and in 2004 the final report of the Future of Family Medicine project reemphasized community medicine’s educational importance. 2 Communityoriented primary care (COPC) provides a structured model with which residents can learn about community health problems and understand their relevance to clinical practice. Thus, many residency programs use COPC programs to meet the RRC requirements and also to address the Accreditation Council for Graduate Medical Education systems-based practice competency.3 Evaluating and disseminating the outcomes from residency COPC programs is important because it allows us to demonstrate to our learners, colleagues,
community partners, and funding sources that such projects represent effective use of time and resources. However, when we sought guidance from the literature to evaluate our COPC curriculum, we found few articles demonstrating the benefit of and outcomes from residency COPC programs and only one validated evaluation tool. In this article, we address this scarcity of evaluation literature in three ways. First, we share the findings and conclusions from our literature review of family medicine COPC programs since 1969. Second, we identify barriers to evaluating COPC programs in medical education. Third, we offer recommendations to strengthen the evaluation of residency COPC programs. Use of these recommendations may assist medical educators in successfully implementing and evaluating COPC programs in their own residency programs.
From the Department of Family and Community Medicine, University of Texas, Southwestern (Dr Dobbie); School of Nursing, University of Missouri-Kansas City (Dr Kelly); and Schools of Medicine and Nursing (Ms Sylvia) and Department of Family Medicine (Dr Freeman), University of Kansas.
Methods—Literature Review Search Strategy and Selection Criteria We searched Medline and ERIC through Ovid using the search term “community-oriented primary care,”
June 2006
Study Categorization We identified 16 articles that described or evaluated COPC programs in US family medicine residencies since 1969. We divided the articles into descriptive and evaluative and categorized them by research methodology and by outcomes using Kirkpatrick’s four levels of evaluation, adapted for medical education.4,5 For a paper to be classified as evaluative, the authors had to report some quantitative or qualitative data that corresponded to one of Kirkpatrick’s four levels of evaluation. Two investigators categorized independently and then met with a third investigator to decide on disagreements. Our search and categorization process is summarized in Table 1. Results Prevalence of COPC Activity in Family Medicine Residencies Since 1969 Four surveys, spanning 25 years, report remarkably stable rates of COPC training in family medicine residencies. In a 1979 survey, 38% of 122 residency directors (who were known to offer community medicine curricula) reported teaching COPC, although this study was limited by selection bias and a low response rate (also 38%).6 In 1994, 37% of 120 residency program directors taught COPC (response rate 81%).7 In 2002, two large surveys of 470 and 244 family medicine residency directors, respectively, with response rates greater than 70%, reported COPC residency training at 44%6 and 38.3%.8,9
residents performing group and/or individual COPC projects,10-21 ranging from 1 month to 2 years in length. Only three papers, including two describing the same program, reported COPC projects involving interdisciplinary collaboration,11,12,18 although one author planned such collaboration,14 and another provided residents with high-quality support from mentors in other disciplines.16 Four authors did not provide details about their COPC teaching and learning activities.22-25 Reported COPC Program Evaluation Methodologies Of the 16 papers describing COPC teaching programs in family medicine residencies, we classified eight as evaluative and eight as descriptive. The eight evaluative papers described six programs. No authors conducted randomized controlled trials. In a well-designed study to develop an attitudes scale for COPC, Oandasan (1999) conducted a quasi-experimental control group study with pretests and posttests of attitudes and focus group analysis.22 In 1998, Donsky10 used a pretest and posttest of residents’ COPC knowledge and attitudes and also conducted a focus group and semi-structured interviews with five residents. In two studies in 1996 and 1997, Thompson11,12 surveyed residents on their attitudes and self-reported behavior change after a COPC curriculum, conducted chart reviews to confirm behavior change, and carried out semi-structured interviews with community sites. Six authors described their evaluation methods but did not report any data,13-15,17,21,23 and two authors did not describe any evaluation methodology.24,25 Evaluation methods described without accompanying data in-
Table 1 Selection Process of Articles for Review Initial Search 112 Articles
Articles Concerned COPC in Family Medicine Residencies
↑ ← ←←↑
including publication years 1969–2005 and applying the limits “English language” and “human.” Our selection criteria were that the study had to describe and/or evaluate COPC teaching in a US family medicine residency program. A hand search of the reference lists of these papers failed to yield any more articles than those identified in the initial search. Two authors independently decided on inclusion or exclusion of articles and resolved disagreement (on two articles) by consensus. We included evaluative and descriptive studies but excluded letters and comments unless these described a COPC program. Since our focus is US family medicine residency education, we excluded studies that described medical student COPC programs, projects that occurred in other specialty residencies, or programs outside the United States.
Family Medicine
↑
400
Yes
22 Articles
90 Articles Surveys 6 Articles
↑
Program Description and/or Evaluation—16 Articles Evaluative 8 Articles
↑
The Nature of COPC Teaching and Learning Activities The 16 papers describing COPC programs in family medicine residencies included nine reports of monthlong COPC block rotations,10-18 usually in more than one residency year. Five papers described longitudinal halfday COPC experiences extending over periods from a few months to 3 years.10-14 Twelve authors described
No
Descriptive 8 Articles
Residency Education
Vol. 38, No. 6
cluded video journals and reflective portfolios,13 direct resident observation and patient satisfaction surveys,21 and written reports.15,17 Reported Educational Outcome Measures for COPC Educational Programs We categorized the educational outcome measures from residency COPC projects under Kirkpatrick’s four levels of evaluation4 adapted for medical education,5 as shown in Tables 2 and 3. In the only study directly reporting residents’ satisfaction with the COPC educational process (Level 1 data), Thompson reported from focus group analysis that residents enjoyed the interdisciplinary experience and working with community sites and that 75% stated that the COPC training increased the probability of their doing community work in the future.12 In that same paper, and in another article the following year,11 Thompson also measured residents’ attitudes about COPC (Level 2A data) in a posttest survey and by semi-structured interviews. Finally, Thompson also presented data on self-reported positive changes in residents’ behavior (eg, referral to community agencies), which were not confirmed from a subsequent chart review (Level 3 data).11 In 1999, Oandasan22 developed and validated a 20item COPC attitudes survey (Level 2A data). In this study, the survey results revealed no difference in the attitudes of the intervention and control groups about COPC after the educational intervention. In fact, inter-
401
vention residents’ attitudes toward the practicality of practicing COPC declined. In 1998, Donsky measured the changes in residents’ COPC attitudes (Level 2A) and knowledge (Level 2B) in a pretest and a posttest, focus group, and semi-structured interviews.10 These authors reported no validation data on the pretest and posttest questionnaire, which they adapted from a self-test from the American Academy of Family Physicians’ 1989 COPC monograph.26 Using this survey, residents’ COPC knowledge improved, but their attitudes declined. Four studies reported patient outcomes and/or impact on graduating residents’ practice (Level 4 data).16,18-20 In two studies describing the same residency COPC projects, Baker and Harper reported mixed patient outcomes. For example, they described a decreased teen chlamydia infection rate but a high, but stable, teen pregnancy rate.19,20 In 2003, Fisher reported impressive Level 4 outcomes by determining residents’ future practice. Of 10 graduating residents, six chose fellowships or academic careers related to community medicine/public health, and two entered practice in underserved areas.16 However, the author is open about the program’s likely selection bias in favor of residency candidates with community/underserved interests. Almost 20 years ago, in 1986, Strelnick reported equally impressive Level 4 outcomes, in that more than 50% of his program’s 187 primary care graduates were
Table 2 Kirkpatrick Table for Planning COPC Project Evaluation Kirkpatrick’s Four Levels of Evaluation, Adapted for Medical Education* Level
Description
Examples of Evaluation Methods
Level 1 Reaction
Residents’ reactions/satisfaction data
Focus groups, semi-structured interviews, satisfaction surveys, program evaluation data
Level 2a Learning
Modification of residents’ attitudes and perceptions
Focus groups, semi-structured interviews, before and after attitudes surveys, reflective essays, and journals
Level 2b Learning
Residents’ acquisition of knowledge and skills
Pretests and posttests of community medicine knowledge, focus groups, standardized patient cases, written vignettes
Level 3 Transfer
Change in residents’ behavior
Chart review, observational studies, self-reported change in surveys, interviews and focus groups, worked case studies
Level 4a Results/Outcomes
Change in practice graduates, residencies, or community agencies
Change in practice due to project outcomes, permanent adoption into residency curriculum or agency activities
Level 4b Results/Outcomes
Benefits to patients or communities
Patient outcome data, surveys (self-report), chart review, CQI projects, maintenance of certification projects, billing data
COPC—community-oriented primary care CQI—continuous quality improvement * Adapted from Kirkpatrick’s Four Levels of Evaluation24
402
June 2006
Family Medicine
Table 3 Family Medicine Residency COPC Papers Since 1969, Sorted by Reported Evaluation Level Author Baker19 2002
Subjects (#) Patients at FM residency clinic (# not stated)
Activities Five COPC projects— preschool immunization, family-centered birth, intimate interpersonal violence, teen pregnancy/STD prevention, HIV screening
Harper20 2000
Residents in same program as above (# not stated)
Same five projects as above. No details on length of projects, residency year, or number of residents
Fisher16 2003
FM residents— total # not stated but 10 graduates described
Block and longitudinal team and individual projects. Residents expected to fund projects through grants and obtain IRB approval. High-quality multidisciplinary support available to residents
Strelnick18 1986
61 FM, 72 IM, and 53 pediatrics residents. Medical students (# not stated)
R1 1-month orientation block with group project, up to five more 1-month elective blocks, presentation of resident’s own practice, individual projects, faculty projects
Thompson11 35 FM residents 1997 over 3 years. 12 out of 16 residents in 1 year completed evaluation
Interdisciplinary COPC team—FM faculty, Health District epidemiologist, social workers, health nurses, sociologist (School of Allied Health). Community organizations Year 1—3.5 day orientation sessions at community sites Years 2 and 3—other community activities, not specified
Methodology / Evaluation Method Chart review and audit, lab test reviews, Health Department reports, key informant interviews, analysis of billing data
As above—patient outcomes measured but no educational evaluation—paper states that residency didactics were “not evaluated” No educational evaluation, but highquality outcomes in terms of completed projects and graduate careers involving community medicine/ COPC
No educational evaluation, highquality outcomes in terms of graduate careers involving community medicine/ COPC. Projects described but not patient outcomes Posttest only given 10 months after the 3.5 day experiences. 5-item survey mostly attitudinal. One item elicited self-reported behavior change. Chart review also performed to confirm behavior change
Outcomes / Key Findings Clinical indicators—eg, improved HIV prenatal screening rates, decreased teen chlamydia rate, improved domestic violence screening, teen pregnancy high but stable, but preschool immunization rates declined, preterm births increased Same outcomes as above
Evaluation Level 4B—patient outcomes
4B—patient outcomes
The authors focus on “lessons learned” for residency education but report no resident or educational evaluation
Project outcomes, eg, 70% of gay men at risk of Hep B received two shots. Career outcomes: 6 of 10 residents chose academic careers (faculty or fellowship) related to community medicine, and two entered underserved community practice More than 50% of graduates to date of paper working in medically underserved areas
4B—patient outcomes 4A—impact on graduates’ practice
Impressive outcomes may be subject to selection bias of residency applicants predisposed to community medicine/ COPC. No educational evaluation
4A—impact on graduates’ practice
True interdisciplinary collaboration. Author open regarding selection bias of residents with community focus, eg MPH degree
Statistics not formally reported. Ten of 12 residents selfreported improvement in identifying and referring addictions, not confirmed by chart review
Levels 3 (behavior change) and 2A (attitudes change)
True interdisciplinary collaboration in COPC modeled to residents. Study limited by small numbers, limited nature of intervention, and poor-quality evaluation.
(continued)
Notes No educational program evaluation reported
Residency Education
Vol. 38, No. 6
403
Table 3 (continued) Author Oandasan22 1999
Thompson12 1996
Donsky10 1998
Brill13 2002
Outcomes / Key Findings Residents’ attitudes toward COPC— validated a 20-item scale to measure residents’ attitudes about COPC. Results: no difference in attitudes between intervention group and control group before and after intervention. Intervention residents’ attitudes toward practicality of COPC declined. 20 residents of Same as reference 9— Different survey from Survey statistics not same program description of same reference 9—19 formally reported. as reference 9— orientation program items. Themes from the 7 first-year First- and second-year Semi-structured evaluation were: residents, 7 residents also did small interviews with 1. Residents and second-year projects over 3 months. community sites. community sites enjoyed residents, 6 third- Third-year residents Formal program and working together, 2. year residents learner evaluation by Residents enjoyed interdisciplinary interdisciplinary team. experience (although only 25% achieved it), 3. Residents found the COPC model useful— 75% stated the training increased the probability of future community work, 4. Didactics insufficient to train residents on the COPC process 26 FM residents Longitudinal Group Pretest and posttest On the pretests and post in each of Project. of residents’ attitudes tests, residents’ COPC two projects Introductory activities and knowledge. knowledge improved described in year 1—seminar plus Focus group with but their attitudes community tours. residents, five semideclined. 2-year longitudinal structured interviews Qualitative themes: group COPC project— with residents. • Lack of ownership 1/2 day per week, 12 • Frustration with slow weeks each year, total COPC process 60 hours. • Loss of continuity Group contains three • Value of group PGY-3s and three PGYlearning 2s, plus two to three • Increased interest faculty in working with community • Increased knowledge and confidence in content area Residents Year 1—4-week Posttest Not stated. One project (# not stated) community medicine Video journal described. rotation. Years 2, 3— Reflective portfolio Longitudinal clinical Progression matrix care plus project Subjects (#) FM residents 27 pretest 22 posttest
Activities Resident COPC activities not described. Did not report nature or time period of intervention.
Methodology / Evaluation Method Quasi-experimental control group study Pretests and posttests Focus groups
(continued)
Evaluation Level Level 2A— modification of attitudes and perceptions
Notes Small #, large proportion of control groups lost to followup. Qualitative results not described in detail
Level 2A (attitudes), Level 1 (satisfaction)
Again, study limited by small numbers and inadequate evaluation. Survey not sufficiently described or reported. No methodology reported for qualitative data gathering and analysis
Level 2A (attitudes change) and 2B (knowledge change)
Some positive community reactions also reported. COPC knowledge questionnaire attached.
0
Innovative evaluation methods mentioned but no results reported
404
June 2006
Family Medicine
Table 3 (continued) Zoorob14 1999
Primary care residents (# not stated)
3-year managed care curriculum Year 1—Primary care/ orientation month Year 2—Urban and community medicine/ managed care month Year 3—Practice management and managed care month Longitudinal COPC projects in years 2 and 3.
Planned chart audits, questionnaires, casebased learning Direct observation Faculty and staff evaluations Patient satisfaction
Not yet measured
0
Paper mainly about managed care curriculum, but COPC involved. Curriculum described but not yet implemented—no results to report
Zweifler21 1998
FM residents (# not stated)
Direct resident observation Patient satisfaction surveys
Not stated. Evaluation not yet in place.
0
COPC teaching is part of wider curriculum on cultural competency
Nutting23 1991
Patients and household members of a 24-slot FM residency program
Described practice population demographics Identified priority health problems for population
None. Only described 0 first two steps of COPC
No educational process or outcomes described
Werblun17 1979
Residents (# not stated)
Year1—1-month community medicine rotation Resident community project completion and presentation required, details not given 1. Surveyed 35 consecutive patients 2. quality of care practice audit 3. compared expected versus observed index of diseases Year 1—Visit and assess a community Years 2/3—One project from following: • Health care services research • Policy analysis and decision making • Community service • Community health education • Case study
Year 1—Written community report Years 2/3—Written and presented project, report from community
None reported. Two projects described
0
No evaluation or outcomes data presented
Summerlin15 1993
Residents and Trainees complete medical students a community report (# not stated) during a 1-month rural elective Residents in a Residents work and CHC training train in a CHC where program it is stated that COPC (# not stated) principles are applied Residents and Curriculum not medical students described for residents (# not stated) Three matrices introduced for applying COPC principles to domestic violence— community medicine, clinical, and education
Written reports
Trainees self-report a unique opportunity to understand the health system of a community Not stated
0
No evaluation data presented
0
No evaluation data presented
The matrices are the key outcomes. No educational outcomes reported
0
No evaluation data presented
Prislin24 1996
Baker25 1995
FM—family medicine STD—sexually transmitted disease HIV—human immunodeficiency virus IM—internal medicine COPC—community-oriented primary care PGY—postgraduate year CHC—community health center
Not stated
Educational methodology not described. Resident evaluation stated as by pretests and posttests and self-reports
Residency Education currently practicing in medically underserved areas.18 Again, the author openly stated that his several residencies actively recruited candidates with established community interests and skills, eg an MPH degree. Influence of Residency COPC Programs on Graduates’ Future Practice and/or the Health of Patients and Communities As described above, a few reports provide some evidence that residency COPC programs can influence the health of patients and communities,16,19,20 as well as residents’ future practices.16,18 However, when drawing conclusions, we must bear in mind that these studies used nonvalidated instruments, and the studies were subject to resident selection bias. There is also some evidence that residency training can influence primary care physicians’ community involvement after graduation. In 1999, Steiner surveyed 500 recently graduated primary care physicians (44% family practice, 31% internal medicine, 25% pediatrics), with a 66% response rate.27 He found that subjects generally reported little community-related training. However, three factors that did correlate with current community involvement were residency training in community medicine, having a rural medicine experience, and having a mentor in the community. There also may be different levels of community involvement between academic and non-academic physicians. In 2002, 95% of 770 Society of Teachers of Family Medicine members reported participating in some communityengaged activity, not specifically COPC (response rate 58%).28 In contrast, in a different 2002 survey with an identical response rate (58%), only 6.7% of practicing physicians reported practicing COPC.9 This 6.7% may represent a decrease in COPC activities since 1994, when 75% of 200 board certified family physicians surveyed stated that they had never heard of COPC, but 16% reported practicing it.7 Discussion COPC provides a structured method to teach community medicine that modifies the public health/preventive medicine approach in a way that is relevant to primary care physicians. COPC’s five-step model of identifying community problems and developing interventions to address them can be applied to primary care practice populations. As such, the COPC model has resonated among family medicine educators. Despite this resonance, we found a paucity of evidence describing and supporting the educational benefit of residency COPC programs at any level, from resident satisfaction through patient outcomes. Our conclusions are limited by the small number of published studies and the variable, but generally poor, quality of the reported evidence.
Vol. 38, No. 6
405
Our review of 16 articles, essentially the extant literature on teaching COPC in US family medicine residencies, indicates that approximately 40% of programs state that they are teaching COPC, a number that has remained stable since 1979. No teaching methods or curricular interventions emerge as the gold standard for COPC teaching, although practical project experience is almost universally reported,10-21,23 and programs reporting higher-level outcomes16,18 report longitudinal curricula spanning all 3 years as well as interdisciplinary collaboration. Programs that have implemented COPC most extensively are more likely to report high-level outcomes than those with more-limited involvement in the area. Two of these programs16,18 that reported impressive Level-4 outcomes report bias in their selection of residents who have a community focus. However, this selection bias is justified if the residency’s goal is to graduate community-involved physicians. Even from programs reporting graduate outcomes, we lack data demonstrating that COPC-trained and committed physicians consistently obtain better health outcomes for their communities. Common limitations in the 16 papers included low survey response rates (which limit our ability to draw conclusions), weak educational methodologies, nonvalidated instruments, and inadequate statistical analyses leading to poorly reported or absent results. From the literature, we have nonetheless identified several barriers to evaluating COPC projects that may contribute to the generally poor quality of these peer-reviewed reports. Barriers to Evaluating COPC Educational Programs Reported barriers to successfully evaluating COPC programs in educational settings include limited curricular time,15,29,31 lack of faculty expertise in COPC and/or evaluation/curriculum development,18 lack of resources, including funding,15,29,31 inadequate community partnering,29 and poor evaluations from learners.15,29,31 A further barrier to successfully implementing COPC in educational settings may be the fragmented nature of the US health system. COPC success in other countries may be associated with health care systems in which physicians care for a (usually geographically) defined population. These barriers do not prevent approximately 40% of family medicine residency programs teaching COPC, but they may contribute to the paucity of peer-reviewed evaluation literature. Using the limited evidence from our literature review as a starting point, we propose recommendations that may assist family medicine faculty to gather evaluation data demonstrating COPC programs’ efficacy in increasing residents’ knowledge of COPC, changing their attitudes, and influencing their practice behavior to the benefit of their communities.
406
June 2006
Recommendations—Designing a COPC Evaluation Plan We recommend that faculty design their evaluation plan before implementing any residency COPC project, because data are more difficult to gather after the event, and the opportunity to gather baseline data is lost. We recommend that faculty design a plan addressing all four of Kirkpatrick’s levels of evaluation, as demonstrated in Table 2. Because of the small numbers of learners in residency programs, it is difficult to use experimental methodologies (eg, randomized controlled trials), but it is practical to gather good baseline quantitative and qualitative data for each evaluation level, as shown below and summarized in Table 2. Level 1 data (reaction) measure course process outcomes, such as whether residents enjoy the learning experience, believe the content and teaching methods to be appropriate and well taught, report the program to be well organized and efficient, and consider it a useful contribution to their training. This information can be gathered with a simple 5-point Likert scale from “strongly disagree” to “strongly agree” using stems such as “I enjoyed my COPC experience.” The data can be enriched by including a free-text opportunity for comments and also by conducting focus groups where residents discuss the strengths and weaknesses of the COPC program. Level 2 data (learning) describe changes in residents’ COPC knowledge and attitudes. In 1999, Oandesan validated a 20-item survey that can serve as a pretest and posttest of residents’ attitudes to COPC. Donsky published a COPC questionnaire in 1998 that can be used as a pretest and posttest of knowledge and attitudes, although this instrument has not yet been validated. Qualitative methods for evaluating learning include focus groups, semi-structured interviews, written case exercises, and reflective essays and journals. Level 3 data (transfer) concern changes in residents’ behavior and/or clinical practice. Behavior change can be measured through direct observational studies, chart reviews, electronic health record searches, and/or selfreports in written or electronic surveys. Self-reported changes in behavior represent much weaker types of Level 3 data than objective measures such as direct observation or chart review. Examples of Level 4 data (results and outcomes) include changes in graduates’ clinical practice resulting from the COPC projects, permanent adoption of the COPC program into the residency curriculum, and measurable effects on community agencies or practice populations. The effect of the COPC program on graduates’ practice behavior can be measured by telephone, electronic, or mailed surveys 1 to 2 years after residency. Effects on community agencies can be measured by semi-structured interviews or focus groups with staff and/or by objective measures such
Family Medicine as increased funding or expanded activities. Effects on the health of practice populations can be demonstrated by chart review and audit projects. Conclusions Since approximately 40% of family medicine residencies have been teaching COPC for up to 30 years, we must assume that more process and outcome evaluation has taken place than has been reported in the peer-reviewed literature. Other important outlets for COPC evaluation data include campus press and local newspaper reports, grant progress reports, community and campus Web sites, and presentations at professional society meetings. These outlets are vital for COPC evaluation, and we do not wish to minimize their important contribution to community-engaged scholarship. However, for COPC to earn recognition as a scholarly activity in the academic environment, we contend that faculty must rigorously evaluate their programs and publish outcomes in the peer-reviewed literature. Such publications may improve the quality of other COPC programs, expand the educational scholarship of COPC, and advance the academic careers of community-engaged faculty. In addition, this increased academic rigor may enhance the educational experience for our residents, who might in turn be more likely to incorporate COPC into their clinical practice. Only by changing the practice of community physicians, and describing the effect on practice populations, can we truly demonstrate any benefit of teaching COPC in family medicine residencies. Acknowledgments: Financial support was received from the Health Resources and Services Administration Award no. D56HP00164. We presented this paper as a lecture-discussion at the Society of Teachers of Family Medicine 2004 Annual Spring Conference in Toronto. Corresponding Author: Address correspondence to Dr Dobbie, University of Texas, Southwestern, Department of Family and Community Medicine, 6263 Harry Hines Boulevard, Dallas, TX 75390-9067. 214-648-1399. Fax: 214-648-1307.
[email protected].
REFERENCES 1. Program requirements for residency education in family practice. www. acgme.org/downloads/RRC_progReq/120pr701.pdf. Page 17. Accessed February 12, 2005. 2. Martin JC, Avant RF, Bowman MA, et al. Future of Family Medicine Project Leadership Committee. The Future of Family Medicine: a collaborative project of the family medicine community. Ann Fam Med 2004;2 Suppl 1:S3-S32. 3. Accreditation Council for Graduate Medical Education Outcomes Project, 2001. www.acgme.org/outcome/comp/compFull.asp#6. Accessed October 22, 2005. 4. Encyclopedia of Educational Technology. Kirkpatrick’s four levels of evaluation. http://coe.sdsu.edu/eet/Articles/k4levels/index.htm. Accessed October 22, 2005. 5. Morrison J. ABC of learning and teaching in medicine: evaluation. BMJ 2003;326:385-7. 6. Donsky J, Massad R. Community medicine in the training of family physicians. J Fam Pract 1979;8(5):956-71.
Residency Education 7. Williams R, Foldy SL. The state of community-oriented primary care: physician and residency program surveys. Fam Med 1994;26(4):232-7. 8. Plescia M, Konen JC, Lincourt A. The state of community medicine training in family practice residency programs. Fam Med 2002;34(3):17782. 9. Longlett SK, Phillips DM, Wesley RM. Prevalence of community-oriented primary care knowledge, training, and practice. Fam Med 2002; 34(3):183-9. 10. Donsky J, Villela T, Rodriguez M, Grumbach K. Teaching communityoriented primary care through longitudinal group projects. Fam Med 1998;30(6):424-30. 11. Thompson R, Haber D, Chambers C, Fanuiel L, Krohn K, Smith AJ. Orientation to community in a family practice residency program. Fam Med 1997;30(1):24-8. 12. Thomson R, Haber D, Fanuiel L, Krohn K, Chambers C. Communityoriented primary care in a family practice residency program. Fam Med 1996;28(5):326-30. 13. Brill JR, Ohly S, Stearns MA. Training community-responsive physicians. Acad Med 2002;77(7):747. 14. Zoorob RJ, Sidani M. A managed care curriculum: developing a managed care curriculum for primary care residents. Med Educ 1999;33(11): 845-9. 15. Summerlin HH Jr, Landis SE, Olson PR. A community-oriented primary care experience for medical students and family practice residents. Fam Med 1993;25(2):95-6. 16. Fisher JA. Medical training in community medicine: a comprehensive, academic, service-based curriculum. J Community Health 2003;28(6):407-20. 17. Werblun MN, Dankers H, Betton H, Tapp J. A structured experiential curriculum in community medicine. J Fam Pract 1979;8(4):771-4. 18. Strelnick AH, Shonubi PA. Integrating community-oriented primary care into training and practice: a view from the Bronx. Fam Med 1986; 18(4):205-9. 19. Baker NJ, Harper PG, Reif CJ. Use of clinical indicators to evaluate COPC projects. J Am Board FAm Pract 2002;15(5):355-60.
Vol. 38, No. 6
407
20. Harper PG, Baker NJ, Reif CJ. Implementing community-oriented primary care projects in an urban family practice residency program. Fam Med 2000;32(10):683-90. 21. Zweifler J, Gonzalez AM. Teaching residents to care for culturally diverse populations. Acad Med 1998;73(10):1056-61. 22. Oandasan IF, Ghosh I, Byrne PN, Shafir MS. Measuring community-oriented attitudes toward medical practice. Fam Pract 2000;17(3):243-7. 23. Nutting PA, Nagel J, Dudley T. Epidemiology and practice management: an example of community-oriented primary care. Fam Med 1991;23(3): 218-26. 24. Prislin MD, Morohashi D, Dihn T, Sandoval J, Shimazu H. The community health center and family practice residency training. Fam Med 1996;28:624-8. 25. Baker NJ. Strategic footholds for medical education about domestic violence. Acad Med 1995;70:982-5. 26. Nutting PA, Garr DR. Community-oriented primary care. Monograph, edition no. 124, Home Study Self-assessment Program. Leawood, Kan: American Academy of Family Physicians, 1989. 27. Steiner BD, Pathman DE, Jones B, Williams ES, Riggins T. Primary care physicians’ training and their community involvement. Fam Med 1999;31:257-62. 28. Beck B, Wolff M, Guse CE, Maurana CA. Involvement of family and community medicine professionals in community projects. J Fam Pract 2002;51(4):369. 29. Klevens J, Valerrama C, Restrepo O, Vargas P, Casabuenas M, Avella MM. Teaching community-oriented primary care in a traditional medical school: a 2-year progress report. J Community Health 1992;17(4):23145. 30. Glasser M, Holt N, Hall K, et al. Meeting the needs of rural populations through interdisciplinary partnerships. Fam Community Health 2003; 26(3):230-45. 31. Unverzagt M, Wallerstein N, Benson J, Tomedi A, Palley T. Integrating population health into a family medicine clerkship: 7 years of evolution. Fam Med 2002;34(10):45-51.
