Collaborating to Integrate Curriculum in Primary Care Medical ... - STFM

S126

January 2004 Supplement

Family Medicine

Collaborating to Integrate Curriculum in Primary Care Medical Education: Successes and Challenges From Three US Medical Schools Catherine F. Pipas , MD; Deborah A. Peltier, MD; Leslie H. Fall, MD; Ardis L. Olson, MD; John F. Mahoney, MD; Susan E. Skochelak, MD; Craig L. Gjerde, PhD Background and Objectives: Traditional medical school department-based clerkship structures can lead to redundancy and/or gaps in curriculum, inefficient administrative systems, and academic isolation for clerkship directors. This paper describes the approaches, successes, and challenges three institutions experienced when implementing an interdepartmental collaboration to create an integrated primary care clerkship experience. Methods: Each school combined family medicine, ambulatory pediatrics, and ambulatory medicine into contiguous clerkship blocks. In all institutions, each clerkship maintained certain distinct features while the integrated aspects contained longitudinal curriculum of certain primary care topics. Results: Evaluations by students demonstrated favorable responses to the new content and integrated methods of teaching, as did results of the Association of American Medical Colleges graduation survey. Faculty at each institution reported that their multidisciplinary approach has stimulated important educational collaborations, many of which require an economy of scale not often achievable within a single clerkship. These included innovative evaluation/documentation efforts; centralization of administrative tasks; enhanced recruitment, retention, and development of community-based faculty; an increase in the active core group of local and national primary care leaders; and an increase in scholarly activities. The collaborations have not occurred without challenges, primarily in the need for identifying sustainable resources for these and future collaborative educational endeavors. Conclusions: The benefits involved in developing an integrated primary care experience include expansion of curriculum content and methods, as well as enhancement of collegial support and resources to community-based and academic faculty. These integrations do, however, bring added challenges, time, and costs to traditional independent clerkships. (Fam Med 2004;36(January suppl):S126-S132.) Traditional department-based clerkship structures may include redundancy and/or gaps in curriculum, inefficient administrative systems, and academic isolation for clerkship directors. In theory, interdisciplinary integration of curricular content can create opportunities for coordinated and expanded teaching of common subjects, shared administrative tasks, and a larger institutional view of the educational mission. To do so, however, requires interdisciplina ry collaboration among faculty. Adult learning theory, which places an emphasis on generalizable knowledge and contextual From the Department of Community and Family Medicine (Dr Pipas), Department of Medicine (Dr Peltier), and Department of Pediatrics (Drs Fall and Olson), Dartmouth Medical School; the Department of Emergency Medicine, University of Pittsburgh (Dr Mahoney); and the Department of Family Medicine, University of Wisconsin (Drs Skochelak and Gjerde).

learning, suggests that teaching core topics across multiple courses is desirable.1 For these reasons, interdisciplinary curricula have now been developed at multiple medical schools.2-7 The practice of medicine in the 21st century will require well-trained primary care physicians who are versed in the practice of multidisciplinary and collaborative medicine. The managed health care environment rewards physicians for applying evidence-based approaches to improve the health of their patient panels and for developing efficient systems of care that address the health needs of populations in their service areas. In responding to this challenge, medical schools have developed interdisciplinary curricula to broadly teach topics such as common problems in primary care, evidence-based medicine, health care systems, and preventive care. Many of these efforts, including those

Section IV: UME-21 and Beyond: Outcomes and Policy Implications for Medical Education described here, have been supported through the Undergraduate Medical Education for the 21st Century (UME-21) demonstration project.8 Two of the largest primary care interdisciplinary projects described to date are the Robert Wood Johnsonsupported Generalist Physician Initiative9-11 and the Interdisciplinary Gener alist Curriculum Project (IGC).12-15 These two programs exposed medical students to primary care role models in practice during the first 2 preclinical years through the combined efforts of family medicine, general internal medicine, and general pediatrics faculties. An Academic Medicine supplement9-11 devoted to the IGC schools described these projects’ focus on collaboration in the preclinical years and implications for other interdisciplinary courses. Collaboration among primary care disciplines was the central functional ingredient for successful implementation, and these faculty relationships required time to nurture and develop. As these collaborative efforts matured, new and more time-efficient models for administration of these complex integrated courses emerged. As a result, many of the IGC schools moved toward a common course administration. The generalist culture that formed, based on a shared value and a common vision, facilitated and promoted collaboration in other spheres, including faculty development efforts and curricular changes.9-15 The UME-21 project followed and promoted exposure to an integration of curricula relevant to all primary care disciplines in third-year clerkships. This paper describes the approaches taken by three institutions, Dartmouth Medical School, the University of Pittsburgh, and the University of Wisconsin to develop interdisciplinary primary care clerkships, the successes of their collaborations, and the challenges faced. Program Development: Three Institutional Approaches Table 1 outlines the key characteristics of each of the three schools. All have an integrated third-year primary care block that builds, to differing degrees, on first- and second-year longitudinal clinical experiences. Integration is defined for this paper’s purpose as the combining of the clerkships in temporal proximity and merging of efforts in the didactic portions of curricular content, evaluation, and faculty teaching methods and efforts. Interaction does not refer to merging of the clinical experiences in clerkship, although this has occurred coincidentally at a few sites. We define collaboration as a primary method in which faculty worked to achieve integration. Although many distinct aspects of each clerkship remain (eg, clerkship sites, clerkship directors, syllabi, grading systems), students participate in a supplemental integrated longitudinal curriculum of primary care topics, as well as integrated orientation, syllabus, coordinator support, director support, evaluation, and in some cases an integrated grade.

Vol. 36, January Supplement

S127

The goal at each institution was to enhance the quality and diminish redundancy of primary care ambulatory education by developing educational units that addressed gaps and overlap in the current departmentally based teaching. The unique approaches used to achieve this goal and the particular characteristics of each institution are described below. Table 2 illustrates how the three schools differed in the degree of integrated curriculum and teaching they undertook and the evaluation processes and administrative supports required for these changes to occur. Dartmouth Medical School At Dartmouth, the impetus for integration was in part the Robert Wood Johnson (RWJ) Generalist Physician Initiative. In 1997, funding from RWJ led Dartmouth’s primary care clerkship directors to examine existing and overlapping curriculum and to develop a unified vision for curricular design during 3 years of medical school. The UME-21 project expanded the primary care teaching agenda in year 3 and accelerated the process of unifying aspects of year 3. The Medical Education Committee at Dartmouth also began to identify and implement vertically integrated working groups16 that facilitated these activities. As a part of UME-21, the three previously independent ambulatory clerkship blocks were replaced by a 16-week integrated primary care block (IPCC). The number of weeks of the individual clerkships (4, 4, and 8, ambulatory pediatrics, ambulatory medicine, and family medicine, respectively) did not change, but one half day per week over the 16 weeks was devoted to new integrated curriculum. As shown in Table 2, the topics included evidence-based medicine, office management, prevention, chronic illness, ethics, and community and managed care. Changes occurred in the number of hours of teaching, content areas, grading, student evaluation, and time and resources needed to administer the course. University of Pittsburgh In 1992, the University of Pittsburgh adopted an integrated curriculum in the first 2 years, which included multidisciplinary non-departmental courses, multimodal evaluation, early introduction to the patient and community, and an emphasis on active learning.17 It also established a centrally governing Curriculum Committee as an implementation arm of the Office of Medical Education.18 Building on the success of these innovations, in 1995 and 1996, the Curriculum Committee convened student and faculty task forces to examine the organization and curriculum of the third and fourth years. Based on the recommendations of these task forces, the Curriculum Committee approved a new curricular plan that enhanced and integrated community and ambulatory learning experiences. This plan provided opportunities

S128


Family Medicine

for meaningful integration during all 4 years of medical school. The majority of inpatient and outpatient clerkships were assembled into combined experiences. The resultant shortened third-year curriculum (48 weeks) was achieved by reducing the length of certain clinical rotations (Table 1). The community/ambulatory medicine clerkship (CAMC), the keystone of this plan, originated with and was funded by UME-21. It incorporated the only elective time in the third year by offering a 3-week selective clerkship in any one of the three primary care disciplines or in emergency medicine. In CAMC, the new content included the nine key content areas of the UME-21 program and additional clinical topics. University of Wisconsin At the University of Wisconsin, the development of an integrated third-year primary care rotation occurred in 1990 in response to a perceived need to increase the curricular time devoted to primary care (Table 1). Prior to 1990, there was no required training in family medi-

cine, and internal medicine and pediatrics rotations did not have outpatient components. Longitudinal themes containing the new content (Table 2) occur at six different times across the 4-year program, including purposeful sequencing of content in years 3 and 4. Receipt of the UME-21 project funding provided the opportunity to expand the primary care content areas, methods, and evaluation tools used primarily in the thirdyear course. As a result of UME-21, in 1998, the University of Wisconsin curriculum was augmented with the addition of increased hours and curricular topics. Faculty and administrative time were established within the Department of Family Medicine to oversee the project. Additional methods for teaching new content included standardized patients and panel discussions. The unique contribution to the integration that the University of Wisconsin makes is its historical application of the integration across all 4 years of their program, rather than the concentrated focus in year 3 that occurred at Dartmouth and the University of Pittsburgh.

Table 1 Key Characteristics of Each Educational Institution Dartmouth Medical School

University of Pittsburgh

University of Wisconsin

Private

Public

Public

Rural Northern New England

Urban Northeastern

Rural Midwest

70–90 *

148

142

67%

13%

17%

Moderate

Moderate

High

Primary care clerkship structure prior to integration

FM—8 weeks Amb IM—4 weeks Amb Ped—4 weeks

FM—4 weeks Amb IM—4 weeks Amb Ped—4 weeks

None, no FM, Amb IM, or Amb Ped existed

Clerkship structure after integration

16-week integrated primary care clerkship (IPCC): FM 1/2 day less didactic Peds and IM 1/2 day less precepting time

12-week community / ambulatory medicine clerkship (CAMC): FM—1 week less IM—1 week less Peds—1 week less 3-week selective from IM, FM, Peds, or EM

8-week primary care clerkship (PCC) FM—4 weeks and Amb Ped—4 weeks or Amb IM—4 weeks

1/2 day weekly didactic sessions

1/2 day weekly didactic sessions and a 3-week selective

1/2 day weekly didactic sessions

Began 1997, expanded with UME-21 in 1998

Began 1999 with UME-21 funding

Began 1990, expanded with UME-21 in 1998

School funding base Setting Class size % of clinical teaching that is community based Vertical integration of year 1 and 2 with year 3 curriculum

Integration time Timing of curricular integration in year 3

•Approximately 20 students at Dartmouth Medical School matriculate to Brown University annually in year 3. FM—family medicine, Amb IM—ambulatory internal medicine, Amb Peds—ambulatory pediatrics, EM—emergency medicine

Section IV: UME-21 and Beyond: Outcomes and Policy Implications for Medical Education


S129

Table 2 Integrated Changes to Curriculum, Administration, and Collaboration

New didactic curriculum hours New curriculum content

Dartmouth Medical School

University of Pittsburgh

University of Wisconsin

24/per block

48/per block

24/per block

• • • •

Evidence-based medicine Office management Chronic illness Disease management of common clinical problems • Quality improvement • Prevention • Managed care principles

Adjusted disciplinespecific teaching time Adjusted disciplinespecific precepting time New student evaluation tools

• • • •

New grading

• • • • • • • • •

Evidence-based medicine Ethics/professionalism Common clinical problems Communication skills Medical informatics Quality improvement Prevention Managed care principles Practice guidelines

• • • • • • • •

Evidence-based medicine Ethics/professionalism Prevention Common clinical problems Communication skills Quality improvement Practice guidelines Managed care principles

No change

None existing prior

No change

Less 1/2 day/week

Less 1/2 day/week

No change

OSCEs every 16 weeks Two EBM projects Two student presentations Final integrated essay

• OSCEs every 12 weeks • Four EBM projects • Integrated final exam

• OSCEs every 8 weeks • Student projects • Integrated final exam

Discipline-specific grades with 20% from each based on integrated content

Integrated grade only; disciplinespecific evaluation submitted for dean’s letter

Integrated grade only

New course evaluation tools

• Focus groups • Integrated evaluation form • PDA-based primary care teaching and learning system*

• Focus groups • Integrated evaluation form • Primary care teaching and learning log

• Focus groups • Integrated evaluation form • Primary care skills checklist

New administrative support requirements

• Coordinator: .5 full-time equivalent (FTE) • Faculty: .6 FTE

• Coordinator: 1.4 FTE • Faculty: .3 FTE

• Coordinator: 1.0 FTE • Faculty: .5 FTE

New collaborative primary care spin-offs

• Joint recruitment, site and faculty development, standardized preceptor benefits • Interdisciplinary database of clinical sites • Active writing group • Joint regional and national presentations • Joint educational research • Joint grantsmanship • Annual leadership retreat

• Joint faculty development, shared site information • Interdisciplinary database of clinical sites • Active writing group • Joint regional and national presentations • Joint educational research • New CAMC Web site • Ongoing monthly leadership meetings

• Master faculty recruitment, faculty development efforts • Interdisciplinary database of clinical sites • Core educational faculty group • Active writing group • Joint regional and national presentations • Joint educational research • Joint grantsmanship • Annual retreat

Curricular Revision at All Three Schools At all three institutions, the use of interdisciplinary faculty teams occurred in design, implementation, and teaching of the integrated curriculum. The new integrated curriculum added between 24 and 48 total additional hours on non-clinical teaching time, resulting in reduction of clinical preceptor time by one half day per week at two schools. At one school, the integration process actually brought about the first ambulatory experiences and at another, it initiated ambulatory elective

opportunities in the third year. In all three schools, it increased the exposure to ambulatory medicine in a continuous experience. Content covered primary care topics, many of which were requirements of the UME-21 project. Methods used for teaching and evaluation included less traditional lectures and more active discussion and presentation by students in small groups, observed structured clinical examinations (OSCEs), use of standardized patients, Web-based learning, panel discussions, laboratory sessions, and self-directed learning.

S130


Administrative Efficiency and Faculty Collaboration At each school, a combined evaluation committee, consisting of clerkship directors, faculty members, and coordinators, designed, implemented, and reviewed evaluation materials, including development of integrated OSCEs. Students’ performances were reviewed jointly to understand the impact of the educational program. Grading methods differed slightly at each school (Table 2). Faculty and coordinators spent between 4 and 8 additional hours monthly in primary care leadership groups. These groups worked on sharing and revising educational goals, developing curriculum, evaluation tools, and jointly preparing presentations and publications. Program Evaluation Curricular Revision Traditional and additional course evaluation tools were used (Table 2). Overall student curricular satisfaction was assessed by integrated evaluation forms and focus group sessions. Additionally, the UME-21 senior student survey and the Association of American Medical Colleges (AAMC) graduation questionnaire (GQ) surveys from 1999, 2000, and 2001 were reviewed for program trends. Students’ evaluations of the individual and integrated clerkships, although previously favorable in prior ambulatory clerkships, improved in all three schools. This resulted in these primary care clerkships being ranked among the top required clerkships at all of the schools. Specifically noted and highly rated throughout was the experience of learning topics in a multidisciplinary manner taught by integrated faculty. The AAMC GQ surveys indicated that these three schools had an overall 13% increase, as compared to -2% in other UME21 schools and 4% in all schools, in students who agree or strongly agree that “The medical school curriculum should include formal content in interdisciplinary teamwork and experiences involving health professionals.” Specific content areas reviewed in the 1999–2001 UME-21 senior student survey showed significant differences in three areas, with a higher percentage of seniors from these three schools reporting using evidencebased medicine databases, likelihood ratios, and participation in ethics rounds at 43%, 8%, and 19% respectively, as compared to 22%, -12%, and 2% in all other UME-21 schools. Although the additional teaching time came from patient care hours, students did not report insufficient patient care exposure in focus groups or survey. At the time of complete integrations of UME-21 curricular changes in 2001, 95% of graduates of these three schools reported “adequate instructional time in care of ambulatory patients,” as compared to 89% at other UME-21 schools and 90% at all other US medical schools.

Family Medicine A theme of “Too much primary care” surfaced from focus groups at all schools, however, the AAMC GQ survey 2001 showed no differences in these three schools, with 96% of students reporting “they had adequate instruction in primary care.” All three schools utilized primary care documentation systems, one of which has published contributions to knowledge about ambulatory teaching and learning.19-22 Administrative Efficiency, or Not Development and maintenance of new material and additional faculty and coordinator administrative time were funded jointly by departments and UME-21 (Table 2). Faculty teaching time also increased as a result of new hours (Table 2), but these costs were primarily carved out of faculty administrative time. Faculty director position was rotated among clerkship directors at one program and shared by all at the other two. It is of note that the faculty percentage supported in all of these projects was inversely proportioned to the coordinator percentage and that the highest faculty time was added in the longest block (.6 FTE in 16 weeks). The structure of the coordinators’ roles evolved during this process. Two of the three schools had a centralized, funded coordinator for the integrated components, while the third shared this role among departmental coordinators. Organizational benefits were derived in newly developed material (scheduling, printing, and distribution of manuals, grading, recruiting faculty), but overall the quantity of work increased, and for schools still maintaining independent clerkship materials and sessions, these were additive. Course costs were also increased at all three sites, including written materials, copying, printing, etc. Faculty Collaboration Faculty members were surveyed informally and questioned in focus groups about satisfaction and suggestions. Overall views were positive, and items noted consistently among schools include: (1) the benefit of evaluation teams and additional tools in objectively evaluating students, (2) the enhanced perception of scholarship in primary care associated with the added rigor of the evidence-based curriculum, (3) last, but certainly not least, a unanimous sense of increased collaborative efforts and resulting primary care presence and institutional commitment. These efforts have led to academic promotions, specifically five primary care dean positions among the three schools, and in one case, an Office of Primary Care Education and Research. Faculty have contributed in all three schools to an increased number of manuscripts, presentations, research projects, and grants. Through sharing of individual department systems, schools report an additional and unforeseen universal benefit in support of community-based faculty. These

Section IV: UME-21 and Beyond: Outcomes and Policy Implications for Medical Education include: (1) minimizing administrative tasks and promoting quality and standardization for communitybased faculty, (2) increase in faculty development sessions, (3) centrally supported teaching databases, and (4) joint preceptor/site recruitment and retention efforts. Challenges and Limitations Two challenges we experienced, particularly after UME-21 grant funds expired, were time and money. Post UME-21, all three schools received institutional support for curricular activities and coordinators’ time, but faculty’s protected administrative time was only sustained at one school. Achieving an optimal balance of autonomy versus integration of the primary care disciplines was and remains a challenge. As far as limitations, we acknowledge a lack of true comparison of before and after curricular material, as well as multiple other factors within schools. We also acknowledge this to be only a sampling of the multiple schools that have introduced primary care integration. A more comprehensive study would include before and after comparison of all schools with and without integrated programs. Discussion Optimizing undergraduate medical student education is the ultimate goal of these UME-21 projects, and determining the best content and methods remains our challenge. We believe that quality materials and methods have been introduced and that the collaboration process and the resulting collegial teams are as beneficial to faculty as curricular enhancements are to students. We found that multi-discipline teams can provide quality review and revision to existing materials but acknowledge that certain content area may be best taught within departments. Willingness of individual departments to integrate may take into account the proposed loss or gain of individual time. We conclude from student course evaluations that the integration has not hurt the individual course ratings. We, as faculty, appreciate the extended length of time in which to mentor students, but we do not know the long-term effects of integration on career choice, particularly when combined with the increased clinical experiences in years 1 and 2. Indeed, some expressed concern about primary care backlash when students were “overexposed” to primary care. Even after the residency match of 2002, where one of our schools had an increase in the number of students entering primary care fields, we do not know what, if any, role these integrated UME-21 efforts have on primary care recruitment efforts, as market forces, debt load, and lifestyle issues are all influences on specialty choices. An anticipated decrease of faculty teaching time from minimizing redundancy was negated since small-group


S131

instruction (requiring more intensive faculty cross training) was a preferred method of teaching, and UME-21 content areas highlighted gaps that required additional faculty efforts to address. The complex task of integrating poses challenges, particularly when clerkships also remain distinct; this added layer of integrated material, in some ways, creates redundancy of administrative efforts. We cannot say which administrative models are more effective but only that competing demands make it essential to have effective administrative support and dedicated project leaders. Faculty agreed that while more meetings occurred, the result of sharing across disciplines was worth the effort, particularly when the effort was funded. Concern existed that without external funding these innovations would not be sustained. Faculty have been pleased and attribute institutional/ departmental funding successes to two factors. First, to ongoing support from department chairs and deans in developing a shared mission. Second, to the number of other collaborative successes, including local and national dissemination, joint faculty development, and community-based teaching support and their individual leadership roles. We as faculty at the involved institutions have gained insight to the overall scheme of institutional curricula by working within a broader team of committed educators. This collaboration need not be limited to primary care, since this collaborative approach can provide more cohesive and seamless curricula, greater faculty presence, and institutional commitment. Future considerations include increased integration and collaboration between undergraduate and graduate medical education surrounding competencies training. As medical science evolves, local and national leaders committed to a collaborative model can help medical education keep pace by continuing to sponsor and promote integrated efforts. Acknowledgments: We express our sincere appreciation to the following: Patricia A. Carney, PhD, associate professor of community and family medicine and assistant dean of research in medical education, for her vision, skill in mentoring, and 100% plus commitment to medical education; David W. Nierenberg, MD, professor of medicine, associate dean for medical education, Dartmouth Medical School, for continuing to be our institutional champion in promoting our integrated primary care activities; Margaret T. Russell, MS, research associate, Department of Community and Family Medicin e, for her multidiscip lin ary adminis trativ e efforts; Karen E. Schifferdecker, PhD, administrative director, Office of Community-Based Education and Research, for her team-building skills, attention to detail, and knowledge of the literature; and Judith Mitchell, administrator, Office of Medical Education, University of Pittsburgh, for her organizational skills and complete commitment to the success of this project. Grant support was received from: “Undergraduate Medical Education in the 21st Century” provided by the Division of Medicine and Dentistry, Health Resources and Services Administration, Department of Health and Human Services, Rockville, Md, under contract #2240-97-0038. Corresponding Author: Address correspondence to Dr Pipas, Dartmouth Medical School, Department of Commun ity and Family Medicine, HB 725 0, H ano ver, NH 0 37 55. 6 03 -65 0-1 96 7. Fax : 60 3-6 50 -11 53 . [email protected].

S132

January 2004 Supplement REFERENCES

1. Knowles MS. The modern practice of adult education: from pedagogy to andragogy. Englewood Cliffs, NJ: Cambridge Adult Education, 1980. 2. Carney PA, Bar-on ME, Grayson M, Klein M. Assessing the impact of early clinical training in medical education: a multi-institutional perspective. Acad Med 1999;74(1 suppl): S59-S66. 3. O’Neill PA, Morris J, Baxter CM. Evaluation of an integrated curriculum usin g problem-based learning in a clinical environ ment: the Manchester experience. Med Educ 2000;34:222-30. 4. Roberts LW, Franchini G, Fielder K. An integrated psychiatry-neurology clerkship within a problem-based learning curriculum. Acad Med 1997;72(5):423-4. 5. Magrane D, Ephgrave K, Jacobs MD, Rusch R. Weaving women’s health across clinical clerkships. Acad Med 2000;75(11):1066-70. 6. Medalie JH, Metlee TM, Stevens DP. An integrated clerkship program for medical students: commentary. Acad Med 1997;72(4):250-1. 7. Rennie J, Fine B, Seabrook M. Integrated clerkships for medical undergraduates. Acad Med 1999;74(5):591-2. 8. Rabinowitz HK, Babbott D, Bastacky S, et al. Innovative approaches to educating medical students for practice in a changing health care environment: the national UME-21 project. Acad Med 2001;76(6):587-97. 9. Brooks WB, Orgren RO, Wallace AG. Institutional change: embracing the initiative to train more generalists. Acad Med 1999;74(1suppl):S3-S8. 10. Ross LL, Appel MH, Kelliher GJ. The role of the generalist physician initiative in the merger of Hahnemann University and Medical College of Pennsylvania. Acad Med 1999;74(suppl 1):S16-S23. 11. Matson CC, Ullian JA, Boisaubin EV. Integrating early clinical experience curricula at two medical schools: lessons learned from the Robert Wood Johnson Foundation’s Generalist Physician Initiative. Acad Med 1999;74(suppl 1):S53-S58.

Family Medicine 12. Kahn N, Davis A, Wartman S, Wilson M EH, Kahn R. The Interdisciplinary Generalist Curriculum (IGC) Project: an overview of its experience and outcomes. Acad Med 2001;76(4 suppl):S9-S12 13. Rivo ML. With conviction, commitment, and creativity: promoting generalism and the preparation of the generalist physician. Acad Med 2001;76(4 suppl):S3-S4. 14. Bazell C, Kahn R. From the primary care organizations consortium’s proposal to the Interdisciplinary Generalist Curriculum Project. Acad Med 2001;76(4 suppl):S13-S18. 15. Blavo C, Steinkohl DC. The Interdisciplinary Generalist Curriculum Project at Nova Southeastern University College of Osteopathic Medicine. Acad Med 2001;76(4 suppl):S104-S108. 16. Nierenberg DW. The use of “vertical integration” groups to help define and update course/clerkship content. Acad Med 1998;73(10):1068-71. 17. Bernier GM, Adler S, Kanter S, Meyer WJ. On changing curricula: lesson s learned at two d issimilar medical schoo ls. Acad Med 2000;75(6):595-601. 18. Reynolds CF III, Adler S, Kanter SL, Horn JP, Harvey J, Bernier GM. The undergraduate curriculum: centralized versus departmentalized. Acad Med 1995;70(8):671-5. 19. Pipas CF, Carney PA, Eliassen MS, et al. Development of a handheld computer documentation system to enhance an integrated primary care clerkship. Acad Med 2002;77(July):600-9. 20. Carney PA, Pipas CF, Eliassen MS, Donahue DA, Kollisch DO, Dietrich AJ. An encounter-based analysis of the nature of teaching and learning in a family medicine clerkship. Teach Learn Med 2000;12(1):21-7. 21. Carney PA, Dietrich AJ, Eliassen MS, Pipas CF, Donahue DA. Differences in ambulatory teaching and learning by gender match of preceptors and students. Fam Med 2000; 32(9):620-5. 22. Carney PA, Pipas CF, Eliassen MS, et al. An analysis of students’ clinical experiences in an integrated primary care clerkship. Acad Med 2002;77:681-7.