Pipeline, Profession, and Practice Program: Evaluating ... - CiteSeerX

Pipeline, Profession, and Practice Program: Evaluating Change in Dental Education Ronald M. Andersen, Ph.D.; Pamela L. Davidson, Ph.D.; Kathryn A. Atchison, D.D.S., M.P.H.; Edmond Hewlett, D.D.S.; James R. Freed, D.D.S., M.P.H.; Judith-Ann Friedman, Ed.D.; Amardeep Thind, M.D., Ph.D.; John J. Gutierrez, B.A.; Terry T. Nakazono, M.A.; Daisy C. Carreon, M.P.H. Abstract: This article describes the conceptual and analytical framework that will be used to assess the effectiveness of the Pipeline, Profession, and Practice: Community-Based Dental Education Program. The evaluation will use a mixed method qualitative and quantitative data collection, analysis, and triangulation. Baseline measures are reported using data from the 2003 ADEA survey of dental school seniors. Baseline measures show the dental schools are confronting a major recruitment challenge that will require short and long pipeline efforts to attract and retain underrepresented and low-income (URM/LI) persons. Gaps were found between the perceptions of URM and non-URM students in the adequacy of the curricula. The majority of all seniors described the current extramural clinical rotations as positive experiences, but URMs were more likely to report the experience improved their ability to care for diverse groups. Dr. Andersen is Professor Emeritus, Department of Health Services, UCLA School of Public Health; Dr. Davidson is Associate Professor, Department of Health Services, UCLA School of Public Health; Dr. Atchison is Professor, UCLA School of Dentistry; Dr. Hewlett is Associate Professor, UCLA School of Dentistry; Dr. Freed is Clinical Professor Emeritus, UCLA School of Dentistry; Dr. Friedman is Director of the Workforce Development Center, West Los Angeles College; Dr. Thind is Associate Professor, Department of Health Services, UCLA School of Public Health; Mr. Gutierrez is Project Manager, Department of Health Services, UCLA School of Public Health; Mr. Nakazono is Programmer Analyst, Department of Health Services, UCLA School of Public Health; and Ms. Carreon is Research Associate, Department of Health Services, UCLA School of Public Health. Direct correspondence and request for reprints to Dr. Pamela L. Davidson, UCLA School of Public Health, Department of Health Services, CHS 31-293, 650 C.E. Young Drive South, Room 31-269CHS, Campus Box 951772, Los Angeles, CA 90095-1772; 310-825-7188 phone; 310-825-3317 fax; [email protected]. Key words: evaluation of dental education, dental health services, dental schools Submitted for publication 11/5/04; accepted 11/24/04

T

his article describes the methods that will be used to assess the effectiveness of the Pipeline, Profession, and Practice: CommunityBased Dental Education Program, sponsored by the Robert Wood Johnson Foundation (RWJF) and the California Endowment (TCE). We will describe the evaluation plan and present baseline data used in the evaluation from the American Dental Education Association (ADEA) survey of dental school seniors. This article will provide the methodological foundation for a series of manuscripts that will report the outcomes of a longitudinal evaluation of the Pipeline project. The Pipeline initiative was developed to address the critical shortage of oral health care services for underserved and disadvantaged populations. The nation’s dental care safety net is limited, and access problems are likely to become more acute in the next decade as the relative supply of dentists declines.1,2 The RWJF funded programs in eleven of the fifty-six U.S. accredited dental schools, and one year

February 2005

■

Journal of Dental Education

later, TCE funded four additional programs in dental schools in California. Additionally, TCE required all five of California’s dental schools to develop a regional recruitment program for underrepresented minorities and a health policy initiative to sustain the Pipeline program after funding ends. In total, the two foundations invested more than $25 million dollars in the Pipeline program and its evaluation. Additionally, the W.K. Kellogg Foundation and TCE contributed $1.6 million for financial aid to underrepresented minority and low-income students recruited by the Pipeline schools. The National Evaluation Team (NET) for the Pipeline project, based at UCLA, will determine if the three Pipeline objectives are achieved: 1) increase recruitment and retention of underrepresented minority and low-income (URM/LI) students; 2) revise didactic and clinical curricula to support communitybased educational programs; and 3) establish community-based clinical education programs that will provide dental students and residents with sixty days

239

of experience in this patient care environment.3 The specific evaluation questions the NET will address are: • Underrepresented and Low-Income (URM/LI) Recruitment: Did the Pipeline schools increase the recruitment, admittance, and retention of underrepresented minorities and/or disadvantaged students? • Revisions to the Curriculum: Was the curriculum revised, and were students prepared to provide culturally competent care to underserved and disadvantaged populations? • Extramural Clinical Rotations: Did students and residents spend an average of sixty days in patient-centered community delivery settings, and was more care delivered to underserved patients? What is the financial and clinical training effect of shifting clinical programs from traditional sites to community sites? • Practice Decisions: Upon graduation, what factors influence the decision to provide care to lowincome and underserved populations? • Health Policy: Did the schools work cooperatively, and were they successful in developing and implementing health policy initiatives to sustain the program? • Sustainability: Are the Pipeline program components developed under the RWJF and TCE grants likely to be sustainable after the program ends?

Conceptual and Analytical Framework The evaluation framework presented in Figure 1 was adapted from previous conceptual and analytical work on health care access.4-8 The framework is based on the assumption that educational programs are influenced by multiple inputs, including the contextual environment and stakeholder groups. The major components of the Pipeline program (recruitment, curricular revisions, and extramural clinical rotations) are expected to influence the structures, processes, and outcomes of dental education. Structure refers to Pipeline plans, personnel, and resources. Processes are the strategies and approaches used to execute Pipeline plans, programs, and services. Outcomes are the intermediate and longer-term results of the Pipeline project. Ongoing monitoring and program management are provided to improve the performance of Pipeline programs including NET feed-

240

back reports and coaching and technical assistance provided by the National Program Office (NPO) at Columbia University. In turn, all of these factors influence the longer-term outcomes, including practice decisions of dentists, program sustainability, and ultimately, sustainable improvements in dental care access for underserved communities.

Inputs The contextual environment influences Pipeline program design and implementation and dental care access in underserved populations. The contextual environment includes the influence of federal and state health policy, the dental care delivery system, university and school policies, and population characteristics (Figure 1). Federal, state, and local health policies influence dental care financing, the percent of the population with health insurance coverage, and resources available for training health professionals. Another policy variable is the percent of racial-ethnic representation in the state legislature, which can influence resources for medical and dental education and the availability of services for vulnerable populations. Delivery system variables include, for example, the number of federally qualified health centers providing dental care to the underserved. Examples of university and dental school contextual variables are the number of URM/ LI students attending the university. Contextual variables are used to understand the characteristics of community residents in a catchment area and their collective effect on access.4 For example, when large numbers of low-income, racial-ethnic minority groups and/or uninsured persons reside in a geographic area, access barriers are magnified for individuals competing for limited services and resources.9 All of these contextual characteristics may influence the structures, processes, and outcomes of the Pipeline program. Stakeholders are individuals, groups, or organizations having a vested interest in the progress of the Pipeline program.10 All of the stakeholders— RWJF, TCE, NPO, dental associations, students, faculty, dentists, allied health professionals, and underserved communities—are viewed as inputs that facilitate or impede changes to dental education and dental care access.

Pipeline Program Components The structure and process measures of URM/ LI recruitment are considered predisposing condi-

Journal of Dental Education ■ Volume 69, Number 2

February 2005

■

Journal of Dental Education

241

Policy Industry University and School Population Characteristics

RWJF TCE NPO Dental Associations Students Faculty Dentists Allied Health Professionals Underserved Communities

STRUCTURE PROCESS INTERMEDIATE OUTCOME

community preventive activities • Volume of community treatment services • Stakeholder satisfaction • Change in revenue • Practice setting selected by recent graduates

• Volume of outreach/

evaluations of courses • Stakeholder satisfaction • Competency objectives for public health dentistry

• Curriculum committee

courses and faculty

• Student evaluations of

• Number minorities accepted • Number minorities enrolled • Number minorities graduated • Satisfaction of graduates

NPO Coaching & Technical Assistance

regional Pipeline schools

• Develop strong partnerships • Established QA program • Established MIS program • Cooperative activities of

• Revisions to curriculum • Instructional methods used

regional Pipeline schools

• Tutor/retention • Number visits to schools • Number applying • Contact with partners • Cooperative activity of

NET Feedback Reports

clinical sites • Responsiveness to community needs assessment • Administrative structure (affiliation agreements)

• Adequate faculty FTE for

Extramural Clinical Rotations

• Number of didactic courses • Number of practical courses • Number elective courses • Learning outcomes • Number and type of faculty

Revisions to the Curriculum

• Partners for recruitment • Budget for recruitment

involved

• Number faculty/staff

diversity

• Comprehensive plan for

Underrepresented Minority & Low-Income (URM/LI) Recruitment

Figure 1. Conceptual and analytical framework

• • • • • • • • •

Stakeholders

• • • •

Contextual Environment

INPUTS

Dental Care Access

Sustainability

Practice Decisions

LONG-TERM OUTCOME

tions for dental care access. The evaluation examines the structure of the recruitment and retention programs, as well as the strategies and approaches used by the schools to increase URM/LI enrollment. The outcome includes the number of URM/LI students who are admitted to dental school, matriculate, and graduate each year. These intermediate term outcomes are linked to longer-term outcomes since increased numbers of minority providers increase the likelihood that underserved populations will receive care.11-13 Similar to recruitment, the structure and process measures of curriculum revision are considered predisposing to improved access for underserved populations. Curricular revisions are designed to improve, for example, provider patient communication skills and cultural competency. Curriculum committee leaders, faculty, and graduating seniors are the sources of information used to assess the perceived strengths/ weaknesses in the existing curriculum and barriers/ facilitating factors to curriculum change. These intermediate outcomes represent potentially beneficial influences on long-term access outcomes since students completing didactic and experiential courses may develop a greater willingness to serve disadvantaged populations as well as better communication skills for interacting with diverse individuals. Moving clinical training from the main school clinic to extramural rotation sites is expected to improve access because more dental students will be providing more services in underserved communities. We will examine the structure and process of developing these clinical rotation sites as well as outcomes of revenue generation and the productivity of students in extramural sites. Annually, the NET provides feedback reports to all Pipeline schools to help them continually improve their programs. The reports describe and compare Pipeline schools in the baseline year and document structure, process, and outcome changes over the duration of the project. In addition to feedback reports, the NET presents an overall progress report at the annual grantees meetings, summarizing evaluation findings and their implications. The NPO provides technical assistance to the funded schools through meetings, information sharing, program website, workgroups, and regular communications with the NET. Each year, the NPO convenes an annual grantees meeting(s) with a focused theme for improving the Pipeline program, for reporting comparative evaluation data, and for exposing grantees to best practices and innovative models

242

for Pipeline program components. The NPO conducts ongoing program monitoring to assess the progress of the schools in achieving their objectives. The other major NPO activity is developing and executing an annual communications plan.

Long-Term Outcomes The Pipeline initiative was developed to help address the critical shortage of oral health care for underserved populations. Thus, critical outcomes of the Pipeline initiative are the practice settings selected by recent graduates and the percentages of underserved populations these entry-level dentists expect to serve in their practice. Determinants of practice decisions will be investigated including demographic, social, and economic characteristics of seniors, their attitudes and beliefs, the influence of Pipeline program components, and the contextual environment. Based on qualitative and quantitative results, evaluators will assess the probability that Pipeline program components will be sustained after foundation funding ends. Additionally, statewide recruitment efforts and health policy interventions will be closely monitored in California and elsewhere to determine implications for sustainability and lessons learned worthy of national replication.

Methods Table 1 presents each data source, time schedule for data collection, and use in addressing the major evaluation questions. It also shows site visit interviews are being conducted to collect baseline data prior to Pipeline program implementation and subsequently to assess progress in achieving Pipeline goals. Interview data are collected using a uniform bank of questions for each of the three major program components: recruitment, curriculum, and extramural clinical rotations. For example, interview data will be used to assess the community-based dental education curriculum at baseline, the content of existing courses, plans for changing the curriculum, and barriers and enabling factors in proposing curricula changes. Implementation reports are being provided annually by all grantees. They include administrative data on the structure, processes, and outcomes of the Pipeline program components. For example, schools provide information on the number of extra-

Journal of Dental Education ■ Volume 69, Number 2

mural site rotations and the number of hours students are providing care in the community. Financial reports, also collected annually, show changes in revenues and expenses related to moving clinical education to community settings. These data have

obvious implications for sustaining communitybased dental education programs. ADEA conducts an annual survey of dental school seniors. The NET works closely with ADEA to modify the survey in an effort to capture changes

Table 1. Data sources Data Source and Description

Time Schedule

Evaluation Questions

Site Visit Interviews: Site visit interviews examine student recruitment and retention, curriculum development and revision, and extramural clinical rotations. Site visit interviews are recorded, transcribed, and coded using a peer-review process. Qualitative interview data are used to interpret quantitative findings and to capture models and strategies emerging from the Pipeline initiative that might not be otherwise uncovered using quantitative methods. The data will be used to develop individual case study reports and to conduct cross-site analysis.

RWJF: 2003, 2005, 2007 TCE: 2004, 2006

Recruitment Curriculum Clinical Services Practice Decisions Health Policy Sustainability

Implementation Reports (NPO/NET): The implementation report is a collaborative data collection effort conducted by the National Program Office (NPO) and the National Evaluation Team (NET). Uniform data collected in the report is used to monitor program development, implementation, and outcome indicators.

Annually (2003-07)

Recruitment Curriculum Clinical Services

Annual Financial Reports: Financial reporting will assess the changing financial structure of the dental education program as clinical training moves from traditional settings to communitybased practice.

Annually (2003-07)

Clinical Services Sustainability

ADEA Survey of Dental School Seniors: Each year the American Dental Education Association (ADEA) conducts a national survey of all dental school seniors in the United States. The survey data will be used to assess and compare changes in the dental school programs as perceived by successive cohorts of dental school seniors.

Annually (2003-07)

Recruitment Curriculum Clinical Services Practice Decisions

ADA Survey of Predoctoral Dental Education: Each year the American Dental Association (ADA) conducts a national survey of all accredited dental schools in the United States. UCLA will use enrollment data as a secondary data source to monitor dental student recruitment in Pipeline schools.

Annually (2003-07)

Recruitment

Faculty Survey: The faculty survey will be self-administered by a random sample of faculty to obtain perceptions on the changes occurring in the dental school related to the Pipeline initiative.

2004, 2006

Recruitment Curriculum Clinical Services Sustainability

Syllabi Collection: The NET will request copies of syllabi reflecting the community-based dental education (CBDE) curricula to describe new and existing courses taught by the Pipeline schools for preparing students for community-based practice.

RWJF: 2003, 2005, 2007 TCE: 2004, 2006

Curriculum

Clinical Information System (NET/NPO) used to determine the impact on services utilization and access as a result of the expansion of community-based clinical care by obtaining information on services provided to patients at the main school and extramural sites.

Annually (2003-07)

Clinical Services

Contextual Variables: Contextual variables measuring policy, delivery system, university, and population characteristics will be used to describe community-level factors associated with successful Pipeline program components.

2003, 2005

Recruitment Curriculum Clinical Services Practice Decisions Health Policy Sustainability

February 2005

■

Journal of Dental Education

243

in perceptions and preferences of graduating seniors related to the Pipeline program. Also, the annual American Dental Association (ADA) Survey of Predoctoral Dental Education serves as a critical source of information on the number of URM students entering dental schools. A faculty survey is being conducted twice during the Pipeline years. The faculty survey reflects items in the ADEA survey of dental school seniors, allowing the NET to assess the gaps between student and faculty perceptions of the Pipeline program. Course syllabi are collected during the site visit years to assess curricula revisions and to identify innovative courses and teaching methods. The NET and NPO provided technical assistance to the schools for developing clinical information systems containing a uniform set of data elements collected at the extramural rotation sites. These data will show the kinds and volume of services provided at the sites. Contextual variables are being constructed to measure the policy, delivery system, university and school, and population characteristics influencing the Pipeline program and outcomes.

Baseline Measures for Longitudinal Analysis In this section, we provide examples of measures that have been collected at baseline and will be followed over the course of the Pipeline Project. These measures are from the ADEA survey of dental school seniors collected annually using a selfadministered questionnaire distributed to graduating seniors in accredited U.S. dental schools. The survey collects information about students’ financing of dental education, graduating indebtedness, practice and postdoctoral education plans following graduation, decision factors that influenced postgraduation plans, and impressions on the adequacy of time that was directed to various areas of predoctoral instruction.14 Each school uses its own distribution and collection system to conduct the survey. Surveys are returned to ADEA for analyses and reporting. Fifty-three accredited U.S. dental schools returned surveys in 2003, resulting in an overall student response rate of 83.2 percent. This response rate is based on the total number of students who completed the survey from the fifty-three schools returning surveys divided by the total number of students graduating from those schools.

244

From this survey, selected questionnaire items were identified that best represent recruitment, curriculum, and clinical services. These are critical baseline measures the NET will track longitudinally for the next five years to examine changes in dental education associated with the Pipeline program. Ethnicity and parents’ income as reported by graduating dental students are used to identify URM/LIs. Hispanics, African Americans, and Native Americans were combined into an underrepresented minority (URM) group, while whites and Asian/Pacific Islanders were kept separate. Parents’ income was divided into less than or equal to $30,000; $30,000-$50,000; and greater than $50,000. Curriculum measures are based on the following questions: 1) do you believe the amount of time devoted to your instruction in each of the following areas was excessive, appropriate, or inadequate for cultural competency and social and behavioral determinants of health? 2) how would you rate how patients were treated as people (excellent, very good, fair, poor, very poor) in the main clinic? and in the extramural clinical rotations? and 3) you are prepared to integrate knowledge regarding cultural differences into treatment planning and care delivery (strongly agree, agree, disagree, strongly disagree). Clinical services measures included mean number of weeks students spent providing care in extramural rotations; whether number of weeks spent in last year at extramural rotations was inadequate; whether extramural clinic experience improved ability to care for diverse groups much or very much; and extramural clinical rotations were positive/very positive experiences. In Table 2, responses to the recruitment, curriculum, and clinical services items are reported for National Pipeline schools (ten funded Pipeline schools excluding those in California), California Pipeline schools (all five dental schools in California), non-Pipeline schools (all other dental schools in the United States and Puerto Rico), and all schools combined. Furthermore, all responses are broken out by race and parents’ income. ADEA data will be collected and analyzed each year to assess progress of the schools in attaining Pipeline objectives inferred from the perspective of graduating seniors. Recruitment Measures. In 2003, blacks (4 percent), Hispanics (5 percent), and Native Americans (1 percent)—the URMs for the Pipeline project— represent one-tenth of the total graduating class that completed the questionnaire; Asian/Pacific Islanders (26 percent) represent about one-fourth; and nonHispanic whites (63 percent) constitute a little less

Journal of Dental Education ■ Volume 69, Number 2

than two-thirds (Table 2). National Pipeline schools have proportionately more black (8 percent) graduates than non-Pipeline schools (3 percent). This is because Howard and Meharry, the two predominantly black dental schools in the United States, are both participating in the Pipeline program. Two percent of the graduates of the other eight National Pipeline schools are black. California Pipeline schools have a smaller proportion of all URMs (4 percent) and a larger proportion of Asian/Pacific Islanders (45 percent) than dental schools in the rest of the country. The Pipeline schools (National and California) have a slightly larger proportion (31 percent-32 percent) of graduates whose parents earned less than $50,000 than the non-Pipeline schools (27 percent). URM graduates are more likely to have parents with lower income, earning less than $50,000 (43 percent) than Asian/Pacific Islanders (37 percent) and especially whites (22 percent). Still, URM status is far from synonymous with low-income status as more than half (57 percent) of URMs’ parents earn more than $50,000. Curriculum Measures. One quarter (25 percent) of all seniors rated the time devoted to instruction in cultural competency as inadequate (Table 2). URMs were most likely to rate the time as inadequate (36 percent) while whites were least likely (21 percent) to do so. Perceptions of URM and white students were fairly comparable among school types (National Pipeline, California Pipeline, and non-Pipeline). Asian/PI students in California Pipeline schools, however, were less likely (24 percent) to believe time was inadequate than their counterparts in the National Pipeline (36 percent) and non-Pipeline (32 percent) schools. Sixteen percent of all seniors regarded time devoted to instruction in social and behavioral determinants of health as inadequate. URMs (23 percent) and Asian/Pacific Islanders (22 percent) were most likely to judge the instruction as inadequate while whites (13 percent) were less likely to do so. In describing how patients were treated as people at the main school clinic, 68 percent of all seniors reported patient treatment as very good/excellent. Results were comparable for the extramural clinic rotations, where 64 percent described this aspect of care as very good/excellent. Seniors in the California Pipeline schools were more likely to describe this aspect of patient treatment as very good/excellent than seniors in the National Pipeline schools for both the main school clinics (69 percent vs. 61 percent) and the extramural rotations (68 percent vs. 63 percent).

February 2005

■

Journal of Dental Education

Most seniors (87 percent) for all schools described themselves as prepared to integrate knowledge of cultural differences into treatment planning and this percentage varied little by race or income. However, URMs in the National Pipeline schools were the most likely (94 percent) and the URMs in the California Pipeline schools somewhat less likely to (81 percent) to describe themselves as prepared. Clinical Services Measures. Key measures of community-based dental education show that the median number of weeks students spent providing care in extramural rotations was three weeks for National Pipeline schools, four weeks for California Pipeline schools, and three weeks for non-Pipeline schools (Table 2). A higher percentage of seniors described the number of weeks spent in the last year at extramural rotations as inadequate in the National Pipeline schools (43 percent) compared to the California Pipeline schools (24 percent) and non-Pipeline schools (29 percent). About one-third (35 percent) of all seniors agreed the extramural clinic experience improved ability to care for diverse groups much/very much. Seniors in the California Pipeline schools were most likely to agree (40 percent). URMs across all schools were also more likely to agree (45 percent) than Asian/Pacific Islanders (33 percent) or whites (35 percent). URMs were generally more likely to report the extramural clinic improved their ability to care for diverse groups much/very much (45 percent) than Asian/Pacific Islanders (33 percent) or whites (35 percent). Finally, 65 percent of all seniors described the extramural clinical rotations as positive/very positive experiences. URM seniors were most likely to describe the experience as positive/very positive in both the National Pipeline schools (81 percent) and the California Pipeline schools (87 percent).

Conclusions This baseline article lays the foundation for a series of manuscripts to follow from the Pipeline program National Evaluation Team (NET). Evaluation questions were posed reflecting the major program components and intermediate and longer-term outcomes of the national program. The conceptual and analytical framework described here will be used to guide data collection and analysis longitudinally. A wide net has been cast to collect data from multiple stakeholder groups, who will report progress of the Pipeline initiative over the next five years. The

245

Table 2. Baseline measures from the ADEA 2003 survey of dental school seniors Pipeline: National (%)

Pipeline: Non-Pipeline California (%) (%)

Total1 (%)

(N=608)

(N=496)

(N=2574)

(N=3678)

8 4 2 27 27 2

0.4 3 1 45 50 2

3 6 1 22 66 2

4 5 1 26 63 2

Parents’ income: >$30,000 $30,001-$50,000 >$50,000

(N=600) 17 14 69

(N=483) 21 11 68

(N=2564) 13 14 74

(N=3647) 15 13 72

Parents’ income by race: URM $50,000 Asian/Pacific Islander $50,000 White $50,000

(N=84) 42 58 (N=159) 45 55 (N=343) 23 77

(N=17) 35 65 (N=215) 39 61 (N=242) 24 76

(N=251) 44 56 (N=557) 35 65 (N=1702) 21 79

(N=352) 43 57 (N=931) 37 63 (N=2287) 22 78

28 (N=574)

22 (N=473)

25 (N=2489)

25 (N=3536)

34 (N=83) 36 (N=138) 23 (N=339)

35 (N=17) 24 (N=210) 19 (N=237)

37 (N=249) 32 (N=540) 21 (N=1642)

36 (N=349) 31 (N=888) 21 (N=2218)

28 (N=174) 27 (N=394)

27 (N=145) 20 (N=317)

28 (N=651) 24 (N=1819)

28 (N=970) 24 (N=2530)

19 (N=573)

15 (N=473)

16 (N=2494)

16 (N=3540)

30 (N=83) 27 (N=137) 13 (N=339)

18 (N=17) 18 (N=210) 13 (N=237)

21 (N=252) 23 (N=541) 12 (N=1643)

23 (N=352) 22 (N=888) 13 (N=2219)

22 (N=173) 17 (N=394)

14 (N=145) 16 (N=317)

20 (N=653) 14 (N=1822)

19 (N=971) 15 (N=2533)

61 (N=565)

69 (N=469)

70 (N=2475)

68 (N=3509)

63 (N=82) 64 (N=132) 60 (N=337)

65 (N=17) 68 (N=208) 69 (N=235)

69 (N=250) 66 (N=536) 71 (N=1633)

67 (N=349) 66 (N=876) 69 (N=2205)

59 (N=169) 62 (N=390)

68 (N=142) 69 (N=316)

68 (N=640) 71 (N=1817)

67 (N=951) 69 (N=2523)

63 (N=441)

68 (N=430)

63 (N=2135)

64 (N=3006)

66 (N=65) 53 (N=104) 66 (N=261)

75 (N=16) 65 (N=197) 71 (N=208)

63 (N=230) 54 (N=463) 66 (N=1394)

64 (N=311) 57 (N=764) 67 (N=1863)

59 (N=145) 66 (N=293)

64 (N=135) 71 (N=285)

61 (N=557) 64 (N=1563)

61 (N=837) 65 (N=2141)

RECRUITMENT Race/ethnicity: URM (Black, Hispanic, Native American) Black Hispanic Native American Asian/Pacific Islander White Other

CURRICULUM Time devoted to instruction in cultural competency was inadequate By race: URM Asian/Pacific Islander White By income: $50,000 Time devoted to instruction in social and behavioral determinants of health was inadequate By race: URM Asian/ Pacific Islander White By income: $50,000 How patients were treated as people at the main school clinic was very good/excellent By race: URM Asian/ Pacific Islander White By income: $50,000 How patients were treated as people at the extramural clinical rotation was very good/excellent2 By race: URM Asian/ Pacific Islander White By income: $50,000

246

Journal of Dental Education ■ Volume 69, Number 2

evaluation will be a mixed method qualitative and quantitative data collection, analysis, and triangulation. ADEA baseline measures were reported for subsequent use in longitudinal analysis to track changes reported by successive cohorts of dental school seniors in Pipeline and non-Pipeline schools.

Not surprisingly, baseline measures show the dental schools are confronting a major recruitment challenge that will require short and long pipeline efforts to attract and retain diverse URM/LI persons. Gaps were found between the perceptions of URM and non-URM students in the adequacy of the cur-

Table 2. Baseline measures from the ADEA 2003 survey of dental school seniors (continued)

Prepared to integrate knowledge of cultural differences into treatment planning By race: URM Asian/ Pacific Islander White By income: $50,000 CLINICAL SERVICES Median number of weeks students spent providing care in extramural rotations By race: URM Asian/ Pacific Islander White By income: $50,000 Percent of students who report number of weeks spent in last year at extramural rotations was inadequate By race: URM Asian/ Pacific Islander White By income: $50,000 Extramural clinic experience improved ability to care for diverse groups much/very much2 By race: URM Asian/ Pacific Islander White By income: $50,000 Extramural clinical rotations were positive/very positive experiences2 By race: URM Asian/Pacific Islander White By income: $50,000

Pipeline: National (%)

Pipeline: Non-Pipeline California (%) (%)

Total1 (%)

86 (N=558)

88 (N=465)

87 (N=2469)

87 (N=3492)

94 (N=81) 80 (N=132) 86 (N=332)

81 (N=16) 91 (N=206) 86 (N=234)

84 (N=249) 88 (N=536) 87 (N=1628)

86 (N=346) 87 (N=874) 87 (N=2194)

79 (N=168) 89 (N=384)

88 (N=140) 88 (N=315)

87 (N=637) 87 (N=1814)

85 (N=945) 87 (N=2513)

3 (N=555)

4 (N=461)

3 (N=2436)

3 (N=3452)

3 (N=79) 3 (N=126) 3 (N=336)

3 (N=16) 4 (N=206) 3 (N=230)

4 (N=248) 3 (N=521) 4 (N=1615)

4 (N=343) 3 (N=853) 3 (N=2181)

4 (N=165) 2 (N=384)

4 (N=142) 3 (N=309)

3 (N=625) 3 (N=1795)

4 (N=932) 3 (N=2488)

43 (N=559)

24 (N=467)

29 (N=2460)

30 (N=3486)

46 (N=80) 39 (N=128) 44 (N=337)

31 (N=16) 22 (N=209) 26 (N=233)

25 (N=247) 28 (N=530) 29 (N=1628)

30 (N=343) 28 (N=867) 31 (N=2198)

38 (N=169) 45 (N=384)

27 (N=143) 23 (N=313)

31 (N=635) 28 (N=1808)

32 (N=947) 30 (N=2505)

36 (N=424)

40 (N=418)

34 (N=2090)

35 (N=2932)

40 (N=62) 28 (N=96) 38 (N=256)

80 (N=15) 41 (N=192) 38 (N=202)

44 (N=226) 31 (N=449) 34 (N=1372)

45 (N=303) 33 (N=737) 35 (N=1830)

33 (N=137) 38 (N=284)

45 (N=132) 38 (N=277)

38 (N=544) 33 (N=1533)

38 (N=813) 34 (N=2094)

69 (N=425)

65 (N=419)

64 (N=2093)

65 (N=2937)

81 (N=62) 57 (N=96) 71 (N=256)

87 (N=15) 64 (N=193) 66 (N=202)

66 (N=226) 58 (N=449) 65 (N=1374)

70 (N=303) 59 (N=738) 66 (N=1832)

60 (N=137) 73 (N=285)

62 (N=133) 68 (N=277)

65 (N=543) 64 (N=1537)

63 (N=813) 65 (N=2099)

1

Total number of surveys returned was n=3697; missing responses were excluded from this table. Those students who 1) did not report any weeks or 2) reported more than 52 weeks in an extramural rotation were excluded from this measure. 2

February 2005

■

Journal of Dental Education

247

ricula, with URM and Asian/PI students reporting the need for more instruction in cultural competency and the social and behavioral determinants of health than whites. The majority of all seniors described the current extramural clinical rotations as positive experiences, but URMs were more likely to report that the extramural clinical experience improved their ability to care for diverse groups.

Acknowledgments We are grateful to our foundation officers who have provided expert advice on the design and implementation of this national evaluation: Laura Leviton, Ph.D., and Mary Ann Scheirer, Ph.D., The Robert Wood Johnson Foundation; and Diane Manuel, Ph.D., The California Endowment. Additionally, Richard Weaver, D.D.S., M.S.D., Associate Director of the Center for Educational Policy & Research at the American Dental Education Association, has generously collaborated with the UCLA evaluation team to redesign the annual survey of dental school seniors to measure changes in students’ perceptions of dental schools during the Pipeline years.

REFERENCES 1. Health Resources Center, American Dental Association. Dental workforce model 2000-2020. Chicago: American Dental Association, 2002. 2. Guay HG. Access to dental care: the triad of estimated factors in access-to-care programs. J Am Dent Assoc 2004;135(6):779-85. 3. Bailit H, Formicola A, Herbert K, Stavisky J, Zamora G. The origins and design of the dental pipeline program. J Dent Educ 2005;69(2):232-8.

248

4. Andersen RM, Davidson PL. Improving access to care in America: individual and contextual indicators. In: Andersen RM, Rice TH, Kominski GF, eds. Changing the American health care system: key issues in health services, policy and management. San Francisco: JosseyBass, 2001. 5. Andersen RM, Davidson PL. Ethnicity, aging, and oral health outcomes: a conceptual framework. Adv Dent Res 1997;11:203-9. 6. Chen M, Andersen R, Barmes D, Leclercq MH, Lyttle CS. Comparing oral health care systems: a second international collaborative study. Geneva: World Health Organization, 1997. 7. Oral health in America: a report of the surgeon general. Rockville, MD: U.S. Department of Health and Human Services, National Institute of Dental and Craniofacial Research, National Institutes of Health, 2000. 8. Davidson PL, Andersen RM, Hilberman DW, Nakazono TT. Transforming health services management education and development: a challenge for the new millennium. J Health Adm Educ 2000;18:63-110. 9. Davidson PL, Andersen RM, Wyn R, Brown ER. A framework for evaluating effects of the safety net and other community-level factors on access to health care. Inquiry 2004;41:21-38. 10. Rossi PH, Lipsey MW, Freeman HE. Evaluation: a systematic approach. Thousand Oaks, CA: Sage Publications, 2003. 11. Mofidi M, Konrad TR, Portefield DS, Niska R, Wells B. Provisions of care to the underserved population by National Health Service Corps alumni dentists. J Pub Health Dent 2002;62(2):102-8. 12. Weaver RG, Haden NK, Valachovic RW. Annual ADEA survey of dental school seniors: 2001 graduating class. J Dent Educ 2002;66(10):1209-22. 13. Weaver RG, Haden NK, Valachovic RW. Annual ADEA survey of dental school seniors: 2002 graduating class. J Dent Educ 2002;66(12):1388-404. 14. Weaver RG, Haden NK, Valachovic RW. Annual ADEA survey of dental school seniors: 2003 graduating class. J Dent Educ 2004;68(9):1004-27.

Journal of Dental Education ■ Volume 69, Number 2