Basic Research Journal of Social and Political Sciences Vol. 1(3) pp. 65-70 November 2012 Available online http//www.basicresearchjournals.org Copyright ©2012 Basic Research Journal
Full Length Research Paper
How masters of medical education program could be evaluated in their working setting 1
Saeideh Ghaffarifar*, 2Abolghasem Amini, 3Fathieh Mortazavi, 4Sedigheh Sadat Tavafian, 5 Fatemeh Sadeghi-Ghyassi
1
Medical education Department, Student of Health Education. Research Center of medical Education, EDC, Tabriz University of Medical Sciences. 2 Department of Medical Education, Research Center of medical education, EDC, Tabriz University of Medical Sciences. 3 EDC, Shahid Beheshti University of Medical Sciences. 4 Department of Health Education, Medical School, University of Tabiat Modarres 5 Iranian Center for Evidence-Based Medicine, Tabriz University of Medical Sciences. *Corresponding author email:
[email protected] Accepted 20 November, 2012
Purpose: Master course of medical education in Iran has been launched since 2002 to improve the quality of health services. Reports received from Iranian medical universities show most of graduates in this field are not aware of their professional role. In this study, we explain how a valid questionnaire for professional role evaluation of medical education masters could be designed and validated. Method: The primary draft of a researcher-maid questionnaire was designed in 2011 based on a literature review. To do validation process, the view points of 20 expert were obtained to determine content and face validity of the questionnaire through Content Validity Ratio (CVR) and Content Validity Index (CVI) at the levels of >0.51 and >0.79 respectively. We applied the index of average expressed variance to report convergence validity of items being acceptable at the level of at least 0.5 doing an exploratory factor analysis with the cut point of 0.5 to determine dimensions of the questionnaire. Results: According to 14 experts’ comments on the quantity and quality of the questionnaire, 9 items which did not meet preassumed criteria were eliminated, so 16 items remained in the questionnaire. Bartlett’s test of Sphericity (Sig: 0.000) approved appropriateness of the data to do exploratory factor analysis that led to extraction of 8 factors with Eigen value more than 0.941. Conclusions: The findings of the study showed the questionnaire could assess the professional role of masters graduated from medical education program in Iran. However, further researches are needed to improve the questionnaire in future. Keywords: Medical education; professional role; graduate; evaluation tool; master
INTRODUCTION Master program of Medical Education in Iran was launched in 2002, and despite being young, very soon, many health policy makers hoped to basically solve the health problems by the help of scholars and graduates of this field. So-called trusts led politicians of health care system to setting objectives like “all Iranians should know all the different aspects of health including physical, emotional, psychological, social and spiritual, and achieve optimal level of health”, and “they should obtain a positive attitude toward health promotion in order to receive the most improved health services necessary to the prevention, treatment and rehabilitation with the best
quality, easiest way and the fastest time when needed”. It seems health policy makers have considered the weight of the objective of getting better quality of services higher than other above-mentioned goals. American Association of Medical Colleges (AAMC) believes that a community-centered health system provides rich and meaningful learning experiences for students of medical sciences. Such health-education systems provide simultaneously integrated ambulatory and hospital cares across the community. 1 With a deep professional look from medical educationists’ points of view on the objectives of launching the master
Ghaffarifar et al. 65
program of medical education, a major requirement reveals itself that medical students should be taught in the line of the real needs of the community in order to learn the three levels of prevention (Primary: concerned with health promotion activities that prevent the actual occurrence of a specific illness or disease , Secondary :promotes early detection or screening and treatment of disease and limitation of disability. This level of prevention is also called HEALTH MAINTENANCE. Tertiary: -directed towards recovery or rehabilitation of a disease or condition after the disease has been developed) and serve them at the community centers. 1 To fulfill this requirement, first, curriculums and course plans in all fields of medical sciences should necessarily be revised. By such doing, highly motivated selected candidates will make significant changes after graduation whether in the learning context, input, output and learning-teaching processes. Second, students during this program should be able to obtain competencies of effective, efficient and targeted learning, and easily achieve learning facilities which they need. In so-called education system, medical students' motivation and intention to acquire the necessary competencies is expected to reinforce even further than which has been anticipated in their educational curriculum, and the teachers issue the license to graduating students through valid, reliable and fair evaluations whether formative or summative. It means, the certificates will be issued to the capable graduates who will solve non-previously faced problems at unfamiliar educational settings. In other words, medical education masters graduated from revised curriculum are expected to make the best decisions with the help of reordering their previous experiences and making relationship with new ones. The question that arises here is that who will be responsible for these logical changes? Or in order to achieve a community- centered health education system, who(s) will seriously reflect positive applied and scientific experiences of universities of medical sciences and share them with others? If the answer accurately points to the graduates of medical education, the next question is, whether graduates in this field acquire the competencies needed for performing this great responsibility during their studying? More importantly, are they well aware of their professional role? Data hiding in the shadows of previous inductive research which reviewed medical education master degree curriculum in Iran in 2010, and reports from other universities in Iran (Dadghostar et al., 2011; Javdani, 2010; Fasihi et al., 2004; Dehghani et al., 2007; Bakhshi et al., 2002; Javadi et al., 2002) created this hypothesis in our mind, that most master graduates of medical education are not aware of their professional roles. Therefore, it is necessary to test this hypothesis, to investigate if master program of medical education have addressed, established and improved the competencies
of medical education graduates to fulfill their professional roles or not. This study describes the process to design and validate a questionnaire in order to evaluate competencies of masters of medical education on their own points of view as a part of a master-thesis.
METHOD In this study, the operational definition of professional role was determined reviewing the literature (Kachur, 2012; Wojtczak, 2012; Whitcomb, 2005, 2007; Coulehan, 2005; Khadem et al., 2008; Farzinpour et al., 2010; Ferguson et al., 2002; Lurie et al., 2009) and a draft of the items of the researcher- maid questionnaire were made available. A panel of 20 experts in the field of medical education judged the content validity of the questionnaire. In this way, those 20 experts were asked to specify the domain that the questionnaire measures .They were also asked to determine the area each item of the questionnaire evaluates. Then, they gave us feedback how the questionnaire could assess the professional role of masters graduated from medical education course, in addition to their comments to revise some items, even to omit a few of them. Qualitative evaluation of the questionnaire was also done based on their feedback receiving necessary corrections regarding grammar, wording, item allocation, and scaling of the items. Content Validity Ratio (CVR) and the Content Validity Index (CVI) were used to evaluate quantitative content validity of the questionnaire. Content validity of the items was rated based on a threepart scale and minimum validity of 0.51. The CVI score related to simplicity, relevance and clarity of each item was calculated on a four-part Likert scale, and items with CVI score higher than 0.79 were considered acceptable for the study. (Rimm and Jerusalem, 1999; Scholz et al., 2002; Thurstone, 1947; Hajizade and Asghari, 2011). The questionnaire was e-mailed to 112 masters of medical education who we had their e-mail address. They were asked to participate in this study and send us the completed one if they were willing to anonymously cooperation. The results of the study were obtained using SPSS.16 (Statistical Package for the Social Sciences, version 16). The convergence validity of the construct of professional role in this research was determined by Average Variance Explained (AVE), and its minimal acceptable level was assumed to be at least 0.5. Exploratory factor analysis (EFA) was used to determine dimensions of the construct of professional role. To assess the suitability of present data to perform an exploratory factor analysis Bartlett’s Test of Sphericity was done. In order to data reduction, the number of factors for extraction was considered eight factors, Varimax rotation method was used, and absolute values less than 0.5 were suppressed. Maximum Iteration for
66. Basic Res. J. Soc. Polit. Sci.
Table 1. Statistics presents content and face validity of the questionnaire to evaluate professional role of 94 graduates of medical education according to a panel of 14 experts Item Number 1 2 3 4 5 6
7 8 9
10 11 12 13 14 15 16
Studying in this program has increased: My……. Skills respecting the honesty in medicine. Skills and capabilities regarding altruism. Skills and capabilities regarding respectfulness to others. Interest to co-operate with university and community service providers. Motivation for learning to respond to the needs of the community. Motivation for proposing new methods of teaching and learning to faculty members. Awareness of my roles and responsibilities in health system. Knowledge about morals of practicing in this profession Active participation in curriculum planning, revisions when necessary in college or university Contribution on making decisions in educational departments. Commitment to make proper changes in the learning environment. Contribution on production of articles in medical education. Contribution on provision of research projects in medical education. Readiness to accept my new professional role. Knowledge about interdisciplinary cooperations. Professional status and popularity.
CVR 0.57
CVI CVI CVI CVI Impact (Simplicity) (Relevance) (Clarity) (Scale) Score 0.785 0.85 0.92 0.85 4.19
1 1
0.92 0.92
0.92 0.92
0.71 0.92
0.85 0.92
4.44 4.67
0.71
0.85
0.64
0.92
0.80
3.30
0.86
0.85
0.71
0.92
0.83
3.76
0.86
0.85
0.92
0.71
0.83
3.30
1
0.92
0.64
0.92
0.82
4.02
0.86
0.92
0.57
0.92
0.80
2.80
0.71
0.785
0.92
0.85
0.85
2.25
0.71
0.92
0.64
0.85
0.80
3.30
0.86
0.785
0.92
0.85
0.85
4.30
0.71
0.92
0.785
0.71
0.80
3.31
0.57
0.85
0.785
0.92
0.85
3.30
0.71
0.92
0.92
0.64
0.83
4.19
0.57
0.92
0.785
0.71
0.80
2.98
0.71
0.785
0.64
0.85
0.758
2.25
convergence was considered 25. (Rimm and Jerusalem, 1999; Scholz et al., 2002; Thurstone, 1947; Hajizade and Asghari, 2011).
RESULTS The draft of our researcher-made questionnaire with 25 items was designed presenting professional characteristics of masters of medical education. Those characteristics were extracted from existing literature in the field of medical education, and were included: altruism, accountability, dutifulness, loyalty, trustworthiness, honesty, respectfulness for others and providing the most efficient and effective service in the health care system based on what she/he has been taught. Equal and more similar items were excluded from the initial questionnaire based on the results of 20 experts’ opinions. Because of the lack of consensus on the judgment of experts, the opinions of 14 experts were used in the rest of study in order to qualitative and
quantitative validation of the questionnaire. In this way, 9 Items of the 25 original items did not meet pre-assumed criteria and were omitted due to lack of a minimum content validity based on judgment of 14 experts. It remained 16 items, and content validity index (CVI) was calculated for each item based on mean CVI for relevance, clarity and simplicity of that item, according to the experts' comments and recommendations. (Table 1) 94 masters of medical education completed the questionnaire in a volunteer and anonymous type of participation. The results of the study were obtained using SPSS.16 based on the Maximum Iteration for convergence equal to12. Statistics of Kaiser-Meyer-Olkin (KMO) in this study was 0.474.The level of Bartlett's test of sphericity was significant (Sig: 0.000) which showed the suitability of the data of the study to perform an exploratory factor analysis. With regard to the extraction of eight factors, eight factors with Eigen values of 0.941 were extracted. Scree plot (Figure1) shows the adequacy of 8 items to evaluate the construct of professional role.
Ghaffarifar et al. 67
Figure 1. Eigen values of 16 items of the questionnaire to evaluate the professional role of masters of medical education
Table 2. Rotated Component Matrix of the construct of professional role for its evaluation in masters of medical education ‡ § ¶ Item q2 q5 q16 q15 q6 q9 q7 q1 q11 q8 q3 q4 q14 q12 q13 q10
Component 1 .763 .745 -.608
2
3
4
5
6
7
8
-.786 .643 .857 .585 .807 -.611 .773 .575 .843 .863 .887
‡ Extraction Method: Principal Component Analysis. § Rotation Method: Varimax with Kaiser Normalization. ¶ Rotation converged in 12 iterations.
Item 13 had more communality (0.813) with the construct of professional role, and the lowest communality belonged to item 16 (0.546).The Total Variance Explained (TVE) was equal to 69.286%. Table 2 shows component matrix after Varimax rotation.14 experts of medical education in our study re-
arranged component matrix, as much as it was possible for them. Table 3 provides re-structured component matrix regarding their points of view. It led to initial designing of the questionnaire of our research to assess the professional role of masters graduated from medical education program in Iran.
68. Basic Res. J. Soc. Polit. Sci.
Table 3. Eight factors which measure the professional role of medical education graduates ¶ Factor 1
2 3
4 5 6 7 8
Item Number 2 5 16 15 6 9 7 1 11 8 3 14 4 12 13 10
Studying in this program has increased: My……. Skills and capabilities regarding altruism. Motivation for learning to respond to the needs of the community. Professional status and popularity. Knowledge about interdisciplinary co-operations. Motivation for proposing new methods of teaching and learning to faculty members. Active participation in curriculum planning, revisions when necessary in college or university. Awareness of my roles and responsibilities in health system. Skills respecting the honesty in medicine. Commitment to make proper changes in the learning environment. Knowledge about morals of practicing in this profession. Skills and capabilities regarding respectfulness to others. Readiness to accept my new professional role. Interest to co-operate with university and community service providers. Contribution on production of articles in medical education. Contribution on provision of research projects in medical education. Contribution on making decisions in educational departments.
CONCLUSIONS On one hand, the findings of this study create a mentality that the items of the designed questionnaire could reflect appropriateness of the tool to evaluate the professional role of masters of medical education. On our points of view, taking some scientific steps supports this idea, while keeping their all strengths and weaknesses in our mind. First, according to evidences, fixing a theoretical (conceptual) and functional (operational) definition of a construct, specifying the domain that a questionnaire should measure in addition to determining the area each item of a questionnaire should evaluate is the first step in designing of a questionnaire. (Rubio et al., 2003). It was the same as we did in this study. Based on a literature review, we determined how each graduate in the field of medical education should behave in a real working setting. In other words, a professional master of medical education in our researcher-maid questionnaire was introduced with traits of altruism, accountability, dutifulness, loyalty, trustworthiness, honesty, respectfulness for others and providing the most efficient and effective service in the health care system based on what she/he has been taught. Second, as evidences express judgment of a minimum of 2 and maximum of 20 professional experts are needed to content validation of a questionnaire. (Rubio et al., 2003; Grant and Davis, 1997). So, we asked a panel of 20 experts to judge on our researcher-made questionnaire draft of 25 items. Third, studies show content validation of a questionnaire requires diagnosis of irrational, nonerelated and none- quality items. (Rubio et al., 2003; Grant and Davis, 1997). Hence, 9 out of the 25 items in this study were eliminated due to lack of content validity based
on judgment of a minimum of 14 experts. On the other hand, Results of construct validation of the questionnaire witness the fact that after the removal of nine inappropriate items, 16 remaining items are able to make a tool to measure the construct of professional role in the behavior of masters of medical education. The total variance explained (TVE) equal to 69.286% confirms convergence validity of this construct. It means the eight extracted factors could express 69% of the changes of the construct of professional role. In all, our research, by using Google and Yahoo search engines in Persian and English language databases, did not end to finding similar questionnaire to evaluate professional role of masters of medical education. It also gives us encouragement to think that the questionnaire was designed in this study could help health policy makers to evaluate masters’ professional role to assess how much medical education curriculum planning and its implementation has met the pre-determined goals and objectives in training capable masters to fulfill community demands. So, when it comes to policy makers’ expectations that “all Iranians should know all the different aspects of health”, this questionnaire could help them if medical education graduates are able to help the people to manage their health problems or not. Besides, this questionnaire would help us to test our previously mentioned hypothesis of that most of masters graduated from medical education program are not aware of their professional role.
Recommendations and limitations Although this study had some good positive points, and led to the development of a valid tool for evaluating the professional role of medical education graduates, but like
Ghaffarifar et al. 69
all other studies was not without any limitation. As noted in the test results statistics of Kaiser-MeyerOlkin (KMO) in this study was low (
