Vice Dean, College of Science â De La Salle University. INTRODUCTION: The data driven decision making (DDDM) process is achievement oriented and may ...
Some Factors Affecting the Student Evaluation Ratings of Biology Faculty at De La Salle University. JSR Carandang VI* and FF Co** Abstract A measure used at DLSU as an indicator of an academic department’s enhanced performance is when 85% of its faculty receives a student evaluation rating of Very Satisfactory. This present paper sought to find out which parameters affect student evaluation ratings and use this information to develop a strategy for the Biology Department to constantly get a Very Satisfactory student rating. Different information contained in the student evaluation survey data of Biology faculty during the last two academic years (AY 2010-2011 and AY 2011-2012) were collected and tested against the corresponding student evaluation rating and one another using the T-test and analysis of variance (1 factor, 2 factors and 3 factors). Results show that faculty status, gender, faculty rank, term, and academic year have no effect on the mean rating. However, the mean rating of classes taught under the transmissive mode is significantly higher than those under the transformative mode. Key Words: student evaluation, measuring academic department performance, data driven decision making * Chair, Biology Department – De La Salle University. Presentor and Corresponding Author ** Vice Dean, College of Science – De La Salle University
INTRODUCTION: The data driven decision making (DDDM) process is achievement oriented and may involve making a decision for tomorrow based on today’s results (Wagner, 2005 and Sherrod et al., 2010). DDDM is based on the premise that data are key basis of information to steer progress at all levels of the education system and to hold individuals and groups responsible (Marsh et al., 2006 and Park and Datnow, 2009). Likewise, DDDM is linked to accountability, school improvement and educational reforms. Highly successful schools and classroom teachers have been using DDDM for years and appreciate the usefulness of reporting their work across all levels of the educational system (Mandinach and Jackson, 2012, California Comprehensive Center, 2006, Katz and Earl, 2006 and Meselt, 2004). DDDM is one of the management tools employed at De La Salle University (DLSU). The principal task of administrators in DDDM is to help maintain a considerate, trusting culture in which data can be collected, analyzed and used to boost student achievement (McREL (2003). Part of maintaining a culture that supports the use of data is making sure that data are frequently assessed (Mandinach and Jackson, 2012). At DLSU, effective teaching performance is a major concern during hiring and promotions. Another key aspect of DDDM involves looking at information over an extended period of time (Salpeter, 2004). At DLSU, student evaluation is conducted online every term for part timers and for full timers, at least once each academic year. An average rating of Very Satisfactory (at least a 4.0000) is required for promotion and tenure. The average rating is derived from the scores given by the students’ evaluation conducted by the
Institutional Testing and Evaluation Office or ITEO (see Table 1 for evaluation ratings and interpretations), peers and the Chair of the department. Table 1. Institutional Testing and Evaluation Office (ITEO) Student Evaluation Ratings
A measure being used by DLSU as an indicator of an academic department’s enhanced performance is when consistently 85% of its faculty receives a rating of Very Satisfactory. This present paper will seek to find out which parameters affect the student evaluation rating of the Biology department and to use this information to develop a strategy to enable the department to consistently get a Very Satisfactory rating from the students. The Problem and Hypothesis An objective of the faculty development program of the Biology Department at DLSU for the AY 2012-2013 is to have 85% of the faculty being able to receive a performance rating from student evaluations a rating of at least 4.0. The results released last 25 April 2012 of the latest ITEO Survey conducted during the 3rd Term of AY 2011-2012 (see Table 2), indicated only one out of nine (88.9%) of the surveyed faculty rated lower than 4.0 in subjects taught using the transmissive learning method but one out of three (66.7%) of those using the transformative method rated at least 4.0. The ITEO Survey is conducted every term and is accomplished on line by the respondents. Aside from the ratings, survey reports also include data on the term and school year when the survey was conducted, the gender of the faculty, the faculty rank, the faculty status (part time or full time), the class type (laboratory or lecture) and the delivery mode (transformative or transmissive). To be able to achieve the objective of 85% of the faculty receiving at least a 4.0 rating, it will be useful to find out which of these parameters has an effect on the rating received by the faculty. The findings can then be used to devise a strategy so that the objective can be achieved. Research Questions 1. Which parameters affect the ITEO Student Evaluation rating received by Biology faculty? 2. In which mode of teaching, transformative or trasnsmissive learning does Biology faculty perform better as indicated by the comparative ITEO Student Evaluation Rating they have received during the last two consecutive academic years?
3. Is the Biology faculty improving in terms of their average ITEO Student Evaluation Rating for the past two consecutive academic years? 4. What approach should the Biology Department take in order to achieve the target of 85% of the faculty receiving at least a rating of 4.0 from the ITEO Student Evaluation conducted every term? Table 2. Percentage of Biology faculty with ITEO Student Evaluation Rating of at least 4.0 for two consecutive academic years.
METHODOLOGY: The results of the ITEO Student Survey of Biology Faculty conducted during the last two academic years (AY 2010-2011 and AY 2011-2012) were collected and subjected to statistical test using Statistica software. The different information contained in the survey report (faculty status, faculty rank, gender, class type, delivery mode, term and academic year) were tested against the ITEO rating and one another using the T-test and analysis of variance (1 factor, 2 factors and 3 factors).
RESULTS AND DISCUSSION: The AY 2010-2011 data on the possible effect of faculty status, gender, class type, and delivery mode were tested against the ITEO rating of Biology faculty and one another using the T-test. The effect of faculty rank and school term were tested against the ITEO rating and one another using the analysis of variance. The following were determined as shown in Table 3: 1. The t-test for equality of two means indicates that faculty status (FT vs. PT) has no significant effect on the mean ITEO rating (p = 0.188350). 2. The t-test for equality of two means shows that gender (F vs. M) has no significant effect on the mean ITEO rating (p = 0.275477).
3. The t-test for equality of two means shows that class type (lecture vs. lab) has a significant effect on the mean ITEO rating (p = 0.028935), with laboratory classes having a significantly higher mean ITEO rating (4.4579) than lecture classes (4.2156). 4. The t-test for equality of two means shows that the mode of delivery (transmissive vs. transformative) has a significant effect on the mean ITEO rating (p = 0.015713), with the transmissive mode having a significantly higher mean ITEO rating (4.4455) than the transformative mode (4.1842). 5. The mean ITEO ratings have no significant difference across the three faculty ranks as shown by the results from the one-factor analysis of variance (ANOVA) (p = 0.877262), wherein the mean ratings are within the narrow range from 4.2834 to 4.3441 (Very Satisfactory). 6. The results from the One-Factor ANOVA show that the mean ITEO rating across the three terms of AY 2010-2011 have no significant difference (p = 0.944205). The mean ITEO ratings are within the range from 4.2667 to 4.3207 (Very Satisfactory). Table 3. Summary table of the results of the statistical test conducted to find out which parameters affect the ITEO Student Evaluation rating received by Biology faculty during the AY2010-2011.
Likewise, the AY 2011-2012 data on the possible effect of faculty status, gender, class type, and delivery mode were tested against the ITEO rating of Biology faculty and one another using the T-test. The effect of faculty rank and school term were tested against the ITEO rating and one another using the analysis of variance. The following were determined as shown in Table 4: 1. The t-test for equality of two means indicates that faculty status (FT vs. PT) has no significant effect on the mean ITEO rating (p = 0.419745).
2. The t-test for equality of two means shows that gender (F vs. M) has no significant effect on the mean ITEO rating (p = 0.639880). 3. The t-test for equality of two means shows that class type (lecture vs. lab) has no significant effect on the mean ITEO rating (p = 0.596005). 4. The t-test for equality of two means shows that the mode of delivery (transmissive vs. transformative) has a significant effect on the mean ITEO rating (p = 0.000024), with the transmissive mode having a significantly higher mean ITEO rating (4.5515 = “Outstanding”) than the transformative mode (4.1901 = “Very Satisfactory”). 5. The mean ITEO ratings have no significant difference across the three faculty ranks as shown by the results from the one-factor analysis of variance (ANOVA) (p = 0.145532), wherein the mean ratings are within the range (4.3122 to 4.6361). Although lecturers tend to have the highest mean ITEO rating (4.6361 = “Outstanding”) compared to the other two categories (Asst Prof: 4.3122 = “Very Satisfactory”; Assoc/Full/Visiting Prof: 4.3564 = “Very Satisfactory”), the sample size of only 5 lecturers prevents us from making a reliable generalization that lecturers have a significantly higher mean ITEO rating than those of the other two categories of faculty ranks. 6. The results from the One-Factor ANOVA show that the mean ITEO rating across the three terms of AY 2011-2012 have no significant difference (p = 0.343074). The mean ITEO ratings are within the range from 4.3136 to 4.4701 (Very Satisfactory). Table 4. Summary table of the results of the statistical test conducted to find out which parameters affect the ITEO student evaluation rating received by Biology faculty during the AY2010-2011.
The combined data of AY 2010-2011 and 2011-2012 were also analyzed. The possible effects of the academic year and term on the ITEO rating of the Biology faculty were analyzed using two factor analysis of variance (see Table 5). Results indicate that the mean ITEO ratings have no significant difference between the two academic years (p = 0.400610) and across the three terms (p = 0.607291) as shown by the results from the two-factor analysis of variance (ANOVA). Furthermore, there is no significant “academic year vs. term” interaction effect (p = 0.638414), wherein the mean ratings are all within the range (4.2667 – 4.4701) or “Very Satisfactory”. Table 5. Summary table of the results of the statistical test conducted to find out whether there is a difference in the ITEO student evaluation rating received by Biology faculty during two consecutive academic years and in different terms.
The possible effects of the delivery mode and academic year on the ITEO rating of the Biology faculty were analyzed using two factor analysis of variance (see Table 6). The results from the two-factor ANOVA indicate that there is no significant interaction effect between the delivery mode and academic year (p = 0.440482) on the mean ITEO rating and only the main effect of delivery mode has a significant effect (p = 0.000005) on the mean ITEO rating. Note from the table of descriptive statistics that the overall mean ITEO rating (AY 2010-11 and AY 2011-12 combined) for the transmissive mode is significantly higher than the TL mode (4.5004 vs. 4.1875). The same pattern can be seen for each academic year for transmissive vs. TL (AY 2010-11: 4.4455 vs. 4.1842 and AY 2011-12: 4.5515 vs. 4.1901). Furthermore, the possible effects of the class type and academic year on the ITEO rating of the Biology faculty were analyzed using two-factor analysis of variance (see Table 7). The results from the two-factor ANOVA indicate that there is no significant interaction effect between the class type and academic year (p = 0.182876) on the mean ITEO rating and only the main effect of class type (p = 0.036705) has a significant effect on the mean ITEO rating. From the table of descriptive statistics, note that the overall mean ITEO rating (AY 2010-11 and AY 2011-12 combined) for the lab classes is significantly higher than lecture classes ( 4.4147 vs. 4.2718). The same pattern can be seen for each academic year for lab vs, lecture classes (AY 2010-11: 4.4579 vs. 4.2156 and AY 2011-12: 4.3840 vs. 4.3297).
Table 6. Summary table of the results of the statistical test conducted to find out whether there is a difference in the ITEO student evaluation rating received by Biology faculty during two consecutive academic years and using two different modes of teaching.
Table 7. Summary table of the results of the statistical test conducted to find out whether there is a difference in the ITEO student evaluation rating received by Biology faculty during two consecutive academic years and in two class types.
Lastly, the possible effects of the class type, delivery mode and academic year on the ITEO rating of the Biology faculty were analyzed using three-factor analysis of variance (see Table 8). The results from the three-factor ANOVA indicate that delivery mode is the only significant factor (p = 0.000064) on the mean ITEO rating at the = 5% significance level. Class type (laboratory vs. lecture) could be considered as a significant factor (p = 0.079364) on the mean ITEO rating at the = 10% significance level. Table 8. Summary table of the results of the statistical test conducted to find out whether there is a difference in the ITEO student evaluation rating received by Biology faculty during two consecutive academic years, using two different delivery modes and two class types.
Answers to the Research Questions 1. Which parameters affect the ITEO Student Evaluation rating received by Biology faculty? Results of the study indicate that only the mode of delivery has an effect on the ITEO rating of the faculty of the Biology Department. Although on the average ratings of the Biology faculty were at least 85% only during the 1st term of AY 2010-2011 and the 2nd and 3rd term of AY 2011-2012, the grand average rating for each academic year studied were both within the Very Satisfactory range. 2. In which mode of teaching, transformative or trasnsmissive learning does Biology faculty perform better as indicated by the comparative ITEO Student Evaluation Rating they have received during the last two consecutive academic years? Results suggest that although on the average the rating received by the faculty of the Biology Department was Very Satisfactory, the scores for transmissive is consistently higher than that for the transformative learning method. The transmissive scores were within the range for Outstanding while the transformative scores were within the Very Satisfactory range (see Table 1 as a guide). 3. Is the Biology faculty improving in terms of their average ITEO Student Evaluation Rating for the past two consecutive academic years?
Results indicate that the ITEO Student survey ratings received by the Biology faculty have not changed when one compares the AY 2010-2011 with the AY 2011-2012 data. 4. What approach should the Biology Department take in order to achieve the target of 85% of the faculty receiving at least a rating of 4.0 from the ITEO Student Evaluation conducted every term? Firstly, it must be mentioned that three times in the last six terms, the Biology faculty was able to comply with the target requirement. However, there is a need for consistency. To achieve the objective of 85% of the Biology faculty getting an ITEO Student Evaluation rating of at least a 4.0, attention should be given to the training of the faculty most especially the new ones on the transformative learning pedagogy.
CONCLUSION: The results show that faculty status, gender, faculty rank, term, and academic year have no effect on the mean ITEO rating. The mode of delivery and, to some extent, the class type affects the mean ITEO rating. Specifically, the mean ITEO rating of classes taught under the transmissive mode is significantly higher than those that were taught under the transformative mode. Laboratory classes were also given significantly higher ITEO ratings than lecture classes but only in AY 2010-2011. It is recommended that to improve the ratings in classes taught using the transformative learning method, faculty most especially the new faculty should be given more training and skills enhancement with the fundamentals of this pedagogy.
LITERATURE CITED: California Comprehensive Center. 2006. Data-Driven Decision Making Based on CurriculumEmbedded Assessment: Findings from Recent California Studies. American Institutes for Research. p 1-5 Katz S and Earl L. 2006. Data Driven Decision Making Study. Toronto Area Professional Network Center. Summary Report. Aporia consulting Ltd. Toronto. Mandinach EB and Jackson SS. 2012. The Context of Data-Driven Decision Making. Chapter 1. In: Transforming Teaching and Learning Through Data-Driven Decision Making. Sage Pub. p 11-22. Marsh JA, Pane JF and Hamilton LS. 2006. Masking Sense of Data-Driven Decision Making in Education. Evidence from Recent RAND Research. Occasional Paper. RAND Corporation. 1-15. McREL. 2003. Sustaining School Improvement. Data –Driven Decision Making. Mid--continent Research for Education and Learning (MREL). Denver, Colorado. p 1-4. Meselt J. 2004. Data Driven Decision Making: A Powerful Tool for School Improvement. A White Paper. Sagebrush Corporation. Minneapolis, Minnesota. p 1-12. Park V and Datnow A. 2009. Co-constructing distributed leadership: district and school connections in data driven decision-making. School Leadership and Management. 29(5): 477-494.
Salpeter J. 2004. Data: Mining with A Mission. Technology & Learning. p 30-36. Sherrod D, McKesson T and Mumford M. 2010. Are you prepared for data driven decision making? Career Scope South Atlantic. Nursing Management. p 61-64. Wagner S. 2005. Data Driven Decision Making In Schools. A Workshop. Data Driven Enterprises. Westminster, Colorado. p 1-6.
