grades in the Performance Based Assessment (PBA) component in the ... scores and PBA grades in year 1, which assessed both written and ... PSL-1 (09-10).
Predicting Students’ Communication Skills in a Pharmacy Program Robert D. Renaud Ph.D.1,2, Chris Louizos B.Sc.(Pharm)2, Cheryl Kristjanson Ph.D.2,3, Sheryl A. Zelenitsky Pharm.D.2 1 Faculty
of Education, University of Manitoba; 2 Faculty of Pharmacy, University of Manitoba; 3Faculty of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
Introduction
Methods
Among admission variables examined in health sciences disciplines, undergraduate GPA (UGPA) is a widely studied predictor of future overall academic performance.1 Summarized in recent reviews in nursing2 and medicine1, the correlation between UGPA and future overall performance is approximately 0.40. This leaves much of the predictability of academic performance remaining unaccounted for. Thus, the next step is to explore other predictors sufficiently divergent from UGPA.
Data consisted of AE scores and subsequent PBA grades in years 1, 2, and 3 for all students over six academic years (n=309). The AE score for each student represented the mean score on one essay graded by at least two evaluators. Because the AE scores were highly skewed (see Figure 2), only AE scores of 17 (out of 30) or higher were retained. The overall PBA grade for each student was the total grade across all assignments within the PBA component of each PSL course. Given the restricted range of AE scores, a correction formula was used to calculate the correlations between AE scores and PBA grades.
Our admissions process (shown in Figure 1) includes incoming adjusted GPA and an admission essay. Continuing our previous research, this study explored the predictive validity of an admission essay (AE), intended to assess communication skills, on subsequent grades in the Performance Based Assessment (PBA) component in the Pharmacy Skills Lab (PSL) stream in years 1, 2 and 3 in the pharmacy curriculum.
Results
Conclusions
Table 1 Correlations between Admission Essay Scores and Performance Based Assessment Grades PSL Section
Figure 2 Distribution of Admission Essay Scores
All Essay Scores
Uncorrected Essay Scores ≥ 17
Corrected* Essay Scores ≥ 17
PSL-1 (07-08)
0.23 (n=49)
0.26 (n=37)
0.39
PSL-1 (08-09)
0.25 (n=48)
0.33 (n=30)
0.58
PSL-1 (09-10)
0.14 (n=52)
0.34 (n=30)
0.63
PSL-1 (10-11)
0.05 (n=55)
0.33 (n=31)
0.71
PSL-1 (11-12)
0.07 (n=53)
0.05 (n=39)
0.08
There was a notably significant correlation between AE scores and PBA grades in year 1, which assessed both written and verbal communication skills. A relationship was not observed for subsequent years possibly due to an increasing emphasis on the application of knowledge. Overall, these findings validate the use of an AE in predicting how well an applicant will perform in relevant parts of the Pharmacy curriculum that assess communication abilities. Given the potential for finding more meaningful relationships between predictor variables and specific academic outcomes, future research should continue exploring the predictability of key learning outcomes such as critical thinking and professionalism.
Mean PSL-1 0.48
Figure 1 Pharmacy Admissions Process
PSL-2 (06-07)
0.16 (n=46)
0.34 (n=27)
0.69
References
PSL-2 (07-08)
-0.12 (n=47)
-0.09 (n=36)
-0.13
PSL-2 (08-09)
-0.13 (n=49)
-0.16 (n=32)
-0.29
1. Siu, E., & Reiter, H. I. (2009). Overview: What’s worked and what hasn’t as a guide towards predictive admissions tool development. Advances in Health Sciences Education, 14, 759-775.
PSL-2 (09-10)
-0.04 (n=55)
-0.25 (n=32)
-0.56
PSL-2 (10-11)
-0.04 (n=53)
-0.12 (n=29)
-0.33
2. Grossbach, A., & Kuncel, N. R. (2011). The predictive validity of nursing admission measures for performance on the national council licensure examination: A meta-analysis. Journal of Professional Nursing, 27(2),124-128.
Mean PSL-2 -0.13
3. Gulliksen, H. (1950). Theory of mental tests. New York: John Wiley.
Results
PSL-3 (06-07)
0.11 (n=45)
0.07 (n=27)
0.18
From the adjusted sample (n=197), the mean and standard deviation of the AE scores were 19.3 and 2.3 respectively. The mean percentage grades across all PSL sections in each year level were: year 1 (73.8 ±6.2), year 2 (74.1 ±8.5), and year 3 (71.1 ±8.1). As shown in Table 1, The mean correlation between AE scores and PBA grades in years 1, 2 and 3 were 0.48, -0.13, and 0.03 respectively.
PSL-3 (07-08)
-0.02 (n=41)
-0.04 (n=31)
-0.06
PSL-3 (08-09)
0.06 (n=48)
0.31 (n=32)
0.51
PSL-3 (09-10)
-0.13 (n=56)
-0.22 (n=32)
-0.51 Mean PSL-3 0.03
.
*Correction based on following formula3 Rxy = Sx rxy / (Sx2 rxy2 + sx2 – sx2 rxy2)½
Acknowledgements • Support of the Dean of Pharmacy, Dr. Neal Davies • The Program Evaluation Committee and Admissions Committee
