Session S2G THE engage PROGRAM: RESULTS FROM ... - CiteSeerX

Session S2G THE engage PROGRAM: RESULTS FROM RENOVATING THE FIRST YEAR EXPERIENCE AT THE UNIVERSITY OF TENNESSEE Fred E. Weber, R.M. Bennett, J.H. Forrester, P.G. Klukken, J. Roger Parsons, Christopher D. Pionke, William Schleter, J. Elaine Seat, D.L. Yoder1 Abstract — An innovative new freshman engineering initiative called engage has been fully implemented at the University of Tennessee. This is a comprehensive approach to meeting the educational and developmental needs of our freshmen. Extensive data on student performance was collected during the two-year phase in of this new program. Results presented include retention rates (engineering college and university), progression through the program, performance on common final exams and performance in sophomore courses. Comparisons are also made between the engage program and nationally administered surveys i.e. Pittsburgh survey on engineering student attitudes. In all these areas students in the new program did as well or better than those in the traditional curriculum.

INTRODUCTION An innovative new freshman engineering initiative, engage, has been fully implemented at the University of Tennessee. This is a comprehensive approach to meeting the educational and developmental needs of our freshmen. It is an integrated approach to the freshman curriculum, with a six semester hour fall course emphasizing problem-solving, teamwork, design concepts, and computer tools (engineering graphics and computer programming), all based around the study of non-calculus based physics. The spring consists of another 6-semester hour course integrating statics and dynamics, while assuming and using mastery of the material from the fall semester. Details of this program have been previously presented. [1] The program includes many different forms of learning opportunities to help accommodate the students’ different learning styles. Concepts are first presented in a traditional lecture. Then a hands-on laboratory “physical homework” experience develops student ownership of the concept [2]. This is followed by a recitation-style working session where students learn computer tools by solving problems using the concepts. Homework assignments are used to provide practice in the concepts and the computer tools. The material is integrated around “design, build, and test” team projects which range from chairs to rubber band powered vehicles to egg-launching catapults [3]. These team activities introduce students to engineering design and allow them to experience the same decision-making processes as practicing engineers.

1

Among the noteworthy curriculum features of engage are a team facilitation course where engineering upperclassmen are trained in team facilitation techniques and placed as coaches with freshman design teams [4,5], and the consolidation of freshman coursework facilities, faculty and staff offices, and support facilities in an Engineering Freshman Village. The engage program was piloted with 60 students during the 1997-98 academic year followed by a transition year with 150 students. Full implementation (464 students) began fall of 1999.

EXPERIMENTAL DESCRIPTION The pilot group of 60 students was chosen by the college to reflect the demographics of the previous year’s freshman class. Previous experience suggested that student demographics only varied slightly from year to year. This was an acceptable method for selecting a pilot group comparable to students in the traditional curriculum. These students were then invited to participate in the pilot program. Almost all these students opted to be in the pilot program. The pilot group was then filled out with students who arrived immediately prior to the fall semester. The demographics of the pilot group are presented in Table 1. The control for the first year consisted of first time freshmen students registered with the college’s freshman advising center. There was a requirement that students in the engage program be enrolled in at least pre-calculus. Due to this requirement, the control group was further restricted to students enrolled in pre-calculus or a more advanced math course. Table 1: Demographics of Pilot & Control Groups Total Students % Female % Caucasian % Afro Am % Asian Average Math ACT Average Comp ACT Average HS GPA

Pilot 60 20% 88% 7% 3% 27.1 26.1 3.55

Control 254 20% 79% 13% 6% 26.2 25.5 3.43

The Engin eering Fundamentals engage Program, The University of Tennessee, Knoxville, TN 37996

0-7803-6424-4/00/$10.00 © 2000 IEEE October 18 - 21, 2000 Kansas City, MO 30 th ASEE/IEEE Frontiers in Education Conference S2G-24

Session S2G The second year a transition group of 150 students was formed. Since this value was estimated to be approximately 1/3 of the entire freshman class, no attempt was made to balance the demographics of the transition group with the entire class. All freshmen were invited to participate in the transition group as they came to summer orientation. The demographics of the transition group and the corresponding control group are presented in Table 2 The control group was again all first-time freshmen students registered with the college’s freshman advising center and who were enrolled in pre-calculus or a later math course. Table 2: Demographics of Transition & Control Groups Total Students % Female % Caucasian % Afro Am % Asian Average Math ACT Average Comp ACT Average HS GPA

Transition 150 24% 79% 14% 4% 26.9 26.2 3.58

Control 244 16% 87% 11% 1% 26.8 25.8 3.43

STUDENT RETENTION/PROGRESSION One of the goals of the engage program was to improve retention rates for engineering freshmen. Table 3 gives the three semester retention results for both the pilot and transition groups compared to their control groups. In this case, retention in the college of engineering is defined as still taking engineering classes after the third semester. Retention in the university is similarly defined as still taking classes somewhere in the university after the third semester. The engage program clearly does a better job of retaining students in engineering and may be improving the retention rate for the university. Table 3 also reports on the student’s progress through the freshman year. In this case, the engage students move dramatically faster through the freshman year with between

nine and twenty-five percentage points more of the engage students completing the freshman year in three semesters compared to their control groups. The engage also passed approximately one more engineering course than the control groups. A common comment on innovative curriculum changes in engineering is that the data improves because the new courses have “watered down” the material. This is clearly not true for the engage program as shown by the average engineering GPA. Grades for the engage program courses are actually slightly lower than the grades given in the traditional program courses. Table 3 gives similar data at three and five semesters for pilot group who entered the university during fall of 1997. This data shows how engage students fare as they progress through the curriculum compared to the control group. After five semesters the college and university retention rates are essentially the same between the engage program and the traditional program. One possible cause of this drop in engage retention is a reaction to traditional engineering pedagogy once they enter a department. This points up the need for departments to adopt a similar type of curriculum to improve retention of students. The number of engage students who completed the freshmen year increased while the traditional students remained essentially constant. This is due to students who failed the engage program repeating courses in the traditional format. Apparently they were better prepared to function in the old curriculum than the control group who repeated courses. After five semesters engage students were able to raise their average GPA in engineering classes to that of the traditional curriculum’s students. This suggests that engage students performed better in departmental courses than the traditional curriculum students. This will be discussed further in the section on performance in the next section. Finally, the engage students continued to keep ahead of the traditional students in terms of progression through the curriculum. After five semesters the engage students were more than one course ahead of the traditional students. All of the data in this table needs to be interpreted with care because of the small sample size of the pilot group. Next Table 3: Retention Results

Total Retained in Engineering Retained at UTK Completed Freshmen Engr Average Engr GPA Average Engr Hours Passed

Fall 1997 Freshmen 3 Semester Retention Pilot Control 51 272 89% 67% 96% 95% 65% 56% 2.99 3.12 13.3 12.9

Fall 1998 Freshmen 3 Semester Retention Transition Control 138 212 88% 74% 97% 92% 81% 56% 2.96 3.16 14.2 10.6

Fall 1997 Freshmen 5 Semester Retention Pilot Control 51 271 55% 52% 86% 82% 71% 58% 3.12 3.12 29.9 25.4

0-7803-6424-4/00/$10.00 © 2000 IEEE October 18 - 21, 2000 Kansas City, MO 30 th ASEE/IEEE Frontiers in Education Conference S2G-25

Session S2G year we will have equivalent data for the larger transition group STUDENT PERFORMANCE Another goal of the engage program was to better prepare students for entry into their departments. A measure for this goal was student performance in their first departmental course. The three-semester performance for both engage and traditional students is presented in Table 4. In general, engage students did at least as well as traditional students in their first departmental course. The large variations in results from year to year are due to the small sample sizes in the pilot group. As noted below, some classes had only one engage student from the pilot section. Table 5 contains similar performance data for the pilot group at three and five semesters. The three semester results are basically the same as the five semester results. Engage students again did better, on the average, than traditional students. Table 4: Performance in First Engineering Courses 3 Semester Performance Fall 1997 Fall 1998 Freshmen Freshmen Pilot Control Transition Control CivilE 210 -2.93 3.50 3.06 ChemE 200 3.70 3.11 3.31 3.55 ECE 201* 0.64 0.83 2.83 2.80 ECE 205 3.13 2.04 3.00 3.05 ESM 231* 2.20 2.44 3.02 2.84 IE 201 -3.00 --MSE 201* 2.95 1.89 2.66 2.86 NE 203 -2.58 3.50 -*Service Course taken by most students -- less than 2 students in course Table 5: Performance in First Engineering Courses Fall 1997 Freshmen 3 Semester 5 Semester Performance Performance Pilot Control Pilot Control CivilE 210 -2.93 -3.04 ChemE 200 3.70 3.11 3.50 2.38 ECE 201* 0.64 0.83 1.81 1.19 ECE 205 3.13 2.04 2.90 2.05 ESM 231* 2.20 2.44 2.79 2.43 IE 201 -3.00 4.00 3.05 ME 331 -2.88 2.81 1.85 MSE 201* 2.95 1.89 2.96 2.10 NE 203 -2.58 -2.46 *Service Course taken by most students -- less than 2 students in course

To further compare performance between the engage and traditional students, common finals in statics and dynamics were given to both groups each year. It was not possible to give common finals for the graphics and computer programming courses due to significant differences in the curricular content between engage and the traditional curriculum. The exams were multiple choice with up to ten different answers for each problem. Possible answers included results of common errors. These included sign errors, common incorrect assumptions, and others. The engage students had a disadvantage with these exams compared to traditional students in that they took both the statics and dynamics exams within a few days of each other. Engage covers both topics during the semester (statics first) while traditional students normally took either statics or dynamics. Table 6 summarizes the groups performance on the statics final. Even though it had been approximately seven weeks since the engage students had covered statics, they consistently did better than the traditional students. The ‘No Errors’ entry is the average for all questions of the percent of students who got a given problem completely correct. The ‘Common Errors’ entry is a similar average for students who made a simple error on the problem that resulted in an incorrect answer. This number mirrors the awarding of partial credit common on engineering exams. In both years, the engage students did considerably better than the traditional students. TABLE 6: PERFORMANCE ON COMMON S TATICS FINAL % Correct Fall 1997 Fall 1998 Pilot Control Transition Control No Errors 44 37 50 36 Common Errors 71 57 69 52 TABLE 7: PERFORMANCE ON COMMON DYNAMICS FINAL % Correct Fall 1997 Fall 1998 Pilot Control Transition Control No Errors 45 37 65 58 Common Errors 45 36 59 60 Similar results for the dynamics final are presented in Table 7. The engage students again performed better than the traditional students in all cases.

ATTITUDE SURVEY The engage program at the University of Tennessee has participated in a cross-institutional study using the

0-7803-6424-4/00/$10.00 © 2000 IEEE October 18 - 21, 2000 Kansas City, MO 30 th ASEE/IEEE Frontiers in Education Conference S2G-26

Session S2G “Pittsburgh” freshman attitude survey to provide comparison data with freshman engineering programs at other institutions [6]. Sixteen institutions participated in this study during the 1998-1999 academic year, administering the survey at the beginning of the freshman year and at the end of the first semester or first year, depending on the institution. Changes in freshmen responses from the beginning to the end of the year that were found to be a statistically significant for the engage students were: 1.

Students thought engineering to be less of an exact science then they did at the beginning of the year. This was the case at 5 of the 16 schools in the study and is thought to be a characteristic of programs that include design and applied problem solving. All other institutions were statistically neutral on this point. 2. Students indicated a higher self-assessed confidence in their background knowledge and skills then they did when they first started engineering in the fall. On this question, 13 schools reported no change, one other school demonstrated a positive trend, and one school had a negative trend. 3. Students indicated a higher self-assessed confidence in their communication and computer skills than they did at the beginning of the year. This was the case at 9 of the 16 schools. 4. Students indicated a higher perception of their engineering attributes at the end of the year. This was the case at 5 of the 16 institutions. We are currently looking at the extensive data provided by this study to try to improve our program. Even a casual perusal of the data leads to the conclusion that there is great room for improvement in freshman engineering programs. When asked to rate their “general impressions of engineering”, no school demonstrated a positive trend from the pre-survey to the post-survey, six schools maintained the students impressions of engineering over the course of the study (including the engage program), and ten schools demonstrated negative trends during the freshman year.

departments sooner, and perform better in the departments. They generally are better prepared to succeed academically. This study also suggests that in order to continue the retention of students once they enter a department, similar curriculum changes are necessary in departmental courses.

ACKNOWLEDGEMENT The Implementation of the engage program is supported by the National Science Foundation through grant number EEC-99.

REFERENCES [1]. Fred T Gilliam, P. Gary Klukken, J. Roger Parsons,

[2]. [3].

[4].

[5].

[6].

Christopher D. Pionke, Tom H. Scott, J. Elaine Seat, Fred Symonds, Fred E. Weber, and Daniel C. Yoder, “The engage Program: Renovating the First Year Experience at the University of Tennessee,” Proceedings of the 1998 Frontiers in Education Conference, November, 1998. Daniel C. Yoder, J. Roger Parsons, Chris D. Pionke, and Fred Weber, “Hands-On Teaching of Engineering Fundamentals ,” Proceedings of ASEE Annual Meeting, June 1998. “Balancing Capability, Enthusiasm, and Methodology in a Freshman Design Program,” Pionke, C.D., Parsons, J. R., Seat, J. E., Weber, F. E., Yoder, D. C., International Journal of Engineering Education, (in review). “Training Engineering Upperclassmen to Facilitate Freshman Design Teams”, Knight, D., Poppen, W., Klukken, G., Parsons, J. R., Seat, J. E., Proceedings of the American Society of Engineering Education Annual Conference, Seattle, Washington, June, 1998. “An Evaluation of a Design Team Facilitator Training Program for Engineering Upperclassmen”, Knight, D., Seat, J. E., Poppen, W., Parsons, J. R., Klukken, P. G., and Hector, M., Proceedings of the ASEE Frontiers in Education Conference, San Juan, PR, November, 1999. Besterfield -Sacre, M. E. and C. J. Atman, “Survey Design Methodology: Measuring Freshman Attitudes About Engineering,” American Society of Engineering Education Conference Proceedings, June 1994, pp. 236-242.

CONCLUSION The engage program is an innovative curriculum designed to meet the changing needs of freshmen engineering students today. It includes much of the traditional engineering topics but also adds training in teaming, communication, engineering perspective, and other skills lacking in traditional programs. By integrating the engineering curriculum into two six-hour courses, enough time is made available to cover these new topics. The engage program has succeeded in improving the three semester retention rates in the college of engineering and the university in general. The students progress through the engineering curriculum more quickly, enter their major 0-7803-6424-4/00/$10.00 © 2000 IEEE October 18 - 21, 2000 Kansas City, MO 30 th ASEE/IEEE Frontiers in Education Conference S2G-27