Session S4B
Work in Progress - Web-Based Implementation of Capstone Engineering Design Assessments Denny Davis, Howard Davis, Steven Beyerlein, Phillip Thompson, Olakunle Harrison, Michael Trevisan
[email protected],
[email protected],
[email protected],
[email protected],
[email protected],
[email protected] Abstract - Assessment of student performance in capstone engineering design is vital to student learning and to program accreditation and improvement. Performances important to capstone design courses include students’ growth in personal capacity for design, development of teamwork, development of design processes, and delivery of solution assets. The Transferable Integrated Design Engineering Education (TIDEE) consortium is creating and testing a capstone design assessment system to measure these student performances and provide rapid personalized feedback. Broad adoption of this assessment system requires ease of use by both students and instructors. This paper describes design features of a web-based implementation of the TIDEE capstone design assessment system. Index Terms – Assessment system, Capstone design, Database, Web-based. INTRODUCTION National leaders are calling for increased attention to educating engineers with a broad set of skills, including abilities to design under realistic constraints [1] - [3]. Without effective assessment, performance targets are illdefined and learning is relatively unguided and unproductive [4] - [5]. Effective assessment feedback to students on their performances in capstone engineering design courses is vital to ensuring preparedness of engineering graduates for their professions. Capstone design course faculty struggle with the assessment of many outcomes to be demonstrated in these courses [6]. Available assessments do not adequately assess all important outcomes [7]. Students in these courses should evidence both quality and continuous improvement of performances in two complementary areas: learner development and solution development [8] - [10]. Consequently, useful assessment requires measurement and tracking of individual and team performances over the duration of their design projects. To support program accreditation, assessment information should also document levels of student performance with regard to program-specific outcomes. This work-in-progress paper describes a web-based assessment system that enables instructors to provide students rapid, personalized feedback while minimizing work associated with data entry, collection, and processing.
ASSESSMENT INSTRUMENTS AND USE The Transferable Integrated Design Engineering Education (TIDEE) consortium is developing an integrated system for assessing outcomes related to students' personal capacity, teamwork, design processes, and solution assets. Instruments are designed for formative use, but also provide summative data. Assessments under development are listed in Table I. TABLE I SUMMARY OF TIDEE ASSESSMENT INSTRUMENTS Assessment Name [Area]*
Focus of Assessment
Personal Growth [P]
Personal strengths and improvement
Professional Practices [P]
Professional strengths, improvement
Team Member Citizenship [T]
Peer/self ratings and coaching
Team Processes [T]
Process strengths and improvements
Problem Scoping Processes [D]
Process description, self-assessment
Concept Generation Processes [D]
Process description, self-assessment
Solution Realization Processes [D]
Process description, self-assessment
Problem Definition [S]
Definition details, self-assessment
Selected Concept [S]
Concept details, self-assessment
Proposed Solution [S] Solution details, self-assessment * P=Personal capacity, T=Teamwork, D=Design process, S=Solution assets
The web-based assessment system enables structured, online communication focused on learning. As seen in Figure 1, instructors specify the assignment and students complete it. Next, instructors review individual, team, and class performances and give feedback. Finally, students receive and interpret feedback. All is accomplished securely online.
FIGURE 1 COMMUNICATION ENABLED BY WEB-BASED ASSESSMENT SYSTEM
978-1-4244-1970-8/08/$25.00 ©2008 IEEE October 22 – 25, 2008, Saratoga Springs, NY 38th ASEE/IEEE Frontiers in Education Conference S4B-1
Session S4B REFERENCES
WEB-BASED ASSESSMENT SYSTEM DESIGN The web-based assessment system manages both assessment instrument content and user (student, instructor, etc.) inputs. The system’s web-based software incorporates a MySQL relational database with pages dynamically generated. Relational tables define the underlying structure of paperbased pilot instruments that are being implemented online. Object-oriented modeling of the relationships between assessment instruments, assignments, assessors, targets of the assessments, and archived data is implemented in Php 5. Figure 2 shows a typical user screen for the Team Member Citizenship assessment. Student responses to this screen are captured and merged with other input data— documenting their responses, teammate responses to the same items, and instructor feedback on this and other items.
FIGURE 2 SAMPLE INPUT SCREEN FOR TEAM MEMBER CITIZENSHIP ASSESSMENT
The web-based system is currently housed on a server maintained by the second author. Access to the system is by request to the TIDEE project director (first author). Upon approval, instructor and class rosters are uploaded to the database and username and password lists are generated to provide individual, role-based (instructor, team leader, student) access to the system. Confidentiality of data records is ensured by keeping no sensitive personal data on the database. Data viewing is controlled by a combination of permission and natural hierarchy of the classroom. For example, instructors can only view their students’ data. The system provides anonymous peer feedback, a repository for project and class data, automated summarization of data, and the ability to generate customized reports. In comparison to paper-based assessments, the TIDEE webbased assessment system offers improved ease of use and considerable time savings along with valuable archived data. ACKNOWLEDGMENT Funding for development of the capstone design assessment system and its web-based implementation were provided by the National Science Foundation, Division of Undergraduate Education grants DUE 0404924 and 0717561.
[1]
The National Academies, “Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future,” Washington DC: National Academies Press, 2006.
[2]
National Academy of Engineering, “The Engineer of 2020: Visions of Engineering in the New Century,” Washington DC: National Academy Press, 2004.
[3]
ABET, “Criteria for Accrediting Engineering Programs,” ABET, Baltimore, MD, (2008), www.abet.org.
[4]
Black, P., & Wiliam, D., “Inside the Black Box: Raising Standards through Classroom Assessment,” Phi Delta Kappan (October, 1998), 139-148.
[5]
Stiggins, Richard, “Student-Centered Classroom Assessment,” (2 ed, 1997), Prentice-Hall: Upper Saddle River, New Jersey.
[6]
McKenzie, Larry, Michael Trevisan, Denny Davis, and Steven Beyerlein, “Capstone Design Courses and Assessment: A National Study,” Proceedings of Annual Conference of American Society for Engineering Education, June 2004.
[7]
Gerlick, Robert, Denny Davis, Steven Beyerlein, Phillip Thompson, Olakunle Harrison, Michael Trevisan, "Assessment Structure and Methodology for Design Processes and Products in Engineering Capstone Courses," Proceedings of Annual Conference of American Society for Engineering Education, June 2008.
[8]
Davis, Denny, Steven Beyerlein, Olakunle Harrison, Phillip Thompson, and Michael Trevisan, "Assessments for Three Performance Areas in Capstone Engineering Design,” Proceedings of Annual Conference of American Society for Engineering Education, June 2007.
[9]
Beyerlein, Steven, Denny Davis, Michael Trevisan, Phillip Thompson, and Olakunle Harrison, “Assessment Framework for Capstone Design Courses, Proceedings of Annual Conference of American Society for Engineering Education, June 2006.
[10] Davis, Denny, Steven Beyerlein, Olakunle Harrison, Phillip Thompson, Michael Trevisan, and Benjamin Mount, “A Conceptual Model for Capstone Engineering Design Performance and Assessment,” Proceedings of Annual Conference of American Society for Engineering Education, June 2006.
AUTHOR INFORMATION Denny Davis, Co-director, Engineering Education Research Center, Washington State University,
[email protected] Howard Davis, Laboratory Instructional Supervisor, Chemical Engineering and Bioengineering, Washington State University,
[email protected] Steven Beyerlein, Professor, Mechanical Engineering, University of Idaho,
[email protected] Phillip Thompson, Chair, Civil and Environmental Engineering, Seattle University,
[email protected] Olakunle Harrison, Associate Professor, Mechanical Engineering, Tuskegee University,
[email protected] Michael Trevisan, Director, Assessment and Evaluation Center, Washington State University,
[email protected]
978-1-4244-1970-8/08/$25.00 ©2008 IEEE October 22 – 25, 2008, Saratoga Springs, NY 38th ASEE/IEEE Frontiers in Education Conference S4B-2
