Proctoring strategies for computer-based and paper

NEW PRACTITIONERS FORUM

Proctoring strategies for computer-based and paperbased tests

A

cademic dishonesty, including plagiarizing and cheating on tests and assignments, is an educational concern for faculty and preceptors.1 Faculty and preceptors play an important role in maintaining academic integrity and preventing cheating by clearly delineating standardized procedures and expectations for tests in course and rotation syllabuses.2 Previous reports have offered guidance for creating, administering, and proctoring traditional paper-based tests.2 However, similar guidance is lacking for computer-based tests (also referred to as online or electronic tests) that are given in a classroom and supervised by a proctor (versus computer-based tests that are completed outside of a classroom and are not proctored or monitored).1 These computer-based tests might be administered on laboratory computers or students’ personal devices (such as a laptop, tablet, or iPad). As a growing number of colleges and schools of pharmacy are implementing computer-based testing, using tools such as ExamSoft (ExamSoft Worldwide Inc., Boca Raton, FL), Blackboard (Blackboard Inc., Washington, DC), and Desire2Learn (Desire2Learn, Inc., Ontario, Canada), proctoring guidelines must be reviewed and revised to address the unique challenges of this testing method. Because faculty and preceptors are commonly responsible for proctoring tests and serve as the primary deterrent to cheating,3 the similarities and differences between the proctoring protocols for computerbased versus paper-based tests should be explained when training proctors. Similarities between proctoring computer-based and paper-based tests. Paper-and-pencil testing involves the use of test questions on paper with or without the use of Scantron (Scantron Corporation, Eagan, MN) grading forms. Alternatively, computer-based tests are delivered and monitored in the classroom and use software or au-

The New Practitioners Forum column features articles that address the special professional needs of pharmacists early in their careers as they transition from students to practitioners. Authors include new practitioners or others with expertise in a topic of interest to new practitioners. AJHP readers are invited to submit topics or articles for this column to the New Practitioners Forum, c/o Jill Haug, 7272 Wisconsin Avenue, Bethesda, MD 20814 (301-664-8821 or [email protected]).

274 

dience response systems, such as clickers. In either format, creating a uniform and standardized testing environment that is clearly explained in writing to students and then enforced by proctors is a key factor for reducing cheating.1 Regardless of the test format, faculty and preceptors are encouraged to revise existing test questions each year. They can accomplish this by using exam statistics to revise test questions and answer choices, add new questions that address new or existing learning objectives, and change the placement of the correct answer (unless they are already randomized, as is the case with computerized exams).4 These procedures decrease the likelihood that students can simply recognize correct answers if they receive guidance from upperclass students about test questions. When proctoring during the test, faculty and preceptors should consult the test administration recommendations offered by Medina and Yuet,2 as the setup and administration of paper-based and computer-based testing are similar. For example, with either format, proctors should ask students to leave personal belongings in the front of the room or another designated area, use a closed-exam policy (where no exams are returned to the students), and use a test seating chart to monitor students in the room and prevent intentional seating arrangements. 2 In addition, as technology advances with smart watches capable of communication, faculty and preceptors should consider procedures to prohibit the use of these devices during an exam. After each exam, faculty and preceptors should conduct exam reviews by highlighting concepts or objectives that were missed rather than reviewing specific test questions with the entire class, though mentored review between a faculty member or preceptor and a student can still occur.2 Suggestions for proctoring computer-based tests. Despite the similarities between paper-based and computerbased tests, there are differences that faculty and preceptors

  AM J HEALTH-SYST PHARM | VOLUME 73 | NUMBER 5 | MARCH 1, 2016

Continued on page 276

NEW PRACTITIONERS FORUM Continued from page 274

should address in their syllabuses regarding testing procedures when using computer-based exams. Programmatic planning and coordination are essential when making the transition from paper-based to computer-based tests and are needed before individual course or rotation test planning can occur in order to create uniformity across the curriculum. It is essential that individuals at the college or school of pharmacy in areas such as the office of student affairs, academic affairs, and assessment office create and distribute to students a detailed student computer policy outlining the type of computer required (laptop versus tablet) and minimum system requirements, as some testing software is specifically designed for delivery on a laptop. The computer policy should also detail computer software requirements, such as Respondus Lockdown Browser (Respondus, Redman, WA) and SofTest (ExamSoft Worldwide), in addition to Adobe Acrobat (Adobe, San Jose, CA), which might be required to read PDF attachments. The college or school must also evaluate the wireless capacity of the testing environment if using personal devices, since the type of software chosen to deliver the test may require an Internet connection to access the test. Too few access points can relate to computer connectivity problems and contribute to increased student and proctor anxiety during the exam. The college or school should also consider a syllabus template that outlines testing procedures and exam expectations in order to standardize procedures and ensure accountability across courses. Examples of such policies include the following: • Students must download RespondusLockdown Browser or SofTest during the first week of the semester in order to take course tests. Students who do not have the software downloaded will not be permitted to take the test and risk receiving a zero on the test. Schools or colleges using ExamSoft should also include a policy that require students to download the test before the start (e.g., 24 hours prior) or risk receiving a zero on the test for failing doing so. • Students are responsible for the maintenance of their device and updates to testing software. • Students who forget their device on a test day risk receiving a 10% deduction from their test score. • Students must arrive 10 minutes before the exam start time to prepare their device for the test. Students who arrive late will not get extra time to take the exam and may miss the exam start window. • Students must turn off their mobile phones and any other electronic devices (e.g., tablets, smart watches) rather than putting them on silent upon room entry to prevent computer crashes during the test if the number of access points for the room is a concern.

276 

• Students are expected to take their device and power cord to their assigned seats for the exam. Once seated, they should close all programs and open the test-taking software to prepare for the exam. • Students will be given a 5–15 minute window to open the quiz or exam and enter the password in class. Passwords should be complex enough to prevent students from accessing the exam outside of the testing room. Students who are late and miss the window may be blocked from opening the exam or quiz and may receive a zero. • Students must save their test after every test question is answered if this does not occur automatically within the software. • Students will be required to acknowledge the school’s honor code and attest to not providing or receiving aid from other students on this test. • If a student experiences malfunctioning equipment, such as software problems, hardware problems, or a loss of power, that student will be given a different device to complete the test (limited availability of extra laptops) or will be given a paper version of the test. Students may or may not be given extra time to complete the test equal to the time lost troubleshooting the technical malfunction. • Students may receive one piece of “scratch” paper to use during the exam. This paper must be signed with the student’s name, date, and test completion time and submitted to the proctor before leaving the exam room or risk receiving a zero on the exam. • Students must submit their test electronically when they are finished and check out of an exam session with an exam proctor by showing that the exam or quiz has been submitted before closing their device. Faculty should also configure test software settings to allow for only one test submission; otherwise, students could leave the testing room and reaccess the test. • All computer-based testing is subject to the academic misconduct policy described in the course syllabus. Any intentional attempt to disable or tamper with the testing security features is considered academic misconduct, and students risk receiving a zero on the test and up to an F in the course along with having their name submitted to the dean or academic misconduct board or both.

Research has shown that students’ and faculty members’ greatest concern with computer-based testing is technology-related difficulties, such as the inability to access an exam and network connectivity issues.5 This illustrates the importance of having a backup testing plan for technical difficulties. Faculty and preceptors should consider bringing extra printed copies of the test and item attachments with them into the testing room as well as extra testing devices in case a student’s computer malfunctions before or during the test. A standardized procedure should be created, documented, and used in the event of technical failures in order to reduce student and faculty

  AM J HEALTH-SYST PHARM | VOLUME 73 | NUMBER 5 | MARCH 1, 2016

NEW PRACTITIONERS FORUM



anxiety. A plan should also be developed about how proctors discuss a technical failure with individual students so that other students are not disrupted. Faculty and preceptors should consider offering a practice quiz covering the testing procedures noted in the syllabus at the beginning of the semester to increase familiarity with the computer-based testing format and testing procedures. To remind students of important testing logistics, consider providing reminders in an e-mail just before the exam (e.g., when you release the exam for download) or an instructions slide in the room of the test or on the preexam notices that are contained within some testing platforms. Faculty and preceptors also need to decide if they should proctor from the front of the room to see student faces or the back of the room to see computer screens. For this reason, multiple proctors should be considered for large groups of students. In the future, tablets may be a better computer-based test delivery option, as tablets can lie flat on a desk. During the exam, faculty and preceptors should not program the testing software to reveal the correct answers to a student after he or she submits the responses, as other students in the room may see the display. Privacy screens on a laptop are costly and may not work effectively enough to prevent other students from seeing someone else’s screen and answers. To help prevent cheating, the randomization of test questions and answers is encouraged.4,6 Similarly, the testing software should not release scores or answers at the end of the exam before the students leave the room. Instead, faculty and preceptors should review the test results and statistics after the exam has closed and make any adjustments before posting the grades to cause the least amount of disruption and grade negotiating. To help facilitate good proctoring during the exam, administrators can provide proctors with a “proctoring guide” that contains the exam password (to help students who did not understand the password when given orally), resume code (if required when restarting an exam due to technical difficulties), support contact numbers (for the software program itself as well as cam-



pus technical support), tips to watch for cheating (wandering eyes, sharing scratch paper, finger tapping), and fixes for common problems such as Internet connectivity and restarting computers during a frozen screen. Summary. Although the creation, delivery, and proctoring of computer-based and paper-based testing are similar, additional procedures must be standardized and implemented when using computer-based exams to maintain academic integrity. It is important that testing environments ensure academic integrity without implying a mistrust of students. 1. Beck V. Testing a model to predict online cheating: much ado about nothing. Active Learn Higher Educ. 2014; 15:65-75.7. 2. Medina MS, Yuet WC. Promoting academic integrity among health care students. Am J Health-Syst Pharm. 2013; 70:7547. 3. Rabi SM, Patton LR, Fjortoft N, Zgarrick DP. Characteristics, prevalence, attitudes, and perceptions of academic dishonesty among pharmacy students. Am J Pharm Educ. 2006; 70:article 73. 4. Conway-Klaassen JM, Keil DE. Discouraging academic dishonesty in online courses. Clin Lab Sci. 2010; 23:194-200. 5. Stolte SK, Richard C, Rahman A, Kidd RS. Student pharmacists’ use and perceived impact of educational technologies. Am J Pharm Educ. 2011; 75:article 92. 6. Bussieres JF, Metras ME, Leclerc G. Use of Moodle, ExamSoft, and Twitter in a first-year pharmacy course. Am J Pharm Educ. 2012; 76:article 94. Melissa S. Medina, Ed.D. The University of Oklahoma College of Pharmacy Oklahoma City, OK [email protected] Ashley N. Castleberry, Pharm.D., M.A.Ed. The University of Arkansas for Medical Sciences College of Pharmacy Little Rock, AR Nancy Letassy, Pharm.D., CDE; and Amy Williams, B.A., are acknowledged for providing their input and guidance about this subject. The authors have declared no potential conflicts of interest. DOI 10.2146/ajhp150678

AM J HEALTH-SYST PHARM | VOLUME 73 | NUMBER 5 | MARCH 1, 2016  277

Copyright of American Journal of Health-System Pharmacy is the property of American Society of Health System Pharmacists and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.