Using Digital Test Preparation Software to Predict and Impact First ...

234 downloads 35092 Views 571KB Size Report
First-Time Pass Rates on National Certification Exams. ISSUES IN IMPROVING POST-SECONDARY LEARNING OUTCOMES. Navigate TestPrep: EMT can ...
ISSUES IN IMPROVING POST-SECONDARY LEARNING OUTCOMES

Using Digital Test Preparation Software to Predict and Impact First-Time Pass Rates on National Certification Exams

In order to obtain certification to practice as an Emergency Medical Technician (EMT), students in most states are required to pass the National Registry of Emergency Medical Technician’s (NREMT) computer-based certification exam. More than 100,000 EMT students sit for the exam each year, with an average pass rate of 70% in 2011. In some areas of the country, the pass rate is as low as 50%, making failure a major concern. Jones & Bartlett Learning offers Navigate TestPrep: EMT (formerly JB TestPrep: EMT-Basic Success), an online program that includes practice exams and simulated certification exams that are highly effective in preparing examinees for the NREMT certification exam. Recently, we surveyed 665 users and the results show:

The First-Time Pass Rate on NREMT Certification Exams Is 14% Higher Among Navigate TestPrep: EMT Users than on Average Nationally

National First-Time Pass Rate*

70%

Navigate TestPrep: EMT Users

30%

84%

0%

10%

20%

30%

40%

16%

50%

60%

70%

80%

90% 100%

Navigate TestPrep: EMT can predict the probability of passing the NREMT certification exam A Logistic Regression Model was used to predict the probability of passing the NREMT certification exam based on students’ best attempt scores for a sample of 572 Navigate TestPrep: EMT users. Prob (1st Time Pass) = 1 / 1 + e^(1.246 - 0.034 * Best Attempt Score)

The table below shows the probability of passing the NREMT certification exam on the first attempt based on students’ best attempt scores on Navigate TestPrep: EMT. Here is a breakdown of our findings: • The probability of a student with a score of 30 on the Navigate TestPrep: EMT simulated certification exam to pass the NREMT certification exam on the first attempt is .44. • The probability of a student with a score of 100 on the Navigate TestPrep: EMT simulated certification exam to pass the NREMT certification exam on the first attempt is .90. • The passing probability for a student with a score of 100 on the Navigate TestPrep: EMT simulated certification exam is almost twice as high as for a student with a score of 30.

Did Not Pass NREMT Exam Passed NREMT Exam N=665

*Source: NREMT 2011 Annual Report http://www.nremt.org/nremt/downloads/2011_Annual_Report.pdf

0

10

20

30

40

50

60

70

80

90

100

.22

.29

.36

.44

.53

.61

.69

.76

.81

.86

.90

Navigate TestPrep Best Attempt Score First-Time NREMT Passing Probability

How did Navigate TestPrep: EMT impact first attempt pass rates?

This probability curve depicts the relationship graphically:

Users said:

“…really helped me I believe. They taught in our program to study hard with [Navigate TestPrep: EMT] and we [would] pass the first time. They told us it was the best way to study for the National Registry Exam. Both my friend and I passed on the first attempt.” “…made a massive difference in my class scores and I firmly believe it was the reason I passed my National Registry Exam on the first try.” “…a very big help in passing the National Registry Exam. I would recommend all EMS students to use this for a study guide and for preparing for the test.” “I felt [it] did help me succeed. Thank you!”

100 First-Time NREMT Passing Probability

“I doubt I would have passed the National Registry Exam on my first attempt without [it].”

80

60

40

20

0

0

20

40

60

80

Navigate TestPrep: EMT Best Attempt Score

100

In addition, the average score on Navigate TestPrep: EMT is significantly higher for those students who passed the NREMT’s certification exam on their first attempt (89 points) than for those who failed on their first attempt (83 points).

After using Navigate TestPrep: EMT Users said: “I recommend all EMT students use this as a study guide and for preparing them for the test.” “…made a massive difference in my class scores…”

Navigate TestPrep: EMT Simulated Certification Exam Best Attempt Score for Students Who...

“I recommend it to all who plan to take the test…” “The [NREMT] test was easy after using Navigate TestPrep.” Passed NREMT Exam

“I would like to use this again for re-certification testing.”

89 (0.5)

Did Not Pass NREMT Exam

83 (1.8)

0

10

20

30

40

JBTestPrep: EMT Users 50

60

70

80

90

Helped me learn

98%

I would recommend it to others

98%

I would use it again

97%

Prepared me well for my certification exam

96%

100

Navigate TestPrep: EMT simulates the NREMT certification exam

Kept me focused on my studying

94%

Helped me organize my studying

93%

Engaged me in my studying

93%

Made a major contribution to my success in the course

90%

Users said: “The questions were worded very similar to how the National Registry Exam was so I was more prepared in how to answer the questions.” “The questions were very similar to the real test, and the different sections were designed to help me learn how to test well.” “Without the assistance of [Navigate TestPrep], I would have struggled with the test format. However, [it] really drove home the concepts I learned and was instrumental in preparing me mentally for the testing format of the National Registry Exam.”

0%

40%

60%

80%

100%

Agree Disagree

N=242

“…set up just like the real exam. It prepared me to study in the exact way and format.”

Overall

76%

22%

Airway and Breathing

77%

Cardiology

74%

23%

Trauma

74%

24%

Medical

74%

24%

“The questions were very similar to the National Registry Exam and that helped a lot.”

How satisfied were Navigate TestPrep: EMT users?

20%

Obstetrics and Pediatrics

24%

72%

Operations

31%

66% 0%

Overall, how satisfied were you with Navigate TestPrep: EMT or JB TestPrep: EMT-Basic Success?

22%

20%

40%

60%

80%

100%

Very Well Adequately

4.5%

N=241

Very Satisfied Satisfied

25.5%

Somewhat Satified or Not Satisfied 70%

N=423

Poorly

Conclusions • Navigate TestPrep: EMT users first-time pass rates are higher than the national average. • Navigate TestPrep: EMT can predict the probability of passing the NREMT exam. • Navigate TestPrep: EMT successfully simulates the NREMT exam. • Students showed that Navigate TestPrep: EMT made a difference in their success rate.

Jones & Bartlett Learning | 5 Wall Street | Burlington, MA | 01803 phone: 1-800-832-0034 | fax: 978-443-8000 | www.jblearning.com

For more information on Navigate, visit www.jblnavigate.com

Suggest Documents