Incorrect Answers in by Topics

Proportions of Corrects/Incorrect Answers in by Topics, Bloom’s Taxonomy, and Knowledge Dimension in Formative and Summative Assessments in a Microbiology Lab Course(MCB3023L, 13354, 2016-1 Term) Prof: Felix E. Rivera-Mariani, PhD December 09, 2016

Summary This report summarizes a data analysis workflow of proportion of correct and incorrect answers by students in the upper division Microbiology Laboratory course (MCB3023L, section 13354 term 2016-1). The purpose was to evaluate the proportions of correct and incorrect in questions administered in formative (quizzes, polling questions) and summative assessments (midterm and final exams). The role of different topics, levels of Bloom’s taxonomy (remembering, understanding, applying, analyzing, evaluating, creating), and Knowledge dimensions (factual, conceptual, and procedural) on the correct and incorrect ratios was evaluated, and the interaction between topics and Bloom’s taxonomy levels was examined through logistic regression models. Based on the findings from the proportions and logistic regression models, interventions were recommended for prospective sections of the Microbiology Laboratory course. Note: This report was prepared through the publicly-available statistical software R. All the codes for the statistical analysis in this report can be found here, and the complete version of this report (including all the codes) can be found here. • Report Outline 1) 2) 3) 4) 5) 6)

Questions to Answer Formative Assessments in the Course Summative Assessments in the Course Data Data Cleaning Data Analysis a) Tables of Proportions b) Logistic regression of correct/incorrect by Bloom’s taxonomy levels and knowledge dimension 7) Conclusion 8) Recommendations

Questions to Answer 1) In which topics did students have the lowest probabilities of correct? 2) In which Bloom’s taxonomy did students have the lowest probabilities of correct? 3) In which Knowledge dimension did students have the lowest probabilities of correct?

1

Formative Assessments in the Course 1) Quizzes. There were 6 in-class lowstakes quizzes, of 6 questions each. These quizzes are administered within the first 20 minutes of class. Students’ grade in these quizzes is calculated based on 5 questions: the remaining question is bonus. Lastly, the questions are mainly short-answer question to better assess their thinking and provide better feedback based on their word-selection. 2) Reef-polling questions. These are questions administered to the students during the course of a class/lab session. Through a smartphone app (Reef-PollingTM ), students answer the questions posted within the slides discussed in class. This allows assessing misconceptions and provide feedback in real-time because the software automatically generates graph based on the students’ answers to the question posted. Because students create an account with this application, the power point slide gets recorded in their accounts for students to go back to the material covered in each of the individual questions for studying purposes.

Summative Assessments in the Course 1) Midterm Exam. I administered a 25-question theoretical midterm, which was contained a combination of questions formats: multiple choice, short-answer, and true/false. I strongly believe that students should have many opportunities to showcase their learning. For this reason, I implemented many forms of assessments, including a summative assessment midpoint through the course (Midterm Exam). The midterm also allows to evaluate which topics and levels of learning may need interventions for the remaining of the course. 2) Unknown project. The way I organize the unknown is as follows: a) Samples with mixed cultures are prepared. Each sample has a coliform and a non-coliform. b) They work on isolating and preparing pure culture of both bacterial species. c) Through a Dichotomous key that they learn to prepare, they carry out physiological tests as needed. d) Once the complete all tests according to their Dichotomous key, they are required to perform a confirmation test to verify their identification was correctly. 3) Lab notebook. The purpose of the lab notebook is for them to practice proper documentation, including how to correct fix written mistakes, write scientific names, and collect data from experiments. 4) Final exam. In the final exam, in addition to the 24 practical questions related to each of the 24 stations, I included 16 additional theoretical questions. The purpose of these additional 16 questions was to assess is there was any change in midterm to final exam performance, provide them more room to apply their learning, and contribute additional data for this report.

Data Find below the codes to upload all the corresponding data for this statistical analysis. There are included to demonstrate reproducibility in the analysis. Nevertheless, the data is only stored in my personal computer.

library(xlsx) # upload package to handle Excel files # create file path for each assignment pathrf