Evaluating LIS Program-Level Student Learning ...

We’re Expected to Think, Aren’t We? Evaluating LIS Program-Level Student Learning Outcomes Elizabeth Lieutenant, Master’s Degree Candidate The Catholic University of America, Washington, DC ✉ [email protected] ✉ 🌐 www.elizabethlieutenant.com 🌐 This poster presents the framework for a quantitative content analysis of master’s library and information studies (LIS) program-level student learning outcomes across all degree programs accredited by the American Library Association (ALA).

Problem Statement

Program-level student learning outcomes are a critical component of all educational programs. They form the essential frame of reference for designing curricula, delivering instruction, and assessing student learning. A broad-based analysis of LIS program-level student learning outcomes has never been conducted (O’Connor & Mulvaney, 2013; Bishop, Grubesic, & Parrish, 2015). This study will measure the intellectual rigor and content of LIS program-level student learning outcomes. The results of this study could be used as an alternative measure of LIS program quality.

Content Analysis Methodology Data Collection Procedure •Locate all ALA-accredited degree programlevel student learning outcomes (PLSLO) statements by reviewing each program’s website. •Capture and archive all PLSLO statements. •Direct confirmation verifying currency of statements and compliance with the 2015 ALA Standards for Accreditation.

Define Units of Analysis •Sampling Units: Individual PLSLO that meet at least one (1) of the criteria in research question 1. •Recording/Coding Units: (a) Measurable action verb denoting specific cognitive processes; (b) description of specific knowledge, skills, and abilities. •Analysis Units: Enumerated coding results.

Research Questions

1. Do LIS programs have program-level student learning outcomes that include: 1. Measurable action verbs that clearly define the cognitive processes students are expected to use? 2. Specific descriptions of the knowledge, skills, and abilities students are expected to learn and demonstrate upon graduation? 2. How are higher- and lower-order cognitive processes represented in program-level student learning outcomes? 3. How are knowledge, skills, and abilities represented in programlevel student learning outcomes? 4. How are the relationships between cognitive processes and knowledge, skills, and abilities expressed in program-level student learning outcomes?

References

American Library Association. (2008). Core Competences of Librarianship. Retrieved from http://www.ala.org/ala/educationcareers/careers/corecomp/corecompetences/finalcorecom pstat09.pdf American Library Association. (2015). Standards for Accreditation of Master's Programs in Library and Information Studies. Retrieved from http://www.ala.org/accreditedprograms/standards Anderson, L. W., and Krathwohl, D. R. (Eds.). (2000). Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Longman: New York, NY. Bishop , B. W., Grubesic, T. H., & Parrish T. (2015). Collaborative development and assessment of student learning outcomes for LIS electives. Journal of Education for Library and Information Science, 56(4), 272-282. doi:10.12783/issn.2328-2967/56/4/1 O’Connor, D., and Mulvaney, P. (2013). ALA accountability and accreditation of LIS programs. Library Journal, 138(14), 440.

Category Development

Coding Procedure

•Inductive development of coding scheme. •Cognitive process categories derived from Revision of Bloom's Taxonomy of Educational Objectives. •Knowledge, skills, and abilities categories derived from ALA Core Competences of Librarianship.

•Hybrid content analysis methodology. •Deductive application of coding scheme to data. •Use MAXQDAplus software for coding and preliminary analysis.

Data Analysis

Establish Reliability

•Tabulation of (a) cognitive processes and (b) knowledge, skills, and abilities to measure strengths and weaknesses (research questions 2 and 3). •Cross-tabulation of (a) cognitive processes and (b) knowledge, skills, and abilities to assess relationships (research question 4).

•Recruit at least one additional coder, preferably with prior knowledge of PLSLO construction. •Provide training, pilot test a preliminary sample of data, then formally measure interrater reliability. •Scott’s 𝛑 (for two coders) or Krippendorff’s 𝛂 (for three or more coders).