Carnegie Mellon University From the SelectedWorks of Denise Troll Covey
January 2011
Open Access and Library Resources Compete in Importance
Contact Author
Start Your Own SelectedWorks
Notify Me of New Work
Available at: http://works.bepress.com/denise_troll_covey/61
Open Access and Library Resources Compete in Importance
Denise Troll Covey Scholarly Communications Librarian Carnegie Mellon University
[email protected] Janel A. Sutkus Director, Institutional Research and Analysis Carnegie Mellon University
[email protected]
Purpose – Few assessments of academic library value have included the value of open access resources, despite widespread agreement that dramatic changes in the scholarly landscape challenge the library’s relevance and viability. This paper reports on faculty and graduate student perceptions of the relative importance of various resources to particular teaching and research activities. Methodology – Faculty and graduate students were surveyed and group difference tests conducted on the findings using independent samples t‐tests and Chi‐square cross tabulations.
Findings – Open access resources compete in importance with full‐text digital resources provided by the library. Both are more important to research than teaching, but open access resources are the most important resource in some teaching activities. Implications – Understanding student and faculty use of open access resources has implications for library acquisitions and instruction. Research is needed to improve our understanding of factors that affect the shift to open access.
Keywords – academic library value, open access 1. Introduction The academic library landscape has changed dramatically over the past decade. Changes in user behavior, driven by technology, and increasing financial pressure, driven by the recession, compel libraries to demonstrate their value to their institution. In this environment, decisions about staffing, services, facilities, and the acquisition of materials must be informed by evidence of need and importance. Libraries are conducting assessments of their value. Some studies endeavor to show the return on investment in the library. Others attempt to show the library’s contribution to the institution’s mission. At Carnegie Mellon, the University Libraries chose the latter approach. Carnegie Mellon is a relatively small, private, research university. Our mission is to create and disseminate knowledge and art through research and creative inquiry and to serve our students by teaching them problem solving, leadership and teamwork skills, and the value of commitment to quality, ethical behavior, and respect for others (2008). To assess and enhance 1
our contribution to this mission, the University Libraries needs both an analysis of trends in the use of library resources and an analysis of the importance of various resources to student and faculty work, regardless of whether we provide these materials. Developing an effective strategic plan and appropriately allocating the Libraries’ human and financial resources to serve the university’s mission requires us to understand our position in the larger information landscape. In 2010 Carnegie Mellon University Libraries surveyed campus faculty and graduate students. The primary goal of the surveys was to assess the relative importance of various resources – provided by the library or freely available (open access) on the web – to particular teaching and research activities. The secondary goal was to understand faculty and graduate student practice of making their work available open access. The surveys were developed in collaboration with and administered and analyzed by the Office of Institutional Research and Analysis. The faculty survey was conducted spring 2010, the graduate student survey fall 2010. 2. Method Several items in the surveys were loosely based on questions posed by the University of Washington Libraries in a 2007 survey of research scientists. Considered leaders in creating a culture of assessment in the academic library community, the University of Washington asked respondents to rate the importance of selected library resources and of the library’s contribution to selected activities. Similar to their survey, we captured demographic data to conduct group difference tests of the importance of various resources by college affiliation and research funding. Unlike the Washington survey, our surveys also enabled us to determine the importance of each resource to each activity and to conduct group difference tests by teaching responsibilities, faculty track, faculty age bracket, and graduate degree program. Appropriately interpreting the findings requires a rudimentary understanding of how Carnegie Mellon operates. This background information is provided as needed throughout this article. The surveys focused on activities on Carnegie Mellon’s main campus in Pittsburgh, Pennsylvania, during the 2009‐10 academic year. All Pittsburgh‐based faculty on the research, tenure, or teaching track, and all non‐student employees who taught or conducted research as their primary function (e.g., postdoctoral researchers) were invited to participate in the faculty survey during the spring semester of 2010. All Pittsburgh‐based graduate students enrolled both semesters of the 2009‐10 academic year were invited to participate in the graduate student survey during the fall semester of 2010. Reminder invitations were sent one and two weeks after the initial invitations. Response rates are shown in table 1. Tables 2 and 3 show the respondents’ demographic characteristics. 2
Table 1. Response.
Initial invitations Initial respondents Initial response rate Reminder invitations Reminder respondents Reminder response rate Reminder invitations Reminder respondents Reminder response rate Total Response rate
Faculty 1,262 169 13.4% 1,093 105 9.6% 988 34 3.5% 308 24.4%
Grad students 1,000 205 20.5% 795 61 7.7% 734 39 5.3% 305 30.5%
Table 2. Faculty respondent demographics.
College
Respondents
CFA CIT HNZ HSS MC SCS TSB
Fine Arts Engineering Public Policy Humanities and Social Sciences Science and Mathematics Computer Science Business and Economics Total
24 62 11 70 71 49 21 308
8% 20% 4% 23% 23% 16% 7%
Taught undergrad course 22 42 3 56 47 15 14 199 65%
Taught graduate course 13 36 9 27 29 26 18 158 51%
Received research funding 5 47 4 31 46 37 13 183 59%
Used research funding 5 54 5 39 54 43 16 216 70%
Age Group
Under 30 31‐36 36‐40 41‐45 46‐50 51‐55 56‐60 61‐65 Over 65 Total
Taught undergrad course 12 22 26 25 25 28 23 19 19 199 65%
Respondents 24 40 39 37 43 40 32 24 29 308
8% 13% 13% 12% 14% 13% 10% 8% 9%
3
Taught graduate course 6 13 20 21 27 21 22 11 17 158 51%
Received research funding 10 29 25 24 27 22 18 14 14 183 59%
Used research funding 21 33 32 28 28 25 19 15 15 216 70%
Table 3. Graduate student respondent demographics.
College
Respondents
CFA CIT HNZ HSS MCS SCS TSB
Fine Arts Engineering Public Policy Humanities and Social Sciences Science and Mathematics Computer Science Business and Economics Total
14 111 34 19 27 52 48 305
5% 36% 11% 6% 9% 17% 16%
Ph.D. 2 80 7 17 26 41 12 185 61%
Research Master’s degree 2 14 3 1 1 4 4 29 10%
Professional Master’s degree 10 17 24 141 0 7 32 91 30%
Taught Used any research course funding 2 2 21 44 6 3 11 6 9 13 10 27 3 6 62 158 20% 52%
Demographic groups responded to the surveys at different rates. To compensate for this disparity and its potential to bias the relationship between respondents’ answers and their demographic characteristics, the collected data were weighted to simulate what might have happened if the entire population responded. Data from the faculty survey were weighted based on a cross‐tabulation of primary college affiliation and age; data from the graduate student survey were weighted based on a cross‐tabulation of college affiliation and program type (Ph.D., research Master’s, professional Master’s) and the weights were applied to all analyses. The primary analytic method for the survey findings was group difference testing. With continuous data (e.g., ratings of importance), independent samples t‐tests were conducted to compare the mean of one group to the mean of another group. With nominal data (e.g., self‐ archiving practices), cross‐tabulations with Chi‐square were used. All group difference tests were conducted strictly among groups of faculty or groups of graduate students. No comparisons were conducted across faculty and graduate student groups. 3. The Importance of Resources We asked faculty and graduate students to rate the importance of various resources to various activities using a unipolar scale with five options from very important to unimportant, and a sixth option of does not apply to me. The scale used in the analysis is shown in table 4 and the mean scores are shown in table 5. Table 4.
Very important Important Moderately important Of little importance Unimportant Does not apply to me
4
4 3 2 1 0 Not included
Table 5. The importance of resources to activities.
Resources Tasks / Respondents Keeping current in their field Finding information in other fields Being productive Being an effective instructor Enriching student learning Saving time Preparing grant proposals
Provided by the University Libraries Digital full‐text Physical Databases materials materials Faculty Grad Faculty Grad Faculty Grad 3.56 3.38 3.39 3.37 2.45 2.49 2.99 2.73 3.19 3.12 2.40 2.53 3.59 3.42 3.36 3.38 2.59 2.83 2.90 2.65 2.70 2.79 2.60 2.59 2.89 2.71 2.69 2.76 2.64 2.53 3.44 3.13 3.25 3.23 2.27 2.26 3.10 2.96 2.97 2.94 1.97 2.26
Open access resources Faculty 3.23 3.39 3.23 3.10 3.09 3.32 2.89
Grad 3.41 3.47 3.34 3.01 3.11 3.40 2.98
We expected digital materials to be more important to faculty and graduate students than physical materials and this turned out to be the case. Respondents rated digital full‐text and databases provided by the library and resources freely available on the web as important or moderately important for all activities. They rated physical materials provided by the library as only moderately important for all activities. The importance of physical materials did not exceed the importance of digital materials on any task. Assuming that graduate students regularly interacted with technology from an earlier age than many faculty, we expected graduate students to value open access resources more than faculty do. This appears to be the case, though the difference is probably not statistically significant. What is noteworthy is the relative importance of open access resources and full‐text digital materials provided by the library. With one exception, graduate students consider open access resources to be more important in the activities assessed than digital full text provided by the library. For three of the seven activities, even faculty rated open access resources as more important than digital full text provided by the library. Faculty and graduate students appear to value different resources for different activities. In terms of keeping current in their field, faculty value the contribution of full‐text digital materials provided by the library significantly more than any other resource; graduate students value the contribution of open access resources slightly more than that of digital full text provided by the library, suggesting that graduate students, more than faculty, use open access resources in their primary work. Faculty and graduate students agree that full‐text digital materials provided by the library make the greatest contribution to their being productive, and that open access resources make the greatest contribution to finding information in other fields, being effective instructors, and enriching student learning experiences. They disagree about what resources are most important to saving time and preparing grant proposals. Faculty rated full‐text digital materials provided by the library as most important to these activities. Graduate students rated open access resources as much more important than digital full text provided by the library for saving time and somewhat more important for preparing grant proposals. Based loosely on age, we expected faculty to value physical resources more than graduate students do. To our surprise, there appears to be little if any significant difference, though 5
students and faculty disagree somewhat about the contributions of physical materials. Both groups value the contribution of physical materials to being effective instructors, but faculty most value the contribution of physical materials to enriching student learning experiences. Graduate students most value the contribution of physical materials to being productive, an unexpected finding. The overall mean scores are of limited usefulness in strategic planning. We assumed that college affiliation, teaching and research activities, research funding, faculty track, faculty age, and graduate student program would play a significant role in how faculty and graduate students valued resources. We therefore calculated several sets of group means and tested them against each other to identify significant differences. Where no significant difference existed, the groups placed equal importance on that resource. 3.1. Findings by College Significant differences by college need to be interpreted cautiously. The survey data associate respondents only with their home college. However, much of the work conducted at Carnegie Mellon is interdisciplinary. Many faculty members have appointments or engage in significant collaborations with faculty in multiple departments or colleges. Many graduate degree programs are likewise collaborative ventures of multiple departments or colleges. Furthermore, within a single college, such as the College of Humanities and Social Sciences, each department can have very different scholarly practices. With that in mind, we expected faculty and graduate students in the College of Engineering (CIT), Mellon College of Science (MCS), and School of Computer Science (SCS) to value full‐text digital materials and databases provided by the library more than their peers in other disciplines. We expected faculty and graduate students in the College of Fine Arts (CFA) and College of Humanities and Social Sciences (HSS) to value physical materials more than their peers. We expected those in disciplines with a self‐archiving culture, such as SCS, to value open access resources more than their peers in disciplines without a self‐archiving culture. With rare exception, the significant differences that surfaced in the survey met our expectations for faculty. Compared with all other faculty, on four or more of the activities assessed: Full‐text digital materials and databases provided by the library are significantly more important to CIT faculty and significantly less important to CFA faculty. Databases provided by the library are also significantly more important to HSS faculty. Physical materials provided by the library are significantly more important to CFA and HSS faculty and significantly less important to SCS faculty. Open access resources are significantly more important to SCS faculty. CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
6
Compared with all other graduate students, on four or more of the activities assessed: Databases provided by the library are significantly more important to HSS graduate students. Physical materials provided by the library are significantly more important to CIT and HSS graduate students. The finding for CIT is unexpected. In general, graduate students appear to value physical materials more than faculty, and CIT graduate students value physical materials significantly more than most of their peers. 3.2. Findings by Other Demographics Most faculty at Carnegie Mellon are appointed to either the tenure, research, or teaching track. Tenure‐track faculty are expected to teach and to conduct research. Research‐track faculty are not expected to teach, but to secure funding and conduct research. Teaching‐track faculty are not expected to conduct research, but to teach. Each track has its own policy and requirements for promotion. Graduate students are enrolled in a Ph.D. program or master’s degree program. Master’s degree programs may be research‐oriented, possibly preliminary to enrolling in a Ph.D. program, or professionally‐oriented. The two types of master’s degree programs are referred to as research master’s and professional master’s degree programs. We expected full‐text digital materials provided by the library to be more important to those who conduct research than to those whose primary responsibility is teaching. The results meet our expectation. Among graduate students, in four or more activities, full‐text digital materials provided by the library are significantly more important to Ph.D. students than to master’s degree students, though surprisingly there was no significant difference between research master’s and professional master’s degree students. Among faculty, in four or more activities, those with research funding value full‐text digital materials provided by the library significantly more than those without research funding. Teaching‐track faculty value digital full text provided by the library significantly less than faculty not on the teaching track. As expected, on four or more activities, open access resources are significantly more important to research‐track faculty than to faculty not on the research track, and significantly less important to faculty over age 65 than to faculty age 65 and younger. Physical resources provided by the library are significantly more important to faculty who teach undergraduate courses than to those who do not teach undergraduates, and significantly less important to faculty with research funding (all research‐track faculty and many tenure‐track faculty) than to those without. CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
7
4. Self‐archiving Practice We also asked faculty and graduate students to indicate whether they self‐archived their work and if so, in which of multiple locations. The results are shown in table 6. We expected at least 40 percent of the faculty to have self‐archived their work (Troll Covey, 2008), but did not know what to expect of the graduate students. To our surprise, 83 percent of faculty and graduate students responded that their work was available open access. As expected, personal and departmental websites are the most popular places to self‐archive. More faculty self‐archive in a disciplinary repository than graduate students. Faculty and graduate students self‐archive in Carnegie Mellon’s institutional repository, Research Showcase, at similarly low rates. Table 6. Self‐archiving practice.
Venue Personal website Departmental website Disciplinary repository CMU Research Showcase Work not available open access
Faculty 46% 27% 19% 5% 17%
Grad 33% 27% 9% 7% 17%
We expected more faculty and graduate students in disciplines with a self‐archiving culture to self‐ archive than those in disciplines without a self‐archiving culture. We expected more of those whose primary responsibility is research to self‐archive than those whose primary responsibility is teaching. We expected those who self‐archive in disciplines with a disciplinary repository to prefer self‐archiving in that repository. Among faculty, the data meet our expectations: Significantly more faculty with research funding self‐archive than those without research funding. Significantly more faculty who teach graduate courses self‐archive than those who do not teach graduate courses. There was no significant difference in self‐archiving practice based on teaching or not teaching undergraduate courses. Compared with faculty in other colleges, significantly more SCS faculty self‐archive on a website, significantly more faculty in MCS self‐archive in a disciplinary repository, and significantly fewer faculty in CIT self‐archive anywhere. Significantly more tenure‐track faculty self‐archive in a disciplinary repository than faculty not on the tenure track. The picture is somewhat different with graduate students: Compared with graduate students in other colleges, significantly more graduate students in SCS and CFA self‐archive on a website; significantly more graduate students in MCS, HSS, and TSB self‐archive in a disciplinary repository; significantly more graduate students in CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
8
MCS and HNZ self‐archive in Carnegie Mellon’s institutional repository, Research Showcase; and significantly fewer graduate students in CIT self‐archive anywhere. Significantly more graduate students with research funding self‐archive than those without research funding. Significantly more graduate students who teach self‐archive than those who do not teach. Significantly more Ph.D. students self‐archive than master’s degree students. Significantly more students enrolled in research master’s degree programs self‐archive than those enrolled in professional master’s degree programs. We also asked graduate students about plans to make their thesis, dissertation, or final project available open access. Respondents could select multiple options. Overall, the most popular choice (43 percent) was depositing their work in a Carnegie Mellon open access repository other than Research Showcase. However, more graduate students (37 percent) plan to deposit their work in Research Showcase than in a non‐CMU repository (23 percent). The least popular choice (20 percent) was making their work available open access in ProQuest. Open access plans varied across colleges. Table 7 indicates venues where significantly more graduate students in each college plan to deposit their work in comparison with their peers in other colleges. Plans in CIT are noteworthy given that significantly fewer CIT graduate students currently self‐archive. In addition: Significantly more Ph.D. students than master’s degree students plan to make their dissertation or thesis available open access in any venue. Significantly more research master’s degree students than professional master’s degree students plan to make their work available open access in Research Showcase or ProQuest. Significantly more graduate students who teach or have research funding, compared to those who do not, plan to make their dissertation or thesis available open access in Research Showcase, another CMU repository, or ProQuest. Table 7. Open access plans for dissertation, thesis, or final project.
Research Other CMU College Showcase repository CFA CIT X HNZ HSS X X MCS X X SCS X TSB
Non‐CMU repository X
ProQuest X X
X
X
CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
9
5. Conclusions Resources provided by the University Libraries contribute to the university’s mission of creating and disseminating knowledge and art. Library resources are important to faculty and graduate students throughout the university, though what resources they value most for various activities differs significantly across academic disciplines and primary responsibilities (teaching or research). Overall, databases and digital full text are more important than physical materials. Faculty and graduate students rated digital full text as the most important resource contributing to their productivity. Full‐text digital materials provided by the library are significantly more important to those whose primary responsibility is research. In contrast, physical materials provided by the library are significantly more important to those whose primary responsibility is teaching. Though rated overall as only moderately important, physical materials provided by the library continue to be important to some demographic groups. Compared with their peers in other disciplines, physical materials are significantly more important to faculty in CFA and HSS and to graduate students in CIT and HSS. Physical materials are also significantly more important to faculty who teach undergraduate courses than to those who do not, but they are not more important to graduate students who teach than to those who do not. Faculty and graduate students disagree about the activities to which physical materials make the greatest contribution. Open access resources compete with library resources in importance. Though the competition is more striking in graduate student responses than faculty responses, for several activities even faculty rated open access resources as the most important resource. Open access resources are significantly more important to research‐track faculty than to other faculty, but they are also important in teaching. Faculty and graduate students rated open access resources as the most important resources contributing to their being effective instructors and enriching student learning experiences. These findings augment the discovery in another study conducted by the University Libraries in 2010 that faculty found roughly 22 percent of the materials they used in their academic work freely available on the web (St. Clair, Troll Covey, and Linke, forthcoming). Open access resources are changing the scholarly information landscape and the library’s position in it. In addition to using open access resources, most Carnegie Mellon faculty and graduate students make their work available open access. Research responsibilities and funding appear to drive self‐ archiving practice, as does disciplinary culture and the availability of disciplinary repositories. Given that key publishers in engineering allow self‐archiving, the unexpected finding that, compared to their peers in other disciplines, significantly fewer faculty and graduate students in CIT currently self‐archive warrants investigation. CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
10
6. Next Steps We need to situate the survey findings and determine next steps within the larger context of changes occurring in the scholarly information landscape. The increasing availability and use of open access resources is an important change. The transition from print to electronic journals is veritably complete, providing much of the digital full text valued by our faculty and graduate students. The transition from print to electronic books is accelerating. Each of these changes is a harbinger of opportunity to reallocate human and financial resources. Significant disciplinary differences identified in the surveys must be investigated in light of the interdisciplinary research and teaching conducted at Carnegie Mellon. Liaison librarians plan to consult their constituencies to determine if the survey findings resonate with their perception of the importance of various resources. The librarians will also explore how faculty and graduate students use physical materials provided by the library, with particular attention to differences between faculty and graduate student use in CFA and CIT. Understanding how different groups use physical materials is critical to developing a strategy and pace for transitioning from physical books to e‐books across the disciplines. The findings on self‐archiving practice and graduate student plans to provide open access to their dissertation or thesis are encouraging. We need to understand the dearth of self‐archiving by faculty and graduate students in CIT and to take aggressive steps to populate our institutional repository. The faculty and graduate student surveys and faculty focus groups conducted in 2010 (Troll Covey, 2011) indicate that we must promote Research Showcase as an open access venue for faculty and graduate student work, harvest material they have self‐archived elsewhere, and provide incentives to garner their direct participation. In the future we plan to investigate what percentage of materials used by faculty and students are found freely available on the web and to conduct a study of undergraduate student perceptions and values related to libraries in general and the University Libraries in particular. These studies and those already conducted will inform our strategic plan and drive our resource allocation. As more materials become available open access and disciplines increasingly rely on these materials, we expect to recover financial resources and human capacity, enabling us to acquire important materials that are not available open access and to develop new services designed to support teaching, learning, and research. References Carnegie Mellon 2008 Strategic Plan, http://www.cmu.edu/strategic‐plan/2008‐strategic‐ plan/2008‐strategic‐plan.pdf (accessed 24 June 2011). CFA = Fine arts CIT = Engineering HNZ = Public policy HSS = Humanities and social sciences MCS = Science and mathematics SCS = Computer science TSB = Business and economics
11
St. Clair, G., Troll Covey, D. and Linke, E. (forthcoming), “A Model for Campus Information Consumption.” Troll Covey, D. (2008), “Faculty Self‐Archiving Practice: A Case Study,” white paper, http://works.bepress.com/denise_troll_covey/35 (accessed 24 June 2011). Troll Covey, D. (2011), “Recruiting Content for the Institutional Repository: The Barriers Exceed the Benefits,” Journal of Digital Information, Vol. 12 No. 3, http://works.bepress.com/denise_troll_covey/56/.
12
