addressing this bottleneck, including training more metadata specialists; automatic ... and discussion spaces, face-to-face workshops, and conference calls.
THE USER-CENTERED DESIGN OF A NON-SPECIALIST METADATA TOOL AND INTERFACE FOR THE INTERNET PUBLIC LIBRARY Michael Khoo, Jung-Ran Park, Xia Lin The iSchool at Drexel University Philadelphia PA 19104 USA 1. Introduction This poster describes the user-centered design and development of a tool to support nonspecialists specialists – specifically, student volunteers and interns with the Internet Public Library (IPL) – to engage in the creation and quality control of Dublin Core metadata records for the Internet Public Library (IPL). Good quality metadata support rich user interactions with a repository, while poor quality metadata hide resources, produce poor search results, and negatively affect user satisfaction (Barton et al. 2004; Beall 2005; Geisler et al. 2002). There is however a ‘metadata generation bottleneck’ between the growing numbers of digital resources requiring description, and the limited numbers of metadata specialists who can create such descriptions (Liddy et al., 2001). This can result in restricted metadata production, and reduce the utility of digital libraries (Wilson, 2007). There are a number of approaches to addressing this bottleneck, including training more metadata specialists; automatic metadata generation (Liddy et al., 2001; Ochoa & Duval n.d.); and the solution discussed in this poster, using non-specialists, to create their own metadata. 2. Non-Specialist Metadata Tools Non-specialist metadata tools have in the past had mixed success (Crystal & Greenberg, 2005). This is partly related to the efficacy with which these tools convey complex metadata concepts to nonspecialists. Wilson (2007) found technical errors to be common in creator-contributed metadata, and she recommended a better understanding of contributors’ thought processes and improved Web interfaces to “allow contributors to produce high-quality metadata records without having a great deal of organizational expertise.” Khoo (2005) describes how the Digital Water Education Library used K-12 teachers to generate a collection of Web-based pedagogical resources using a cataloging tool, online work and discussion spaces, face-to-face workshops, and conference calls. Despite this activity, cataloging was slow, and analysis of the project’s communication showed that many of the teachers had difficulty understanding the basic concepts that should have underpinned their work, such as ‘digital library,’ and ‘metadata.’ Kastens et al. (2005) describe how the Digital Library for Earth Systems Education’s ‘community cataloging’ tool allowed resource creators, educators, and others to catalog online educational resources “through an easy-to-use web interface.” However, “this approach yielded few resources and the community-cataloged metadata often turned out to be incomplete or incorrect.” Greenberg et al.’s (2003) analysis of author-generated records in the National Institute of Environmental Health Sciences found that some users lacked sufficient understanding of the tool and the purpose of metadata, despite the provision of a tutorial. Finally, Crystal and Greenberg (2005) found that “succinctly explaining the fields of a standardized schema such as Dublin Core to metadata novices appears to be a key challenge for designers,” and that “developing a conceptual understanding of metadata records and their use in retrieval was found to be challenging for users.” 3. The Internet Public Library The IPL receives approximately 1 million visits a month from over 200 countries and territories around the globe. It is currently crosswalking its metadata to Dublin Core. For a number of reasons, the quality of the crosswalked metadata will have to be evaluated, and the IPL is therefore developing a metadata quality control tool. As advocated by Crystal and Greenberg (2005), the tool development will follow a user-centered ‘interaction design’ process (Norman 2002; Sharp et al. 2007) that considers not just the functionality of the tool, but users’ needs. It stresses the importance of requirements gathering and iterative prototyping to avoid ‘building in’ mistakes that can require much effort to address later on.
The work has begun with the elicitation of initial design requirements from students in HCI and metadata classes at Drexel University and at iSchool consortium partners. The students reviewed IPL metadata records using existing prototype tools, critiqued these tools, and then developed hierarchical task analyses, personas, scenarios, paper prototypes (Snyder 2003), and then basic functional prototypes. These were evaluated with heuristic evaluation, cognitive walkthroughs, and think-alouds. The outcomes included usability reports, and data of users’ understanding of the task of metadata quality control. All of these data will support the development of a fully functional prototype that will be subject to further user testing. The accuracy and efficacy of the final tool will be assessed through (i) a review of the accuracy of a random sample of completed quality control forms, and (ii) a comparison of completed forms by different users to test for inter-user reliability. Outcomes The new IPL metadata tools will increase the quality of the library’s metadata, positively impact the search service, and directly benefit the IPL’s millions of global users. The research findings will be useful to other libraries involving users in their metadata work, and will have a direct relevance to the digital library and HCI research communities. The activities are integrated into courses taught at the Drexel iSchool, and so we will also report on the significant curriculum and pedagogical outcomes and implications of this research. Acknowledgement This research is supported by a grant from the OCLC-ALISE Library and Information Science Research Grant Program. References Barton, J., Currier, S., & Hey, J. (2003). Building Quality Assurance into Metadata Creation. Dublin Core Conference 2003, 28th Sept - 2nd Oct, Seattle, Washington, USA. Beall, J. (2004). Metadata and data quality problems in the digital library. Journal of Digital Information 6(3). Crystal, A., & Greenberg, J. (2005). Usability of a metadata creation application for resource authors. Library & Information Science Research 27 (2005), 177-189. Geisler, G., Giersch, S., McArthur, D., & McClelland, M. (2002). Creating virtual collections in Digital Libraries: Benefits and implementation issues. JCDL ’02, Portland OR, USA, pp. 210-218. Greenberg, J., Crystal, A., Robertson, W., & Leadem, E. (2003) Iterative design of metadata creation tools for resource authors. 2003 Dublin Core Conference, Seattle, WA, USA. Kastens, K.A., DeFelice, B., Devaul, H., DiLeonardo, C., Ginger, K., Larsen, S., Mogk, D., & Tahirkheli, A. (2005). Questions & challenges arising in building the collection of a digital library for education: Lessons from five years of DLESE. D-Lib Magazine 11(11). http://www.dlib.org/dlib/november05/kastens/11kastens.html. Khoo, M. (2005). The Tacit Dimensions of User Behavior: The Case of the Digital Water Education Library. JCDL 2005, Denver, CO, June 7-11, 2005, p. 213-222. ACM Press. Liddy, E., Sutton, S., Paik, W., Allen, E., Harwell, S., Monsour, M., Turner, A., & Liddy, J. (2001). Breaking the metadata generation bottleneck: preliminary findings. JCDL ’01, p. 464. ACM Press. Norman, D. (2002). The Design of Everyday Things. New York: Basic Books. Ochoa, X., & Duval, E. (n.d.). Towards Automatic Evaluation of Metadata Quality in Digital Repositories. Ms. http://ariadne.cti.espol.edu.ec/M4M/files/TowardsAutomaticQuality.pdf Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design (2nd edition). New York, NY: Wiley. Snyder, C. (2003). Paper prototyping. San Franciso, CA: Morgan Kaufmann. Wilson, A. (2007). Toward releasing the metadata bottleneck: a baseline evaluation of contributorsupplied metadata. Library Resources and Technical Services, vol. 51, No. 1. (January 2007), pp. 1628.
