A User-Centred Approach to Metadata Design Emma Tonkin UKOLN, University of Bath, BA27AY, UK
[email protected] http://www.ukoln.ac.uk/
Abstract. The process of development of metadata elements and structures can be approached and supported in a number of different ways. We sketch a user-centred approach to this process, based around an iterative development methodology, and briefly outline some major questions, challenges and benefits related to this approach. Keywords: Metadata, user-centred design, evaluation.
1
Introduction
Development of application profiles (APs) – a description of an application of a metadata set – and metadata vocabularies is well-documented from many perspectives, with research focusing on aspects such as interoperability concerns, modularity and reuse. We describe work arising from the JISC-funded IEMSR metadata schema registry project. IEMSR attempts to support the process of AP and metadata vocabulary development in Dublin Core (DC) and IEEE Learning Object Metadata (LOM) via desktop and Web-based client software that draw on the central registry as a resource[3]. As the IEMSR progresses from prototype towards evaluation, collection of user feedback and use in real-world contexts, we seek to align the software more closely with present-day best practice in metadata schema development. However, metadata development processes are often relatively informal, drawing on professional experience rather than a formally encoded process model. To inform this development, it was necessary to investigate current development methodologies and to develop a compatible approach, briefly sketched here. 1.1
Iterative Development
Our model is based on the star development life cycle[1], an iterative cycle that emphasises input from stakeholders and frequent evaluation stages, reflecting an underlying assertion that to speak of the usability of metadata is appropriate in a large subset of cases. The accuracy of this assertion depends on the specific context of use; for example, a metadata fragment destined for use only by software acts as an internal data structure, making developers the sole stakeholder group. By contrast, an AP may be in widespread use, often in several widely L. Kov´ acs, N. Fuhr, and C. Meghini (Eds.): ECDL 2007, LNCS 4675, pp. 551–554, 2007. c Springer-Verlag Berlin Heidelberg 2007
552
E. Tonkin
Task/functional analysis
Implementation
Prototyping
Evaluation
Requirements Specification
Conceptual/ formal design
Fig. 1. The star development lifecycle
geographically distributed communities. This distributed usage pattern has a limiting impact on knowledge transfer and the available channels for feedback or repair, leading to circumstances in which usage patterns diverge. Iterative design methods are sometimes used at present for AP developments; most processes follow an essentially sequential model, although most permit ongoing application of minor changes and apply a corresponding versioning process (eg. [7]).
2
Activities
Application of the star methodology imposes prerequisites on the process of AP design. Stakeholders must be identified and approached, requirements (aims and objectives) identified, managed and encoded at each iteration. The evaluation stages of the methodology introduce additional challenges; evaluation of candidate APs or elements requires the identification of an appropriate technique for evaluating metadata structures. Evaluation methods have been proposed and applied for various aspects of digital library systems, eg.[2], [6]. Exploration of ‘paper prototyping’ approaches can permit exploration of metadata usage in context before development takes place, allowing issues to be identified at an early stage. 2.1
Supporting the Design Process
The schema registry may support this process in a number of ways – for example, by: promoting an appropriate methodology, providing the means to develop tools to support analysis of metadata use, and recording and allowing appropriate visualisation of changes made. It may also act as a focal point bringing together widely geographically distributed development communities. 2.2
Evaluation
Evaluation represents a key stage in this process. Marshall and Shipman[5] argue that users may be unwilling or unable to explicitly express information, and thus circumvent structures that require such formalisation. They describe many of
A User-Centred Approach to Metadata Design
553
the difficulties that this may cause, such as cognitive overhead and the need to develop, apply and consult a wealth of tacit knowledge during the process of developing a formalisation. The enforcement of a rigid, potentially misdesigned structure is characterised as a disincentive to make use of a system. They then outline an approach designed to minimise such problems, suggesting the following five principles: – Designers need to work with users to reach a shared understanding of the use situation and the representations that best serve it. – Designers must identify what other services or user benefits the computer can provide based on trade-offs introduced by additional formalization. – Designers should also expect, allow, and support reconceptualization and incremental formalization in longer tasks. – Taking a similar, computationally-based approach, designers may provide facilities that use automatically recognized (but undeclared) structures to support common user activities. – Finally, training and facilitation can be used to help users effectively work with embedded formalisms. We may add the following points: – Designers must identify appropriate metrics for system evaluation. – Metrics are required for detection and handling of systematic or random error in input. Consequences of error should be considered throughout the design phase. – System repair and feedback methods should be considered; Marshall & Shipman invoke the need to handle semantic change. – Approaches to error-handling and correction should be considered. 2.3
Means and Modes of Evaluation
There are various metrics that may indicate that a knowledge representation is failing the user. Dushay et al[4] demonstrate several in their analysis of DC metadata in use. – Missing data – Confusing or inconsistent data – Incomplete data Their approach to discovering flaws was based around a visual graphical analysis tool called Spotfire, and is designed for application by a human analyst rather than an as part of an automated system. However, inconsistencies and missing data fields may generally be identified. For example, inconsistently applied encoding or syntax can be recognised using a variety of approaches, such as the use of sequence classifiers. Building and applying a limited set of partial grammars to describe the contents of a given field is possible; this is part of a general field of research covering the development and application of grammar checking techniques for natural or formal languages.
554
3
E. Tonkin
Future Work
Detailed user testing and evaluation is currently planned for the components making up the IEMSR. This will also enable detailed case studies to be undertaken regarding the development model described here, feeding into the development of a mature policy and guidance framework. We will also evaluate current developments in the area of computer-supported collaborative work, such as Web 2.0, for relevance to registry design. Particularly of relevance to the IEMSR project are developments in the areas of interoperability and architectural requirements.
References 1. Hix, D., Hartson, H.R.: Developing User Interfaces. Wiley, Chichester (1993) 2. Fuhr, N., Hansen, P., Mabe, M., Micsic, A., Sølvberg, I.: Digital Libraries: A generic classification and evaluation scheme. In: Constantopoulos, P., Sølvberg, I.T. (eds.) ECDL 2001. LNCS, vol. 2163, Springer, Heidelberg (2001) 3. Heery, R., Johnston, P., Beckett, D., Steer, D.: JISC Metadata Schema Registry. In: JCDL 2005 (2005) 4. Dushay, N., Hillman, D.I.: Analyzing Metadata for Effective Use and Re-Use. In: Proceedings of the Dublin Core Conference, September 28 - October 2, 2003, Seattle, WA (2003) 5. Shipman, F.M., Marshall, C.: Formality considered harmful: Experiences, emerging themes, and directions on the use of formal representations in interactive systems. In: Proc. Computer Supported Cooperative Work (CSCW) 1999, pp. 333–352 (1999) 6. Kralisch, A., Yeo, A.W., Nurfauza, J.: Linguistic and Cultural Differences in Information Categorization and Their Impact on Website Use. In: Proceedings of the Thirty-ninth Hawaii International Conference On System Sciences. January 4-7. Hawaii, USA (2006) 7. Kulvatunyou, B., Morris, K.C., Buhwan, J., Goyal, P.: Development Life Cycle and Tools for XML Content Models. In: XML Conference 2004 (2004)
