research focuses on the design of citizen science platforms, and the potential implications regarding scientific uses and impact. Human factors in citizen science.
Feature
Human factors in citizen science James Sprinks The information age has transformed the process of collecting and storing scientific data, with increasingly sophisticated instruments able to study the universe around us in amazing detail at an exponentially increasing rate. In fact, this abundance of data is proving a headache for the professional scientific community, with many terabytes of information sitting on hard drives remaining unexplored for years, even decades, with its potential unrealised. However, the current digital revolution has made the sharing of data and information increasingly straightforward, in a way that is unrestricted both geographically and temporally. Over the past five to ten years the scientific community has started to use this to their advantage, sharing their data with the public through specially designed platforms and in return asking them to take on a portion of the analytic workload. As technology continues its relentless advance, it is clear that novel approaches will be needed to deal with the ‘data avalanche’, and ‘citizen science’ will continue to grow as a key part of the solution.
Virtual Citizen Science
ABOUT THE AUTHOR
James Sprinks is a postgraduate researcher with the University of Nottingham’s Geospatial Institute. Currently his research focuses on the design of citizen science platforms, and the potential implications regarding scientific uses and impact.
One method by which scientists are using the public to assist with scientific analysis is through Virtual Citizen Science (VCS) platforms. Allowing the general public to access and annotate varying types of scientific data from the comfort of their own home, it could be argued that this above all else has been responsible for a huge rise in citizen science participation in recent times. Currently there are millions of registered citizen scientists around the world contributing to VCS platforms across a range of scientific disciplines. Marking features on the surface of other planets, transcribing the logbooks of historical shipping voyages and identifying the calls of different bat species are just a few of the scientific endeavours that are being undertaken by volunteers. This diversity of pursuit is only likely to increase as other research areas investigate the gains that can be made through using citizen science. The success of VCS platforms is not restricted to the size of communities they have built, but also stretches to their scientific impact. Huge
4 The Ergonomist May 2015
data catalogues have been produced on the back of volunteers’ results, that either have greater coverage than existing ‘expert’ versions, correct anomalies created by computer-derived analysis, or are completely new resources to be exploited for their scientific worth. Beyond the initial aims of the platform, serendipitous discoveries have also been made through volunteer communications on specially designed forums, which would not have been possible by using the perhaps more single-minded expert community or rigid computer analysis.
Citizen scientist tasks and judgements In order to achieve the vast range of scientific goals set out by VCS platforms, volunteers are asked to perform a number of different psychophysical tasks. They vary in both their complexity and involvement, from simple detection tasks such as is a feature present or not in the data, and identification and discrimination for example, what type of feature is it, which is the brighter, louder etc., to data annotation, that is, marking or highlighting features and making measurements. To complete these tasks, a variety of judgements need to be made, decided by the mechanisms present in the VCS design. Answering yes/no questions or making ‘forced choice’ decisions from a selection of categories are judgements frequently required of citizen scientists, along with more subtle decisions such as rating metrics using a provided scale, or threshold detection – identifying an event at the precise moment it is detectable above the background noise. Other variations that exist across virtual citizen science platforms include the autonomy afforded to the volunteer; how much freedom they have to explore the data and perform the required tasks as they choose, and also the number and variety of tasks and associated judgements they have to complete. What is clear is that there appears to be no pattern or correlation to these variations in task design. This is certainly true when considering the scientific discipline involved. For instance, it could be argued that classifying a galaxy type compared to a known catalogue is very similar to identifying a hieroglyphic compared to the
known alphabet - both being a forced choice even though the disciplines of astronomy and ancient Egyptian history are clearly different. It seems that in the early days of VCS platforms, during their conception and design stages, more importance was placed on the results that could be collated and the science cases that could be addressed, rather than considering the impact of the different possible task designs that allow volunteers to contribute.
What about the citizen scientist? The volunteers of virtual citizen science projects are often the ‘elephant in the room’, with the science case and potential end results usually taking precedence. Due to virtual citizen science being a new area of work, although there has been research into interface HCI and functionality, there has been relatively little attention paid specifically to human factors issues. This is perhaps unsurprising as in the early days such platforms were developed by the scientific teams that required them, but could be considered ironic given the importance of the ‘citizen’ part of the discipline, especially as the effectiveness of a citizen science venture is related to its ability to attract and retain engaged users. When considering the user engagement of virtual citizen science platforms, it appears to follow a power law similar to the Pareto distribution, that is, the typical volunteer only visits maybe once or twice, while those who remain interested end up completing the majority of the work. In order to better understand citizen science communities and their members, studies have been carried out looking at citizen scientist motivations, finding that they centre on the opportunity to contribute to ‘real science’. Others have looked more in depth at the volunteers themselves, considering their educational backgrounds, demographics and related contribution behaviour.
and judgement itself. This may be considered remiss, as over 30 years of human factors research has identified a relationship between motivation, satisfaction, performance and task design. Additionally, while it is important to know more about the communities involved in citizen science platforms and their motivations, it is hard to predict at the design stage. Knowing what science case or genre will capture the public’s imagination and garner increased media attention is virtually an impossible task, but the effect of different task designs can be understood and considered during the build process. As citizen science becomes increasingly used as a method for scientific analysis, and the number of virtual citizen science platforms continues to grow, task design will become an ever more important consideration for developers of such platforms. With this, several questions will have to be answered, such as: Who do we design for? Are tasks and judgements derived for the highly engaged minority who do most of the work, or the majority who visit only once or twice? Alternatively, can a more ambitious tack be taken, where task design is used to try and change the pattern of user engagement? Following on from this, what effect does task design have on the validity and quantity of the results? It is likely that future VCS platform science teams and developers will need to perform a balancing act, weighing up the importance of user satisfaction and community engagement against the data needs of the science case and the resources that can be committed to data reduction, more than likely on a case-by-case basis.
What has not considered in any depth is the form of work activity, the task
May 2015 The Ergonomist 5
