From Citizen Science to Volunteer Monitoring:

From Citizen Science to Volunteer Monitoring: seeking hybridization of models for community science

All photos by ALLARM

Candie C. Wilderman and Jennifer L. Shirk Seventh National Monitoring Conference April 29, 2010

I.

Outline

Taxonomy of community science models II. Evaluation of outcomes rubric III. Strength analysis of models based on three categories of major outcomes IV. Suggestions for hybridization

What is community science? • Community science is partnership between professional scientists (university, agency, or industry) and volunteers (residents) to systematically document and analyze an environmental condition of concern or interest. • The primary goal of community science is to produce “useful” data.

The diversity of operational models in community science • There are a variety of successful operational models for community science. • These models differ in their goals, the nature and scope of the projects, the extent of community control over the definition and implementation of the project, and the outcomes.

Citizen Science Toolkit Conference June 2007 at Cornell Lab of Ornithology

• Community science projects from “worms to stars” were represented • Opportunity for project directors involved in different models for community science to learn from each other • Clear gap in communication between academic researchers involved in citizen science (top down) and the more bottom up volunteer monitoring community

Models have shared challenges • Recruiting and training volunteers • Ensuring data quality • Managing large data sets • Getting volunteer-collected data accepted and used by various audiences.

Models have shared goals • Increasing scientific literacy of the public

Models have differences in terms of goals and outcomes • • • • • •

Geographic scope of project Intended audience for outcomes Scientific value of outcomes Educational value of outcomes Actionability of outcomes Capacity-building value of outcomes

These attributes can be thought of as multidimensional continua along which the models can be “plotted.”

I.

Outline


Developing a taxonomy for the various models for community science can be informed by examining the answers to the following questions (Wilderman 2007): • Who defines the problem? • Who designs the study? • Who collects the data? – Sample collection? – Analysis of samples? • Who interprets the data? • Who communicates the results (tells the story)? • Who takes action? As the model moves from top-down to bottom-up, the number of questions addressed by “community members,” rather than “professional scientists” increases.

The Center for Advancement of Informal Science Education (CAISE) • In response to a request by the National Science Foundation, the Center for Advancement of Informal Science Education (CAISE) established an Inquiry Group to: – describe models for public participation in scientific research (PPSR); – understand and describe the educational impacts of PPSR projects; and – make recommendations for conceptualizing and developing future ISE activities that will enhance public participation in scientific research.

http://caise.insci.org/

The product:

Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J., and Wilderman, C.C.

http://caise.insci.org/uploads/docs/PPSR%20report%20FINAL.pdf

CAISE taxonomy: similar to Wilderman (2007) Steps in Scientific Process

Steps in which volunteers participate: Contributory Projects

Consulting Projects

Co-created Projects

Choose or define questions of study

X

X

Gather information and resources

(X)

X

Develop hypotheses

(X)

X

Develop study design Collect data

X X

X

Analyze samples

X

Interpret data and draw conclusions

X

Disseminate conclusions/translate results into action Discuss results and ask new questions

(X)

X

X

(Adapted from Bonney, Ballard, Jordan, McCallie, Phillips, Shirk, and Wilderman, 2009)

Contributory Projects (Citizen-Science) Steps in Scientific Process

Steps in which volunteers participate: Contributory Projects

Collect data

X

Disseminate conclusions/translate results into action

(X)

Weather monitoring stations Backyard bird counts

• Top-down, scientist-driven • Issues studied usually have a wide geographic range • Volunteers are primarily data collectors

CONTRIBUTORY

ALLARM acid rain project, Spotting the Weedy Invasives (Rutgers U), and The Birdhouse Network (Cornell Lab of Ornithology)

CONTRIBUTORY

Monarch Larvae Monitoring Project (U Minn) and CoCoRaHS (Community Collaborative Rain, Hail and Snow Network (CO State Univ )

CONSULTING model projects Steps in Scientific Process

Steps in which volunteers participate: Consulting Projects


X


(X)

Develop hypotheses

(X)

European Science Shops

Community Health Effects of Industrial Hog Operations

Consulting model lends itself to doing community science within a university course framework

Watershed-based Integrated Field Semester (Luce Semester)

From the Bay to the Bayou

Using the consulting or “science shop” model, all students do an independent research project on an research question defined by ALLARM’s partner groups.

CO-CREATED, Community-based Participatory Research (CBPR), or Participatory Action Research (PAR) projects Steps in Scientific Process

Steps in which volunteers participate: Co-created Projects


X


X

Develop hypotheses

X

Develop study design

X

Collect data

X

Analyze samples

X

Interpret data and draw conclusions

X

Disseminate conclusions/translate results into action

X

Discuss results and ask new questions

X

•Bottom up, community-driven •Issues are usually local •Volunteers participate in all steps of the scientific process

Importance of Service Providers • In this model, partnerships with scientists are critical to producing valid, credible data • Role of scientists is to provide capacity-building programmatic and scientific technical assistance to groups – to guide them to reach their own goals • Scientists can also do validation studies and QA/QC to provide data credibility • Two examples of service providers: – PA Consortium for Scientific Assistance to Watersheds, funded by the PA DEP – University of Florida and LAKEWATCH

I.

Outline


Assessment Rubric for Describing Impacts of Public Participation in Scientific Research Projects Impact category Awareness, knowledge, and/or understanding of: Content (Concepts) Process Nature of science Careers Community Engagement of interest in: Content (Concepts) Process Community Careers Skills Asking questions Study design Data collection Data analysis Data interpretation Discuss results Disseminate results Using technology Writing Community

Stated goal

Potential indicators

Measured outcomes

Inferred outcomes

Impact category Attitudes Toward science Toward content Toward people About activities Toward species About careers About theories About community Behaviors Time engaged Time ourdoors Lifestyle changes Within community Community involvement Citizen action Responsible environmental behavior New participation Other Social capital Community capacity Economic impact

Stated goal

Potential Measured Inferred indicators outcomes outcomes

Need for evaluative tools • We quickly discovered that few PPSR projects have been evaluated comprehensively. • In the end, our sample of 10 projects included most of the USA-based PPSR projects that we could locate that included measured and reported outcomes or for which we felt that we could infer outcomes from available information. • The need for evaluative tools is critical.

I.

Outline


Major outcomes of community science: the three-legged stool

Research findings

Community action Science education

GRAPHIC DEPICTION OF STRENGTH ANALYSIS OF OUTCOMES OF THREE MODELS

Research Results Contributory

Science Education Community action Research Results Consulting

Science Education Community action

Research Results

Co-created

Science Education

Community action

I.

Outline


Hybridization • Projects at the extremes have particular strengths, but also limitations that must be acknowledged • Combining attributes of the extreme end models results in maximizing the potential for realizing strengths in all three outcome categories

Steps in which volunteers participate:

Maybe put examples ofSalah hybrids here, Harvest Steps in Scientific Process Contributory Co-created Sustainability Projects that such exist. Projects just to demonstrate Project Choose or define questions of study

X


X

Develop hypotheses

X

X

Develop study design

X

X

X

X

X

X

(X)

X

Collect data

X

Analyze samples Interpret data and draw conclusions Disseminate conclusions/translate results into action Discuss results and ask new questions

(X)

X

(X)

X

Example of an existing “hybrid”* Salal Harvest Sustainability Study (2001-2004)

• Sponsored by the Community Forestry and Environmental Research Partnership and the Northwest Research and Harvester Association. • The project involved harvesters of salal (Gaultheria shallon) in research to determine the effects of harvest intensity on the plant’s growth and biological and commercial production on Washington’s Olympic Peninsula.

*The CAISE report calls this a “collaborative” model

• This project is considered a hybrid because, while the project scientist had the final say on all steps in the process, participants who had a wealth of local information played a key role in certain steps, including developing study hypotheses, designing the study, and interpreting the data.

Suggestions for enhancing contributory projects

Suggestions include ways to expand the public involvement in the scientific process, thereby strengthening the educational and community action outcomes.

• Involve public in study design • Hold regional training workshops to develop skills of data interpretation and analysis • Make data more available and provide tools for simple visualization and analysis • Have participants develop their own questions and support more local projects that are spawned by the larger study • Train volunteers to utilize data for action such as reporting to local newspapers, blogs, and informing local decisions about land use planning

Example: Urban Bird Gardens • A project that was initiated by the Cornell Lab of Ornithology because of the evidence that the traditional contributory approach deterred participation by urban residents, particularly Latino families, and limited the educational and scientific reach of associated projects. • Project leaders opened the project design to community input: – Interviews with community leaders revealed that projects should address issues of community relevance, through components such as enhancement of urban green spaces and intergenerational learning

• These ideas are, however, challenging the capacity of project staff and suggest that new programmatic infrastructure needs to be adapted to achieve this.

Suggestions for enhancing co-created projects

Suggestions include ways to expand the geographic scope of the monitoring and to encourage the publication of scientific data; thereby enhancing the research strength of the project.

• Work together to design and develop regional and national databases that would allow participants to compare data and findings across larger regions and over longer spans of time • Enhance educational support materials such as online videos and identification keys that can be used over a broad geographic area by many local groups • Partner with professional scientists to encourage publication of results in scientific journals and to create new projects to expand the scope of activities

Example: Ecological Monitoring and Assessment Network Coordinating Office* • Environment Canada’s Ecological Monitoring and Assessment Network oversees a rich set of local and regional monitoring programs that engage the public in documenting diverse indicators of environmental health. • The Coordinating Office was established to develop a standardized suite of protocols and a common data management framework, two attributes which additionally addressed the needs of ecosystem assessment at larger scales. • The Coordinating Office also developed mechanisms for timely communication between network partners which allowed decisions to be made at all scales within the context of understanding larger ecosystem trends. • These large ecosystem trends resulted in scientific publications. *Despite creating an integrated network of monitoring initiatives operating at multiple scales, funding cuts forced the closure of EMAN CO in 2008.

Conclusions • The variety of models for community science can be categorized based on the extent of community control over the definition, implementation and outcomes of the project. • Outcomes for community science include research findings, science education, and community action; the strength of these outcomes differ among community science models. • Models must be chosen based on the goals of the project. • Although there has been little communication between the communities of practice of the different models, we suggest that collaboration across project types could be of benefit, leading to stronger volunteer monitoring programs and more meaningful outcomes for the public participating in citizen science projects.

NSF-ISE: possible funding source for volunteer monitoring programs? • Relatively few of the thousands of community science projects have been funded by the Informal Science Education (ISE) program at the National Science Foundation (NSF). • Most of the projects funded by the NSF-ISE program have been developed by the Cornell Lab of Ornithology which uses the contributory model. • The CAISE Inquiry Report on PPSR suggests that projects which involve members of the public in the largest number of steps or categories of research yield the greatest impact in terms of understanding science process, developing skills and changing behavior. These are the bottom up models, including most water quality volunteer monitoring programs. • There is a clear opportunity for future funding for volunteer monitoring programs with specific informal science education goals from this major source. http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5361&org=DRL&from=home

For more information: •

Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J., and Wilderman, C.C. 2009. Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report. Washington, D.C.: Center for Advancement of Informal Science Education (CAISE)

•

Ely, Eleanor, 2008. Volunteer Monitoring & the Democratization of Science, The Volunteer Monitor, 19(1), pp.1,3-5.

•

Wilderman, C.C., 2007. “Models of Community Science: Design Lessons from the Field,” pp. 83-97 in McEver, C., Bonney, R., Dickinson, J., Kelling, S., Rosenberg, K., and Shirk, J. (Eds). Proceedings of the Citizen Science Toolkit Conference. Cornell Laboratory of Ornithology, Ithaca, NY, June 20-23, 2007. www.birds.cornell.edu/citscitoolkit/conference/presentations

•

Wilderman, C., A. Barron and L. Imgrund, 2003, “The ALLARM program: growth, change, and lessons learned,” The Volunteer Monitor, 15(1), pp.1-4.

•

Wilderman, C.C., and J. Vastine, 2005. “Breaking the Code: Data Analysis Workshops,” The Volunteer Monitor, 17(1), pp. 11-14.