Open Book Publishers logo Open Access logo
  • button
  • button
  • button
GO TO...
Contents
Copyright
book cover
BUY THE BOOK

Introduction

© W. Sutherland et al., CC BY 4.0 https://doi.org/10.11647/OBP.0179.13

This book has been created to help you make decisions about practical conservation management by providing an assessment, from the available scientific evidence, of what works and what does not work in conservation. It also tells you if no evidence has been found about whether or not a conservation intervention is effective. This is the 2019 edition of What Works in Conservation, which was first published in 2015 and is updated annually.

Who is What Works in Conservation for?

This book is for people who have to make decisions about how best to support or conserve biodiversity. These include land managers, conservationists in the public or private sector, farmers, campaigners, advisors or consultants, policymakers, researchers or people taking action to protect local wildlife. What Works in Conservation and the associated synopses summarize scientific evidence relevant to conservation objectives and the actions that could be taken to achieve them. What Works in Conservation also provides an assessment of the effectiveness of interventions based on available evidence.

We do not aim to make decisions for people, but to support decision-making by providing what evidence there is (or is not) about the effects that your planned actions could have. It is important that you read the full details of the evidence, freely available online at www.conservationevidence.com, before making any decisions about implementing an intervention.

The Conservation Evidence project

The Conservation Evidence project has four parts, all of which are available from our website conservationevidence.com:

  1. An ever-expanding searchable database of over 5,500 summaries of previously published scientific papers, reports, reviews or systematic reviews that document the effects of interventions.
  2. Synopses of the evidence captured in part 1) relating to particular species groups, habitats or conservation issues. Synopses bring together the evidence for all possible interventions. Synopses are also available to purchase in printed book form, or can be downloaded for free as electronic material.
  3. What Works in Conservation provides an assessment of the effectiveness of interventions based on available evidence. It contains both the key messages from the evidence for each conservation intervention from the relevant synopses, and an assessment of the effectiveness of each intervention by expert panels.
  4. An online, open access journal Conservation Evidence that publishes new pieces of research on the effects of conservation management interventions. All our papers are written by, or in conjunction with, those who carried out the conservation work and include some monitoring of its effects.

Alongside this project, the Centre for Evidence-Based Conservation (http://www.cebc.bangor.ac.uk) and the Collaboration for Environmental Evidence (http://www.environmentalevidence.org) carry out and compile systematic reviews of evidence on the effectiveness of particular conservation interventions. We recommend carrying out a systematic review, which is more comprehensive than our summaries of evidence, when decisions have to be made with particularly important consequences. Systematic reviews are included in the Conservation Evidence database.

Which conservation interventions are included?

Lists of interventions for each synopsis are developed and agreed in partnership with an advisory board made up of international conservationists and academics with expertise in the subject. We aim to include all actions that have been carried out or advised for the conservation of the specific group of species or habitat or for the specific conservation issue.

The lists of interventions are organized into categories based on the International Union for the Conservation of Nature (IUCN) classifications of direct threats and conservation actions (https://www.iucnredlist.org/resources/classification-schemes). Interventions are primarily grouped according to the relevant direct threats. However, some interventions can be used in response to many different threats and so these have been grouped according to conservation action.

How we review the literature

We gather evidence by searching relevant scientific journals from volume one through to the most recent volume. Thirty general conservation journals are regularly searched by Conservation Evidence. Specialist journals are also searched for each synopsis (248 have been searched so far). We also search reports, unpublished literature and evidence provided by our advisory boards. Two of the synopses used systematic mapping exercises undertaken by, or in partnership with, other institutions. Systematic mapping uses a rigorous search protocol (involving an array of specified search terms) to retrieve studies from several scientific databases. Evidence published in languages other than English is included when it is identified. Evidence from all around the world is included in synopses. One exception is farmland conservation, which only covers northern Europe (all European countries west of Russia, but not those south of France, Switzerland, Austria, Hungary and Romania). Any apparent bias towards evidence from some regions in a particular synopsis reflects the current biases in published research papers available to Conservation Evidence.

The criteria for inclusion of studies in the Conservation Evidence database are as follows:

  • A conservation intervention must have been carried out.
  • The effects of the intervention must have been monitored quantitatively.

These criteria exclude studies examining the effects of specific interventions without actually doing them. For example, predictive modelling studies and studies looking at species distributions in areas with long-standing management histories (correlative studies) are excluded. Such studies can suggest that an intervention could be effective, but do not provide direct evidence of a causal relationship between the intervention and the observed biodiversity pattern.

For each study we summarise the results that are relevant to each intervention. Unless specifically stated, results reflect statistical tests performed on the data within the papers.

What does What Works in Conservation include?

What Works in Conservation includes only the key messages from each synopsis, which provide a rapid overview of the evidence. These messages are condensed from the summary text for each intervention within each synopsis. For the full text and references see www.conservationevidence.com

Panels of experts have assessed the collated evidence for each intervention to determine effectiveness, certainty of the evidence and, in most cases, whether there are negative side-effects (harms). Using these assessments, interventions are categorized based on a combination of effectiveness (the size of benefit or harm) and certainty (the strength of the evidence). The following categories are used: Beneficial, Likely to be beneficial, Trade-off between benefit and harms, Unknown effectiveness, Unlikely to be beneficial, Likely to be ineffective or harmful (for more details see below).

Expert assessment of the evidence

The average of several experts’ opinions has been shown to be a more reliable and accurate assessment than the opinion of a single expert. We therefore ask a panel of experts to use their judgement to assess whether evidence within the synopsis indicates that an intervention is effective or not. They are also asked to assess how certain they are of the effectiveness given the quality of evidence available for that intervention (certainty of the evidence). Negative side-effects described in the collated evidence are also assessed (harms). They base their assessment solely on the evidence in the synopsis. We use a modified Delphi method to quantify the effectiveness and certainty of evidence of each intervention, based on the summarized evidence. The Delphi method is a structured process that involves asking a panel of experts to state their individual opinion on a subject by scoring anonymously. They can then revise their own scores after seeing a summary of scores and comments from the rest of the panel. Final scores are then collated. Scores and comments are kept anonymous throughout the process so that participants are not overly influenced by any single member of the panel.

For each intervention, experts are asked to read the summarized evidence in the synopsis and then score to indicate their assessment of the following:

Effectiveness: 0 = no effect, 100% = always effective.

The score uses an assessment by independent experts of the effectiveness of this action based on the summarized evidence (0% = not effective, 100% = highly effective). This score is based on the direction and size of the effects reported in each study. Actions with high scores typically have large, desirable effects on the target species/habitat in each study. There is some variation between actions, e.g. 100% effectiveness in adding underpasses under roads for bat conservation will likely have different impacts to 100% effectiveness in restoring marsh habitat. The effectiveness score does not consider the quantity or quality of studies; a single, poorly designed study could generate a high effectiveness score. The effectiveness score is combined with the certainty and harms scores to determine the overall effectiveness category (for more details see https://www.conservationevidence.com/content/page/79)

Certainty of the evidence: 0 = no evidence, 100% = high quality evidence; complete certainty. This is certainty of effectiveness of intervention, not of harms.

An assessment by independent experts of the certainty of the evidence for this action based on the summarized evidence (0% = no evidence, 100% = high quality evidence). How certain can we be that the effectiveness score applies to all targets of the intervention (e.g. all birds for an action in the bird synopsis)? This score is based on the number, quality and coverage (species, habitats, geographical locations) of studies. Actions with high scores are supported by lots of well-designed studies with a broad coverage relative to the scope of the intervention. However, the definition of “lots” and “well-designed” will vary between interventions and synopses depending on the breadth of the subject. The certainty score is combined with the effectiveness and harms scores to determine the overall effectiveness category.

Harms: 0 = none, 100% = major negative side-effects to the group of species/ habitat of concern.

An assessment by independent experts of the harms of this action to the target group of species/habitat, based on the summarized evidence (0% = none, 100% = major undesirable effects). Undesirable effects on other groups of species/habitats are not considered in this score. The harms score is combined with the effectiveness and certainty scores to determine the overall effectiveness category.

Categorization of interventions

After one or two rounds of initial scoring, interventions are categorized by their effectiveness, as assessed by the expert panel. The median score from all the experts’ assessments is calculated for the effectiveness, certainty and harms for each intervention. Categorization is based on these median values i.e. on a combination of the size of the benefit and harm and the strength of the evidence. The table and figure overleaf show how interventions are categorized using the median scores. There is an important distinction between lack of benefit and lack of evidence of benefit.

Once interventions are categorized, experts are given the chance to object if they believe an intervention has been categorized incorrectly. Interventions that receive a specified number (depending on the size of the panel) of strong objections from experts are re-scored by the expert panel and re-categorized accordingly. Experts did not see the categories for the farmland synopsis or for the ‘Reduce predation by other species’ section of the bird synopsis and so those categories are based on the second round of scoring.

How to use What Works in Conservation

Please remember that the categories provided in this book are meant as a guide and a starting point in assessing the effectiveness of conservation interventions. The assessments are based on the available evidence for the target group of species for each intervention and may therefore refer to different species or habitat to the one(s) you are considering. Before making any decisions about implementing interventions it is vital that you read the more detailed accounts of the evidence, in order to assess their relevance to your species or system. Full details of the evidence are available at www.conservationevidence.com.

There may also be significant negative side-effects on the target groups or other species or communities that have not been identified in our assessment. A lack of evidence means that we have been unable to assess whether or not an intervention is effective or has any harmful impacts.

Table of categories of effectiveness

Category

Description

General criteria

Thresholds

Beneficial

Effectiveness has been demonstrated by clear evidence. Expectation of harms is small compared with the benefits

High median benefit score

High median certainty score

Low median harm score

Effectiveness: >60%

Certainty: >60%

Harm: <20%

Likely to be beneficial

Effectiveness is less well established than for those listed under ‘beneficial’

OR

There is clear evidence of medium effectiveness

High benefit score

Lower certainty score

Low harm score

OR

Medium benefit score

High certainty score

Low harm score

Effectiveness: >60%

Certainty: 40–60%

Harm: <20%

OR

Effectiveness: 40–60%

Certainty: ≥40%

Harm: <20%

Trade-off between benefit and harms

Interventions for which practitioners must weigh up the beneficial and harmful effects according to individual circumstances and priorities

Medium benefit and medium harm scores

OR

High benefit and high harm scores

High certainty score

Effectiveness: ≥40%

Certainty: ≥40%

Harm: ≥20%

Unknown effectiveness (limited evidence)

Currently insufficient data, or data of inadequate quality

Low certainty score

Effectiveness: Any

Certainty: <40%

Harm: Any

Unlikely to be beneficial

Lack of effectiveness is less well established than for those listed under ‘likely to be ineffective or harmful’

Low benefit score

Medium certainty score and/or some variation between experts

Effectiveness: <40%

Certainty: 40–60%

Harm: <20%

Likely to be ineffective or harmful

Ineffectiveness or harmfulness has been demonstrated by clear evidence

Low benefit score

High certainty score

(regardless of harms)

OR

Low benefit score

High harm score

(regardless of certainty of effectiveness)

Effectiveness: <40%

Certainty: >60%

Harm: Any

OR

Effectiveness: <40%

Certainty: ≥ 40%

Harm: ≥20%

Categories of effectiveness based on a combination of effectiveness (the size of the benefit and harm) and certainty (the strength of the evidence). The top graph refers to interventions with harms <20% and the bottom graph to interventions with harms ≥20%.