Introduction

This book has been created to help you make decisions about practical conservation management by providing an assessment, from the available scientific evidence, of what works and what does not work in conservation. It also tells you if no evidence has been found about whether or not a conservation intervention is effective. This is the 2018 edition of What Works in Conservation, which was first published in 2015 and will be updated annually.

Who is What Works in Conservation for?

This book is for people who have to make decisions about how best to support or conserve biodiversity. These include land managers, conservationists in the public or private sector, farmers, campaigners, advisors or consultants, policymakers, researchers or people taking action to protect local wildlife. What Works in Conservation and the associated synopses summarize scientific evidence relevant to conservation objectives and the actions that could be taken to achieve them. What Works in Conservation also provides an assessment of the effectiveness of interventions based on available evidence.

We do not aim to make decisions for people, but to support decision-making by providing what evidence there is (or is not) about the effects that your planned actions could have. It is important that you read the full details of the evidence, freely available online at www.conservationevidence.com, before making any decisions about implementing an intervention.

The Conservation Evidence project

The Conservation Evidence project has four parts, all of which are available from our website conservationevidence.com:

1. An ever-expanding searchable database of over 5,400 summaries of previously published scientific papers, reports, reviews or systematic reviews that document the effects of interventions.
2. Synopses of the evidence captured in part 1) relating to particular species groups, habitats or conservation issues. Synopses bring together the evidence for all possible interventions. Synopses are also available to purchase in printed book form, or can be downloaded for free as electronic material.
3. What Works in Conservation provides an assessment of the effectiveness of interventions based on available evidence. It contains both the key messages from the evidence for each conservation intervention from the relevant synopses, and an assessment of the effectiveness of each intervention by expert panels.
4. An online, open access journal Conservation Evidence that publishes new pieces of research on the effects of conservation management interventions. All our papers are written by, or in conjunction with, those who carried out the conservation work and include some monitoring of its effects.

Alongside this project, the Centre for Evidence-Based Conservation (http://www.cebc.bangor.ac.uk) and the Collaboration for Environmental Evidence (http://www.environmentalevidence.org) carry out and compile systematic reviews of evidence on the effectiveness of particular conservation interventions. We recommend carrying out a systematic review, which is more comprehensive than our summaries of evidence, when decisions have to be made with particularly important consequences. Systematic reviews are included in the Conservation Evidence database.

Which conservation interventions are included?

Lists of interventions for each synopsis are developed and agreed in partnership with an advisory board made up of international conservationists and academics with expertise in the subject. We aim to include all actions that have been carried out or advised for the conservation of the specific group of species or habitat or for the specific conservation issue.

The lists of interventions are organized into categories based on the International Union for the Conservation of Nature (IUCN) classifications of direct threats and conservation actions (http://www.iucnredlist.org/technical-documents/classification-schemes). Interventions are primarily grouped according to the relevant direct threats. However, some interventions can be used in response to many different threats and so these have been grouped according to conservation action.

How we review the literature

We gather evidence by searching relevant scientific journals from volume one through to the most recent volume. Thirty general conservation journals are regularly searched by Conservation Evidence. Specialist journals are also searched for each synopsis (231 have been searched so far). We also search reports, unpublished literature and evidence provided by our advisory boards. Two of the synopses used systematic mapping exercises undertaken by, or in partnership with, other institutions. Systematic mapping uses a rigorous search protocol (involving an array of specified search terms) to retrieve studies from several scientific databases. Evidence published in languages other than English is included when it is identified. Evidence from all around the world is included in synopses. One exception is farmland conservation, which only covers northern Europe (all European countries west of Russia, but not those south of France, Switzerland, Austria, Hungary and Romania). Any apparent bias towards evidence from some regions in a particular synopsis reflects the current biases in published research papers available to Conservation Evidence.

The criteria for inclusion of studies in the Conservation Evidence database are as follows:

• A conservation intervention must have been carried out.
• The effects of the intervention must have been monitored quantitatively.

These criteria exclude studies examining the effects of specific interventions without actually doing them. For example, predictive modelling studies and studies looking at species distributions in areas with long-standing management histories (correlative studies) are excluded. Such studies can suggest that an intervention could be effective, but do not provide direct evidence of a causal relationship between the intervention and the observed biodiversity pattern.

We summarise the results of each study that are relevant to each intervention. Unless specifically stated, results reflect statistical tests performed on the data within the papers.

What does What Works in Conservation include?

What Works in Conservation includes only the key messages from each synopsis, which provide a rapid overview of the evidence. These messages are condensed from the summary text for each intervention within each synopsis. For the full text and references see www.conservationevidence.com

Panels of experts have assessed the collated evidence for each intervention to determine effectiveness, certainty of the evidence and, in most cases, whether there are negative side-effects (harms). Using these assessments, interventions are categorized based on a combination of effectiveness (the size of benefit or harm) and certainty (the strength of the evidence). The following categories are used: Beneficial, Likely to be beneficial, Trade-off between benefit and harms, Unknown effectiveness, Unlikely to be beneficial, Likely to be ineffective or harmful (for more details see below).

Expert assessment of the evidence

The average of several experts’ opinions has been shown to be a more reliable and accurate assessment than the opinion of a single expert. We therefore ask a panel of experts to use their judgement to assess whether evidence within the synopsis indicates that an intervention is effective or not. They are also asked to assess how certain they are of the effectiveness given the quality of evidence available for that intervention (certainty of the evidence). Negative side-effects described in the collated evidence are also assessed (harms). They base their assessment solely on the evidence in the synopsis. We use a modified Delphi method to quantify the effectiveness and certainty of evidence of each intervention, based on the summarized evidence. The Delphi method is a structured process that involves asking a panel of experts to state their individual opinion on a subject by scoring anonymously. They can then revise their own scores after seeing a summary of scores and comments from the rest of the panel. Final scores are then collated. Scores and comments are kept anonymous throughout the process so that participants are not overly influenced by any single member of the panel.

For each intervention, experts are asked to read the summarized evidence in the synopsis and then score to indicate their assessment of the following:

Effectiveness: 0 = no effect, 100% = always effective.

Certainty of the evidence: 0 = no evidence, 100% = high quality evidence; complete certainty. This is certainty of effectiveness of intervention, not of harms.

Harms: 0 = none, 100% = major negative side-effects to the group of species/ habitat of concern.

Categorization of interventions

After one or two rounds of initial scoring, interventions are categorized by their effectiveness, as assessed by the expert panel. The median score from all the experts’ assessments is calculated for the effectiveness, certainty and harms for each intervention. Categorization is based on these median values i.e. on a combination of the size of the benefit and harm and the strength of the evidence. The table and figure overleaf show how interventions are categorized using the median scores. There is an important distinction between lack of benefit and lack of evidence of benefit.

Once interventions are categorized, experts are given the chance to object if they believe an intervention has been categorized incorrectly. Interventions that receive a specified number (depending on the size of the panel) of strong objections from experts are re-scored by the expert panel and re-categorized accordingly. Experts did not see the categories for the farmland synopsis or for the ‘Reduce predation by other species’ section of the bird synopsis and so those categories are based on the second round of scoring.

How to use What Works in Conservation

Please remember that the categories provided in this book are meant as a guide and a starting point in assessing the effectiveness of conservation interventions. The assessments are based on the available evidence for the target group of species for each intervention and may therefore refer to different species or habitat to the one(s) you are considering. Before making any decisions about implementing interventions it is vital that you read the more detailed accounts of the evidence, in order to assess their relevance to your species or system. Full details of the evidence are available at www.conservationevidence.com.

There may also be significant negative side-effects on the target groups or other species or communities that have not been identified in our assessment. A lack of evidence means that we have been unable to assess whether or not an intervention is effective or has any harmful impacts.

Table of categories of effectiveness

Category	Description	General criteria	Thresholds
Beneficial	Effectiveness has been demonstrated by clear evidence. Expectation of harms is small compared with the benefits	High median benefit score High median certainty score Low median harm score	Effectiveness: >60% Certainty: >60% Harm: <20%
Likely to be beneficial	Effectiveness is less well established than for those listed under ‘beneficial’ OR There is clear evidence of medium effectiveness	High benefit score Lower certainty score Low harm score OR Medium benefit score High certainty score Low harm score	Effectiveness: >60% Certainty: 40–60% Harm: <20% OR Effectiveness: 40–60% Certainty: ≥40% Harm: <20%
Trade-off between benefit and harms	Interventions for which practitioners must weigh up the beneficial and harmful effects according to individual circumstances and priorities	Medium benefit and medium harm scores OR High benefit and high harm scores High certainty score	Effectiveness: ≥40% Certainty: ≥40% Harm: ≥20%
Unknown effectiveness (limited evidence)	Currently insufficient data, or data of inadequate quality	Low certainty score	Effectiveness: Any Certainty: <40% Harm: Any
Unlikely to be beneficial	Lack of effectiveness is less well established than for those listed under ‘likely to be ineffective or harmful’	Low benefit score Medium certainty score and/or some variation between experts	Effectiveness: <40% Certainty: 40–60% Harm: <20%
Likely to be ineffective or harmful	Ineffectiveness or harmfulness has been demonstrated by clear evidence	Low benefit score High certainty score (regardless of harms) OR Low benefit score High harm score (regardless of certainty of effectiveness)	Effectiveness: <40% Certainty: >60% Harm: Any OR Effectiveness: <40% Certainty: ≥ 40% Harm: ≥20%

Categories of effectiveness based on a combination of effectiveness (the size of the benefit and harm) and certainty (the strength of the evidence). The top graph refers to interventions with harms <20% and the bottom graph to interventions with harms ≥20%.