In a new paper from PhD research by Luiz Ribas, we tried to answer the question: how do different methods of impact evaluation estimate the effectiveness of protected areas in avoiding deforestation? This study is a systematic review that searched for quantitative measurements of protected areas’ (PAs’) effectiveness around the world.
Until recently in ecology and conservation, a few designs were applied to estimate impacts and measure the effectiveness of different interventions. Typically, these designs disregarded causal mechanisms when selecting control areas that would be compared with areas where conservationists’ actions were applied. Thus, these approaches tended to bias impact estimates of these actions. More recently, counterfactual thinking has been applied to overcome these issues. Counterfactual methods try to compare apples-to-apples when considering similarities between units of analysis submitted to a certain intervention and potential control units. These methods result in more accurate impact estimates.
In our study, we showed differences in impact estimates of one of the most common interventions to circumvent biodiversity loss (PAs) in terms of their most commonly evaluated outcomes (rates of deforestation). As our main result, we found that traditional methods tended to overestimate the impact of PAs in avoiding deforestation when compared with counterfactual methods.
Figure. Effect sizes estimated by both counterfactual and traditional (naive) impact evaluation methods by sample (paired group). Negative values indicate that PAs avoided deforestation, values equal to zero indicate no effect of protection, and positive values indicate more deforestation within PAs than outside. CM: covariate matching; PSM: propensity score matching. * CM, PSM, and their variants are considered to be counterfactual methods.
