The United States and the European Union want to reach net-zero emissions by 2050, China by 2060, and India by 2070. By December 2022, 138 countries had already made net-zero pledges covering more than 80% of global emissions1. Similarly, many large corporations – including Amazon and Volkswagen – have promised to reach carbon neutrality by mid-century or earlier 1, and various firms claim that they are already ‘carbon neutral’ today. Yet, many of these claims entail the purchase of carbon offsets. These are “reduction, avoidance or removal of a unit of greenhouse gas (GHG) emissions by one entity, purchased by another entity to counterbalance a unit of GHG emissions by that other entity.”2 Offsets in the voluntary carbon market today almost exclusively rely on reducing or avoiding emissions through, for instance, more fuel-efficient cookstoves or improved forest protection. While offsets based on carbon removal are growing, they only constitute a minor share of current voluntary carbon markets and are not the focus of this study.
What explains the major role that offsets are playing in corporate strategies is the implicit assumption that carbon offsets are economically efficient, as emissions reductions are achieved where they are cheapest 3. However, for an offset project to contribute to emissions reductions, offsets need to conform to environmental integrity criteria, such as additionality4,5 (i.e., reduction/removals would not have occurred without the project), durability (i.e., reduction/removals are not subject to near-term reversal or renewed at fixed intervals), and not leading to leakage (i.e., merely displacing emissions elsewhere). In addition, many carbon offset projects aim to create additional positive environmental and socioeconomic co-benefits, such as enhanced biodiversity or poverty alleviation.2
Carbon offsets have come under considerable criticism, however, as the underlying projects may not lead to actual emissions reductions6. Carbon offsets are commonly issued by comparing the actual carbon reductions of a project to a hypothetical baseline scenario if the project had not been implemented. This counterfactual baseline scenario is typically based on extrapolating historical emission trends. Yet, historical baselines are commonly an imperfect guide to future emissions. It is, therefore, critical to contrast the ex-ante estimated emissions reductions to the ex-post achieved emissions reductions by offset projects. This allows us to gauge what the offset achieved relative to what has been claimed ex-ante. We call this the ‘offset achievement ratio’ (see Methods for detailed explanation), which is the share of achieved emissions reductions based on credible academic studies relative to the claims made by project developers ex-ante.
While several studies have assessed the actual emissions reductions that were realized in individual offset projects relative to the expected reductions claimed by the verifiers6,7, systematic and large-scale evidence of the actual reductions covering the full range of offset sectors is missing (for definition and a full list of sectors, see Table 1). In line with conventional systematic review methodology8 and based on a Context-Intervention-Mechanism(s)-Outcome(s) logic (CIMO) 8, the central question of this analysis is therefore: ‘What is known in the scientific literature about the differences between ex-ante estimates and ex-post outcomes of individual carbon offsetting projects adopted to enable the transition towards a net-zero emission economy across multiple sectors?’
We proceed in four steps. First, we define keywords to identify potentially relevant scientific studies across all major carbon offset sectors. As many offsetting sub-sectors only constitute a fraction of a per cent, we focus on the largest sub-sectors which, combined, make up more than 90% of credits issued in the voluntary offset market (Figure 1). Second, we use the artificial-intelligence-supported systematic review tool AS Review9 to filter for relevant studies (e.g., using experimental or rigorous observational research methodologies) from 64,993 potentially relevant studies identified in the first step (Supplementary Figure 1 and Supplementary Table 1 & 2 for search terms). Third, we download the full text of the studies identified using AS Review and manually check for relevance (see Supplementary Table 3 for criteria). Fourth, two researchers independently extract the ex-post computed emissions reductions from individual projects and other relevant aspects of the study detailed in our Codebook. Lastly, for each project, we compute an offset achievement ratio. For field interventions that did not officially issue offsets, we compute a ‘synthetic’ offset achievement ratio (i.e., the ratio of achieved emissions reductions if these projects had used assumptions of similar, real-world projects to issue offsets; see Methods for detailed approach). In total, our final sample comprises more than 2,000 offset projects, and 130 effect sizes from 61 studies (see Extended Data 1).
Our analysis extends the existing literature in two major ways: First, we provide the first cross-sectoral, quantitative assessment of the offset achievement ratio of carbon offset projects in the peer-reviewed literature10 and highlight insights on durability, co-benefits, and other relevant factors from these studies (see Supplementary Table 4 for previous meta-analyses in the non-peer reviewed literature). Second, we complemented the evidence on offset projects with 51 ex-post evaluations from field interventions that tested interventions similar to offset projects and jointly comprise 1.2 million observations. For instance, less than half of projects that attempt to reduce deforestation from deforestation and forest degradation (REDD+) have issued offsets11. Yet, there is a large, high-quality literature that investigates the underlying effectiveness of such interventions12,13 allowing us to assess whether assumptions made by project developers using these interventions for offset projects are realistic.
Offset projects and field interventions
We conducted a systematic review of the offset achievement ratio of offset projects. In total, our set of studies includes offset sectors that jointly issued around 90% of carbon offsets (Figure 1). These contain 7 main sectors and 14 sub-sectors as defined by the Berkeley Carbon Trading Project14 (Table 1).
Table 1: Sectors, sub-sectors and descriptions of offset sectors. Directly cited and text shortened from Berkeley Voluntary Registry Offsets database scope & types document (Version April 2021).12
Forestry & Land Use
|
REDD+
|
Reducing deforestation and forest degradation in the global south. Many REDD+ projects bundle several activities (e.g., improved forest management, afforestation/reforestation). The “+” in REDD+ refers to the many project co-benefits (e.g., biodiversity).
|
Improved Forest Management
|
Applying practices which increase above and below-ground carbon stocks including reducing timber harvest levels, extending timber harvest rotations, designating reserves, fuel load treatments, enrichment planting, and stand irrigation or fertilization
|
Afforestation & Reforestation
|
Planting trees and reducing barriers to natural regeneration in non-urban areas.
|
Renewable Energy
|
Wind
|
Installing wind turbines for grid-connected electricity generation replacing traditional, fossil-fuel or natural gas combustion for electricity production
|
Hydropower
|
Installing large and small-scale hydroelectric power plant (HEPP) turbines to generate electricity through regular dam flow operations or additions to multipurpose reservoirs
|
Solar
|
Installing solar modules as electricity production for grid-connected energy use.
|
Biomass
|
Generating heat, electricity (grid-connected or direct use), and/or biogas from renewable biomass, commonly utilizing agricultural waste biomass.
|
Waste management
|
Landfill/wastewater methane
|
Landfill: Reducing and combusting methane from landfills including municipal, industrial, and other solid waste facilities. Wastewater: Treating wastewater to capture and flare methane, process with anaerobic digesters, and/or dewater sludge by drying before disposal
|
Chemical processes
|
Ozone-depleting substances
|
Collecting and destroying refrigerants that are ozone-depleting substances with high GWP from discarded equipment such as air conditioners, refrigerators, and foam. We also include the recovery and destruction of SF6 and HFC-23 in this category.
|
N2O destruction in nitric acid production
|
Installing abatement measures and catalytic reduction units to destroy N2O emissions from nitric acid factories and caprolactam production plants. Nitric acid (HNO3) and caprolactam are crucial components of fertilizer and synthetic fibre production.
|
Household and community
|
Cookstoves
|
Building improved cookstoves to replace or minimize the use of dung or firewood for cooking. Carbon benefits are realized in the form of reduced emissions from burning biomass as well as reducing deforestation. Less smoke leads to improved health benefits.
|
Industrial manufacturing
|
Mine methane capture
|
Capturing and destroying or using mine methane that would otherwise be released to the atmosphere from active and abandoned coal, trona, and precious and base metal mines.
|
Natural gas electricity production
|
Constructing new natural gas-fired grid-connected electricity generation plants replacing higher greenhouse gas intensity fuels like coal. The fuel sources for the plants are fossil fuel natural gas, not renewable natural gas harvested through decomposition processes.
|
Carbon capture and storage
|
Carbon Capture and Enhanced Oil Recovery
|
Capturing carbon dioxide from industrial processes followed by compression, transport and injection for permanent storage underground while also enhancing oil recovery.
|
Projects from forestry and renewable energy projects dominate the voluntary carbon market and constitute 74% of issued credits (Figure 1a/b). Industrial manufacturing, waste management, chemical processes, and household and community jointly constitute 23% (we use household and cookstoves interchangeably throughout the text as there is only one category). Carbon Capture and Storage (CCS), agriculture, and transportation together account for around 3%. Each sector is composed of sub-sectors. For instance, the forestry sector contains projects related to REDD+, forest management, and afforestation (though for forestry there is substantial overlap between these categories as REDD+ is a broad term). In turn, the renewable energy sector contains projects from wind, solar, biomass, and hydropower.
Following the typology of the Berkeley Carbon Project, we classify each of the 61 studies in our review into one of seven sectors with 14 sub-sectors (Figure 1). We differentiate between studies that investigate projects that officially issued carbon offsets and those projects that used a similar field intervention but did not officially issue offsets. We found 10 studies investigating 2,244 offset projects across four sectors (Figure 1a) and 51 studies investigating field interventions without issued carbon credits with a total of 1.2m observations (Figure 1b). For the other 3 main sectors (waste management, industrial manufacturing, and carbon capture and storage), we could not find any ex-post studies using a credible control group.
We have the strongest concentration of offset evaluations in the forestry sector, followed by renewable energy and chemical processes (Figure 1a/b). Offset evaluations are split between different geographies (apart from Africa with 0 studies). Similarly, for field interventions, most studies focus on forestry and are mainly focused on Latin America as most forestry projects are being implemented in tropical forests. Overall, both offset and field interventions mainly rely on rigorous observational studies (e.g., difference-in-difference and propensity score matching methodologies). In contrast, only 8 of 61 studies use randomised controlled trials (mainly evaluating the impact of fuel-efficient cookstoves, with one exception in forestry15).
The offset achievement ratio
The central question of this review is: what is known in the scientific literature about the differences between ex-ante estimates and ex-post outcomes of individual carbon offsetting projects? To operationalise this question, we introduce a new, simple metric, which we call the offset achievement ratio (see Methods Section for detailed description). The offset achievement ratio compares ex-post estimates from empirical studies with ex-ante estimates made by offset project developers. Hence, if a project reduced only half of what was originally claimed, the offset achievement ratio would be 50%.
For assessing offset achievement ratios, we only include empirical impact evaluations that contain a credible control group. A credible control group has similar characteristics as the treatment group. For instance, if a project seeks to avoid deforestation, then the deforestation trends within the conservation project would be compared to a forest with similar biophysical (e.g., type of forest, distance to forest edge) and socio-economic (e.g., distance to roads) characteristics that was not protected by the offset project.
This counterfactual approach stands in stark contrast to offset verifiers, which rely on simplistic comparisons of the offset project against a historical baseline to determine whether the project achieved its intended goals. For instance, in projects Reducing Emissions from Deforestation and Forest Degradation (REDD+), historical deforestation trends are commonly used, but these are often unsuitable to gauge the impact of the project for two main reasons 16. First, changes in underlying political and economic conditions may lead to reductions in deforestation that are wrongfully attributed to the offset project, as likely happened with Brazil’s policy effort to thwart deforestation post-2004 6,16. Second, project developers have an incentive to inflate deforestation baselines to benefit from the sale of a larger number of offset credits, which results in questions regarding the actual emissions reductions6.
Overall, we find that offset projects achieved considerably lower emissions reductions than claimed ex-ante. We find the lowest values for the offset achievement ratio in the renewable energy (0%) and household (0.4%) sector, followed by forestry (25.0%) and chemical processes (27.5%) (Figure 3a). In contrast to offset projects, estimates from field interventions show higher results for cookstoves (17.1%) and forestry (39.2%) but not for renewable energy (no data on chemical processes) (Figure 3b). For our estimates in Figure 3, we use the central estimates from the studies. For studies that only report an upper bound, we do not include them in our main estimates (but show them graphically in Figure 3a) as the authors make clear that the results could be as low as zero17 (see Methods section for discussion). We discuss issues of permanence in Section leakage, durability, and co-benefits.
The offset achievement gap
When generalizing the estimates from offset project studies in Figure 3a, we estimate that only 12% of the total volume of existing credits constitute real emissions reductions. Hence, 88% of the current voluntary carbon market across the main four sectors may not achieve the claimed offset goals. These non-achieved emissions reductions claimed by offsets are sizeable: the volume corresponds to almost twice the annual CO2 emissions of the entire German economy.
Forestry and renewable energy credits account for around 90% of the current market (Figure 4a). Most renewable energy credits are likely not achieving the claimed goals, whereas a share of forestry credits likely represents actual emissions reductions. (Figure 4b). While industrial credits have a higher offset achievement ratio, their overall share in the voluntary carbon market is relatively low.
Field interventions show a higher degree of ‘synthetic’ offset achievement ratio, but even applying these more optimistic estimates from field interventions, almost 80% of the current market would not constitute actual emissions avoidance or reductions. We delve into the external validity of our findings, the potential reasons for the observed low achievement and the divergence between offset projects and field interventions in the discussion section.
Leakage, durability, and co-benefits
Although the offset achievement ratio across sectors is the central focus of this study, we also evaluated whether offset project and field intervention studies address other important considerations related to carbon offsets. Our results show that studies investigating the emissions reduction potential of carbon offset projects or field interventions only partly consider leakage, durability, and co-benefits.
Some carbon offset projects may only displace carbon emissions instead of avoiding them. Only in the forestry sector, do some studies consider leakage (for which it is arguably the biggest risk) (Figure 5a). Within this sector, around ¼ of studies analyse leakage. For those that analyse leakage, 73% of these studies find no evidence of leakage and the rest a mixed picture. Leakage effects can be positive as one forestry study found additional conservation effects in nearby areas to field interventions.18
Another key consideration for carbon offset projects is durability, which denotes the time that the carbon offset projects avoid, reduce, or remove emissions. Avoided emissions are not per se permanent, as the avoidance may only be temporary if, for instance, a protected forest is later cut down. On average, studies in our sample analyse on average 6.5 years of intervention, with the shortest average timeframes found in cookstove studies (2 years), whereas chemical processes, renewables and forestry investigate longer intervention periods (7-11 years) (Figure 5b). Many cookstove studies rely on randomised controlled trials. Since these are costly to implement, they tend to be more short-term in nature. In addition to the relatively short intervention periods studied, almost none of the sectors considers post-intervention effects (e.g., once the payments run out). The only exceptions are a few studies in the forestry sector (13% of all forestry-related studies), which tend to show that once payments run out, conservation effects are likely to be reversed.
Lastly, co-benefits/harms are also important considerations for offset projects to assess whether a project’s impacts go beyond carbon reductions. For example, these include positive effects of cookstove projects on health (co-benefit) or an increase in poverty levels (co-harm) due to a forestry conservation project. For chemical and renewable projects, no study investigates these effects (Figure 5c). In contrast, 22% of forestry projects and 64% of cookstove projects investigate co-benefits/harms. Cookstoves projects find neutral to positive effects (especially on time saving in collecting fuelwood19 and cooking20, and reductions in indoor air pollution7). In forestry studies, co-benefits also tend to be neutral to positive (especially on socio-economic factors such as participants’ subjective wellbeing21 and poverty alleviation2223 as well as ecological factors, such as improved agricultural productivity24 and hydrological services2526,27). Only one forestry study found negative effects on the subjective well-being of project participants, mainly related to frustrations around project implementation.28