Show Summary Details

Page of

Printed from Oxford Research Encyclopedias, Environmental Science. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 30 September 2022

Use of Experimental Economics in Policy Design and Evaluation: An Application to Water Resources and Other Environmental Domainsfree

Use of Experimental Economics in Policy Design and Evaluation: An Application to Water Resources and Other Environmental Domainsfree

  • Simanti BanerjeeSimanti BanerjeeUniversity of Nebraska–Lincoln

Summary

Economics conceptualizes harmful effects to the environment as negative externalities that can be internalized through implementation of policies involving regulatory and market-based mechanisms, and behavioral economic interventions. However, effective policy will require knowledge and understanding of intended and unintended stakeholder behaviors and consequences and the context in which the policy will be implemented. This mandate is nontrivial since policies once implemented can be hard to reverse and often have irreversible consequences in the short and/or long run, leading to high social costs. Experimental economics (often in combination with other empirical evaluation methods) can help by testing policies and their impacts prior to modification of current policies, and design and implementation of new ones. Such experimental evaluation can include lab and field experiments, and choice experiments. Additionally, experimental policy evaluation should pay attention to scaling up problems and the ethical ramifications of the treatment. This would ensure that the experimental findings will remain relevant when rolled out to bigger populations (hence retaining policy makers’ interest in the method and evidence generated by it), and the treatment to internalize the externality will not create or exacerbate societal disparities and ethical challenges.

Subjects

  • Policy, Governance, and Law
  • Management and Planning
  • Sustainability and Solutions
  • Environmental Economics

Introduction

Economics conceptualizes harmful environmental effects as negative externalities, which can be internalized through regulatory and market-based mechanisms, and behavioral economic interventions such as nudges. Regulatory mechanisms include command-and-control policies, which are top-down and mandate specific types of behavior changes from the harming agent. Market-based instruments create incentive and disincentive structures (Hahn & Stavins, 1992), while nudges leverage human psychology without changing any incentives associated with choices (Carlsson et al., 2021). For example, under the Clean Air Act, policy instruments such as technology standards and emissions trading have led to significant improvements in air quality in the United States. Similarly, global policies regulating ozone layer depletion have been spectacularly successful, as is evidenced by the unanimous signup to the Montreal Protocol by 197 countries, and national-level action such as in the United States through amendment of the Clean Air Act and at the European Union (EU) level through implementation of the EU Ozone Regulation. Also, as described later in the sub-sections on water and energy conservation, electricity and water utilities have included information about resource use in different formats via information nudges to generate resource conservation.

Yet, in the United States for instance, despite presence of the Total Maximum Daily Load (TMDL) mechanism implemented under the Clean Water Act, the problem of nonpoint source water pollution remains. Again, policies to internalize externalities have also achieved the opposite of what is desired. For example, again in the United States, regulations under the Endangered Species Act preventing the “taking” of listed species on private property led to landowners adopting the “shoot and shovel” strategy whereby they preemptively killed valuable species on their land to escape the burden of government regulation (Brown & Shogren, 1998).

Hence, effective policy design, implementation, and evaluation will require knowledge and understanding of the context and stakeholders, as well as the direct and indirect costs and benefits of the interventions, while accounting for unintended behaviors and consequences. This mandate is nontrivial but essential since policies once implemented may be difficult to reverse and can have reversible and irreversible consequences both in the short and/or long run, leading to high social costs. Thus, it would be ideal if there was a method to evaluate economic performance of existing policies and to assess new ones to identify reasons that influence their current and possible future performance. These reasons can be used to inform policy modifications and development in similar and related contexts as is relevant.

Economic experimentation can serve as one such method and help address policy inefficiencies and failure by allowing an analyst to evaluate current and future policies in controlled “testbeds” to identify factors that influence behavior and performance. Through study of behavior, and economic, environmental, and social outcomes in response to the treatment variable, experiments allow us to draw meaningful conclusions regarding how best to address negative externalities and maximize the net benefits of policy making (Bouma, 2021; Colen et al., 2016; Higgins et al., 2017). The value of this method in policy evaluation has been elevated through recognition of researchers such as Vernon Smith, Elinor Ostrom, Alvin Roth, Abhijit Banerjee, Esther Duflo, and Michael Kremer, who have received the Nobel Prize for their experimental work. Also, groups such as the Abdul Latif Jameel Poverty Action Lab (J-PAL), Ideas42, Research Network on Economic Experiments for the Common Agricultural Policy (REECAP), and Center for Behavioral & Experimental Agri-Environmental Research (CBEAR) work extensively with public and private for-profit and not-for-profit organizations that evaluate interventions related to environmental policies in multiple domains such as water, energy, climate change, waste management and conservation policies.

Finally, many administrative units have been established at the government level such as the Behavioral Insights Team in the United Kingdom and the Mind, Behavior, and Development Unit at the World Bank to run experiments involving behavioral economics nudges to address inefficiencies both within and outside the environmental domain. These developments demonstrate the interest of academic researchers in conducting experimental policy evaluations, and of policy makers to use experiments to guide their policy making (Al-Ubaydli et al., 2021), and the presence of communication channels through which novel academic research can percolate to the policy sector to inform and facilitate design and evaluation of existing and new policies.

This article describes the different types of economic experiments and how they can be beneficial for policy evaluations. This is followed by a description of experimental studies that have been used to understand and inform effective policy interventions to reduce the social costs associated with environmental externalities, in multiple domains. Finally, the article discusses the scaling-up problem and steps to avoid or mitigate it, and concludes with some final thoughts on ethically and socially responsible experimental design and implementation.

Economic Experiments and Policy Evaluation

Economic experiments are human subject experiments in which randomly selected participants respond to one or more randomly implemented treatment manipulation(s). Depending upon the goal and context, and features of the participants, experiments can be classified into different types. In terms of the goal, induced value experiments study behavior under a stylized setting, which is a simplified but accurate representation of an empirical situation (Smith, 1976). Home-grown value experiments, on the other hand, elicit participants’ valuation for specific objects, attributes, or experiences. In this category fall choice experiments (Adamowicz et al., 1998), which have been used extensively in environmental economics research. In terms of context, lab experiments are those in which the context is tightly controlled, while field experiments (including lab-in-the-field, framed field, and randomized control trials [RCTs]) involve varying the degree of real-life context included and experimenter control imposed within the experimental setting (Harrison & List, 2004). Lab experiments are conducted with university student subjects, while field experiments involve stakeholders relevant to the empirical setting under investigation.

Experiments individually or in combination with other empirical methods such as simulations, case studies, and analysis of observational data can help understand behavior change stemming from policy modifications and introductions. Through the creation of new data, experiments set up counterfactual scenarios that are difficult to obtain with other empirical methods but are essential for comprehensive evaluation of policies (Colen et al., 2016). In fact, without this counterfactual, policy evaluation is limited to monitoring and tracking outcomes, which does not provide a comprehensive analysis of the full policy impact (Ferraro, 2009). Moreover, experiments establish the internal and external theoretical validity of different policy initiatives (Roe & Just, 2009), without which any subsequent implementation and/or evaluation is moot. Here, it is important to make the distinction between ex-ante and ex-post policy evaluations (Colen et al., 2016). Ex-ante evaluations are associated with modifications to existing policies and those being considered for future implementation. Ex-post evaluations are used to assess net social impact of policies post-implementation and are critical to set the stage for future policy agenda and ex-ante evaluations. Lab, field, and choice experiments play an important role in assessing the economic efficiency, equity, and overall public acceptance of different policy components under these two types of evaluations.

Experimental Evaluation in Different Environmental Domains

This section, focuses on experimental evaluation of various policies and institutions in different environmental domains ranging from water resources to climate change. In doing so, both lab and field studies of public- and private-sector initiatives in different countries are discussed. This section demonstrates the different ways in which economic experimentation has and continues to add to the understanding of and solutions to key environmental challenges. These include documenting inefficiencies and unintended consequences, identifying opportunities for intervention, and in general building up the knowledge base for improving ongoing and future policy design and performance. A snapshot of this description is provided in Table 1 with the companion Figure 1 indicating the geographical location of different field studies discussed in this paper.

Figure 1. Map of field experiment locations.

Table 1. Experiments In This Paper: Snapshot of Design Features and Evaluated Performance Metrics

Environmental Domain

Study

Experiment Type

Experimental Setting for Field Experiments

Metrics for Policy Performance Evaluated With Experiments

Water markets and policy

Lab experiment

Increased trading in water quality markets through elimination of upfront commitment requirements

Efficient pollution abatement under multiple policies with changes to features of related decision context

Improved efficiency through bid submission with more market players and sophisticated information processing and feedback

Improved efficiency through elimination of institutional constraints

Lab-in-the-field experiment

Mexico

Increased resource reserve under multiple policies

Lab-in-the-field experiment

India

Increased resource reserve through community-level institution building

Randomized control trial

South Africa

United States

Colombia

Evaluating cost-effectiveness of nudges relative to pecuniary price-based mechanisms for water conservation by households

Conservation and payment for ecosystem services (PES) schemes

Framed field experiment

Australia

Increased cost-effectiveness of auctions to generating environmental benefits with attention to ecological features

Lab experiment

Framed field experiment

Bolivia and Peru

Randomized control trial

United States

Increased producer enrollment

Emissions trading

Lab experiment

Identifying most efficient mechanism to distribute pollution permits with attention to reducing consumer cost pass-through

Energy conservation

Randomized control trial

United States

Evaluating cost-effectiveness of nudges relative to pecuniary price-based mechanisms for increasing energy conservation by households

Sweden

India

Waste management

Randomized control trial

Peru

Increased in household recycling program enrollment and recycling rate

Survey-based choice experiment framed field experiment

Germany

Generation of consumer awareness and identification of demand for new products

Climate change

Survey experiment

United States

Increased public awareness about climate change through choice of information format

Framed field experiment

France

Increased environmentally friendly including climate-friendly consumption through choice of information format

Lab experiment

Successful coordination by stakeholders to avoid climate change threshold

Higher cooperation rates across heterogeneous groups with attention to physicochemical features of pollutant and socioeconomic realities of stakeholders

Water Markets and Conservation

Water is a scarce resource with most water sources being public goods or common pool resources, leading to issues of overextraction, contamination, and depletion. Thus, research on water conservation has focused on effective design and implementation of policies, which can improve/maintain water quality and conserve supply.

For example, the TMDL mechanism supports water quality trading to ensure that water pollution reduction goals can be met in a cost-effective manner through trades executed between multiple point-source polluters (Shortle, 2013). Yet not many trades have been executed in the United States, which is in surprising contrast to the success of air quality trading programs (Willamette Partnership, Forest Trends, & the National Network on Water Quality Trading, 2018).

A laboratory experiment conducted by Jones and Vossler (2014) provided an opportunity to compare the effectiveness of four types of water quality trading market structures and gain understanding of the reasons for poor performance. The authors considered a cap-and-trade program, a Pigouvian tax/subsidy market, and two types of baseline-credit (BAC) markets. Across these incentive schemes, participants had options for investing in pollution abatement technology or not. In the two BAC markets, participants generated tradeable credits beyond their committed baseline levels of abatement. Finally, in one type of BAC market, their commitment was binding while in the other it was not. Study results indicated that in the absence of abatement cost technology investments, institutions perform at equal efficiency in achieving water quality improvements. However, when technology investment is required prior to participation in the incentive scheme as is the often the case, performance is somewhat lower and more variable across policies. Here, the Pigouvian tax/subsidy performs better than the other three mechanisms. Moreover, risks associated with interfirm trading influenced policy performance. These results indicate that since economic agents do not like to incur upfront costs and are usually risk averse, the need for upfront investment in abatement technology and making commitments to pollution reduction (as is the case in practice) can create barriers for resolution of the negative externality, although in theory, these features can maximize realized water quality improvement benefits. Thus, if markets are to realize their theoretical potential in internalizing the externality of water pollution, the high cost of and risk associated with market participation will need to be addressed. This finding is further highlighted by the report by the Willamette Partnership, Forest Trends, and the National Network on Water Quality Trading (2018).

Focusing on choice across multiple policy instruments with theoretically equivalent economic and environmental outcomes, economic experimentation can be useful for identifying the factors that influence stakeholder preferences for different policy instruments. This information is essential for policy makers since efficiency improvements obtained in an experiment will not translate into reality if it does not have stakeholder buy-in. Palm-Forster et al. (2019) in a lab experiment compared performance of different ambient pollution reduction subsidy schemes given a water quality target when experimental subjects can retain agricultural subsidy payments if they invest in technology that reduces emissions. They found that factors other than the type of policy instrument such as the degree of environmental damage from water quality reduction and cost of pollution reduction technology influence instrument attractiveness. In an agricultural context, this lab finding is key since in real life, there might be significant political barriers to implementing or modifying existing policies that impact producers, not to mention their unwillingness to participate in experiments (and surveys) in the first place (Rosch et al., 2021). In that situation, carefully crafted lab experiments with some baseline results might be the only evidence available to inform agricultural policy making or, in the best case, serve as the starting point to inform field experimentation with producers (if possible).

In the domain of water quantity conservation, governments have instituted water markets such as water banks, bulletin boards, and double auctions to execute trades. These markets are, however, not easy to modify owing to challenges associated with transaction costs of trading, water storage, unintended consequences, and climate change impacts. Experiments are useful to study current and new market mechanisms to obtain a baseline understanding of the impacts these institutions can have on efficient water usage. For example, Murphy et al. (2000) used a lab experiment calibrated to California conditions to evaluate the performance of a sealed bid double auction for the trading and allocation of water use rights. These auctions utilize sophisticated computer algorithms to process bids and asks from water rights buyers and sellers, as well as transporters to efficiently clear the market even when supply is uncertain, markets are thin, and there is significant volatility in trading. Results of this and other experiments provide foundational evidence for introduction and continual use of sealed bid double auctions in publicly run exchanges in Australia such as Watermove in Victoria, SunWater Exchange in Queensland, and the non-profit-run Murrumbidgee Water Exchange in New South Wales (Hadjigeorgalis, 2009).

Garrido (2007) employed a lab experiment to evaluate the impact of two aspects of Spanish water regulations on market performance given water law reform in Spain in 1999. These included studying the efficiency impacts of restricting senior water rights holders from trading with junior users and the role of water storage facilities on water price volatility. These are important issues since water needs of different users vary over time and also because historically any water remaining in water reservoirs if not sold is considered common-property resource, leading to overextraction from open access. His experiments reveal that allowing unrestricted trading and allowing for utilization of storage opportunities has positive impacts on water supply, identifying some of the ways in which Spanish water policies can be modified to ensure efficient water allocation and usage (Palomo-Hierro et al., 2015).

Water Use and Conservation

This section, discusses interventions to encourage water conservation at the community and household level, with the intention to introduce policies and/or develop informal and formal institutions to promote water stewardship. Work by Foster et al. (2016) considered lab-in-the-field experiments with Mexican producers in León, Guanajuato, evaluating three types of economic incentives to regulate water extraction and promote the health of a groundwater aquifer. The researchers compared the status quo where producers received electricity subsidies for pumping groundwater with three settings in which this subsidy was eliminated, lowered, or replaced by a new policy that provided cash transfers to producers independent of the volume of water pumped but based on their past subsidy payments. Experiments included a baseline of 15 rounds where no subsidy change was implemented followed by 15 treatment rounds pertaining to the three policy interventions, in multiple sessions. A set of control sessions were also conducted in which for all 30 rounds, there was no change to subsidy values. The results indicate that all three policy interventions led to increase in the height of the water table, with elimination having the most impact, relative to the status quo. This finding is not surprising since the subsidy exacerbates the resource dilemma in the first place. But what is key is that the experiment established cash transfers as a viable water conservation policy, which could potentially replace existing subsidies without having political and social repercussions.

A feature of the Foster et al. (2016) paper is that it does not permit communication between participants. Work by Meinzen-Dick et al. (2018) has leveraged the ubiquitous nature of communication to evaluate the effectiveness of a collective action game in facilitating understanding and subsequently influencing crop choice and water extraction. In a lab-in-the-field field experiment, producers in the Indian state of Andhra Pradesh played a game in which they had to make a choice of crops that varied in their water intensity and for which water needed to be extracted from a ground water aquifer. In their experiments, they focused on the role participant communication and information provision on water extraction behavior. Baseline rounds of the game were played without communication followed by treatment rounds in which producers communicated with each other before making their extraction choices. The researchers found that groups achieved sustainable extraction levels in the communication treatments. After experimental implementation, they also invested in water registers and created institutions to govern groundwater extraction in their communities. These and other experimental results from India (Falk et al., 2021) highlight the effectiveness of low-cost interventions such as behavioral games in promoting water conservation, especially when there are problems of collective action and lack of formal institutions to promote water stewardship.

Experiments involving nudges have also identified different ways of promoting water conservation, often in tandem with other policies while also uncovering long-term impacts that would be difficult to isolate with observational studies. For example, Ferraro and Price (2013), as part of an RCT, worked with a water utility in the city of Atlanta, Georgia in the United States to test the relative effectiveness of social norms and technical information regarding water use conservation, in promoting resource stewardship. The social norm treatments involved a “weak” treatment in which the household was provided with their water use behavior and asked not to waste water and work with everyone to save water, and a “strong” treatment where in addition to this information, water use data of a neighbor was provided. The findings suggest significant although small improvements in water conservation rates in the strong social norm treatment.

In the context of extreme and continuously increasing water scarcity and significant social inequality, as part of a RCT, Brick et al. (2017) nudged households in Cape Town, South Africa, with different types of information about water scarcity. Nudges considered include one type that provided information about water use associated with different types of household behaviors in both a loss and gain frame, a second type that ensured social recognition when the household reached a target level of water usage, a third that appealed to people’s intrinsic motivation to save water, and finally one that emphasized the public good nature of water resources by suggesting that everyone save as much water as possible since it is beneficial for all. The results note that all nudges had positive impacts, with the social recognition and public good nudges performing the best. This was the case even in the presence of extant pecuniary incentives and regulations such as water tariffs and water use restrictions.

Brent et al. (2020) did a similar RCT in the United States with a local water utility in Reno, Nevada, with a focus on factors causing household-level differences in treatment effects. They tested the efficacy of social comparison nudges after the utility had already adopted media and other information campaigns to incentivize water use, to find similar positive impacts on water conservation with the effect varying by household type—households who had high water use prior to the intervention engaged in more water conservation. Torres and Carlsson (2018) conducted a study in Colombia and found that not only were nudges impactful in promoting lower water usage, but they also led to information spillovers for households who were not nudged but still reduced water use in the first 6 months following the intervention.

The effectiveness of the nudges presented in these studies should be considered in light of the fact that the welfare benefits of the nudges are small, the reductions in quantity of water used fall over time, and the effects are heterogeneous across households. A benefit-cost analysis conducted by Nauges and Whittington (2019) in the specific context of the municipal water sector in both developed and developing nations indicates that long-term water conservation can generate positive net benefits more effectively if pursued through pecuniary price instruments such as a water price increase rather than nonpecuniary social norm nudges. Nudges can indeed be effective but should most likely be reserved for emergency scenarios such as water use reduction during droughts when such information messaging has a high chance of capturing the public’s imagination and lead to much-needed immediate water conservation. Thus, in essence, which instrument (and policy) will work requires paying attention to the policy and social context, behavioral dynamics in the short and long term, and careful consideration and customization of the nudges themselves (Ferraro & Miranda, 2013).1

Conservation and Payment for Ecosystem Services (PES) Schemes

In the domain of environmental conservation, Payment for Ecosystem Services (PES) schemes have been designed and studied with attention to the factors that can increase participation and generate ecosystem services benefits cost-effectively while paying attention to ecological principles.2

One of the earliest PES schemes is the Conservation Reserve Program (CRP) in the United States, which implements a reverse auction to select producers to retire land from intensive agriculture in order to generate ecosystem services. However, the CRP has not been cost-effective (Kirwan et al., 2005). Similar issues have been observed in other auction-based PES schemes such as the Bush Tender Trial (Stoneham et al., 2003) in Australia, giving rise to experiments to evaluate mechanisms to address this cost-effectiveness challenge. For example, as part of a field trial in the Desert Uplands of Australia, Windle et al. (2009) found that presence of opportunities for landholders to revise their bids led to bid reductions of about 34% and improved auction cost-effectiveness. In the United States, in the context of the Wetland Reserve Program, approximately $820,000 in cost savings was realized from implementing a two-round pilot iterative auction for procurement of conservation easement projects, relative to a single-round one (U.S. Congress, 2009). These results build up the knowledge base for modifications to current conservation auction designs for improved economic performance.

Auctions related to PES schemes have also been used to pursue spatially contiguous land management to generate biodiversity and wetland conservation benefits. For example, in a lab experiment, Banerjee et al. (2021) evaluated the performance of a PES auction that permits joint bidding by neighboring participants on the landscape to procure spatially adjacent projects. They found that if joint bidding by neighboring producers is incentivized with bonus payments higher than those rewarded for neighboring bids submitted individually, auction performance is lower unless bid revisions are permitted. Similarly, Narloch et al. (2017) conducted framed field experiments in two Andean farming communities in Bolivia and Peru and found that the performance of auctions with joint bidding depends upon the degree to which communities are willing to work collectively. If groups have a cooperative norm, then the joint-bidding auction can lead to higher ecosystem services benefits with minimal negative impacts on cost-effectiveness. This research objective is aligned with the need to study the efficacy of policies that have been recently adopted in countries such as the Netherlands, allowing producer collectives to participate in PES policies (Westerink et al., 2017), despite negative experiences in other (albeit very different) contexts such as Costa Rica where, joint participation–based policies were discontinued owing to free-riding within groups (Porras et al., 2013).

Experiments have also been utilized to evaluate mechanisms to increase producer enrollment in PES schemes in a cost-effective fashion. For example, in an RCT, Wallander et al. (2017) mailed out letters to U.S. producers with expiring CRP contracts and those with eligible non-enrolled lands. These letters included different wording representing various nudges to increase enrollment. In some treatment letters, peer comparison information was provided while in others, information related to stable risk-free nature of the CRP payments was included. The results indicated that rather than the type of the nudge, what mattered for producers with expiring contracts is that they received a letter—the “letter effect.” For those with eligible non-enrolled land, letters of any type did not have any impact on enrollment. Czap et al. (2019) conducted an RCT in the context of increasing enrollment in the Conservation Stewardship Program (CSP), in the state of Nebraska in the United States They sent out different types of letters with nudging messages to producers with eligible but non-enrolled land in addition to the letter with standard language. One type of letter included wording to nudge producers’ empathetic tendencies in addition to reminding them about the impact of their production decisions on the environment, and current and future generations. An additional nudging statement was also included in these letters, which was photocopied in one treatment and handwritten in another. Their results also provided evidence on the “letter effect” with the number of producers submitting CSP applications the highest in the handwritten nudge condition. However, there was no difference in impacts comparing the standard letter to any of the two nudging letters, although the handwritten nudge significantly outperformed the photocopied one.

Emissions Trading Toward Pollution Reduction

Policy interventions to reduce harmful impact of air pollution have taken the form of numerous emissions trading schemes implemented by the United States, the EU, New Zealand, Australia, South Korea, Kazakhstan, and China (Jotzo & Loschel, 2014). These schemes regulate multiple pollutants such as sulfur dioxide and nitrogen dioxide under the Acid Rain Program (Siikamäki et al., 2012) and greenhouse gases under the Regional Greenhouse Gas Initiative (RGGI), both in the United States and EU Emissions Trading Schemes (ETS). The design and effective implementation of many of these programs have evolved over time given policy realities and impacts. For example, after experiences with permit auctions under the Acid Rain Program, policy makers used auctions to disburse pollution permits under the RGGI. Herein, economic experiments have been valuable to understand the impact of auction format and permit distribution rules on combatting air pollution efficiently.

Results of lab experiments by Cason and Plott (1996) indicated that the discriminatory price call auction implemented under the Acid Rain Program was less efficient compared to the uniform price call auction. Given that the RGGI implemented an auction in 2008, Burtraw et al. (2009) used lab experiments to evaluate policy performance under uniform and discriminatory price single-round and clock auction formats with special attention to bidder collusion. Their results showed that the uniform and discriminatory price auctions were more resistant to bidder collusion than the clock auction regardless of opportunities for bidder communication, providing evidence to support the ongoing use of sealed bid uniform price auctions to sell permits under the RGGI. This focus on collusion is policy relevant since permit auctions involve a small set of players (often within a given industry such as the power sector) and are repeated multiple times at regular intervals providing opportunities for bidder learning and eventual collusion. Collusion can lower the total auction revenue generated. Lower revenues are a problem because the money generated from these auctions has double dividend properties.

Another issue that experiments have permitted researchers to study is the extent to which permit distribution schemes (auctions or free distribution) can influence the prices that firms end up charging for their products. For example, in the first round of the EU ETS, significant cost pass-through (and public backlash) was observed when the German firm RWE made windfall profits of about U.S.$6.4 billion by charging its customers the value of the permits they had received for free through grandfathering (Goeree et al., 2010). Evidence by Cludius et al. (2020) demonstrates significant cost pass-through in the cement, steel, and iron industries in multiple EU countries. Goeree et al. (2010) used lab experiments to study this phenomenon. In their study, large polluters who receive many permits for free through grandfathering had the option of selling these permits to other small and large emitters in a secondary spot market. The experiment indicated that these large firms exercise market power in the secondary spot permit markets by withholding permit supply, which increased the price at which they are willing to sell their permits. Permits bought at such high prices by other polluters, or not traded at all, constrict supply in the output market, leading to significant cost pass-through and commodity price increases. These issues do not exist with auctions as firms should not be bidding for and buying more permits than they need (at least in theory).

The EU policy makers are currently deliberating whether to bring the transportation and heating sectors under the ambit of the EU ETS (Abnett & John, 2021). However, this policy expansion can be contentious because pricing carbon from these sectors could lead to increases in the price that consumers pay for transportation and household heating fuel. Furthermore, this policy expansion could have varying impacts on different sections of society with socio-economically disadvantaged consumers being disproportionately impacted if they cannot reduce their fuel use. Herein, the research findings on permit auction type and pass-through should prove valuable in guiding the way emissions from these sectors should be regulated (and traded) and compensations paid to prevent the inefficiencies encountered in the first round of EU ETS.

Nudging for Energy Conservation

Efficient energy use has been incentivized through differential energy pricing assuming that economic agents will consume energy to the point where marginal benefits equal marginal costs. Yet, socioeconomic factors, including lack of information about energy wasted and long-standing personal habits and cognitive inertia associated with changing habits and behaviors, often lead to significant energy wastage despite high utility bills (Frederiks et al., 2015). Experiments are routinely used to establish the efficacy of behavioral economic nudges in cost-effectively promoting energy conservation without requiring significant expenditure of cognitive resources by consumers to process the information contained in the energy-saving nudge.

For example, Allcott (2011) partnered with the company Opower for a large RCT on energy conservation in the United States. He studied the efficacy of nudging utility customers with information included in their Home Energy Reports (HERs) about their own energy use and that of a comparable peer. The experiment indicated that nudges reduced energy use by up to 2%, an effect that can be replicated by price-based incentives only if electricity prices go up by 11% to 20%. Similarly, in 2012, the Swedish utility company E.On conducted an RCT with some of its customers to install smart meters in their homes and evaluated the effectiveness of nudging to reduce their energy consumption over a 12-month period. The peer control group involved households who did not have meters and were not nudged about their energy use with any feedback information (provided through a smartphone application). At the end of this period, the control group had increased their energy consumption by 1.5% while the treatment group had reduced consumption by 0.7% (Pyrko & Uggmark, 2014). Sudarshan (2017) tested the efficacy of a similar energy nudge in India with an additional treatment involving the opportunity to earn money or pay a penalty depending upon whether a household’s energy use was lower or higher compared to their peer. The author found similar outcomes highlighting nudge efficacy in a developing country context but also that combining nudges and pecuniary incentives can crowd out energy conservation owing to growing distrust for the energy utility.

The results of these large-scale experiments have led to initiatives that work in combination with current incentive-based programs to promote energy conservation. For example, Opower has implemented significant innovations to the HER format for better user experience, which can make the decision to transition to energy-saving habits less cognitively burdensome (Oracle, 2020). In Sweden, E.On has launched electricity meters and a smartphone application that allow consumers to get real-time feedback about their energy use and subsequently engage in energy-saving behaviors.

Recycling and Waste Management

Waste management policies in different parts of the world have targeted households and firms, and involved regulations imposed and recycling fees levied in addition to providing information regarding waste sorting and disposal. Nudges have also been used extensively in this domain.

For example, Chong et al. (2015) worked with a large nongovernmental organization (NGO) in Peru to conduct an RCT to test the efficacy of nine types of prosocial messages on nudging households to enroll in a recycling program. Surprisingly, messaging had no impact on enrollment, which is attributed to in-person visits made by the NGO staff marketing the program to the non-enrolled households. In essence, these visits nullified any possible impact that messaging could have had on new enrollment. However, reducing the costs of recycling by providing recycling bins to already enrolled participants (so that they do not have to purchase recycling bags) significantly improved recycling rates. The second intervention is similar to waste disposal policies in the United Kingdom, Canada, and Japan, where the local municipal authorities provided waste disposal bags to households to reduce their costs of disposal and regulate the total amount of waste produced. The current study demonstrated this finding in a developing country context. It also indicated that reducing the costs of recycling can increase rates at the intrinsic margin and ongoing marketing visits to increase awareness can increase enrollment at the extrinsic margin. Overall, the experiment demonstrated that contextual factors play a key role in determining the efficacy of interventions. That an intervention has worked in one setting should not be used to justify the use of that or a similar intervention in another setting without a clear understanding of the decision setting and upfront testing, as that can lower policy efficacy.

Solid waste management policies with the aim to reduce greenhouse gas emissions have also taken the form of encouraging and/or mandating producers to use new products in their operations that have a low carbon footprint. For example, on January 1, 2019, Germany introduced the new Packaging Act—VerpackG to encourage companies to take actions to cover some of the environmental costs of packaging (Wensing et al., 2020). Such extended producer responsibility (EPR) policies have been introduced and/or enhanced in many countries, such as South Korea (Kengo, 2021a), Kenya (Langhill, 2021), and the United States (Quinn, 2021), and are being considered in others such as India (Kengo, 2021b). Yet, such new packaging may come at a higher cost to the producers, making them unwilling to adopt, especially if sales do not compensate them for some or all of this extra cost. In this context, Wensing et al. (2020) used an online choice experiment to calculate German consumers’ willingness to pay (WTP) for cherry tomatoes in bio-based plastic containers when they are exposed to six different types of nudges. These nudges involved information about bio-based plastics, social norm messaging, and information about natural amenities in different formats to prime participants’ environmental sensibilities. Given these treatments, the authors found that nudges increased WTP for bio-based plastics when they matched respondents’ cognitive style.3 That consumers have a higher WTP identifies a potential market for products packaged in bio-based plastic and indicates that producers using novel albeit costly packaging products can pass on some of the increased cost of these new materials to consumers without the prospect of significant backlash (at least in Germany).

However, some issues with such new product-based EPR policies remain largely owing to information gaps on the consumer side. Utilization of bio-based and other types of environmentally friendly product packaging requires that consumers are aware of how to dispose of these products so that the environmental benefits of using new materials are in fact realized. In a lab-in-the-field field experiment, Taufik et al. (2020) find that German consumers are unclear about how to dispose compostable versus recyclable bio-based plastics, relative to their understanding of disposing conventional plastic. Thus, experiments can prove valuable in assessing the severity of problems that might arise from new policies and provide guidance regarding how to resolve them in tandem with the new policy or ahead of time prior to the new policy intervention.

Climate Change

Given the magnitude, and geographical and temporal variation of the irreversible and reversible impacts of climate change, policy interventions with a focus on mitigation and adaptation will require coordinated action by a heterogeneous group of actors with a focus on how inter- and intra-nation priorities and relationships impact policies and decisions. In this regard, policy making to effectively combat climate change has faced significant challenges because of the inherent collective action nature of the problem, geographical disparities, and complex nature of impacts felt, not to mention the misunderstanding of impacts of climate change and overall lack of public consensus regarding whether climate change is in fact happening, and the role that human beings play in causing it.

The Stern Review (2007) argued that climate change can be conceptualized as a market-failure at the global level, meaning that the economic literature on game theory involving public goods and coordination games can provide guidance regarding experimental policy evaluation. For example, Barrett and Danenberg (2012) use game theory to study climate policy in the backdrop of the consensus that was reached under the Copenhagen Accord in 2009, regarding the temperature threshold beyond which climate change impacts would be dangerous. Their lab experiment finds that knowledge of this threshold, low costs of avoiding it, and ability of experimental subjects to communicate and negotiate with each other about how to avoid the threshold significantly increased the chances of avoiding the threshold by changing the strategic setting to a coordination game (from a public goods game). This finding is valid even if the impacts of going beyond the threshold are uncertain. However, lack of consensus about the threshold retains the strategic setting as a global Prisoners’ Dilemma causing policy failure, making tackling climate change impossible. These results indicate the need to continuously monitor and maintain consensus on the climate change threshold but also to facilitate global dialogue for effective policy making. The latter has in fact been achieved at the Paris Agreement in 2015 when 196 countries signed on to work collectively to coordinate their climate change policies.4

While willingness of different nations to work together is a step in the right direction, climate policies are often derailed as there are significant disparities regarding how climate change impacts are felt by developed and developing countries. Herein, experiments have been used to study behavior and climate change policy performance with a heterogeneous group of actors. For example, in a lab experiment, Burton-Chellew et al. (2013) considered two treatments in a threshold public goods game—variation in the resources available to mitigate climate change reflecting the existence of poor and rich countries, and differences in the impact of climate change on different subjects, which reflects the fact that the Global South is expected to be impacted more seriously than the Global North. Their results indicated that when subjects with lower resources faced higher risk, cooperation failed since subjects who have more resources fail to invest to avert the dilemma.5 These results indicate that economic disparity between nation-states can make achievement of voluntary cooperation to address climate change impacts challenging, suggesting the need for long-term climate policy to devote resources to reducing inter-nation economic disparities as well.

As the knowledge base on analysis and implementation of climate change policies grows, researchers are taking the opportunity to run economic experiments, which introduce complexities to the economic setting while mirroring current and long-run realities. For example, Calzolari et al. (2018) used lab experiments to study international cooperation between countries under three scenarios of pollution accumulation—a persistent condition reflecting the nature of pollution caused by greenhouse gases, a scenario where the pollutant accumulates by one half the amount, and a final one where the pollutant has no stock effects. This research agenda is starkly different from past experimental work, which is agnostic about the long-term dynamic impacts of pollution accumulation but which is essential to the study of climate change policy given irreversible impacts on nature and society, and scholarly opinion that pollution persistence can reduce the extent to which countries are willing to collaborate. The experimental results noted that persistence in pollution impacts leads to higher levels of initial collaboration, although this effect erodes over time. This result suggests that any climate change policy needs to consider both current and future impacts, which is not trivial given continual political roadblocks. Significant differences exist between actions of poor and rich players as well, but these are not large. This outcome reflects current realities where both rich and poor nations are willing to tackle climate change, although many rich nations still exhibit strong inertia regarding action as that could negatively impact their GDP (at least in the short term).

Climate migration is the strongest form of voluntary adaptation measure, and current evidence indicates that it is expected to continue (Cattaneo et al., 2019). In this regard, the lab experiment by Marotzke et al. (2020) included poor and rich subjects who participated in a threshold public goods game to generate enough funds to prevent the social dilemma (i.e., climate change). In the experiment, poor subjects had the option to migrate while rich subjects had the option to prevent it, a feature similar to current global political landscape. Treatments considered include a control setup that is a traditional threshold public goods game with rich and poor players, one treatment session where the poor face a climate event with 10% or 20% probability, and a second one that is the same as the first treatment except that returns from investing money in the private good are higher for both the rich and the poor. The results of the experiment suggest that migration cannot be prevented in the long run and that both the rich and poor subjects are willing to contribute toward reaching the threshold with the poor subjects’ efforts being predicated on whether the rich subjects contribute or not. These results are important since they reiterate the fact that climate migration is expected to continue, suggesting the need for global climate policies to be sensitive to economic disparities and prioritize action by developed and developing countries in a way that vulnerable groups (mostly in developing nations) have resources necessary to survive both in their own country or the country that they migrate to.

Finally, experiments have been used to identify effective ways to communicate information about effects of climate change to the public and build consensus on climate change action so that policy makers can move forward with policy implementation. For example, survey-based experiments by Van der Linden et al. (2014) find that simple messages and pie charts can effectively communicate information about scientific consensus on climate change as well as influence public opinion about the worrisome nature of the problem and the need to act, compared to metaphorical representations.

The Scaling-Up Problem

Throughout this paper, various types of experiments have been described that have been conducted to evaluate and improve policy performance along the environmental and economic dimensions. In general however, there are, many examples of policy interventions, which after spectacular success in the trial phase significantly dropped in efficacy when rolled out to larger populations. This is referred to as the scaling-up problem (Al-Ubaydli et al., 2017) and can derail policy makers’ interest and enthusiasm in taking experimental findings and using them to inform the modification to or introduction of new policies. Thus, if experiments are to achieve their fullest potential in informing policy design and implementation in general and in the environmental domain, attention to the scaling-up problem is warranted to identify ways to mitigate or eliminate it. This can be done by focusing on the following factors, which have been discussed extensively by Al-Ubaydli et al. (2021).

Replication, Precision, and Publication Incentives

Experimental research in the social sciences, including psychology (Open Science Collaboration, 2015) and economics, has come under scrutiny owing to their failure to replicate findings and because of lack of precision of empirical estimates (Camerer et al., 2016; Ferraro & Shukla, 2020). Thus, if policy interventions are to be scaled up, protocols should be designed with attention to statistical power corrected for multiple hypothesis testing, significant effort should be directed toward replication of findings, and both null and significant results should be published. In fact, if null results are obtained, experimenters should ensure that it is not an artifact of loss of control and/or lack of internal validity but because the treatment did not work. Such practices give rise to precise and generalizable findings with clear expectation about potential outcomes. Yet, while effective protocol design is a matter of improved research practice, replication research and publication of null results will require a sea change to current research incentives. These include journals and other academic outlets promoting publication of replication studies and those with null results, as well as encouraging preregistration plans (some already do) and research unit heads (such as deans and academic department heads) valuing this type of research in applicants’ portfolio when considering their professional advancement (Ferraro & Shukla, 2020).

Inclusion of Diverse Experimental Subject Groups

If the negative externality being considered has features that are tied to the socioeconomic and cultural features of the group to be impacted by the policy, experiments should ideally involve subjects selected from the impacted group rather than from unrelated populations. This is not a trivial issue since experimenters might often be constrained by resources whereby they have limited ability to conduct the study with groups who would be impacted by resolution of the negative externality, not to mention that the people who will be impacted by the policy may not want to participate. In this case, researchers might end up randomly sampling from an unrelated but conveniently accessible population. This issue of subject pools was brought to the forefront by work by Henrich et al. (2010), who indicated that an exhaustive study of the human psychology will need more research with subjects who are not from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies.

Another problem arises from choice of participants that goes beyond the WEIRD subject issue. First, as noted by Rosch et al. (2021), some groups such as agricultural producers might not want to participate in any experimental policy evaluation. Second, even if conducted with a diverse and contextually relevant subject pool, often those who participate in the experiment are the ones who stand to gain the most (or lose the least) from the intervention, and are those who have a relationship of trust with the research groups whereby they might introduce experimenter demand effects in the results (Zizzo, 2010), despite the researchers’ full-faith effort to randomize. Such strategic participation and demand effects are a threat to randomization and can contribute to the voltage effect represented by a drop in the treatment effect magnitude when the intervention is implemented with other groups (List, 2022), compromising the scientific accuracy of findings and popularity of the intervention. Thus, full effort should be made to conduct and replicate the studies with a randomly selected and representative set of subjects from all policy-relevant groups before some concrete policy steps to address the environmental problems are taken, especially given the disproportionate impact of environmental problems on different groups. At the minimum, when these issues cannot be resolved, they should be explicitly acknowledged in publications so that policy makers have a complete picture of the experimental scenario and findings when considering the policy for potential role out.

Researcher Involvement, and Comprehensive Development and Adherence to Experimental Protocol

Research groups are extremely vested in ensuring that the experiment is well implemented. Yet when they hand over the experimental protocol over to policy makers, the protocol might not be followed diligently, which can contribute to loss of study control, and the voltage effect, thus compromising policy performance. Hence, scaling up of policy interventions by policy makers should try to involve one or more research professionals from the original team. On the researchers’ side, if this is not possible, then the utmost effort should be taken to document and standardize different aspects of the experimental protocol and ensure that this information is passed on to policy makers. Finally, a lot of scaling-up concerns can be mitigated or resolved if researchers design their protocols keeping these issues in mind. As Al-Ubaydli et al. (2021) present, researchers will be best served to “backward induct” such that they are able to factor into their protocols different features of the context in which the policy is going to be implemented after experimental evaluation.

Conclusion

This paper demonstrates the important role that economic experiments can play in informing environmental policy design and evaluation. Many examples of situations where experiments have been used effectively to understand agent behavior and assess policy performance and internalize or mitigate impacts of negative externalities have been provided. These examples are followed by a description of key issues that need to be considered and addressed if economic experiments are to deliver on the promise as an effective method for policy evaluation. Here, some final words are warranted.

First, policies do not operate in a vacuum. As the findings of Chong et al. (2015) indicated, the context is important in ensuring whether an intervention that has worked in other contexts will work in the current one or not. Second, the work by Wallander et al. (2017) and Czap et al. (2019) present that while new interventions might generate environmental benefits (often cost-effectively), depending upon the context, they might not be needed at all if the policies are administered effectively in their current form (which in the PES enrollment context is synonymous with ensuring that all eligible producers receive a reminder letter). Finally, the social and political settings within which a policy is introduced to internalize a negative externality may end up intensifying divisions and disparities between different groups of people and have far-reaching unintended consequences even if the negative externality is internalized. For example, as part of a new policy trial, denoting a biodiversity-rich area as a protective area and prohibiting local community access to resources may have conservation benefits in the short term but might create social unrest within the community, rob people of their way of life, and in the long term exacerbate illegal logging, poaching, and poverty, irreversibly changing the cultural ethos of the community. Thus, even if well intentioned, interventions being implemented as part of current and new policies should be chosen such that they are ethical and respect and recognize the cultural norms, so as not to exacerbate or introduce inequalities between treatment and control groups and trigger unintended consequences during the experimental trial itself and in the long run (Baele, 2013; Sindzingre, 2019). This is not a trivial task and will require that experimental researchers consider the ethical ramifications of treatments before implementing them, especially in the field. Institutional review boards at many institutions can serve as a check on such issues, but the onus first and foremost should always be on the economic researcher(s).

Further Reading

  • Dolan, P., & Galizzi, M. M. (2014). Getting policy-makers to listen to field experiments. Oxford Review of Economic Policy, 30(4), 725–752.
  • Ferraro, P. J., & Agrawal, A. (2021). Synthesizing evidence in sustainability science through harmonized experiments: Community monitoring in common pool resources. Proceedings of the National Academy of Sciences, 118(29), 1–12.
  • Hansen, P. G., & Jespersen, A. M. (2013). Nudge and the manipulation of choice: A framework for the responsible use of the nudge approach to behaviour change in public policy. European Journal of Risk Regulation, 4(1), 3–28.
  • Thoyer, S., & Préget, R. (2019). Enriching the CAP evaluation toolbox with experimental approaches: Introduction to the special issue. European Review of Agricultural Economics, 46(3), 347–366.

References

  • Abnett, K., & John, M. (2021). Analysis: Europe’s carbon push stokes backlash fears, Reuters.
  • Adamowicz, W., Boxall, P., Williams, M., & Louviere, J. (1998). Stated preference approaches for measuring passive use values: Choice experiments and contingent valuation. American Journal of Agricultural Economics, 80(1), 64–75.
  • Ahsanuzzaman, Palm-Forster, L. H., & Suter, J. F. (2022). Experimental evidence of common pool resource use in the presence of uncertainty. Journal of Economic Behavior & Organization, 194, 139–160.
  • Allcott, H. (2011). Social norms and energy conservation. Journal of Public Economics, 95(9–10), 1082–1095.
  • Al-Ubaydli, O., Lee, M. S., List, J. A., Mackevicius, C. L., & Suskind, D. (2021). How can experiments play a greater role in public policy? Twelve proposals from an economic model of scaling. Behavioral Public Policy, 5(1), 2–49.
  • Al-Ubaydli, O., List, J. A., LoRe, D., & Suskind, D. (2017). Scaling for economists: Lessons from the non-adherence problem in the medical literature. Journal of Economic Perspectives, 31(4), 125–44.
  • Baele, S. J. (2013). The ethics of new development economics: Is the experimental approach to development economics morally wrong? The Journal of Philosophical Economics, 8(1), 2–42.
  • Banerjee, S., Cason, T., Vries, F. D., & Hanley, N. (2021). Spatial coordination and joint bidding in conservation auctions. Journal of the Association of Environmental and Resource Economists, 8(5), 1013–1049.
  • Barrett, S., & Dannenberg, A. (2012). Climate negotiations under scientific uncertainty. Proceedings of the National Academy of Sciences, 109(43), 17372–17376.
  • Bouma, J. (2021). Evaluating environmental policy: The use and usefulness of experiments. Journal of Environmental Economics and Policy, 10(4), 468–480.
  • Brent, D. A., Lott, C., Taylor, M., Cook, J., Rollins, K., & Stoddard, S. (2020). What causes heterogeneous responses to social comparison messages for water conservation? Environmental and Resource Economics, 77(3), 503–537.
  • Brick, K., De Martino, S., & Visser, M. (2017). Behavioural nudges for water conservation: Experimental evidence from Cape Town. Preprint.
  • Brown, G. M., & Shogren, J. F. (1998). Economics of the endangered species act. Journal of Economic Perspectives, 12(3), 3–20.
  • Burton-Chellew, M. N., May, R. M., & West, S. A. (2013). Combined inequality in wealth and risk leads to disaster in the climate change game. Climatic Change, 120(4), 815–830.
  • Burtraw, D., Goeree, J., Holt, C. A., Myers, E., Palmer, K., & Shobe, W. (2009). Collusion in auctions for emission permits: An experimental analysis. Journal of Policy Analysis and Management, 28(4), 672–691.
  • Calzolari, G., Casari, M., & Ghidoni, R. (2018). Carbon is forever: A climate change experiment on cooperation. Journal of Environmental Economics and Management, 92, 169–184.
  • Camerer, C. F., Dreber, A., Forsell, E., Ho, T. H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436.
  • Carlsson, F., Gravert, C., Johansson-Stenman, O., & Kurz, V. (2021). The use of green nudges as an environmental policy instrument. Review of Environmental Economics and Policy, 15(2), 216–237.
  • Cason, T. N., & Plott, C. R. (1996). EPA’s new emissions trading mechanism: A laboratory evaluation. Journal of Environmental Economics and Management, 30(2), 133–160.
  • Cattaneo, C., Beine, M., Fröhlich, C. J., Kniveton, D., Martinez-Zarzoso, I., Mastrorillo, M., Millock, K., Piguet, E., & Schraven, B. (2019). Human migration in the era of climate change. Review of Environmental Economics and Policy, 13(2), 189–206.
  • Choi, A. S. (2021). How do attitudes and perceptions affect environmental preferences? Oxford Research Encyclopedia of Environmental Science.
  • Chong, A., Karlan, D., Shapiro, J., & Zinman, J. (2015). (Ineffective) messages to encourage recycling: Evidence from a randomized evaluation in Peru. The World Bank Economic Review, 29(1), 180–206.
  • Cludius, J., de Bruyn, S., Schumacher, K., & Vergeer, R. (2020). Ex-post investigation of cost pass-through in the EU ETS—An analysis for six industry sectors. Energy Economics, 91, 104883.
  • Colen, L., Gomez y Paloma, S., Latacz‐Lohmann, U., Lefebvre, M., Préget, R., & Thoyer, S. (2016). Economic experiments as a tool for agricultural policy evaluation: Insights from the European CAP. Canadian Journal of Agricultural Economics, 64(4), 667–694.
  • Czap, N. V., Czap, H. J., Banerjee, S., & Burbach, M. E. (2019). Encouraging farmers’ participation in the Conservation Stewardship Program: A field experiment. Ecological Economics, 161, 130–143.
  • Dai, H., Milkman, K. L., & Riis, J. (2014). The fresh start effect: Temporal landmarks motivate aspirational behavior. Management Science, 60(10), 2563–2582.
  • Dessart, F. J., Barreiro-Hurlé, J., & van Bavel, R. (2019). Behavioural factors affecting the adoption of sustainable farming practices: A policy-oriented review. European Review of Agricultural Economics, 46(3), 417–471.
  • Falk, T., Zhang, W., Meinzen-Dick, R. S., & Bartels, L. (2021). Games for triggering collective change in natural resource management: A conceptual framework and insights from four cases from India (IFPRI Discussion Paper 1995). International Food Policy Research Institute.
  • Ferraro, P. J. (2009). Counterfactual thinking and impact evaluation in environmental policy. New Directions for Evaluation, 2009(122), 75–84.
  • Ferraro, P. J., & Miranda, J. J. (2013). Heterogeneous treatment effects and mechanisms in information-based environmental policies: Evidence from a large-scale field experiment. Resource and Energy Economics, 35(3), 356–379.
  • Ferraro, P. J., & Price, M. K. (2013). Using nonpecuniary strategies to influence behavior: Evidence from a large-scale field experiment. Review of Economics and Statistics, 95(1), 64–73.
  • Ferraro, P. J., & Shukla, P. (2020). Feature—Is a replicability crisis on the horizon for environmental and resource economics? Review of Environmental Economics and Policy, 14(2), 339–351.
  • Foster, E. T., Rapoport, A., & Dinar, A. (2016). Alternative policies to manage electricity subsidies for groundwater extraction: A field study in Mexico University of California, Riverside School of Public Policy Working paper #16-05.
  • Frederiks, E. R., Stenner, K., & Hobman, E. V. (2015). The socio-demographic and psychological predictors of residential energy consumption: A comprehensive review. Energies, 8(1), 573–609.
  • Garrido, A. (2007). Water markets design and evidence from experimental economics. Environmental and Resource Economics, 38(3), 311–330.
  • Goeree, J. K., Palmer, K., Holt, C. A., Shobe, W., & Burtraw, D. (2010). An experimental study of auctions versus grandfathering to assign pollution permits. Journal of the European Economic Association, 8(2–3), 514–525.
  • Hadjigeorgalis, E. (2009). A place for water markets: Performance and challenges. Review of Agricultural Economics, 31(1), 50–67.
  • Hahn, R. W., & Stavins, R. N. (1992). Economic incentives for environmental protection: Integrating theory and practice. The American Economic Review, 82(2), 464–468.
  • Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.
  • Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466(7302), 29.
  • Higgins, N., Hellerstein, D., Wallander, S., & Lynch, L. (2017). Economic experiments for policy analysis and program design: A guide for agricultural decisionmakers (ERR-236, No. 1477-2017-3947). U.S. Department of Agriculture, Economic Research Service.
  • Jones, L. R., & Vossler, C. A. (2014). Experimental tests of water quality trading markets. Journal of Environmental Economics and Management, 68(3), 449–462.
  • Jotzo, F., & Löschel, A. (2014). Emissions trading in China: Emerging experiences and international lessons. Energy Policy, 75, 3–8.
  • Kirwan, B., Lubowski, R. N., & Roberts, M. J. (2005). How cost-effective are land retirement auctions? Estimating the difference between payments and willingness to accept in the conservation reserve program. American Journal of Agricultural Economics, 87(5), 1239–1247.
  • List, J. A. (2022). The voltage effect: How to make good ideas great and great ideas scale. Penguin Random House.
  • Mamine, F., Fares, M., & Minviel, J. J. (2020). Contract design for adoption of agri-environmental practices: A meta-analysis of discrete choice experiments. Ecological Economics, 176, 106721.
  • Marotzke, J., Semmann, D., & Milinski, M. (2020). The economic interaction between climate change mitigation, climate migration and poverty. Nature Climate Change, 10(6), 518–525.
  • Meinzen-Dick, R., Janssen, M. A., Kandikuppa, S., Chaturvedi, R., Rao, K., & Theis, S. (2018). Playing games to save water: Collective action games for groundwater management in Andhra Pradesh, India. World Development, 107, 40–53.
  • Muller, L., Lacroix, A., & Ruffieux, B. (2019). Environmental labelling and consumption changes: A food choice experiment. Environmental and Resource Economics, 73(3), 871–897.
  • Murphy, J. J., Dinar, A., Howitt, R. E., Rassenti, S. J., & Smith, V. L. (2000). The design of “smart” water market institutions using laboratory experiments. Environmental and Resource Economics, 17(4), 375–394.
  • Narloch, U., Drucker, A. G., & Pascual, U. (2017). What role for cooperation in conservation tenders? Paying farmer groups in the High Andes. Land Use Policy, 63, 659–671.
  • Nauges, C., & Whittington, D. (2019). Social norms information treatments in the municipal water supply sector: Some new insights on benefits and costs. Water Economics and Policy, 5(3), 1850026.
  • Open Science Collaboration. (2015). Reproducibility project: Psychology.
  • Palm‐Forster, L. H., Suter, J. F., & Messer, K. D. (2019). Experimental evidence on policy approaches that link agricultural subsidies to water quality outcomes. American Journal of Agricultural Economics, 101(1), 109–133.
  • Palomo-Hierro, S., Gómez-Limón, J. A., & Riesgo, L. (2015). Water markets in Spain: Performance and challenges. Water, 7(2), 652–678.
  • Porras, I., Barton, D. N., Miranda, M., & Chacón-Cascante, A. (2013). Learning from 20 years of payments for ecosystem services in Costa Rica. International Institute for Environment and Development.
  • Pyrko, J., & Uggmark, M. (2014). The experiment has shown-set your own goal and save electricity! SMARTGREENS, 14, 3–4.
  • Roe, B. E., & Just, D. R. (2009). Internal and external validity in economics research: Tradeoffs between experiments, field experiments, natural experiments, and field data. American Journal of Agricultural Economics, 91(5), 1266–1271.
  • Rosch, S., Raszap Skorbiansky, S., Weigel, C., Messer, K. D., & Hellerstein, D. (2021). Barriers to using economic experiments in evidence‐based agricultural policymaking. Applied Economic Perspectives and Policy, 43(2), 531–555.
  • Shortle, J. (2013). Economics and environmental markets: Lessons from water-quality trading. Agricultural and Resource Economics Review, 42(1), 57–74.
  • Siikamäki, J., Burtraw, D., Maher, J., & Munnings, C. (2012). The US Environmental Protection Agency’s Acid Rain Program (pp. 1–15). Resources for the Future Backgrounder.
  • Sindzingre, A. N. (2019). Experiments in economics and their ethical dimensions: The case of developing countries. In W. Dolfsma & I. Negru (Eds.), The ethical formation of economists (1st ed., pp. 92–113). Routledge.
  • Smith, V. L. (1976). Experimental economics: Induced value theory. The American Economic Review, 66(2), 274–279.
  • Stern, N. (2007). Stern review: The economics of climate change. Cambridge University Press, 2007.
  • Stoneham, G., Chaudhri, V., Ha, A., & Strappazzon, L. (2003). Auctions for conservation contracts: An empirical examination of Victoria’s BushTender trial. Australian Journal of Agricultural and Resource Economics, 47(4), 477–500.
  • Sudarshan, A. (2017). Nudges in the marketplace: The response of household electricity consumption to information and monetary incentives. Journal of Economic Behavior & Organization, 134, 320–335.
  • Taufik, D., Reinders, M. J., Molenveld, K., & Onwezen, M. C. (2020). The paradox between the environmental appeal of bio-based plastic packaging for consumers and their disposal behaviour. Science of the Total Environment, 705, 135820.
  • Torres, M. M. J., & Carlsson, F. (2018). Direct and spillover effects of a social information campaign on residential water-savings. Journal of Environmental Economics and Management, 92, 222–243.
  • U.S. Congress. (2009). Hearing to review the 2007 Farm Bill proposals of the US Department of Agriculture: Hearing before the Committee on Agriculture, House of Representatives, One Hundred Tenth Congress, First Session, February 14, 2007.
  • Van der Linden, S. L., Leiserowitz, A. A., Feinberg, G. D., & Maibach, E. W. (2014). How to communicate the scientific consensus on climate change: Plain facts, pie charts or metaphors? Climatic Change, 126(1), 255–262.
  • Wallander, S., Ferraro, P., & Higgins, N. (2017). Addressing participant inattention in federal programs: A field experiment with the conservation reserve program. American Journal of Agricultural Economics, 99(4), 914–931.
  • Wensing, J., Caputo, V., Carraresi, L., & Bröring, S. (2020). The effects of green nudges on consumer valuation of bio-based plastic packaging. Ecological Economics, 178, 106783.
  • Westerink, J., Jongeneel, R., Polman, N., Prager, K., Franks, J., Dupraz, P., & Mettepenningen, E. (2017). Collaborative governance arrangements to deliver spatially coordinated agri-environmental management. Land Use Policy, 69, 176–192.
  • Willamette Partnership, Forest Trends, & the National Network on Water Quality Trading. (2018). Breaking down barriers: Priority actions for advancing water quality trading.
  • Windle, J., Rolfe, J., McCosker, J., & Lingard, A. (2009). A conservation auction for landscape linkage in the southern Desert Uplands, Queensland. The Rangeland Journal, 31(1), 127–135.
  • Zizzo, D. J. (2010). Experimenter demand effects in economic experiments. Experimental Economics, 13(1), 75–98.

Notes

  • 1. The theory of mental accounting might be valuable here in guiding customized messaging at specific time points for the households to generate a “fresh start effect,” which restarts their water-conserving behavior after the impact of the initial nudge has fallen. This effect was introduced by Dai et al. (2014) in a health context.

  • 2. For a description of the use of choice experiments to study PES scheme attribute choice and design, see Mamine et al. (2020). Here, it is important to note that attitudes and psychological traits play a key role in preferences for both PES schemes and environmental goods in general as noted by Choi (2021) and Dessart et al. (2019).

  • 3. The impact of cognitive style on nudge effectiveness reiterates the importance of customizing nudge content as noted by Ferraro and Miranda (2013).

  • 4. For recent work on this topic of uncertain thresholds, see Ahsanuzzaman et al. (2022).

  • 5. Low contribution behavior is further exacerbated for subjects who are climate-change skeptics outside of lab settings, tying these results back to the body of evidence on communicating impacts of climate change to stakeholders in a way to ensure sustained and effective policy implementation and success.