Study overview
We fielded a survey experiment through Qualtrics in January 2023 to a nationally representative sample of 2847 American adults. To reduce social desirability bias, which results from the respondents’ desire to answer survey questions in a way they believe is morally or socially acceptable, we omitted information about the purpose of the research. Instead of informing the respondents that the purpose of the survey experiment was to examine how their attitudes related to energy and the environment are affected by different types of communication, we informed them that we conducted a study about how various issues related to science and technology can be communicated more clearly. We randomly assigned one of seven short videos to each respondent.
-
The control: A video that described blockchains, a topic related to technology but not to climate change or renewable energy.
-
The first treatment: A climate change video that described results from scientific research related to climate change.
-
The second treatment: A solar PV video that described how solar PV technology works and discussed some of its benefits, such as mitigating climate change.
-
The third treatment: A blue marble video that described the fragility of the Earth as it is viewed from space.
-
The fourth treatment: A blue marble and climate change video that described the fragility of the Earth viewed from space and results from scientific research related to climate change.
-
The fifth treatment: A blue marble and solar PV video that described the fragility of the Earth as it is viewed from space and also explained how solar PV technology works and how it can mitigate climate change.
-
The sixth treatment: A blue marble, solar PV, and ambassador video that described the fragility of the Earth as it is viewed from space, explained how solar PV technology works and how it can mitigate climate change, and included a testimony from an astronaut (Scott Kelly) who is a high credibility ambassador for solar PV technology and environmental protection.
We deliberately edited all videos and used language and images that are as close as possible to the language and images used on websites of scientific agencies (e.g. NASA) and online documentaries about climate change science and solar PV technology produced by reputable media sources (e.g. British Broadcasting Corporation, Deutsche Welle), thus creating realistic videos. To ensure that the respondents watched the video, we created a timer which did not allow moving to the next section of the survey experiment before the video ended. Additionally, we checked for comprehension by asking respondents to answer the question “the video presented to me was about…”, which had one correct answer and three incorrect answers. Only respondents who answered correctly were allowed to move to the next section.
Next, we asked respondents who correctly answered the comprehension question several questions organized in different sections. In one section, respondents were asked to answer several questions about the information presented in the videos and their responses to this information. We asked the questions “What are your thoughts after watching this video?”, “What are your feelings after watching this video?”, and “Would you say that you have learned anything from this video?” Respondents were asked to briefly answer these questions by writing between 1 and 3 sentences.
In the following section we assessed respondents’ attitudes related to climate change. We asked respondents to indicate to what degree they agree with the following statements: “Human activity, such as the burning of fossil fuels, contributes to global climate change” (labeled “climate change human activity”), and “I personally worry about climate change” (labeled “climate change personal worry”). For these questions respondents could choose from “agree 0 percent” to “agree 100 percent” in 10 points increments.
In the following section we gauged respondents’ attitudes towards solar power and other types of energy. To further reduce the possibility that respondents would deduce the real purpose of the study, we also asked several questions that were not related to environment or energy (these questions are described in the Methods section). We asked respondents to indicate to what degree they support expanding solar panel farms, wind turbine farms, offshore drilling, nuclear power plants, expanding natural gas, and expanding coal mining (“Please indicate to what degree you support expanding [solar panel farms/wind turbine farms/offshore drilling/nuclear power plants/natural gas/coal mining]”. For these questions respondents could choose from “support 0 percent” to “support 100 percent” in 10 points increments (labeled “Support solar PV energy”). We asked respondents to indicate to what degree they agree with the statement “I am planning to install solar panels on my own home as soon as possible” (labeled “Plan install solar PV”). For these questions respondents could choose from “agree 0 percent” to “agree 100 percent” in 10 points increments. Additionally, we assessed willingness to pay (WTP) for renewable electricity by asking “Would you be willing to pay more each month on your electricity bill to purchase all of your electricity from clean, renewable sources (such as wind and solar)?” (labeled “Pay more for RE”). Respondents could choose different options, from 0 dollars to 45 dollars and above, in five-dollar increments. Finally, we asked respondents questions about their age, gender, level of education, race, and political views. Table 1 provides descriptive statistics for the five dependent variables: Climate change human activity; Climate change personal worry; Support solar PV energy; Plan to install solar PV; Pay more for renewable energy.
Table 1
Summary statistics for dependent variables.
Variable | Mean | SD | N |
1. Climate change human activity. “Human activity, such as the burning of fossil fuels, contributes to global climate change” (0 = agree 0%; 10 = agree 100%) | 7.31 | 3.05 | 2,846 |
2. Climate change personal worry. “I personally worry about climate change” (0 = agree 0%; 10 = agree 100%) | 6.47 | 3.40 | 2,846 |
3. Support solar PV energy. “Please indicate to what degree you support expanding solar panel farms” (0 = support 0%; 10 = support 100%) | 7.40 | 2.80 | 2,846 |
4. Plan install solar PV. “I am planning to install solar panels on my own home as soon as possible” (0 = agree 0%; 10 = agree 100%; missing = I already have solar panels) | 4.27 | 3.58 | 2,755 |
5. Pay more for RE. “Would you be willing to pay more each month on your electricity bill to purchase all of your electricity from clean, renewable sources (such as wind and solar)?” (0 = No; 10 = $45 and above) | 3.57 | 3.37 | 2,846 |
The Methods section describes the data collection process and the ordered logistic regression used in our models. The Supplementary Materials documents provides (1) descriptive statistics and correlation coefficients for the control variables; (2) the script used in each video and links to each video; (3) quotes illustrating positive and negative responses; (4) additional results from text analyses; (5) tables of results from regression analyses; (6) results from additional variables that were not included in the main text due to lack of space.
We present the results from the survey experiment in four parts. First, we examine whether the treatments had an influence on respondents’ emotions. Next, we present the effects of the six treatments and their interactions with political orientation on climate change attitudes, followed by the effects of the treatments and interaction effects with political orientation on solar energy attitudes. Finally, we present the effects of the treatments and interaction effects with political orientation on willingness to pay for renewable energy.
The effects of treatments on respondents’ emotions
We examine whether the treatments that attempted to induce blue marble awe had an influence on respondents’ emotions by examining responses to the question “What are your feelings after watching this video?” The word maps from Fig. 1 present the frequency of words (size of the bubble) and the association between words (shown as the spatial proximity between words and calculated using Jaccard’s coefficient) for treatments T1-T3. This figure shows that respondents who received the first treatment (climate change video) frequently mentioned words related to negative emotions such as sad (ranked as number 4 in frequency), concerned (10), worried (19), and scary (28), but not awe. Respondents who received the second treatment (solar PV video) frequently mentioned words related to positive emotions such as good (ranked as number 4 in frequency), hope (11), hopeful (17), positive (18), great (19) and happy (20). Respondents who received the third treatment (blue marble video) frequently mentioned words related to blue marble awe such as awe (ranked as number 12 in frequency) and amazed (25), positive emotion words such as fine (9) and good (13), and a few negative emotion words such as sad (17) and concerned (34).
The word maps from Fig. 2 present the frequency of words and the association between words for treatments T4-T6. Respondents who received the fourth treatment (blue marble and climate change video) frequently mentioned words related to negative emotions such as sad (ranked as number 3), worried (11) and scared (16), but also words related to positive emotions such as good (17), and words related to blue marble awe such as beautiful (24) and fragile (43). Respondents who received the fifth treatment (blue marble and solar PV video) mentioned primarily words related to positive emotions such as good (ranked as number 9 in frequency), love (15), great (32) and hope (33), hopeful (34), and happy (36), but they also mentioned words related to negative emotions such as worried (38) and sad (44). Respondents who received the sixth treatment (blue marble, solar PV, and solar ambassador video) mentioned primarily words related to positive emotions such as good (ranked as number 8 in frequency) and happy (24), but also words related to negative emotions such as sad (23) and worried (44), and words related to blue marble awe such as amazed (45).
Individual statements also illustrate that many respondents had positive emotional responses, but some had negative responses. We present examples and additional results from text analyses in the Supplementary Material section.
The effects of treatments and interaction effects on attitudes toward climate change
Next, we examined the efficacy of different communication approaches for shaping climate change attitudes. Figure 3 shows the effects of treatments and interaction effects on the variable “climate change human activity”. This figure presents the effects of the six treatments (T1-T6) and of interaction effects between treatments and political orientation (T1XPO, etc.) on personal belief that climate change is caused by human activity. Figure 4 shows the effects of treatments and interaction effects on the variable “climate change personal worry”. This figure presents the effects of the six treatments (T1-T6) and of interaction effects between treatments and political orientation (T1XPO, etc.) on personal concern about climate change. The dots in these figures show odds ratios from ordered logistic regressions, and the lines show 95% confidence intervals. Because we are showing the odds ratios, values above 1 indicate that the effect is positive, and values below 1 indicate that the effect is negative. For simplicity in interpretation, we color coded the values of odd ratios as follows: the blue dot indicates that the odds ratio is significant at p < .05 or higher. The regressions include all treatments and control for respondent’s age, gender, level of education, race, and political orientation.
Results from Fig. 3 show that the first, second, fourth, fifth and sixth treatments (T1, T2, T4, T5, T6) have significant positive effects on personal belief that climate change is caused by human activity. Due to lack of space, we interpret results only for one of these measures. For example, for respondents who watched the climate change video (T1) the odds of agreeing 100 percent versus agreeing 90 percent or less with the “climate change human activity” statement are 1.74 times greater (p < .001) than for respondents who watched the control video. Additionally, results show that the interaction between the first treatment and political orientation (T1XPO) is significant (p < .01) and positive, indicating that those with a liberal political orientation are more likely to believe that climate change is caused by human activity after watching the climate change video than those with a conservative political orientation. However, the interaction between the third treatment and political orientation (T3XPO) is significant (p < .01) and negative, indicating that those with a conservative political orientation are more likely to believe that climate change is caused by human activity after watching the blue marble awe video than those with a liberal political orientation.
Results from Fig. 4 show that the first, fourth, fifth and sixth treatments (T1, T4, T5, T6) have significant positive effects on personal concern about climate change. For example, for respondents who watched the climate change video (T1) the odds of agreeing 100 percent versus agreeing 90 percent or less with the “climate change personal worry” statement are 1.42 times greater (p < .01) than for respondents who watched the control video. Results from Fig. 4 also show that the interaction between the first treatment and political orientation (T1XPO) is significant (p < .01) and positive, indicating that those with a liberal political orientation are more likely to be personally concerned about climate change after watching the climate change video than those with a conservative political orientation. Moreover, the interaction between the third treatment and political orientation (T3XPO) is significant (p < .01) and negative, indicating that those with a conservative political orientation are more likely to personally worry about climate change after watching the blue marble awe video (T3) than those with a liberal political orientation. Similarly, the interaction between the fourth treatment and political orientation (T4XPO) is significant (p < .01) and negative, indicating that those with a conservative political orientation are more likely to personally worry about climate change after watching the blue marble awe and climate change video (T4) than those with a liberal political orientation.
Taken together, results from Figs. 3 and 4 suggest that watching a climate change video (T1) had a significant positive effect on both climate change attitudes, and that watching a video that combine blue marble awe and either climate change or solar technology facts (T4, T5, T6) also had a significant positive effect on climate change attitudes. Additionally, watching the blue marble awe (T3) video has a stronger effect for conservatives than for liberals on both climate change attitudes.
The effects of treatments and interaction effects on attitudes toward solar energy
We also examined the efficacy of different communication approaches for shaping solar energy attitudes. Figure 5 shows the effects of treatments and interaction effects on the variable “support solar PV energy”. Figure 6 shows the effects of treatments and interaction effects on the variable “plan to install solar PV”. The interpretation of these figures is similar to the interpretation of Figs. 3 and 4.
Results from Fig. 5 show that the second, fourth, fifth and sixth treatments (T2, T4, T5, T6) have significant positive effects on personal support for solar PV energy. Due to lack of space, we interpret results only for one of these measures. For example, for respondents who watched the blue marble awe, solar PV technology, and solar ambassador video (T6) the odds of agreeing 100 percent versus agreeing 90 percent or less with the “support solar PV energy” statement are 2.01 times greater (p < .001) than for respondents who watched the control video. Additionally, results from Fig. 3 show that the interaction between the first treatment and political orientation (T1XPO) is significant (p < .05) and positive, indicating that those with a liberal political orientation are more likely to support solar PV technology after watching the climate change video (T1) than those with a conservative political orientation. Yet, the interaction between the third treatment and political orientation (T3XPO) is significant (p < .05) and negative, indicating that those with a conservative political orientation are more likely to support solar PV technology after watching the blue marble awe video (T3) than those with a liberal political orientation.
Results from Fig. 6 show that the second, fifth and sixth treatments (T2, T5, T6) have significant positive effects on personal intention to install solar PV. Due to lack of space, we interpret results only for one of these measures. For example, for respondents who watched the blue marble awe, solar PV technology, and solar ambassador video (T6) the odds of agreeing 100 percent versus agreeing 90 percent or less with the “plan to install solar PV” statement are 1.83 times greater (p < .001) than for respondents who watched the control video. However, results from Fig. 6 show that none of the interactions between the treatments and political orientation are significant.
Results from Figs. 5 and 6 suggest that watching a video about climate change facts (T1) does not have a significant positive effect on both solar energy attitudes, but that watching a video about solar technology facts (T2) has a significant positive effect on both solar energy attitudes. Similarly, watching a video that combines blue marble awe and solar technology facts (T5, T6) had a significant positive effect on both solar energy attitudes. Watching the blue marble awe (T3) video has a stronger effect for conservatives than for liberals on respondents’ support for solar energy, but not on their plan to install solar PV. Finally, we note that the sixth treatment (the blue marble awe, solar PV technology, and solar ambassador video) had a stronger effect than all other treatments on both solar energy attitudes.
The effects of treatments and interaction effects on willingness to pay for renewable energy
Finally, we investigated the efficacy of different communication approaches for the willingness to pay for renewable energy. Figure 7 shows the effects of treatments and interaction effects on the variable “pay more for renewable energy”. The interpretation of this figure is similar to the interpretation of previous figures.
Results from Fig. 7 show that the fourth and sixth treatments (T4, T6) have significant positive effects on willingness to pay more for renewable energy. Due to lack of space, we interpret results only for one of these measures. For example, for respondents who watched the blue marble awe, solar PV technology, and solar ambassador video (T6) the odds of willing to pay more than 45 dollars versus willing to pay less than 45 dollars are 1.53 times greater (p < .001) than for respondents who watched the control video. Additionally, results from Fig. 7 show that the interaction between the fourth treatment and political orientation (T4XPO) is significant (p < .05) and negative, indicating that those with a conservative political orientation are more likely to be willing to pay more for renewable energy after watching the blue marble awe and climate change science video (T4) than those with a liberal political orientation.
Results from Fig. 5 suggest that watching a video about climate change facts (T1) does not have a significant positive effect on willingness to pay more for renewable energy, but that watching a video that combines blue marble awe and climate change facts (T4) has a significant positive effect. Watching a video that combines blue marble awe and climate change facts has a stronger effect on conservatives’ willingness to pay more for renewable energy than on liberals’ willingness to pay. Additionally, watching a video that combines blue marble awe, climate change facts, and a message from a solar ambassador (T6) has the strongest effect on willingness to pay more for renewable energy.