Aggregate ratings across understandability, believability and convincingness.
A main effect of message type, F(9, 953)=4.49, p=.001, ƞp2=.041, was shown. TPD1, TPD2, and relative risk messages (RRM) 11, 5, 20 and 7 did not differ significantly (all ps>.05). TPD1 and TPD2 scored significantly higher than RRM12, RRM8 and RRM25, whilst RRM9 scored significantly lower than TPD1 but not TPD2 (see Table 3 for mean [SE] ratings for the overall sample and Figure 2 for mean [SE] ratings per smoking status group). No main effect of smoking/vaping status was found, F(2, 953)=1.32, p=.268, ƞp2=.003. A significant message type by smoking status interaction was shown, F(18, 953)=2.33, p=.001, ƞp2=.042. Non-smokers attributed higher scores to TPD1 compared to dual users (M=6.04, 95%CI[5.66-6.42] vs. M=5.14, 95%CI[4.72-5.56]) p=.006), whilst dual users rated RRM12 “This product is a safer alternative to smoking” more favourably than non-smokers (M=5.44, 95%CI[5.10-5.77] vs. M=4.32, 95%CI[3.92-4.73]) p<.001).
Message understandability ratings
There were no main effect of message type, [F(9, 953)=0.88, p=0.54)], smoking status [F(2, 953)=0.81, p=.44] or message type by smoking status interaction [F(18, 953)=1.17, p=.28] for understandability (see Figure S1 supplementary materials for means [SE]).
Message believability ratings
Main effects for type of message [F(9, 953)=6.16, p<.001, ƞp2=.055)], and smoking/vaping status [F(2, 953)= 3.16, p=.04, ƞp2=.007] were found. Mean believability scores for TPD1 and TPD2 were highest followed by RRM11 ”Completely switching to e-cigarettes lowers your risk of smoking related disease”, RRM5 “Use of this product is much less harmful than smoking” and RRM20 “Completely switching to e-cigarettes is a healthier alternative to smoking”; these 5 messages did not differ statistically from each other (all ps >.05). TPD 1 was rated as the most believable and differed significantly from RRM7 (p = .028), RRM8 (p < .001), RRM9 (p = .001), RRM12 (p = .001) and RRM25 (p<.001). TPD2 rated second most believable and differing significantly from RRM8 (p=.005), RRM9 (p = .008), RRM12 (p=.006) and RRM25 (p<.001). See Figure S2 supplementary materials for mean [SE] ratings.
A significant message type by smoking/vaping status interaction [F(18, 953)=1.78, p=.023, ƞp2=.033] was shown. Dual users rated RRM12, “This product is a safer alternative to smoking”, as more believable (M=5.33, 95%CI[4.90-5.76] compared to non-smokers (M=4.11, 95%CI[3.57-4.65], p=.001). Smokers rated RRM5, “Use of this product is much less harmful than smoking”, significantly less believable compared to dual users (M=4.52, 95%CI[3.98-5.05] vs. M=5.32, 95%CI[4.88-5.75], p=.05).
Message convincingness ratings
Significant main effects for message type [F(9, 953)=5.75, p<.001, ƞp2=.052], and smoking/vaping status [F(2, 953)=3.76, p=.02, ƞp2=.008] were shown. The mean score for TPD1 (M=5.52, 95%CI[5.24-5.79]) was the highest followed by TPD2 (M=5.33, 95%CI[5.01-5.64]) then RRM11 “Completely switching to e-cigarettes lowers your risk of smoking related diseases” (M=5.07, 95%CI[4.79-5.35]) and RRM5 “Use of this product is much less harmful than smoking” (M=4.89, 95%CI[4.61-5.17]). TPD1 rated as significantly more convincing than all other messages (all p<.05) except from TPD2, and RRM11. RRM11 in turn, was rated as significantly more convincing than RRM8, RRM9, RRM12 and RRM25 (p<.05).
A significant message type by smoking/vaping status interaction [F(18, 953)=2.49, p=.001, ƞp2= .045] was shown. Non-smokers rated TPD1 as more convincing and RRM25, ”Using an e-cigarette doubles your chances of quitting smoking”, as less convincing compared to dual users (TPD1: M=5.00, 95%CI[4.52-5.48] vs. M=6.16, 95%CI[5.78-6.54], p=.001; RRM25: M=4.20, 95%CI[4.08-5.05] vs. M=5.29, 95%CI[4.92-5.65], p=.005). Non-smokers rated RRM12, “This product is a safer alternative to smoking”, as less convincing than dual users (M=3.78, 95%CI[3.29-4.258] vs. M=5.21, 95%CI[4.82-5.59], p<.001) and smokers (M=4.69, 95%CI[4.27-5.10], p=.012). See Figure S3 supplementary materials for mean (SE) ratings.