Timeline for the Field-Based Experiment and Implementation Study
Figure 3 details the dates of the field-based experiment, the implementation of the best warm-glow message and the dates for the pre-post implementation analysis. The methodology of the field based experiment and implementation trial is detailed below.
Field-Based Experiment of Warm-Glow and Positive Affect Messages
Sampling and sample: A sample of 5,821 of new whole blood donors with A-, A+, O- and O + blood types across Australia who made their first whole blood donation six weeks previously were recruited using the following criteria: (1) made their first whole blood donation six weeks ago, and (2) had not donated previously. For each week of the field-based experiment, all donors who met these eligibility criteria were selected and randomly allocated to conditions. Thus, the whole eligible population was sampled each week. This process continued until the pre-determined numbers per condition were achieved. Twenty donors were excluded: twelve due to email bounces and 8 due to being permanently deferred from donating blood after their initial donation (Fig. 4).
Procedure: New A-, A+, O- and O + donors were recruited to the trial when they made the first donation between the 16th of April 2018 to the 8th July 2018, with the first message issued from the 22nd of May 2018. These donors were randomly allocated (using simple randomization) to one of four active arms and a control arm. The control arm contained no message. All other communications from Lifeblood, including an SMS reminder at 12 weeks that the donor could donate whole blood again, were identical across the arms of the trial. Thus, the only difference was the warm-glow and positive affect messages with or without an identity prime. The four active arms crossed a warm-glow vs positive affect message with a prime (“… that’s when you became a blood donor …”) or no prime for cooperative identity (Fig. 5: Supplementary File S1 contains the full emails sent).
Measures: We collected the following data on the donors: (1) age, (2) sex (0 = male, 1 = female), (3) blood group (ABO: A-, A+, O-, O+, O) and (4) if they had rebooked to make their next donation immediately after their 1st donation (scored 1) or had not initially rebooked (scored 0)
Outcome: The main outcome was a verified attendance at the donor centre to make a donation (whole blood or plasma) three months after becoming eligible to donate. These data were collected using eProgesia. The data analysts who extracted the attendance data were blind to the experimental condition that the donor was assigned to. Attending to donate is a clear behavioural act of wishing to cooperate to help a stranger and the study is powered for attendance to donate as the outcome.
Power Analysis
Warm-glow has a small effect size with respect to predicting blood donor attendance [56, 58]. In a simple regression model with 8 predictors (e.g., arms, age, sex, blood group, rebooked and interaction of rebooked with arms) to achieve 80% power, with an alpha of 0.05 and a small effect size requires 757 donors per arm. Thus, we aimed for 1,000 new donors per arm to allow for any exploratory analyses.
Pre-Registration: The trial was pre-registered on the Open Science Framework prior to ‘prior to data collection commencing’ (OSF reference: https://osf.io/5m69k).
Consenting: All donors, when they attend to make a donation, sign an individual general declaration consenting to assist blood donor research.
Ethics: The field-based experiment was approved by the Australian Red Cross Lifeblood Ethics Committee on 7th May 2018, with the first randomized message sent out on the 22nd of May for those who had made a donation on the 16th of April (Reference: Davison 04052018).
Implementation Study
Design. When recruitment for the field-based experiment stopped, as the specified number of donors per arm had been achieved, the most effective message, based on marketing click through data (click to open) was selected to be rolled out nationally on the 9th of July 2018. Once the field-based experiment follow up data were collected and analysed it showed that the field-based experiment results confirmed the click rate choice. This correspondence allowed us to conduct the implementation study.
The implementation samples consisted of all new donors across Australia who were whole blood donors with autologous and therapeutic donors excluded. We collated aggregate data on whether or not they had attended to make a 2nd donation (WB or apheresis) within 3 months and whether or not they had initially rebooked in the centre or not. We compared the frequency of attending to make a second donation (whole blood or plasma) in a 3 year window prior to the message roll out to the frequency in the year after the message roll out, across two time windows. The pre-implementation period covered 3 time windows consisting of the 16th of April to the 15th of April for years (1) 2015-2016, (2) 2016-2017 and (3) 2017-2018 and the post-implementation period covered two slightly overlapping time windows: (1) the 9th of July 2018 to the 8th of July 2019 and (2) the 16th of April 2019 to the 15th of April 2020. The first post implementation time window covers the year from the moment the most effective warm-glow message went live across Australia and the second window covers the April to April window that is consistent with the April to April pre-implementation time window.
Ethics: As these data for the implementation analysis are aggregated at the population level, and all donors when they attend to make a donation sign a general declaration consenting to assist blood donor research, no specific additional ethical approval was needed for these analyses.
Statistical Analyses: Field Based Experiment and Implementation Data
All data were analysed using standard statistical packages (IBM SPSS v26, ZumStat, Psychometrica). All tests are two-tailed and effects sizes for all analyses are reported as Cohen’s D. Cohen’s D was derived for comparison across multiple group means using the procedures described in [80] and from Z scores using procedure described in [81], with both implemented in Psychometrica [82]. Odds Ratios are converted to D using procedures described in [83]. Comparison of percentage across groups, including interactions, used procedure detailed in [84–85].