Study design
This study used the Delphi method technique, which has become increasingly embraced in health science disciplines. The method was employed based on two modified rounds of online surveys (herein referred to as the eDelphi survey) to achieve the study objective. The Delphi technique involves iterative or structured processes designed to collect the opinions, views, or feedback of experts on a particular research question where uncertain, complex, or incomplete knowledge and information exist [26, 27]. The aggregated group answers in each round are sent back to participants so that they can reconsider their opinions or judgments on the basis of the feedback received and revise them if necessary to resolve any disagreements before arriving at group consensus or salient recommendations [26]. As such, we considered the Delphi method appropriate to ultimately attain the consensus of multi-stakeholder experts from within and outside China and across diverse professional fields on interventions for enhancing GRC for older adult Chinese in managing or living with disabling health conditions induced by floods.
While some studies have proposed some guidelines for conducting Delphi studies, there are still no specified criteria or guidelines about the number of Delphi rounds, sample size, or how to organize them. In this case, we only conducted two rounds of eDelphi expert consultations, whereby the first one was informed by the findings of a systematic review conducted a priori.
Development of the eDelphi survey
The survey was designed and distributed online using the QualtricsXM platform (Provo, USA) in both Chinese and English by the research team, drawing on their prior experience and knowledge in conducting mixed-method research, including Delphi methods and systematic reviews in fields like disaster risk reduction (DRR), health-related rehabilitation, disability, and emergency medicine [28–30]. Aside from the participants’ demographic and informed consent sections, the survey consisted of 4 domains of statements about the proposed interventions in prior systematic review to address a) cognitive and intellectual; b) physical; c) mental and psychosocial; and d) chronic and terminal conditions among Chinese older adults in contexts of flooding. The demographic part aimed at capturing the information of eDelphi participants, for example, about their age, gender, nationality, province or location, educational level and background, current work position, and years of experience.
The study team jointly constructed and refined the statements on GRC interventions in the first draft of the eDelphi survey, which contained 44 statements based on four categories of disabling conditions that were identified in an earlier systematic review. Afterwards, the 2nd version of the multilingual eDelphi survey (both Chinese and English) was pilot-tested with 4 international academics (excluded from participating in R1 and R2) who are actively involved in research related to the study theme. This helped to modify and improve the structure and readability of some statements, as well as determine whether some were to be either added or removed. As a result, 13 statements were removed, and the final version of the eDelphi survey that was disseminated to the experts in R1 consisted of a list of 31 statements (S). On top of that, following the pilot-testing of the eDelphi survey, these statements were listed in general, unlike their previous grouping under the four domains as aforementioned.
The survey was designed in such a way that, after consenting, participants were also required to provide their email(s) so that the eDelphi survey in the two rounds (R1 and R2) could be emailed to them and contacted in case of any further clarifications. A provision for a free-text response was included in the eDelphi survey for participants to elaborate on and explain their rating or make any comments they felt were necessary on any statements. It is worth noting that although three rounds were originally planned to be conducted, the third was foregone because a majority consensus on statements was achieved in the first two rounds. Additionally, since the statements were constructed from the findings of a systematic review conducted a priori, two survey rounds were also deemed enough for eDelphi participants to reflect on their rating before reaching optimal consensus.
The eDelphi participants were required to independently rank the statements based on a 5-point Likert scale (agree, somewhat agree, disagree, somewhat disagree, and neither agree nor disagree). The Likert scale has been disclosed to be favorable and reliable for Delphi studies because it measures the extent to which participants agree with a statement and determines the degree to which they agree with each other [31]. The entire process of drafting, pilot testing, reviewing, revising, and coming up with the last version of the eDelphi survey happened between September and October 2023. All authors actively participated in these processes, and disagreements about the statement formulation were resolved through consensus. Each eDelphi round was estimated to take between 10 and 15 minutes and be completed within a month. Two email reminders were to be sent to respondents during either round to complete the eDelphi survey if no responses had not been received from them two weeks after launching the round, and the second reminder one week before the round closure.
Recruitment of the eDelphi experts: The distinguished experts the research team identified through their online publications and biographies, and also through a snowballing approach, were invited by email to participate in the study with a short explanation about the study background. As the criteria for their selection, we explicitly recruited those who are knowledgeable and whose scholarship, research, or policymaking undertakings, particularly in China, revolved around the fields of ageing, gerontology, rehabilitation, disability, DRR, climate change, public health, and others, as shown in Table 1. They also had to be aged 18 years and older, widely published in renown journals, possess the requisite work experience spanning at least 3 years, be fluent in Chinese or English, be affiliated with notable policy, academic, research, or organizational institutions within and outside China, and agree to participate in all two rounds of the eDelphi survey.
Table 1
Characteristics of eDelphi experts who participated in R1 and R2
| | Round 1 (n = 50) | Round 2 (n = 44) |
Age | Min | 25 | 25 |
Max | 74 | 74 |
Mean | 42.46 | 42.6 |
Gender | Male | 32 (64%) | 27 (61.4%) |
Female | 18 (36%) | 17 (38.6%) |
Nationality | Chinese | 33 (66%) | 31 (70.5%) |
Others | 13 (26%) | 10 (22.7%) |
American | 2 (4%) | 1 (2.3%) |
British | 1 (2%) | 1 (2.3%) |
Korean | 1 (2%) | 1 (2.3%) |
Province | Shanxi | 1 (2%) | 1 (2.3%) |
Liaoning | 1 (2%) | 1 (2.3%) |
Jilin | 1 (2%) | 1 (2.3%) |
Jiangsu | 3 (6%) | 3 (6.8%) |
Zhejiang | 2 (4%) | 2 (4.5%) |
Anhui | 2 (4%) | 2 (4.5%) |
Fujian | 1 (2%) | 1 (2.3%) |
Shandong | 1 (2%) | 1 (2.3%) |
Henan | 2 (4%) | 2 (4.5%) |
Hubei | 4 (8%) | 4 (9.1%) |
Guangdong | 1 (2%) | 1 (2.3%) |
Sichuan | 18 (36%) | 13 (29.5%) |
Yunnan | 1 (2%) | 1 (2.3%) |
Shaanxi | 1 (2%) | 1 (2.3%) |
Gansu | 1 (2%) | 1 (2.3%) |
Qinghai | 1 (2%) | 1 (2.3%) |
Others | 9 (18%) | 8 (18.2%) |
Educational background | Gerontology | 4 (8%) | 4 (9.1%) |
Disaster risk reduction | 19 (38%) | 16 (36.4%) |
Climate change | 4 (8%) | 4 (9.1%) |
Rehabilitation | 1 (2%) | 1 (2.3%) |
Disability | 4 (8%) | 3 (6.8%0 |
Administration & policymaking | 5 (10%) | 5 (11.1%) |
Medical sciences or public health | 5 (10%) | 4 (9.1%) |
Others | 8 (16%) | 7 (15.9%) |
Educational level | Bachelors | 7 (14%) | 7 (15.9%) |
Masters | 11 (22%) | 9 (20.5%) |
PhD (Doctorate) | 32 (64%) | 28 (63.6%) |
Current job position | University professor | 24 (48%) | 21 (47.7%) |
Policymaker | 4 (8%) | 4 (9.1%) |
Researcher | 13 (26%) | 12 (27.3%) |
Medical specialist | 2 (4%) | 2 (4.5%) |
Others | 7 (14%) | 5 (11.4%) |
Years of work experience | <=5 years | 14 (28%) | 12 (27.3%) |
6–10 years | 13 26%) | 12 (27.3%) |
11–15 years | 5 (10%) | 3 (6.8%) |
16–20 years | 10 (20%) | 10 (22.7%) |
21–25 years | 2 (4%) | 2 (4.5%) |
26–30 years | 1 (2%) | 1 (2.3%) |
>=31 years | 5 (10%) | 4 (9.1%) |
As aforementioned, although there are no guidelines on the recommended sample size for Delphi studies [26, 28, 32], what is important is to recruit participants with sufficient knowledge and expertise about the subject matter under investigation. However, some earlier Delphi studies established that at least a minimum sample size of 10 representative experts is adequate for content validity and reaching consensus [32, 33]. For this reason, a non-probability purposive sample of 150 experts within and outside China was initially invited via email to participate in R1. Due to the non-response, attrition, or dropout rate that was projected at 20% in both rounds, as has been witnessed in previous Delphi studies [33, 34], 50 additional experts were identified and contacted to participate in the eDelphi survey. In total, 200 experts were recruited and considered appropriate to achieve the final consensus in the study. It should be noted that issues of diversity and representativeness when recruiting them were prioritized by taking into account the above-mentioned fields or specialties, gender, and at least one participant from each province or autonomous region of China.
The eDelphi survey procedures
The author (JKB) led the research team in administering the Qualtrics-generated survey via a link that was emailed to eDelphi experts who consented to participate in both eDelphi R1 and R2. However, experts who consented but did not complete R1 were not eligible to participate in R2, whereas those who did not complete the survey in R2 were also excluded.
RI: In this round, which was implemented in November 2023, using a 5-point Likert scale, participants were asked to independently rate a total of 31 statements that were formulated in line with the proposed interventions for addressing the four categories of disabling conditions as noted above.
R2: This round was implemented in December, and it commenced immediately after R1 had ended. Since providing respondents with feedback on how their peers rated or voted for consensus building is the most important component of the Delphi methods [26, 33, 35], participants in R2 were able to view the aggregated response rate on 31 statements from R1. Again, a link to the Qualtrics-generated survey was emailed to participants so that they could reconsider and evaluate their previous rating in R1 still based on the 5-point Likert scale. Where possible, they also had an opportunity to state or elaborate on the rationale for their rating under the provision of free-text responses in the eDelphi survey. On a particular note, in R2, no additional statements were added or removed. This round's overall aim was to validate, revise, and ultimately acquire final consensus or any other vital suggestions from the eDelphi participants. In the end, this would help in identifying and prioritizing the most significant interventions for enhancing GRC for older adults within and outside China in managing or living with their disabling health conditions induced by floods.
Data analysis
QualtricsXM was used to generate data reports, which informed the subsequent analyses. The level of participants’ consensus on each statement in R2 was determined by adopting a grading system denoted by U, A, B, and C for unanimous (100%), 90–99%, 78–89%, and 67–77% consensus, respectively, that has recently been used in a COVID-19-related Delphi study [36]. A minimum cut-off on a statement from the consensus of all the eDelphi participants was at ≥ 50%, and statements that attained the consensus below this percentage were excluded for analysis in R2. Positive or negative consensus was considered to be attained at ≥ = 70% if all participants agreed and strongly agreed or disagreed and strongly disagreed with statements in two rounds. Using Microsoft Excel (version 16), descriptive statistics were generated to present the demographic characteristics of participants and ratings for each statement, as well as calculate the percentage changes in statement rating between the two rounds.