Study design and setting
A two round modified Delphi technique (with a potential third) was employed in order to explore subject-expert consensus and identify in-situ use of variables to define major trauma in the absence of ISS through systematically collating questionnaire responses.
Grant, Booth (12) recommend that the Delphi should conclude after predetermined multiple iterations or when consistency between rounds is stable with unchanging opinion.
Definition of consensus
Mubarak, Hatah (13) highlight that 100% agreement can be achieved among experts and that an arbitrary percentage should be set prior to undertaking the study. Within our Delphi design, Likert type scales were used which give the option of a neutral response. With this in mind the research team set the arbitrary percentage of 70% agreement (positive or negative) as subject-expert consensus where the neutral score was not considered. The exception to this would be if the group agreement was more than or equal to a 70% neutral response.
The exploratory nature of the Delphi study allowed for feedback to be provided to the expert panel using group responses. This was believed to provide new information that may generate new perspectives to achieve a group consensus.
The survey was designed to reflect the outcomes of a literature review (5) and the output from three focus groups (14). This included:
- clinician factors, such as experience and exposure,
- patient factors, such as physiology, outcome measures and pre-trauma factors’
- situational factors, such as mechanism of injury.
Question domains were designed in order to ascertain potential clustering factors including both observable (e.g. profession, experience and age) and unobservable factors (e.g. values, attitudes, opinions and preferences).
Sampling of study participants (expert panel)
The expert panel members, who will be referred to as participants within this study, were from a broad range of professional groups who are exposed to and manage major trauma patients within their everyday workplace. Panel members were purposively selected based upon diversity of experience and expertise within a single trauma network.
Whilst there are no absolute guidelines as to the number of participants that may contribute to the Delphi process (11), the aim was to have at least three individuals from each relevant professional group within the Northern Trauma Network (NTN) which covers the North East and Cumbria areas of England.
Data collection and management
Ethical approval was granted through Integrated Research Application System (IRAS project ID: 237977).
We utilised a Delphi method with two iterations of questionnaires. The survey was conducted using the online system SurveyMonkey Inc. (San Mateo, California, USA). Panel members remained anonymous to one another throughout the data collection and analysis process. The Delphi study commenced on 12 December 2018 and ran through to 5 November 2019.
All data collected were stored electronically in a secure and password protected folder and anonymised prior to analysis.
Validity and Reliability
Sackman (15) suggested that the Delphi processes fail to meet standards of reliability and validity ‘normally set for scientific methods.’ However, careful scrutiny of Sackman’s assertions reveal that his concerns relate more to the methodological shortcomings of particular studies rather than overall methodological approach per se.
Anonymised results are believed to prevent attrition of panel members who may have a minority opinion (16). A short Pilot Study was carried out to refine the wording of the surveys and to remove potential ambiguities and ensure reliability of responses. All responses were anonymised and peer reviewed prior to any analysis and sharing with the panel members at repeated iterations between surveys.
Data Analysis
All quantitative data analysis was undertaken using the Software Package for the Social Sciences (SPSS; Version 26, IBM Inc.; Armonk, NY, USA).
After initial descriptive analysis of variables, Kruskal-Wallis tests were used to determine statistically significant differences (p <0.05) in response to the Delphi statements between the professional groups within the sample e.g. Doctors, Paramedics, Nurses and others which included managers, academics and administrators. The term ‘other’ was used to prevent unique individuals within specialised professional groups from being easily identified.
The Kruskal–Wallis test is a statistical method for ascertaining the significance of differences between the median values for K+ sub-groups from within the same sample: this is the test of choice when analysing ordinal data such as that generated by the Delphi instrument.
As no consistent patterns of opinion emerged in relation to professional group membership (Doctor / Paramedic / Nurse / other), a hierarchical cluster analysis was undertaken in order to identify patterns of similarity and difference of response within the data. Yim and Ramdeen (17) identified that ‘Cluster analysis refers to a class of data reduction methods used for sorting cases, observations, or variables of a given dataset into homogeneous groups that differ from each other.’ Cases (individual participants) are clustered based upon chosen characteristics – in this instance, similarity in the way they scored selected Delphi statements – and NOT their professional grouping. Cases in each specific cluster share many characteristics but are also dissimilar to those not belonging to that cluster. A three-cluster solution proved most economical and was ascertained using Ward’s method and squared Euclidian distance as a means to determine cluster membership. This process minimises variance within each cluster.
Cluster analyses are data reduction methods that are used for sorting cases, observations, or variables of a given dataset into homogeneous groups that differ from each other (17). In the current study, the analysis procedures identified clusters based upon similarity in response to the Delphi statements. Arranging response patterns together and classifying these as belonging to different broader groups provides a means of applying some organisation to individual Delphi responses, which at first sight might appear highly individualised or even chaotic. The technique of cluster analysis originated in biology and ecology (18) and although the technique has been reasonably widely employed in social science analysis, it has not (to date) gained the same level of application in health research.
Free text data generated by questionnaire responses were managed and analysed using NVivo qualitative data analysis software, QRS International Pty Ltd., Version 11, 2015. Data were coded and reviewed to identify emerging themes (19).
A grounded theory approach to qualitative analysis of the data allowed for potentially multiple iterations of the Delphi process to be influenced by the generated data and themes identified. This inductive approach allowed for theoretical insights to be generated as the process was undertaken rather than testing preconceived hypotheses (20). Within the context of this study it allowed for a thematic framework of nine areas.