This study was part of the PRECISION-TBI project17. Three different groups consisting of clinicians and research experts in the field were formed to establish a consensus based Australian quality indicator set for the optimum acute management of msTBI. Participation in this study was voluntary and no financial support was provided.
Steering Committee
The steering committee consisted of three intensive care clinicians (TJ, RR, AAU) and one TBI/Trauma researcher (CRB). Their role was to manage the overall conduct of the study, create a preliminary indicator set and identify recognised experts in the field as members for the advisory committee (AC).
Advisory committee
The AC was composed of clinicians and researchers with specific expertise in one or more stages of the care pathway of msTBI. The members of this group were also selected to be representative of the various regions of Australia. The AC was convened and comprised of one neurocritical care nurse (CB), one neurosurgeon (RJ), one trauma care and systems evaluation researcher (BJG), one pre-hospital trauma care specialist (SR), one emergency physician (GOR), two trauma surgeons (CB, ZJB) and three senior intensivists (OF, LTC, DJC). The AC undertook key responsibilities including refinement of the preliminary indicator set, compilation of a list of experts for the Delphi panel, oversight of the Delphi process and interpretation of the results and approval of the final indicator set.
Delphi panel
The steering and advisory committees agreed that the Delphi panel should consist of experts in the field of msTBI from across Australia including pre-clinical scientists, acute care physicians and nurses from the emergency department, neurology, neurosurgical and intensive care specialties and specialists in neurorehabilitation. To qualify for participation in the Delphi group, members needed a minimum of three years of professional experience in either clinical management of msTBI patients or TBI clinical research. Experts were identified from professional networks of the committee members, principal investigators, and collaborators from the PRECISION-TBI project17 and the BONANZA-GT study (ACTRN12619001328167). Given the technical nature and requirement for specific clinical knowledge to assess the quality indicator set, consumer representatives did not form part of the Delphi panel.
Design of the indicator set
Design of the preliminary indicator set was undertaken by the steering committee. As a collection of variables does not necessarily constitute a valid or comprehensive indicator set19, target construct, content and measurement domains (Table 1) were identified explicitly to facilitate assessment of the overall structure of the indicator set by the AC. In line with Huijben et al.’s15 assessment of the importance of ICU care, outcomes from msTBI, the ICU management content domain contained the largest number of indicators. In a departure from this methodology the steering committee concluded that the homogeneity of health care provision and approaches to msTBI management in the Australian context allowed for the inclusion of two sub-groups of process indicators addressing: 1) specific logistical process indicators such as time to primary CT brain and 2) clinical management process indicators such as burden of intra-cranial pressure. Where evidence was available to support specific benchmarks e.g. incidence rate of pre-hospital hypotension 20 these were set. Where no evidence was available to support a benchmark value, indicators were left as measures which could be used to establish future benchmarks.
Refinement of the indicator set
Following the distribution of an introductory document outlining the rationale for the study, the Delphi process, and the preliminary indicator set, the AC met via video conference to discuss the roles and responsibilities of the advisory group and the preliminary variables. The AC was then asked to complete an online questionnaire (REDCap database) assessing the rationale (as defined by the targeted construct) and structure (as defined by the content domains and measurement domains)19 of the indicator set. The questionnaire also offered the opportunity for advisory group members to suggest alternative constructs, content domains and measurement domains. As a continuation of this questionnaire the AC members were then asked to review each indicator in the context of its content domain (e.g. emergency department management) according to the following criteria:
- Content coverage: do the suggested indicators cover all important aspects of msTBI care that may impact outcome at this stage of the care pathway?
- Proportional representation: does the number of indicators for each stage of the care pathway accurately reflect the impact of this stage on patient outcomes?
- Contamination: do any of the indicators influence or overlap with other indicators and render either unnecessary or misleading?
The AC members were also given the opportunity within the questionnaire to advocate for the inclusion of additional indicators or the removal/alteration of existing indicators and to nominate experts to form the Delphi panel.
Delphi group assessment of indicator set
The third stage of the project was the circulation of the AC approved indicator set to the wider Delphi group. The indicator set was provided alongside an introductory document outlining the aims and objectives of the project. Delphi panel members were asked to score each of the indicators according to the selection criteria set out in Table 2 using a 5-point Likert scale allowing scores between 1 (strongly disagree) to 5 (strongly agree). Delphi members were also given the opportunity to provide comments on each of the proposed indicators and to suggest alternative indicators.
Inclusion of an indicator in the final indicator set was contingent upon agreement and consensus scoring. Agreement was defined as a median Likert score of 4 (agreement) or 5 (strong agreement). Consensus was defined as an interquartile range (IQR) of <1 on scores for validity and <2 for all other criteria. The higher threshold for consensus on validity scoring stems from the recognition that validity is the most important characteristic of a quality indicator19 and is in line with previous literature15.
Statistical analysis
Descriptive statistics were employed to determine the median and IQR of each indicator and to calculate the demographics of the Delphi panel members. All statistical analyses were performed using PRISM 10 (GraphPad). Questionnaires were developed and distributed using REDCap21.