Point-of-Care Ultrasound Curriculum in Acute Care Medicine: A Protocol for a Systematic Review and/or Meta-Analysis

doi:10.21203/rs.3.rs-519510/v1

Download PDF

Protocol

Point-of-Care Ultrasound Curriculum in Acute Care Medicine: A Protocol for a Systematic Review and/or Meta-Analysis

https://doi.org/10.21203/rs.3.rs-519510/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Beginning with the guidance of central line insertion, point-of-care ultrasound (POCUS) has evolved into a more inclusive skill set to aid in the examination and management of the acutely ill patient. Published evidence, including original literature and consensus recommendations support an array of applications in the multi-disciplinary arena of acute care medicine. In parallel, we have seen multiple professional societies’ call for more POCUS training in residency. While POCUS has been received with enthusiasm in acute care medicine, there are a number of challenges to ensuring trainees can competently perform POCUS in the acute care environment. There is inconsistent evidence to support optimum practices in curriculum design, implementation, assessment, and evaluation. To help explore this gap, we are conducting a systematic review and meta-analysis of current evidence regarding POCUS curricula.

Methods

We will search electronic databases: MEDLINE, Embase, Cochrane Library, CINAHL, Ovid ERIC, Science Citation Index, and Conference Proceedings Citation Index. Further, we will search the ClinicalTrials.gov register, hand search key proceedings and check references from relevant systematic reviews. Title, abstract and full text screening for inclusion of eligible papers will be performed in duplicate, in accordance with the PRISMA statement. Included publications will be evaluated for internal validity using the Medical Education Research Study Quality Instrument (MERSQI) scale for educational studies. Data abstraction will be conducted using standardized forms with focus on learner population, number of participants, setting, POCUS application, methods of instruction, duration of intervention, methods of assessment, and program evaluation. Further to this, emphasis will be placed on validity arguments of assessment tools using Kane’s framework. Primary analysis will be qualitative in nature. When possible, homogenous studies will be pooled for quantitative meta-analysis.

Discussion

Our systematic review will summarize the current evidence base for POCUS curriculum implementation, evaluation and assessment validity for acute care applications. We anticipate that our review will fill a critical knowledge gap, providing a sound platform for future evidence-based curriculum development.

Systematic Review Registration

Our systematic review was registered with the International prospective register of systematic reviews (PROSPERO) on September 19, 2018 with registration number: CRD42018105973.

Critical Care & Emergency Medicine

Point-of-care

ultrasound

medical education

curriculum

meta-Analysis

systematic-review

Kirkpatrick

Kane-framework

Alongside the many advances that herald the 21st century, the dawn of point-of-care ultrasound (POCUS) may be one of the most practice-changing. Point of care ultrasound (POCUS) is a focused ultrasound performed and interpreted at the bedside by a treating healthcare provider. In this manner, POCUS can enable a practitioner to answer a specific question that traditionally may not be adequately assessed by the physical examination. As POCUS has gradually established itself within the clinical examination, experts advocate that it be placed as the 5th pillar of examination following inspection, palpation, percussion and auscultation. (1). The potential to improve patient care is compelling; over the last decade the interest and enthusiasm for the practice of POCUS has garnered broad support (2) In tandem, adoption of POCUS has been further fueled by increasing machine portability, machine access, availability of educational resources, and a burgeoning expert community. Such factors have provided a fertile environment for POCUS to transcend disciplines; consequently, it is establishing roots in almost every specialty (3).

While POCUS can be performed for a spectrum of clinical problems, nowhere is the need greater than in the acute care context(1). POCUS has been shown beneficial in undifferentiated shock (4–6), predicting fluid responsiveness (7), undifferentiated respiratory illness (8), and a valuable asset in procedural guidance (9–11). The potential ramifications, then, of successful application of POCUS, could include improved diagnostic decision-making, reduced time to appropriate intervention, and reduced procedural complications, among others. Such promises could indeed revolutionize how sick patients are cared for. This has been recognized by a number of professional societies such as the Canadian Critical Care Society, the Canadian association of Emergency Physicians, and the Canadian Internal Medicine Ultrasound (CIMUS) group (12–14), who have all individually called for increasing POCUS training during residency.

The potential benefits of POCUS in caring for acutely unwell patients are clear; what is less clear is how we ensure practitioners can competently perform this skill. POCUS is a user-dependent cognitive and technical skill (15). It requires a clear pedagogical approach with rigorous assessment and evaluation. Indeed, quality training is required to ensure POCUS users are performing to a high standard of safety and accuracy. Nonetheless, there is little synthesized evidence to guide educators on best practices in learning strategies, learner assessment, and program evaluation. A systematic review by Kanji et al. (2016) serves as a practical example of this. Kanji and colleagues studied 15 examples of published hemodynamic critical care ultrasound curricula; they found disparate modules of curriculum delivery, assessment, and evaluation. These authors conclude that curricula should focus on simplicity, include a quality web-based tool and a hands-on teaching component (16).. We agree that these are useful strategies, however, they lack granularity—educators cannot use these findings to engineer precise learning strategies, learner assessment, and curriculum evaluation.

Presently, there are two protocols registered with PROSPERO for systematic reviews on acute care ultrasound training (17, 18). The first review by PoSaw et al, limited to emergency medicine has been completed, but remains unpublished (17). The second review by Rajamani et al. specific to critical care medicine, is still ongoing (18). To date, there has been no published reports that systematically investigate acute care POCUS curricula, especially appraising learner assessment, learner assessment validity, and curricular evaluation.

As POCUS continues to grow across acute medicine specialties, we need to re-evaluate how we best train practitioners of the future. It is clear that more robust information and synthesized evidence is desperately needed to help POCUS educators select from the available evidence to strategically and effectively develop educational interventions. From development of learning outcomes and selecting the ideal learning strategy to choosing the most appropriate method of assessment and planning curricular evaluation. Accordingly, we will carry out a systematic review and meta-analysis of current evidence related to POCUS training curricula for the acute care professional.

In this review, we will provide a comprehensive overview of published acute care POCUS curricula and explore key pillars of curriculum design with the following objectives:

Outline current methods of learner assessment and summarize arguments around assessment-validity.

Characterize existing strategies in the evaluation of curriculum evaluation and/or curricular outcomes.

This is a critical first step towards exploring mechanisms of curricular implementation, assessment, assessment-validity, and curricular evaluation. This review will distill evidence to help guide educators who are tasked with developing POCUS curriculum.

Study design:

A systematic review will be performed using guidelines from the Cochrane Handbook on Systematic Reviews for Intervention, and described according to the Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocol (PRISMA-P) guidelines (19, 20). If adequate homogeneity is observed, a Meta Analysis of the data will be performed.

Study registration:

In accordance with PRISMA-P guidelines, our systematic review was registered with the International Prospective Register of Systematic reviews (PROSPERO) on September the 19th 2018 with registration number: CRD42018105973.

Data source and search methods:

The search strategy will be developed and performed by an experienced information specialist and peer-reviewed by a second research librarian (21). The strategy will combine subject headings (e.g., MeSH), title and abstract terms for concepts related to ultrasound, point-of-care testing and training programs. We will search the following electronic databases: Ovid MEDLINE (1946-), Ovid Embase (1974-), Cochrane Library (inception-), CINAHL via EBSCOhost (1937-), Ovid ERIC (1965-) and Science Citation Index via Web of Science (1900-). We will also search the ClinicalTrials.gov register and Conference Proceedings Citation Index (1900-). All reference lists from relevant systematic reviews identified by the search will also be screened. Finally, we will hand search conference proceedings from the past two years from CHEST, Canadian Critical Care Forum, European Society of Intensive Care Medicine, and Society of Critical Care Medicine. No language or date limits will be applied. Search results will be exported in RIS format to Mendeley reference manager (by Mendeley Ltd, the Relx™ group, Elsevier) for primary screening. Duplicates were removed prior to screening. See appendix for the MEDLINE strategy.

Inclusion criteria:

We will include full text publications describing POCUS curriculum implementation, assessment, and evaluation with the following characteristics: retrospective, prospective observational and interventional studies, systematic reviews, and meta-analysis examining POCUS to detect acute medical and surgical conditions including cardiac, pulmonary, abdominal, and venous pathology. For the purposes of generalization, we are seeking to examine common POCUS applications across acute care medical specialties: cardiac, pulmonary, abdominal, and venous pathology and the ultrasound-guided procedures of thoracentesis and paracentesis. While there are myriad applications of POCUS, we believe examining these “core” modalities will yield critical details and be comprehensive, but ensure this review remains technically feasible. For the same reason, we will limit procedural POCUS to thoracentesis and paracentesis.

Healthcare professionals will include undergraduate medical students, postgraduate trainees (residents, registrars) and attending physicians in the acute, inpatient setting: Critical care, emergency medicine, anesthesiology, pediatrics, surgical and internal medicine.

Exclusion criteria:

We will exclude studies examining POCUS curriculum implementation, assessment, and evaluation with the following characteristics: in the outpatient setting, those which do not report the outcomes of participant assessment, if designed to exclusively validate a training simulator, studies involving a training program whereby the emphasis is already on diagnostic imaging modalities like cardiology or radiology, and those exploring non-POCUS modalities.

We will also exclude those studies evaluating ultrasound-guided procedures other than thoracentesis and paracentesis, and where the training population includes physician assistants, nurse practitioners, nurses, paramedics, or other allied healthcare providers. Primary literature published as abstracts only, will be excluded.

Screening and study selection:

Screening and study selection will occur in two phases. The first phase will be conducted by two independent authors, through screening titles and abstracts identified in the primary search. Articles screened for possible inclusion, will be tracked using Mendeley reference manager (by Mendeley Ltd, the Relx™ group, Elsevier). Disagreements will be resolved through discussion and if unresolved, a third party will be involved.

The second phase will involve full text screening. Articles identified as eligible during the initial screening, will be retrieved, and independently assessed by the same two authors. Full text screening will be done using the Covidence online platform (Covidence.org, Melbourne, Australia – a Cochrane technology platform). Disagreement will be resolved through discussion and if unresolved, a third party will be involved. To keep the literature review relevant and up to date, the search strategy and screening could be repeated should enough time lapse prior to the conclusion of the Systematic Review.

Methodological quality assessment and data extraction:

At this stage, we anticipate that a 3rd reviewer will be added to manage the expected volume of included full text articles as we transition to methodological quality assessment and data extraction.

Data extraction tools will be designed, using standardized forms, which will be divided into 2 major extraction categories: Quality assessment and general extraction with assessment validity. Data extraction will be done by 3 independent reviewers after calibration. Calibration will be accomplished by individually extracting data on the first 10% of studies in each category, followed by reviewing the same 10% as a group. Differences will be discussed and if unresolved, a 4th reviewer will be consulted. Once adequate correlation is achieved, data extraction will continue independently.

Quality assessment will be undertaken by applying the Medical Education Research Study Quality Instrument (MERSQI) score, specifically designed for quality assessment of educational studies (22). Studies will be scored using the MERSQI score categories: Study design, sampling, data type, validity of evaluation instrument, data analysis and finally outcomes. Scores range from 5–18, with 18 being the highest possible (22). Each study will be scored independently between 5 and 18.

Further data extraction, with focus on the learner population, number of participants, setting, POCUS application, methods of instruction, duration of intervention, methods of assessment, and program evaluation will be extracted using standardized forms. The evaluation of learning will be categorized according to the four levels of the Kirkpatrick model (22): Reaction, learning, behavioural change, and organization performance. Additionally, assessment validity data, using the key elements in the validity argument of Kane as our framework (23) will be extracted. Kane’s framework is founded on arguments for validity regarding: Scoring, generalization, extrapolation, and implication of assessment tools employed.

If additional educationally significant data elements are identified during our review, they will be added on a post hoc fashion, given the expected heterogeneity of studies. Extraction of quantitative and qualitative data will be done by a single reviewer with verification by a second. Disagreement will be resolved through discussion. In the case of unresolved matters, a third party will be involved.

Data synthesis and analysis:

The primary analysis of the qualitative data will be a narrative synthesis with an emphasis on the learner population, setting, POCUS application, methods of instruction, duration of intervention, methods of assessment and program evaluation. The curriculum evaluation and outcomes will be categorized according to the four levels of the Kirkpatrick model (22): Reaction, learning, behavioural change, and organization performance. Where available, we will report on assessment validity employing the key elements from Kane’s framework for validity arguments on scoring, generalization, extrapolation, and implication of assessment. Graphs and tables will be used as necessary in the qualitative analysis.

Studies will, where possible, be pooled in analysis by using 2x2 tables constructed from MERSQI quality score, population, duration of intervention, methods of instruction, methods of assessment, outcomes, Kirkpatrick level and validity arguments. We will use random effects model to pool effect sizes for exposures amenable to analysis, as it associated with the primary outcomes. Dichotomous outcomes will be reported as pooled odds ratios and 95% CI’s based on the DerSimonian-Laird random-effects model. Continuous outcomes will be standardized using calculated weighed standardized mean differences with the 95% Cis. Using the I² statistics and chi squared test, we will assess the heterogeneity among studies. The thresholds for interpretation of the I² will be in accordance with the definitions presented in the Cochrane Handbook for Systematic Reviews of Interventions. The presence of publication bias will be explored using funnel plots.

Offering the prospect of refined bedside diagnosis and enhanced patient safety, POCUS is increasingly advocated by many sub-specialties as a necessary adjunct to the practice of acute medicine. Despite the overwhelming interest in advancing POCUS education, critical knowledge gaps may be under-appreciated. We know very little about what constitutes “best practices” in designing educational interventions in POCUS including how to instruct, how to assess, and how to evaluate.

Although many sub-specialty societies have published consensus statements, bringing together, and synthesizing the wide array of published educational interventions may bring clarity to a challenging issue. As education is paramount to POCUS competency, so are pedagogical practices; we need to know more about how to engineer learning intervention to maximize their effectiveness. As such, this systematic review aims to distill and summarize the literature regarding POCUS curriculum implementation, assessment validity and evaluation.

Limitations:

Our systematic review faces several possible limitations, especially study sizes, study types, pedagogical approaches, and publication bias. First, we expect many investigations to be relatively small in population size and without control groups—as is common in published educational interventions. This may limit the overall quality of the evidence as populations are small and uncontrolled, leading to less reliable and lower quality results, as was seen by Kanji et al (16) in their systematic review. Second, instructional approaches, assessment techniques, and curricular evaluation are likely to be, at best, diverse—but, at worst, disparate, which may limit our ability to performa rigorous quantitative analysis.. Finally, we may be limited by publication bias as unpublished curricula are likely being taughtThese may limit our analysis to a qualitative synthesis, but our hope is that there will be enough homogeneity that would allow for quantitative analysis.

Our systematic review will summarize the current evidence base for POCUS curriculum implementation, learner assessment, and curricular evaluation for acute care applications. We anticipate that our review will fill a critical gap and provide a sound platform for future evidence-based curriculum development.

Declarations

Data availability statement:

Since this manuscript is a protocol, all data analyzed and used in its creation is available and listed in the bibliography. Once the manuscript for the planned review is prepared, the datasets used and/or analysed, will be available from the corresponding author on reasonable request.

Ethics approval and consent to participate:

Need for approval waived

Declarations

Conflict of Interest:

The authors declare that they have no competing interests.

Funding:

This research will receive no specific grant from any funding agency in the public, commercial or not-for-profit sectors. However, this work is supported by the Alberta Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit Knowledge Translation Platform, which is funded by Alberta Innovates and the Canadian Institutes of Health Research.

Author Contributions:

LB: Manuscript writing and major editing.

Narula J, Chandrashekhar Y, Braunwald E. Time to Add a Fifth Pillar to Bedside Physical Examination: Inspection, Palpation, Percussion, Auscultation, and Insonation. JAMA Cardiol. 2018;3(4):346–50.
Moore CL CJ. Point-of-Care Ultrasonography. N Engl J Med. 2011;(364):749–57.
Lane N, Lahham S, Joseph L, Bahner DP, Fox JC. Ultrasound in medical education: listening to the echoes of the past to shape a vision for the future. Eur J Trauma Emerg Surg. 2015;41(5):461–7.
E Jones A, S Tayal V, Matthew Sullivan D, Kline J. Randomized, controlled trial of immediate versus delayed goal-directed ultrasound to identify the cause of nontraumatic hypotension in emergency department patients. Vol. 32, Critical care medicine. 2004. 1703–1708 p.
Kanji HD, McCallum J, Sirounis D, MacRedmond R, Moss R, Boyd JH. Limited echocardiography-guided therapy in subacute shock is associated with change in management and improved outcomes. J Crit Care [Internet]. 2014;29(5):700–5. Available from: http://dx.doi.org/10.1016/j.jcrc.2014.04.008
Lanctôt JF, Valois M, Beaulieu Y. EGLS: Echo-guided life support. Crit Ultrasound J. 2011;3(3):123–9.
Zhang Z, Xu X, Ye S, Xu L. Ultrasonographic measurement of the respiratory variation in the inferior vena cava diameter is predictive of fluid responsiveness in critically ill patients: Systematic review and meta-analysis. Ultrasound Med Biol. 2014;40(5):845–53.
Lichtenstein D. Lung Ultrasound in acute respiratory failure an introduction to the BLUE-protocol. Minerva Anestesiol. 2009;75(5):313–7.
Karakitsos D, Labropoulos N, De Groot E, Patrianakos AP, Kouraklis G, Poularas J, et al. Real-time ultrasound-guided catheterisation of the internal jugular vein: A prospective comparison with the landmark technique in critical care patients. Crit Care. 2006;10(6):1–8.
Perazzo A, Gatto P, Barlascini C, Ferrari-Bravo M, Resumo AN. A ultrassonografia pode reduzir o risco de pneumotórax após toracocentese?* Can ultrasound guidance reduce the risk of pneumothorax following thoracentesis? J Bras Pneumol. 2013;40(1):6–12.
Mercaldi CJ, Lanes SF. Ultrasound guidance decreases complications and improves the cost of care among patients undergoing thoracentesis and paracentesis. Chest. 2013;143(2):532–8.
MD SA, MD MB, MD SC, MD JC, MD CG, MD NG, et al. Expert Consensus on a Canadian Internal Medicine Ultrasound Curriculum. Can J Gen Intern Med [Internet]. 2014;9(3). Available from: http://csim.ca/csim-journal/index.php/csim/article/view/15/75
Expert Round Table on Ultrasound in ICU. International expert statement on training standards for critical care ultrasonography. Intensive Care Med [Internet]. 2011 Jul;37(7):1077–83. Available from: https://doi.org/10.1007/s00134-011-2246-9
Henneberry RJ, Hanson A, Healey A, Hebert G, Ip U, Mensour M, et al. Use of point of care sonography by emergency physicians. Can J Emerg Med. 2012;14(2):106–12.
Jamniczky HA, Mclaughlin K, Kaminska ME, Raman M, Somayaji R, Wright B, et al. Cognitive load imposed by knobology may adversely affect learners’ perception of utility in using ultrasonography to learn physical examination skills, but not anatomy. Anat Sci Educ. 2015;8(3):197–204.
Kanji HD, McCallum JL, Bhagirath KM, Neitzel AS. Curriculum Development and Evaluation of a Hemodynamic Critical Care Ultrasound: A Systematic Review of the Literature. Crit Care Med. 2016;44(8):e742-50.
PoSaw L, Lee S, Bell G HH. Teaching point of care ultrasound in emergency medicine: a systematic review of current methods and competency assessment. PROSPERO 2018 CRD42018086839.
Rajamani A. Training processes for point of care ultrasound in critical care: a systematic review of the literature . PROSPERO 2018 CRD42018094033.
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ WV (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. [Internet]. Available from: Available from www.training.cochrane.org/handbook.
Moher D. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015statement. Open access [Internet]. 2012;207(January):1–9. Available from: www.systematicreviewsjournal.com/content/4/1/1
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol [Internet]. 2009;62(9):944–52. Available from: http://dx.doi.org/10.1016/j.jclinepi.2008.10.012
Cook DA, Reed DA. Appraising the Quality of Medical Education Research Methods. Acad Med. 2015;90(8):1067–76.
Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: A practical guide to Kane’s framework. Med Educ. 2015;49(6):560–75.

Download PDF

Version 1

posted

You are reading this latest preprint version

Point-of-Care Ultrasound Curriculum in Acute Care Medicine: A Protocol for a Systematic Review and/or Meta-Analysis

Status:

Version 1

Abstract

Introduction

Methods

Study design:

Study registration:

Data source and search methods:

Inclusion criteria:

Exclusion criteria:

Screening and study selection:

Methodological quality assessment and data extraction:

Data synthesis and analysis:

Discussion

Limitations:

Conclusion

Declarations

Declarations

Declarations

Funding:

Author Contributions:

References

Supplementary Files

Status:

Version 1