Aims/objectives
The aim of our systematic review and Evidence and Gap Map (EGM) is: first, systematically identify and describe CDSS-P in primary care, their functional an operational characteristic, the evidence about their effectiveness, their implementation process and the planed evaluation, secondly, to inform developers and decision makers for about design, implementation, evaluation and maintenance of CDSS-P and future research
Study design
This protocol has been developed following the PRISMA-P (25) and PRISMA-ScR (26) guidelines, using the methodology described in the Joanna Briggs Institute Reviewer’s Manual (27) as well as those recently published relating to EGMs (24).
Protocol and registration
This study has been registered in Open Science Framework (https://osf.io/g3mdy/?view_only=54c2497bc6f04ad8b1cf549dc7ac6299).
Patient and public involvement
We did not involve patients or the public in the conduction of this protocol.
Eligibility criteria
Eligible studies should meet the following criteria: English or Spanish language; health care provided by the prescribing professionals – the end-users of the CDSS-P being investigated – should happen in a primary care setting; population will include both home-dwelling patients as well as nursing home patients; the studies could be focused either on the description of the systems, in whole or in part, or their utility, effectiveness or impact; primary and descriptive studies will be included, along with intervention studies that compare the support system to the usual clinical practice.
Studies that do not include a description of the functioning of the system will be excluded from the EGM. Non-scientific reviews and opinion articles will be excluded.
Research questions
Research questions are as follows:
-
Which CDSS-P in primary care have been described in the scientific literature?
-
What are their main characteristics, from an operational viewpoint?
-
What impacts have they shown in improving prescribing and health outcomes? Do they achieve the objectives for which they were designed?
-
What types of studies have described these systems, and how many studies of each type have been found about each system?
-
What gaps in knowledge have been identified regarding these systems?
Information sources
Medline – through the OvidSP platform –, Embase, The Cochrane Library and Web of Science database will be searched from January 2010 to present day:
Search strategy
The search strategies have been developed by one member of the research team, who has extensive experience in structured searches and handling information sources. Annex 1 details the complete search strategy that will be carried out in the Embase database. The complete strategies used in the other databases (Medline, The Cochrane Library and Web of Science) have not been included in this Annex, but can be requested from the corresponding author.
If a systematic review were identified, the studies included in it will be analysed to incorporate them into the search. In order to identify additional studies, a new search will be performed using the reference lists of all selected reports and articles for identification of additional relevant studies. Furthermore, before finalising the data extraction process, a search update, will be performed in order to identify studies that may have been published between the search closing date and the end of the data extraction process.
Reference and full text for all documents identified through the literature search will be imported into Mendeley® reference management tool and Excel® spreadsheet, where they will be compiled in an ad hoc table, classifying them as included, excluded or duplicated.
Selection of sources of evidence
Two reviewers working independently will screen title and abstract to decide for inclusion. Disagreement will be resolved through discussion and consensus. In order to increase the consistency between the reviewers, both of them will examine the same first hundred publications. Full manuscript of potentially relevant citations will be obtained and the criteria re-applied.
The list of articles that will be selected at first but then rejected after a full-text revision will be included in a specific Annex, accompanying the final publication.
Data charting
A data charting form will be jointly developed by two reviewers to determine which variables to extract. A pilot test will be carried out with five studies, and the chosen variables were included in a .csv file. The two reviewers independently will chart the data, discuss the results, and continuously update the data charting form in an iterative process. Data extraction from the selected studies will be carried out by four members of the research team, working in pairs and using a pre-defined empty table. Any discrepancies that may arise will be resolved by each pair through discussion and consensus. In the case that no agreement is reached, a third researcher will be included in the discussion, and, ultimately, a vote will be carried out.
Data items
The following data will be collected from each study: the date and geographic area where the study was carried out (Europe, USA, ROW), the type of health care organization in which the system was developed, the study type and objectives, the size of the included population, the aim of the system being evaluated, the point of comparison it is being evaluated against, the evaluation variables, and the authors’ main results and conclusions. Furthermore, data will be collected to describe the system being evaluated in each study: a general description of its purpose, its level of complexity, whether the intervention carried out in the study was single- or multi-component, if it focuses on medications or pathologies, if the system is independent or is integrated in a specific electronic prescribing system, whether it is considered an intelligent system, and the main data sources it interacts with (prescriptions, diagnoses, laboratory data, functional testing, etc.).
Critical appraisal of individual sources of evidence
Quality appraisal and risk of bias assessment are optional but not mandatory steps in scoping reviews or EGMs, and are not often conducted (26). So, if we finally decide to carry it out, we will describe which methods and tools will be implemented. The rationale for this decision and the reasons for choosing the pertinent assessment tools will be given.
Synthesis of results and visualisation
We do not expect to find data relevant for conducting a meta-analysis. All information will be categorized and a narrative and qualitative evidence synthesis will be conducted. Tables and figures (ie, bubble plot) will be used to display the evidence landscape and to elucidate clusters and gaps. We have developed a conceptual model of system categories through a process of debate and consensus taking into account the characteristics described in the literature cited in the introduction, regarding prescribing support systems in hospitals (5–14) and primary care (15–22).
Tables
According to the conceptual model results will be grouped by: (a) application focus (medications, pathologies, or prescription adequacy to protocols or guidelines); (b) system functionality; (c) level of system complexity (grouped into five categories from lesser to greater complexity). The description of the system will also include level of integration in the electronic prescribing system (in the event that it is integrated, the electronic prescribing system in which it is integrated will also be described), the databases with which it interacts and whether it can be considered an intelligent system.
Every study will be described in terms of: (1) geographic area where it was implemented; (2 ) type of health care organization that developed the support system; (3) objectives, design and population (number of observed patients, distinguishing between the intervention and control groups, when relevant; (4) characteristics of the identified CDSS-P; (6) comparator/control; (7) outcomes; (8) study’s main conclusions; (9) methodological quality and risk of bias, if applicable.
When appropriate, data related with the type of study, number of subjects involved, and outcome results will be included. The tables will also specify whether any cost-effectiveness analysis was identified. Time-course and geographic differences will be analysed.
The results of the comprehensive search will be presented using a PRISMA flow diagram. Finally, a table summarizing study methodological quality and risk of bias of each study will be provided as supplement.
Evidence and Gap Maps
EGMs will be produced including only primary studies that assess intervention effectiveness.
An interactive table will be designed to provide an overview of the existing evidence and to graphically highlight the evidence gaps and the time that it will show a summary of the studies. Colums will display system complexity and study outcomes (health outcomes, use of health resources, potentially inadequate prescribing/medication errors avoided, and acceptance), and rows will display the purpose or context of the decision support system (these will be defined based on search results).
Additional dimensions will be added using different colours, shapes and sizes to plot studies on the map. Each table cell will show studies sharing design and quality features represented as separate symbols. If we finally perform a methodological quality evaluation, a traffic light colour-coding system will be used to display the results about risk of bias of included studies as green, yellow and red corresponding to high, medium and low confidence findings, respectively. The colour transparency effects, symbol directions (up: favours the new intervention; down: favours the standard system) and colour intensity (colour: significant p < 0.05; grey: not significant p > = 0.05) of each plot will represent the magnitude, direction and significance of interventional effects. A series of pop-up brief text messages will be displayed when the user scrolls over each cell. Finally, the map will allow the user to filter the information and display only certain subgroups of studies; for example, filtering by study design, geographic location, result or direction of the effect.
For the presentation of the maps, the information relating to the identified systems and their characteristics will be entered in a dynamic and interactive platform. The data will be organised in columns according to the results in health outcomes identified in the studies and the level of complexity of the support systems, and in rows according to the subcategory of the system’s purpose. The cells in the table will contain different geometric shapes according to the study type, and different colours according to the results variables and quality of the study, and the cell’s size will be proportional to the number of patients included in the total of the included studies. The location in the cell will give information on the direction of the effect. As the maps will be interactive, the user will be able to click on each of the figures and obtain a list of the relevant studies. From this list, they can then click on each of the studies to access a URL of the study in question.
The data model will be designed in such a way that it exhaustively compiles all of the possibilities existing to this day and will allow for the detection of gaps in knowledge for each of them. The identified knowledge gaps will be laid out in specific tables.