Healthcare Providers’ Implementation of Patient-Centred Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs) in Practice: A Protocol for a Mixed Method Systematic Review

doi:10.21203/rs.3.rs-89904/v1

Download PDF

Protocol

Healthcare Providers’ Implementation of Patient-Centred Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs) in Practice: A Protocol for a Mixed Method Systematic Review

https://doi.org/10.21203/rs.3.rs-89904/v1

This work is licensed under a CC BY 4.0 License

You are reading this older preprint version

Read the latest preprint version →

Background: Substantial literature has highlighted the importance of patient-reported outcome and experience measures (PROMs and PREMs, respectively) to collect clinically relevant information to better understand and address what matters to patients. The purpose of this systematic review is to synthesize the evidence about strategies for implementing individual-level PROMs and PREMs data by healthcare providers in their everyday practice.

Methods: This mixed method systematic review protocol describes the design of our synthesis of the peer-reviewed research evidence (i.e., qualitative, quantitative, and mixed methods), systematic reviews, organizational implementation projects, expert opinion, and grey literature. Keyword synonyms for “PROMs,” PREMs,” and “implementation” will be used to search eight databases with limiters of English from 2009 to 2019. Study selection criteria includes implementation at the point-of-care by healthcare providers in any practice setting. Eligible studies will be critically appraised using validated tools (e.g., Joanna Briggs Institute). Guided by the review questions, data extraction and synthesis will occur simultaneously to identify biographical information and methodological characteristics as well as classify study findings related to implementation processes and strategies. As part of the narrative synthesis approach, two frameworks will be utilized: (a) Consolidated Framework for Implementation Research (CFIR) to identify influential factors of PROMs and PREMs Implementation and (b) Expert Recommendations for Implementing Change (ERIC) to illicit strategies. Data management will be undertaken using NVivo 12^TM.

Discussion: Data from PROMs and PREMs are critical to adopt a person-centred approach to healthcare. Findings from this review will guide subsequent phases of a larger project that includes interviews and a consensus-building forum with end users to create guidelines for implementing PROMs and PREMs at the point of care.

Systematic Review Registration: PROSPERO protocol registration number (42013005938).

Health Economics & Outcomes Research

Nursing

Patient reported outcome measures (PROMs)

Patient reported outcomes (PROs)

Routine outcome monitoring (ROM)

Patient reported experience measures (PREMs)

Knowledge translation (KT)

Patient-centred care (PCC)

Clinical decision-making

Clinicians

Healthcare providers

Implementation

Shifting from traditional, biomedical, disease-focused, and scientific models of healthcare toward patients being drivers of their care is essential to improve health and clinical outcomes(1). Person-centred care that focuses on what matters to patients occurs when healthcare providers (HCPs) include and listen to the voice of patients. To do this, data collected from patient-reported outcome and experience measure (PROMs and PREMs, respectively) can be used by HCPs to incorporate patients’ voice and inform their care (2). Our aim is to conduct a mixed methods systematic review to synthesize existing knowledge on effective methods for implementing individual-level PROMs and PREMs data by HCP in their everyday practice. The protocol for this systematic review is part of a larger study to create user guidelines that support clinical decision-making and care planning, wherein HCP interviews and a deliberative dialogue, consensus-building forum with key stakeholders will also contribute to guideline development. The patient-centred measurement (PCM) implementation guidelines for HCPs will support routine collection, interpretation, and integration of these data in everyday practice, ultimately contributing to effective, quality health care.

PCM is an umbrella term used in this systematic review to include tools for measuring patients’ experiences or outcomes (3). Substantial literature has highlighted the importance of PCM measures to collect clinically relevant information from patients to better understand and address what matters to them. PROMs are questionnaires (also referred to as assessment tools or instruments) to measure patients’ health status in a standardized and quantitative manner at a point in time (3). These inquiries may capture how patients function or feel with respect to their health; disease condition and its treatment; or functional status, quality of life, or mental wellbeing (4, 5). In contrast, PREMs focus on how patients feel with respect to their health care or illness experience. PREMs are commonly used to solicit information about patient satisfaction with service delivery in a clinical setting, or describe patient experience of a therapy or plan of care (3). To inform and plan patients’ care, PROMs and PREMS can be collected by HCPs for various purposes: screening, assessment, monitoring, planning, and evaluating care (e.g., interventions, treatment, referrals, and tests) as well as creating decision aids (6, 7). Both PROMs and PREMs are intended to provide assessment data about patients’ health thereby complimenting HCPs’ use of clinician-based outcomes (CBOs), biological measures, and physical examination. When developed with input from patients and regularly integrated into clinical practice, PROMs and PREMs data can encourage conversations between patients and HCPs. As a result, such conversations lead to shared decision-making, improved patient-clinician communication, detection of overlooked problems, and tailored process monitoring thereby ensuring quality individualized care (4, 8–10).

The use of PROM and PREM data in healthcare has been a robust area of research at individual, organizational, and system levels around the world. Considerable emphasis has been placed on the use of aggregated PCM data to inform program evaluation, quality improvement, benchmarking, value-based healthcare, and to some degree managerial decision making (11, 12). Over several decades, developers and users have examined PROM and PREM applications to clinical practice (e.g., (6, 10, 13–16)). Although structures and processes exist to support the use of aggregated PCM data, the routine integration of individual-level PCM data by HCPs in clinical practice is challenging worldwide for various reasons, with calls for additional research to understand best practices for effective implementation strategies with a focus on end users (3, 17–20). Eight systematic reviews (5, 10, 11, 21–25) and one scoping review (26) have explored PCM implementation by HCPs (see Table 1). While this number of systematic reviews would typically offer a robust body of evidence, a closer inspection reveals a paucity of literature on effective knowledge translation strategies for PCM implementation by HCPs at the point of care. Five reviews identified barriers and facilitators of PCM use as one of several aims (5, 10, 23, 25, 26), with only three stating this as the primary topic often in a specific area of practice (e.g., oncology) (11, 22, 24). Notably, Bantug and colleagues (11) differed from other reviewers in their examination of effective methods for the interpretation of patient-reported data using various graphic displays. Our systematic review will synthesize existing evidence about the everyday use of individual-level PCM data by HCPs in all practice settings or healthcare context.

Table 1

Summary of Published Systematic and Scoping Reviews about Using PCM
Author Name in Chronological Order	Objectives	Phenomenon of Interest Pertinent to this Review And Clinical Area	Types of Evidence Included and Synthesis Method	Databases Searched and Search Dates	Search Terms for Patient-Reported Measures	Included # Articles (Search Results)	Inclusion Criteria Pertaining to HCPs	Quality or Bias Assess-ment Conducted
Duncan 2012² (32)	What are the barriers and facilitators to routine outcome measurement by allied health professionals in practice? (p. 2)	Barriers and facilitators to routine outcome measurement Area: Any setting	No restriction on study design Narrative synthesis	MEDLINE, CINAHL, PsycINFO ≤ 2010	13 synonyms	15 (960)	Concerned with identifying or researching factors which acted as facilitators and/or barriers in the routine use of outcome measures by allied health professionals in practice.	Yes
Antunes, 2014¹ (5)	1. Identify barriers and facilitators to the systematic implementation of PROMs in palliative care clinical practice. 2. Identify needs and other comments of clinical teams regarding the routine use of PROMs. 3. Identify lessons learned on the process of implementation of PROMs in clinical practice (p. 160).	Barriers and facilitators to implementation of PROMs Lessons learning on the PROMs implementation process Area: Palliative	All study designs Narrative synthesis	MEDLINE, CINAHL, PsycINFO, Embase, British Nursing Index 1985-Mar 2011	Example for British Nursing Index Outcome measure* PROs PROMs Patient reported outcome? Patient reported outcome measure? Self-reported measure? (Appendix 1)	31 (3863)	Not stated	Yes
Boyce 2014¹ (11)	Investigated the experiences of healthcare professionals with the use of PROMs. .. . and to synthesise findings about the barriers and facilitators to their use (p. 509).	Use of PROMs Barriers and facilitators of PROMs use Area: Any setting	Qualitative Thematic analysis	PubMed, CINAHL, PsycINFO ≤ Aug 2013	12–15 synonyms depending on database	16 (8344)	Healthcare professionals. Professionals’ views of PROMs after receiving PROMs feedback about individual patients or groups of patients.	Yes
Howell 2015¹ (26)	1. Which PROMs does the published, English literature show have been implemented for use in routine cancer clinical practice and in what phases of the trajectory? 2. What are the barriers and enablers influencing clinical uptake of PROMs in routine care? 3. What is impact of the routine use of PROMs on outcomes at the patient, provider, and system levels? (p. 1846-47)	Barriers and enablers influencing clinical update of PROMs Area: Cancer/oncology	Quantitative, qualitative, or systematic literature reviews Scoping review	MEDLINE, CINAHL, PsycINFO 2003–2013	Examples given included self-report questionnaire self-assessment PRO PROMs PROMIS Outcomes	30 (2447)	Evaluated outcomes at the patient, clinical practice, or care process or system-level or barriers/ enablers to the uptake or use of PROMs	Not specified
Bantug 2016¹ (21)	Review existing literature examining the interpretation of PRO data displayed graphically (p. 484)	Interpretation of PRO data displayed graphically Area: Any setting	Quantitative or qualitative Integrative approach	MEDLINE 1999–2014	Patient-reported outcome routine clinical care health-related quality of life health outcomes (p. 485)	9 (242)	Group- or individual-level PRO data for clinicians or patients	Not specified
Foster 2018¹ (24)	To conduct a systematic review of reviews to identify the facilitators and barriers to implementing PROMs in organisations delivering health related services (p. 2–3)	Facilitators and barriers to impacting the implementation of PROMs Area: Any setting	Systematic reviews Framework synthesis	MEDLINE, CINAHL, PsycINFO, Embase, Cochrane Database of Systematic Reviews ≤ 2017	PROM, PROMS, patient reported outcom, PROs, PRO, PREM, PREMs, patient reported experience measure, ePROs, ePRO, or outcome measure*	6 (2047)	Focus on health-related services irrespective of the type of provider.	Yes
Yang 2018¹ (10)	1. Who uses PROs as a tool to improve communication with patients and which topics are discussed? 2. Through what mechanisms do PROs facilitate communication between clinicians and patients? 3. What factors act as barriers or facilitators to PRO use and improved patient-clinician communication? (p. 42)	Barriers or facilitators to PRO use Area: Cancer/oncology	Not specified Realist synthesis	MEDLINE, CINAHL, PsycINFO, Embase, Cochrane Database of Systematic Reviews, Cab Direct ≤ 2016	Example MEDLINE PROM PROMs PROMIS patient reported outcome* patient* outcome* measure* self report adj1 (questionnaire or measure) Exp Health Status Indicators/ Questionnaires/ Psychometrics/ Self Report/ Health Surveys/	43 (610)	PROs were used as an intervention, not as a study outcome measurement tool	Yes
van Egdom 2019¹ (23)	1. How have PROMs been administered in breast cancer care? 2. What is the impact of PROM administration on patients, care providers, and healthcare services or processes? 3. What are the facilitators and barriers that influence the integration of PROM collection in routine breast cancer clinical practice? (p. 1198)	Facilitators and barriers that influence the integration of PROM Collection Area: Cancer/oncology	Excluded case report, editorial, review or study protocols Not specified	MEDLINE, CINAHL, PsycINFO, Embase, Cochrane Central, Web of Science ≤ 2017	patient-reported outcome'/de OR 'patient reported outcome measure'/de OR (('self report'/de) AND ('outcome assessment'/de OR 'quality of life'/exp OR 'quality of life assessment'/exp OR 'complication'/de OR 'symptom'/de OR 'wellbeing'/de OR 'psychological well-being'/de OR 'health status'/de )) OR PROM OR PROMs):ab,ti)	34 (2311)	No HCP criteria specified	Yes
Gelkopf 2020^1,2	1. To ascertain the goals of the PROM/ROM projects. 2. To identify the setting, diagnoses and services provided in PROM/ROM projects. 3. To identify the instruments used to measure outcome, the domains measured and the process that led to the choice of instruments. 4. To gather information on the frequency of evaluation and the methods of data collection. 5. To gain information about the feasibility of ROM through the follow-up data collection rate; the training needs and type of feedback provided. 6. To gather information on the barriers and limitations encountered to implement ROM and ways to overcome those barriers (p. 2).	Barriers and limitations encountered to implement ROM Area: Adult mental health	Peer-reviewed publications including specific initiatives or implementation of PROM data Unnamed, categories with consensus and triangulation	PsycINFO PubMed (including MEDLINE, Biomed Central, Embase Psychiatry, Elsevier’s Science Direct) (cited by Roe, Mazor & Gelkopf, 2019) 2000–2018	14 synonyms (cited by Roe, Mazor & Gelkopf, 2019)	103 (19589)	Exclusion of articles on the views, knowledge, attitudes and perceptions of consumers, providers, or other stakeholders regarding the use of PROMs/ROMs.	Not specified
¹PROMs or PROs
²Routine outcome monitoring

The lack of apparent evidence regarding implementation methods and knowledge translation strategies that are most effective across practice areas justifies the need for our proposed systematic review. According to evidence-based practice guidelines for systematic reviews (27, 28) and electronic searching strategies (29), limitations of a systematic review can be traced, for example, to the review’s objectives/questions, search strategy, or selection criteria (i.e., inclusion and exclusion). First, getting the question right is of utmost importance in setting up and directing the entire review (27, 30). It was identified that past reviews did not commonly employ the population, issue/interest, context/comparison, and outcome (PICO) format and often had several objectives in addition to PCM implementation by HCPs. Inter-related yet divergent objectives may have influenced the search results for relevant citations putting into question the overall validity and thoroughness of the synthesis results (30).

Search terms in combination with relevant databases are fundamental to conducting a literature search at the appropriate level of specificity (i.e., precision and recall). This often elusive balance, when struck, provides the appropriate data to answer the review question (30). Search terms in previous reviews (see Table 1) have been narrow in scope thereby eliciting a low yield of relevant citations. For example, one author included less than five keywords for patient-reported measures (21) while others missed numerous synonyms and appropriate related terms to capture the concept of patient-reported measures (5, 10, 24, 25) or excluded publishing the search terms making the search irreproducible (31). Notably, one review included PREM search terms (24). Furthermore, search terms did not appear to capture the process of implementation in its entirety (10, 11, 23–25) and were limited, in some instances to two words: barriers and facilitators (5, 32). Lastly, some reviews (e.g., (24, 25, 31) had limited reporting of evidence-based electronic search strategies for systematic reviews such as truncation, alternate spelling, and proximity locators (29). When systematic review search strategies do not include search terms at an appropriate level of specificity, the retrieval of relevant citations can be threatened due to low yield. Likewise, robust systematic reviews seeking to balance recall and precision in the retrieval process must also attend carefully to database selection (30).

Selecting the right bibliographic databases to recall a maximum number of relevant sources is another critical aspect of the search process (30). In previous syntheses, the number of relevant citations identified and subsequently screened as eligible have been quite low, which may be associated with the databases used (see Table 1). Four reviews searched three or fewer databases with the most common being MEDLINE/PubMed, CINAHL, and PsycINFO (11, 21, 26, 32). Given the nature of the previous systematic review questions, a broader scope of databases aligned with the population (i.e., all disciplines of HCPs) and the phenomena of interest is warranted to result in a higher yield of relevant citations from which to draw conclusions (30).

PICO elements of review question(s) are utilized to formulate selection criteria (i.e., inclusion and exclusion) to screen the resulting citations for relevance and determination of eligibility (30). Explicit criteria based on the review question(s) determines which citations are included in the reviews and mitigates systematic selection bias; both which are necessary for accurate conclusions to be made (33). Regarding the population element to explore PCM implementation in everyday clinical practice, some limitations were noted in previous reviews (see Table 1). Only three studies specified selection criteria about HCPs (e.g., allied health professionals) (11, 21, 32). Another review included a broad range of stakeholders extending beyond clinicians such as commissioners (e.g., local and national policy makers) and health service managers (24). In terms of context, in half of the published reviews (see Table 1) the selection criteria were limited to only specific practice settings (e.g., palliative care (5); cancer care (10, 23, 26), and adult mental health (25).

Equally important to selection criteria are careful considerations of the types of evidence to include in the review so as to adequately represent the implementation experiences of providers (34). The purpose of mixed methods reviews is to conduct a comprehensive synthesis that includes the different types of evidence necessary to inform clinical practice, policy, or organizational decisions (35). The published systematic reviews (see Table 1) have included a mixture of qualitative and quantitative research-designed studies. However, only Gelkopf et al. (25) stipulated the inclusion of initiatives or projects (e.g., quality improvement and knowledge translation) exploring “real-world” PROM implementation. In recent years, there have been increasing numbers of published quality improvement, organization implementation, and knowledge translation projects exploring PROMs implementation in clinical practice. As such, there is a need to include peer-reviewed evidence about implementation projects and expert opinion because these contribute to a breadth and depth of understanding by capturing everyday experiences of end-users and that might confirm or dispute evidence (35). To date, no single review has captured the voice of end-users, the HCPs, vis-à-vis their experiences of implementing individual-level PCM data at the point of care.

This protocol builds on the work of the aforementioned reviews. Given the cumulative evidence concerning the barriers and facilitators faced by HCPs implementing PCMs in everyday practice, our project will be informed by knowledge translation and implementation science. To this end, our aim is to conduct a comprehensive mixed methods systematic review of a wide range of peer-reviewed literature, including all healthcare settings. The Population, Phenomenon of Interest, Context (PIC) format (see Table 2) review question guiding the protocol is “How do healthcare providers implement individual-level patient-centred measurement/assessment tools (and the resultant data) as a routine part of their everyday practice?” This includes the following secondary review questions:

What are HCPs’ experiences in applying these tools in clinical practice?
How do HCPs interpret and integrate these tools to inform patient care?
What are the factors (barriers and facilitators) that influence PCM implementation by HCPs at the point of care?

Table 2

Definitions of the review question elements using the PIC format
PIC Element	Definition
In this review, patients complete a PROM or PREM in a given practice setting/health service area and then HCPs act on the resultant data in the provision of patient care.
Population (P)	Healthcare provider(s) Refers to individuals from any health discipline or profession that provide direct health services to patients, clients, and/or families. HCPs are regulated or licenced healthcare professional; however, this may vary by country. HCPs may be referred to as clinicians. Common HCPs include but are not limited to registered nurses, nurse practitioners, physiotherapists, occupational therapists, physicians, social workers, dieticians, psychologists, pharmacists, and midwifes.
Phenomena of interest (I)	Implement individual-level, patient-centred measurement/assessment tools (and the resultant data) In a broad sense, implementation is the process or act of making something active or effective. In the context of PROMs other synonyms include: employ, apply or application, utilize, use, integrate, interpret, draw on, make use of, and act on. The act of implementing PCMs in this review also captures the experiences, views, attitudes habits, practices, and routines of HCPs. Patient-centred measurement as an umbrella term refers to standardized assessment tools or questionnaires classified as PROMs or PREMs that every patient is eligible to complete. Thus, the questions are completed by the patient and are about outcomes that matter to them. The results are individual level numerical or textual data that indicate the patient’s current state or experience.
Context (C)	Routine part of their everyday practice Is an expression to describe an action that has been taken as a regular or common aspect of the HCP’s role. It is usually in relation to a HCP’s direct interaction between them and the patient/client/family. For example, in acute care or hospital settings this term used maybe point-of-care or bedside where patient and providers interact on a regular basis, such as daily, weekly or monthly. The context includes any practice setting or type of health service (e.g., acute care versus community care; private versus publicly funded, and community hospitals versus academic centers/teaching hospitals).

The systematic review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for Mixed Methods Systematic Review (MMSR) (35) and the Preferred Reporting Items for Systematic Review and Meta-Analyses Protocols (PRISMA-P) (36) [see Additional file 1]. To find relevant research evidence, the population, phenomenon of interest, and context (PIC) format has been applied to formulate the review questions, devise a search strategy, and guide study selection criteria. The project will utilize an iterative process undertaken primarily by AW with input from members of the research staff and larger research team based on areas of expertise. A written record will be organized according to the Matrix Method (37).

Search strategy

The proposed search will be conducted in accordance with the checklist for Peer Review of Electronic Search Strategies (PRESS) Guideline for systematic reviews (29) to achieve a balance of recall and precision. With academic librarian support (DD), the search strategy, including all identified keywords will be adapted for each database. An initial limited search of two databases (MEDLINE and CINAHL) was undertaken to identify articles on the topic. Since controlled vocabulary is unique to each database, keywords (rather than subject headings) have been identified as the most reliable approach for suitable recall. We determined that keyword searching, with the generation of all synonyms, plurals, and alternate spellings (e.g., centred and centered), for the phenomenon of interest (i.e., PROM, PREM, and implementation) [see Additional file 2] produced high yield. Titles, abstracts, and keywords of relevant articles were used to assist in the identification of synonyms for each keyword. Although it is common to include a third keyword to represent the population or context elements of the PIC question, we found that it limited the precision of our results. As such, we decided to exclude a third concept for searching and instead include them as part of our selection criteria. Findings from our preliminary search informed the search for the project.

The evidence to answer our question will be retrieved by searching for the published literature between January 2009 and December 2019 using eight databases with the EBSCO platform that covers the subjects of nursing, allied health, health sciences, psychological literature, physical therapy, occupational health, nutrition, kinesiology, and evidence-based reviews (i.e., MEDLINE, CINAHL, PsycINFO, Web of Science, Embase, SPORTDiscus, Evidence-based Medicine Reviews, and ProQuest (Dissertation and Theses)). The reason for the 10-year date range was two-fold: first, to keep the yield of records manageable and second, to synthesize evidence based on the current state of the field. Limiters being used include: (a) scholarly/peer reviewed citations, (b) English language, and (c) date range. The two keywords will be utilized in conjunction with appropriate truncation (e.g., asterisks) to include multiple variants using the same truncated root or stem. In some circumstances the boolean “NEAR” or proximity locators will be used to link terms that may not be adjacent (e.g., barrier* n4 implement* and facilitat* n4 implement*) (30). All identified search terms will be linked using Boolean operators. The boolean “OR” operator will be used to link search terms as a union for each concept for the purpose of expanding and broadening a search. The interaction of these concept searches will then be combined with “AND” to narrow the search (30). Search histories [see example Additional file 3] from all databases will be retained.

Our exploration will be supplemented by using other searching strategies to carry out a comprehensive search and counterbalance the limits of keyword database searching (30). This includes: footnote chasing (i.e., scanning the references of “keeper” articles), author searching of those publishing extensively in their field, and backward/forward citation searching of related systematic reviews and other seminal articles (e.g., large studies with numerous publications or those frequently cited) (30). Additionally, a search of authors most frequently publishing in the field will be conducted. The ProQuest database will be used to search for eligible dissertations and theses. Upon completion of our database search, we will search the unpublished literature to lessen publication bias and to retrieve difficult to find literature and information regarding implementation projects. To avoid bias, we will use two approaches. First, a judicious examination of the “grey” literature (e.g., research reports, practice guidelines, and user guides) will be conducted using the Google© search engine in 2020 from a university internet protocol address with the most cited search terms of the PIC concepts identified during our database searches. Second, we will access websites of credible organizations, agencies, and associations that may produce and publish knowledge translation documents supporting PCM implementation (30). These websites will be identified by seeking the opinions of experts in the field. A PRISMA diagram (28) will be created to reflect the search strategy and study selection to reveal how the included studies were determined.

Types of Evidence

The MMSR methodology combines an assortment of evidence to create a breadth and depth of understanding of the review questions posed to inform practice and policy. The inclusion of all available evidence, regardless of type, allows for the degree of agreement or discrepancies between sources of evidence as well as validating or triangulating the findings. Various aspects of a phenomenon of interest can be examined and the available data can contextualize the findings (35). Furthermore, given that implementation at the point-of-care requires a variety of knowledge to inform practice, diverse evidence types will be sought. This review will consider peer-reviewed literature: quantitative, qualitative, and mixed methods studies in addition to reviews (i.e., systematic and literature), organizational implementation projects (e.g., quality improvement, knowledge translation project, implementation of PROMs, program evaluation, or pilot/feasibility project), and expert opinion (e.g., an individual, group or learned body that draws on their practical experience and understanding of the knowledge). We will include not only the evidence on the effectiveness of strategies for implementing PCM (‘knowing what’ type of evidence), but also evidence related to subjective experiences, attitude, behaviours and/or the accepted discourse at the time of practice (‘knowing how’ type of evidence) (34). Opinion-based evidence will be included when derived from expert opinions. That is, the opinions from experts in the field that were gained through some form of consensus building process (e.g., conference, think tank, special interest group, panel, and current discourse) (34). Inclusion of the unpublished grey literature is unique to a systematic review of this nature.

Selection Criteria

The next step to finding relevant evidence for inclusion in this review is to define the selection criteria. To be included in the review, the literature needs to meet the following eligibility criteria (see Table 3): (a) HCPs in a clinical setting; (b) information pertaining to PROM or PREM implementation (e.g., HCP experiences, strategies for integrating into practice, influential factors, or attitudes towards use); (c) data were at the individual level; and (d) any study design. Articles will be excluded if (see Table 3): (a) articles focused exclusively on decision-makers (e.g., managers) or patients; (b) information pertaining to when and why PROMs and PREMs are used as well as impact of their use; (c) studies about instrument development, testing and selection; or (d) implementation of aggregated data. Furthermore, to determine inclusion of studies in the review we will apply the criteria in a specific order (Gough). After each citation is confirmed as written in English and within the date limit, we will ensure it meets the study design criterion. Next, the phenomenon of interest criterion will be applied followed by screening for population and context.

Table 3

Selection Criteria
Topic	Inclusion	Exclusion
Population (P)	• Healthcare providers	• Decision-makers exclusively • Patients exclusively
Phenomena of interest (I)	Studies about PREMs or PROMs and • experiences of applying or implementing • methods or strategies for integrating and interpreting (e.g., processes, logistics, tools, or workflow) • factors (barriers and facilitator) influencing implementation • views or attitudes towards their use	Studies about PREMs or PROMs and • impact or effectiveness • mechanisms by which they work (e.g., patient-provider communication) • ways used (e.g., screening, assessment, improve communication) • measurement development, testing, and selection • suitability for specific patient populations • a focus solely on patient-centred care
Context (C)	Studies concerning data at the individual (micro) level with patients: • routine clinical care • point-of care • everyday clinical practice • directly inform patient care or care planning • clinical decision-making • real-world application	Studies concerning aggregated data for purposes such as: • performance indicators or accreditation • value-based medicine • quality improvement or quality control • resource allocation, service provision, and economic evaluation • clinical registries • reimbursement and payer issues • benchmarking • drug development
Study Design	Published scholarly work including research, pilot or feasibility projects, evidence-based implementation/quality improvement, systematic reviews, literature reviews, and expert opinion	Published literature such as editorials, opinion or position papers, commentary, study protocols, conference proceedings or abstracts, and theory. Insufficient information reported on study design

Study Selection Procedure

Following the completion of these searches, all identified citations will be loaded into EndNote X9© (Version 9.3.3) (38) and duplicates removed. AW will provide a thorough orientation to those involved in the selection process to ensure rigor. Given the large quantity of records anticipated to be retrieved (e.g., greater than 20000) the first 100 record titles and abstracts will be screened by two independent reviewers for assessment against the inclusion criteria to be identified as relevant, not relevant, or maybe relevant. Following that, AD will screen all other records to determine relevancy. AW to confirm eligibility will rescreened those identified as potentially relevant. To ensure validity of the selection criteria, in the EndNote© library for this project, AW will conduct keyword searches (e.g., outcome measure, patient outcome, patient-reported) of record titles of those deemed irrelevant to reapply the selection criteria. Finally, all relevant studies will be retrieved in full text and their citation details independently reviewed (AW and AD) against the selection criteria to confirm inclusion. Reasons for further exclusion of all studies will be recorded. Any disagreements that arise between the reviewers at each stage of the study selection process will be resolved through discussion (33, 37).

Assessment of Methodological Quality

Critical appraisal of included studies will determine the level of evidence and methodological quality as a basis for our confidence to act on the recommendations from our synthesis. Two independent reviewers, blinded to each other’s assessments, will retrieve all included citations, and applicable supplemental files, in full text format for assessment. AW will provide a robust orientation to primary reviewers (AD, DG, SH, FH, LE, SL, LM, and two undergraduate research assistants) of the process and appraisal checklists to ensure rigor. Authors will be contacted to request missing or additional data for clarification, where required. Any disagreements that arise between the reviewers will be resolved through consensus discussions among select team members, or a blinded third reviewer.

We will use the following standardized JBI critical appraisal instruments for assessing quality (see Table 4): systematic review, qualitative, cross-sectional, prevalence, case report, and text and opinion (27, 39). To evaluate organizational implementation projects, three questions from the JBI case report checklist (40) were combined with questions from the Johns Hopkins’ organizational experience checklist for non-research evidence (41) and questions for quality improvement interventions (42, 43). Similarly, the JBI checklists for analytical cross-sectional (40) and prevalence survey (44, 45) studies were modified to include four additional questions about the research questions, research methods, ethical approval, and justified conclusions (46–48). JBI checklists do not exist for mixed methods studies or literature reviews. AW conducted an extensive review of the literature to locate other standardized tools of high reliability and validity. Based on a parsimonious set of core criteria, the mixed method checklist focuses on both the effective integration of the quantitative and qualitative components of studies, as well as the provision of a rationale for using a mixed methods design (49–51). The checklist used for literature reviews will be based on the Johns Hopkins’ form for non-research evidence (41).

Table 4

Summary of the Critical Appraisal Checklists by Research Design and Level of Evidence
Type of Evidence	Critical Appraisal	Level of Evidence
Systematic review	JBI Systematic Review Appraisal Tool (52)	1
Qualitative	JBI Qualitative Appraisal Tool (53)	3
Analytical Cross-sectional	JBI Analytical Cross Sectional Appraisal Tool with others (above)	3
Survey	JBI Prevalence Appraisal Tool in combination with others (above)	3
Mixed method	Mixed Method Appraisal Tool (51)	3
Organizational implementation	JBI Case Report with others (above)	4
Expert opinion	JBI Text and Opinion Appraisal Tool (39)	5
Literature Review	JH Non-research Evidence (Literature Review) Appraisal Tool (41)	5

All checklists contain a series of criteria (range 8 to 15 questions) scored as being “met” or “not met” or “unclear” and, in some instances, as “not applicable.” Following critical appraisal, all studies will be given a percentage score with higher scores indicating a greater percentage of the quality criteria were met. The research team decided not to set a quality threshold to exclude evidence. Rather, once the data are synthesized we will determine the confidence to act based on the quality and level of the evidence. A modified version of the JBI levels of evidence (54, 55) for meaningfulness will be used as it best aligns with our review questions and the nature of the evidence. The five levels are:

Quantitative or mixed-methods systematic review;
Qualitative or mixed-methods synthesis and single experimental-based quantitative study;
Single qualitative and descriptive or observational quantitative study;
Systematic review of expert opinion and organizational implementation project single study (e.g., evidence-based practice, quality improvement, and knowledge translation); and
Expert opinion and literature review.

Table 4 cross-references the types of evidence with levels of evidence. Each citation will be assigned a level during the extraction process to be subsequently used during data synthesis. Given the evidence in this review is explorative, descriptive, and interpretative in nature, the JBI Grades of Recommendation will be the criteria used to define the overall strength of the recommendation (i.e., strong or weak) (55, 56).

Data Extraction

The data extraction step provides the means by which the most pertinent information about the topic (i.e., study characteristics and findings) can be summarized and culled from the primary studies. All source documents will be loaded into the data management software NVivo™ (Version 12.6) (57). Using this software, a Review Matrix will be generated to maximize efficiency and create “order out of chaos” (37, p. 150). Column topics for the matrix will be defined according to the purpose of the proposed systematic review to capture pertinent bibliographic information, methodological characteristics, and content-specific characteristics (e.g., implementation theory) of each included citation (see Table 5) (37). Column topics for which there is a discrete response option (e.g., methodology) will be extracted using the NVivo™ file classification function. The NVivo™ codes function will be used to identify the column topic response for items that have more than one response option (e.g., studies conducted in multiple settings or involving multiple HCPs). These data will offer contextual and methodological data to support the data synthesis results (37). Select team members will be involved in assembling the extracted data from all included articles with relevant accompanying illustrations (e.g., participant quotes or statistical test values). Notes on the definition of column topics and response options as well as the overall extraction process will be kept to ensure consistency amongst extractors.

Table 5

Bibliographic Information and Study Attributes to be Abstracted
Bibliographic Information	Study Attributes
• Authors • Year of publication • Article title • Keywords • Digital object identifiers	• Country/ies of study • Methodology • Research design • Implementation theory • Context or Setting • Practice area • Sample population/profession of healthcare providers • Sample size • Sampling method • Level of evidence • PROM and PREM instruments used

The next step will be the extraction of the pertinent study findings, specifically from the results and discussion sections of each citation. Using NVivo™ (57), the process of synthesis begins as the study findings will be extracted into specific codes. All study findings from the included citations will be coded for analysis as textual descriptions. Qualitative data will be composed of themes or subthemes with corresponding illustrations (e.g., quotations, tables, and figures). The quantitative data (e.g., descriptive or inferential statistics) will be converted into “qualitized data.” This process will involve the transformation of all quantitative data into textual descriptions or narrative interpretation in a way that answers the review questions. When necessary, corresponding statistical test results can be captured as part of the coding process. As per the narrative synthesis approach (58), code names will be based on a theoretical framework. In our study, we will use the Consolidated Framework for Implementation Research (CFIR) (59). The CFIR is an evidence-based guideline used to assess multiple contexts and identify factors that might influence the process and effectiveness of the implementation of a specific intervention, which, in our review, is PCM. The five major domains are intervention characteristics (8 items), inner setting (5 items), outer setting (4 items), characteristics of individuals involved (5 items), and implementation process (4 items) (59). A further framework will be used to code the identified implementation processes or actions to support a practice change. For this, we will use the validated Expert Recommendations for Implementing Change (ERIC), which is a compilation of 73 discrete strategies in nine clusters (60, 61). Codes not represented in either framework will be created, as determined, by AW to answer the review questions. In this manner, extraction and initial synthesis occur simultaneously. To reduce coding error during data extraction, we will develop a coding protocol, provide coder training, leverage our substantive expertise amongst team members, and use the NVivo coding comparison feature to improve reliability (30). In summary, the overall extraction process of transforming and coding these data will facilitate each element of the narrative synthesis to integrate the existing evidence and answer the review questions (35).

Data Synthesis

The synthesis will follow a convergent integrated approach as per the JBI methodology for MMSR. In this manner, data from all types of evidence will be simultaneously extracted and synthesized into meaningful codes. Furthermore, this integrated approach means that the transformed “qualitized” data will be combined to identify patterns across all the studies as well as explore relationships of the data between and within the studies (35). The integration of these data will be guided by a narrative synthesis approach (58), which is well suited for MMSR that utilize diverse types of evidence and has sample heterogeneity (35). Moreover, this approach allows for the use of theoretical frameworks to shape the analysis. In our case, the analysis will use two implementation science frameworks allowing us to focus broadly on the implementation process as well as effective strategies to implement and sustain changes in HCP’s behaviour (58). Popay et al. identifies four iterative elements to a narrative synthesis.

Element 1: The role of theory in evidence synthesis (p. 12). Contributing to knowledge translation theory on how PCM implementation works, why, and for whom we will use the CFIR (59) and ERIC strategies (60, 61) that are based on theories of change. With the use of NVivo™ for extraction, the process of synthesis begins as the theory contributes to the interpretation of study findings and determines how widely applicable the findings may be. In this way theory building and theory testing can be incorporated as a key aspect of the proposed systematic review (58).
Element 2: Developing a preliminary synthesis (p. 13). A preliminary synthesis is conducted to understand the codes identified and summarize the results of included studies. This will be achieved by defining patterns of findings simultaneously across all the studies. An initial description of the findings will evolve based on similarity in meaning to produce an integrated synthesis (58). Furthermore, using NVivo™ to identify codes within each citation will subsequently lead to overarching categories and themes.
Element 3: Exploring relationships of the data between and within the studies (p. 14). The purpose of the third element is to identify reasons that might explain any differences in the findings regarding the successful implementation of PCMs. The emerging patterns identified in the pooled data will be further analysed to identify factors, study characteristics, and context explaining differences. Comparing and contrasting relationships across studies is important to this stage of the synthesis as a means to explore the influence of heterogeneity (58). The NVivo™ relationship and query features will aid in our exploration of associations.
Element 4: Assessing the robustness of the synthesis (p. 15). This element allows for the integration of the quality assessments to determine the strength of the evidence and support with the trustworthiness of the synthesis products (e.g., answers to the study questions and recommendations). Using NVivo™ the included studies will be assigned both a level of evidence, and a quality score that will be cross-linked to the products of the synthesis. From this, a final determination of the strength of the evidence to support conclusions draw from the synthesis process can be made (58).

Thus, the results of this narrative synthesis approach will provide a critical analysis to determine effective methods for PCM implementation by HCPs in everyday practice.

The protocol for this systematic review meets an internationally clinically driven need to conduct a comprehensive synthesis of various types of evidence on (a) effective methods for HCP interpretation and integration of individual-level PCM data, (b) experiences of HCPs applying these tools, and (c) identification of relevant factors influencing PCM implementation. This systematic review is one part of a larger three-phased project that will be followed by HCP interviews and a consensus-building forum to elicit input from key knowledge users. The outcome of the overall project is the development of evidence-informed guidelines that outline effective implementation strategies for integrating PCM data into HCPs’ everyday practice for shared decision-making and care planning with patients. These guidelines further build on other user guides for the implementation of PROMs in clinical practice (7, 62, 63). Overall, this project is intended to promote “shared ownership” of individual-level PCM data to better understand and address what matters to patients.

CBOs Clinician-based outcomes

CFIR Consolidated Framework for Implementation Research

CINAHL Cumulative Index to Nursing and Allied Health Literature

EBSCO Elton B. Stephens Company

EMBASE Excerpta Medica Database

ERIC Expert Recommendations for Implementing Change

HCP Healthcare provider

JBI Joanna Briggs Institute

MEDLINE Medical Literature Analysis and Retrieval System Online

PCM Patient-centred Measures

PIC Population, phenomenon of interest, and context

PICO Population, issue/interest, context/comparison, and outcome

PREMs Patient reported experience measures

PRESS Peer Review of Electronic Search Strategies

PRISMA Preferred Reporting Items for Systematic Review and Meta-Analyses

PRISMA-P Preferred Reporting Items for Systematic Review and Meta-Analyses Protocols

PROs Patient reported outcomes

PROMs Patient reported outcome measures

PsycINFO Psychological Index

ROM Routine outcome monitoring

SPORTDiscus None available

Ethical approval and consent to participate

Certificate of approval received from the Trinity Western University Human Research Ethics Board (File No. 19F08)

Consent for publication

Not applicable

Availability of data and material

Not applicable

Competing interests

The authors declare no competing interests.

Funding

British Columbia Support for People and Patient-Oriented Research and Trials (BC Support) Unit, conditions of award administered by the British Columbia Academic Health Science Network (Award ID PCM-004). The funding body is not involved in the design, conduct, and publication of this project.

Author’s contribution

AW conceived, designed and leads all aspects of the three-phase study, and coordinated the study team. AD assisted with study coordination and participating in protocol steps. SH, DG, LE, and SL assessed methodological quality of included articles. All authors all assisted with the protocol design. All authors read and approved the final manuscript. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work.

Acknowledgements

Undergraduate research assistants Rachel Jerome and Alexa Huebner. Thanks to manuscript feedback provider by Dr. Richard Sawatzky.

World Health Organization (WHO). Framework on integrated people-centred health services: 69th World Health Assembly; 2016. Available from: https://apps.who.int/gb/ebwha/pdf_files/WHA69/A69_39-en.pdf?ua=1&ua=1.
World Health Organization (WHO). Framework on integrated people-centred health services: An overview.: World Health Organization; n.d. Available from: https://www.who.int/servicedeliverysafety/areas/people-centred-care/Overview_IPCHS_final.pdf?ua=1.
Canadian Institute for Health Information (CIHI). Patient-centred measurement and reporting in Canada: Launching the discussion toward a future state. Ottawa, ON: CIHI; 2017.
Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Affairs. 2016;35(4):575-82.
Antunes B, Harding R, Higginson IJ. Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliative Medicine. 2014;28(2):158-75.
Sawatzky R, Laforest E, Schick-Makaroff K, Stajduhar K, Reimer-Kirkham S, Krawczyk M, et al. Design and introduction of a quality of life assessment and practice support system: Perspectives from palliative care settings. Journal of Patient Reported Outcomes. 2017;2:36.
International Society for Quality of Life Research, prepared by Aaronson N, Elliott T, Greenhalgh J, Halyard M, Hess R, et al. User’s guide to implementing patient-reported outcomes assessment in clinical practice2015. 47 p.
Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: Lack of impact or lack of theory? Social Science & Medicine. 2005;60(4):833-43.
Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: A literature review. Journal of Evaluation in Clinical Practice. 1999;5(4):401-16.
Yang LY, Manhas DS, Howard AF, Olson RA. Patient-reported outcome use in oncology: A systematic review of the impact on patient-clinician communication. Supportive Care in Cancer. 2018;26(1):41-60.
Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research. BMJ Quality & Safety. 2014;23(6):508-18.
Gleeson H, Calderon A, Swami V, Deighton J, Wolpert M, Edbrooke-Childs J. Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open. 2016;6(8):e011907-e.
Lohr KN, Zebrack BJ. Using patient-reported outcomes in clinical practice: Challenges and opportunities. Quality of Life Research. 2009;18(1):99-107.
Dickinson F, McCauley M, Smith H, van den Broek N. Patient reported outcome measures for use in pregnancy and childbirth: A systematic review. BMC Pregnancy Childbirth. 2019;19(1):155-63.
Haverman L, van Oers HA, Limperg PF, Hijmans CT, Schepers SA, Sint Nicolaas SM, et al. Implementation of electronic patient reported outcomes in pediatric daily clinical practice: The KLIK experience. Clinical Practice in Pediatric Psychology. 2014;2(1):50-67.
Absolom K, Gibson A, Velikova G. Engaging patients and clinicians in online reporting of adverse effects during chemotherapy for cancer: The eRAPID System (Electronic Patient Self-Reporting of Adverse Events: Patient Information and aDvice). Medical Care. 2019;57 Suppl 5 Suppl 1:S59-S65.
Clinical Oncology Society of Australia. Implementing monitoring of patient-reported outcomes into cancer care in Australia: A COSA think tank. 2018.
Anatchkova M, Donelson SM, Skalicky AM, McHorney CA, Jagun D, Whiteley J. Exploring the implementation of patient-reported outcome measures in cancer care: Need for more real-world evidence results in the peer reviewed literature. Journal of Patient Reported Outcomes. 2018;2(1):64.
Greenhalgh J. The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research. 2009;18(1):115-23.
Javier M, Kim JY, Toone E, Granger BB. Overcoming barriers to using patient-reported outcomes for clinical inquiry. AACN Advanced Critica Care. 2016;27(2):230-5.
Bantug ET, Coles T, Smith KC, Snyder CF, Rouette J, Brundage MD. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Education & Counseling. 2016;99(4):483-90.
Duncan PW, Abbott RM, Rushing S, Johnson AM, Condon CN, Lycan SL, et al. COMPASS-CP: An Electronic Application to Capture Patient-Reported Outcomes to Develop Actionable Stroke and Transient Ischemic Attack Care Plans. Circulation: Cardiovascular Quality & Outcomes. 2018;11(8):1-11.
van Egdom LSE, Oemrawsingh A, Verweij LM, Lingsma HF, Koppert LB, Verhoef C, et al. Implementing Patient-Reported Outcome Measures in Clinical Breast Cancer Care: A Systematic Review. Value in Health. 2019;22(10):1197-226.
Foster A, Croot L, Brazier J, Harris J, O’Cathain A. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. Journal of Patient-Reported Outcomes. 2018;2(1):46.
Gelkopf M, Mazor Y, Roe D. A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers. International Journal for Quality in Health Care. 2020.
Howell D, Hack TF, Green E, Fitch M. Cancer distress screening data: Translating knowledge into clinical action for a quality response. Palliative and Supportive Care. 2014;12(1):39-51.
Aromataris E, Munn Z. Joanna Briggs Institute reviewer's manual. Australia: The Joanna Briggs Institute; 2017.
Moher D, Liberati A, Tetzlaff J, Altman DG, Group. TP. Preferred reporting items for systematic review and meta-analysis: The PRIMSA statement. PLoS Medicine. 2009;6(7):1-6.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. Journal of Clinical Epidemiology. 2016;75:40-6.
Cooper H, Hedges LV, Valentine JE. The handbook of research synthesis and meta-analysis. 2nd ed. New York: Russell Sage Foundation; 2009.
Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Annals Of Oncology: Official Journal Of The European Society For Medical Oncology. 2015;26(9):1846-58.
Duncan EA, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res. 2012;12:96.
Gough D, Oliver S, Thomas J. An introduction to systematic reviews. Thousand Oaks, CA: Sage; 2012. 288 p.
McArthur A, Klugarova J, Yan H, Florescu S. Chapter 4: Systematic reviews of text and opinion. In: Aromataris E, Munn Z, editors. Joanna Briggs Institute Reviewer's Manual. Australia: The Joanna Briggs Institute; 2017. p. 136-77.
Lizarondo L, Stern C, Carrier J, Godfrey C, Rieger K, Salmond S, et al. Chapter 8: Mixed methods systematic reviews. In: Aromataris E MZ, editor. Joanna Briggs Institute Reviewer's Manual. Australia: The Joanna Briggs Institute; 2017. p. 272-310.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. British Medical Journal. 2015;349:1-25.
Garrard J. Health sciences literature review made easy. 5th ed. Minneapolis, MN: Jones and Bartlett Learning; 2017. 240 p.
Clarivate Analytics. EndNote X9 (Version 9.3.3) [Computer software]. 2020.
McArthur A, Klugárová J, Yan H, Florescu S. Innovations in the systematic review of text and opinion. International Journal of Evidence-Based Healthcare. 2015;13(3):188-95.
Moola S, Munn Z, Tufanaru C, Aromataris E, Sears K, Sfetcu R, et al. Chapter 7: Systematic reviews of etiology and risk. In: Aromataris E, Munn Z, editors. Joanna Briggs Institute Reviewer's Manual. Australia: The Joanna Briggs Institute; 2017.
Dang D, Dearholt S. Johns Hopkins nursing evidence-based practice: Model and guidelines. 3rd ed. Indianapolis, IN: Sigma Theta Tau International; 2017.
Hempel S, Shekelle PG, Liu JL, Sherwood Danz M, Foy R, Lim Y, et al. Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): A tool for critical appraisal of quality improvement intervention publications. BMJ Quality & Safety. 2015;24(12):796-804.
Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. Squire 2.0 (Standards for Quality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. American Journal of Critical Care. 2015;24(6):466-73.
Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. International Journal of Evidence-Based Healthcare. 2015;13(3):147153.
Munn Z, Moola S, Riitano D, Lisy K. The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence. International Journal of Health Policy & Management. 2014;3(3):123-8.
Center for Evidence Based Management. Critical appraisal checklist for cross-sectional study2014.
Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open. 2016;6(12):1-7.
Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW. Critical appraisal of the health research literature: Prevalence or incidence of a health problem. Chronic diseases in Canada. 1998;19(4):170-6.
Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education for Information. 2018a;34(4):285-91.
Hong QN, Pluye P, Fabregues S, Bartlett G, Boardman F, Cargo M, et al. Improving the content validity of the mixed methods appraisal tool: A modified e-Delphi study. Journal of Clinical Epidemiology. 2019;111:49-59 e1.
Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, M. C, et al. Mixed methods appraisal tool (MMAT) version 2018 user guide: McGill University; 2018b. Available from: http://mixedmethodsappraisaltoolpublic.pbworks.com.
Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare. 2015;13(3):132-40.
Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: Methodological guidance for systematic reviewers utilizing meta-aggregation. International Journal of Evidence-Based Healthcare. 2015;13(3):179-87.
Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party. JBI levels of evidence Australia: Joanna Briggs Institute; 2013 [Available from: https://joannabriggs.org/sites/default/files/2019-05/JBI-Levels-of-evidence_2014_0.pdf.
Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party. Supporting document for the Joanna Briggs Institute levels of evidence and grades of recommendation: The Joanna Briggs Institute; 2014 [Available from: https://joannabriggs.org/sites/default/files/2019-05/JBI%20Levels%20of%20Evidence%20Supporting%20Documents-v2.pdf.
Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party. JBI grades of recommendation. Australia: Joanna Briggs Institute; 2013. Available from: https://joannabriggs.org/sites/default/files/2019-05/JBI-grades-of-recommendation_2014.pdf.
QSR International. NVivo (Version 12.6) [Computer software]. 2019.
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Mark Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews: ESRC Methods Programme; 2006. 92 p.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science. 2009;4:1-15.
Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ. Specifying and comparing implementation strategies across seven large implementation interventions: A practical application of theory. Implementation Science. 2019;14(1):32.
Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: Results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science. 2015;10:109.
International Society for Quality of Life Research, prepared by Chan E, Edwards T, Haywood K, Mikles S, Newton L. Companion guide to implementing patient reported outcomes assessment in clinical practice2018. 15 p.
Chan EKH, Edwards TC, Haywood K, Mikles SP, Newton L. Implementing patient-reported outcome measures in clinical practice: A companion guide to the ISOQOL user's guide. Quality of Life Research. 2019;28(3):621-7.

Download PDF

Editor assigned by journal
16 Feb, 2021
Editorial decision: Reject before peer review
16 Feb, 2021
Editor invited by journal
16 Feb, 2021
Submission checks completed at journal
08 Oct, 2020
First submitted to journal
07 Oct, 2020

You are reading this older preprint version

Read the latest preprint version →

Healthcare Providers’ Implementation of Patient-Centred Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs) in Practice: A Protocol for a Mixed Method Systematic Review

Status:

Version 1

Abstract

Background

Methods

Search strategy

Types of Evidence

Selection Criteria

Study Selection Procedure

Assessment of Methodological Quality

Data Extraction

Data Synthesis

Conclusion

List Of Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1