Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol

doi:10.21203/rs.3.rs-185385/v1

Download PDF

Protocol

Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol

https://doi.org/10.21203/rs.3.rs-185385/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Citizen engagement in research is an emerging practice that involves members of the general public in research processes such as priority-setting, planning, decision-making, research conduct, implementation, evaluation, and dissemination. Engaging citizens in research, particularly health research, increases the relevance of study findings, minimizes waste by facilitating stewardship over resources, and builds public trust in the research. While several existing frameworks guide the application of citizen engagement principles to health research, it is unclear how citizen engagement can be utilized to maximize benefits and minimize risks and challenges in health research. To address the gaps in knowledge around citizen engagement in health research, we propose a scoping review to synthesize the state of knowledge on methods to incorporate and evaluate citizen engagement in research. A protocol is presented in this manuscript.

Methods: The methodology for our scoping review is guided by Arksey and O’ Malley’s framework for scoping reviews, and additional recommendations by Levac and colleagues. We will include peer-reviewed and grey literature that report on citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health) and report method(s) to conduct, measure, or evaluate citizen engagement. We will systematically search electronic databases (MEDLINE, EMBASE, CINAHL, JSTOR, PsycINFO, Scopus, and Science Direct) and relevant organizations’ websites for additional studies, frameworks, and reports on citizen engagement. Title and abstract and full-text citations will be screened independently and in duplicate. Data will be extracted independently and in duplicate, including: document characteristics, citizen engagement definitions and goals, and outcomes of citizen engagement (e.g., impact, barriers, facilitators).

Discussion: This review will synthesize the definitions, goals, methods, outcomes, and significance of citizen engagement in health research, as well as any potential barriers, facilitators, and challenges outlined in existing literature. The findings will provide an evidence-based foundation for developing new or improved guidance for citizen engagement in health research. Overall, we anticipate that our scoping review will be a preliminary step to meaningful engagement of citizens in research and strengthen the relationship between the scientific community and the public through transparency and collaboration.

SYSTEMATIC REVIEW REGISTRATION: Not applicable.

Health Economics & Outcomes Research

Citizen engagement

public participation

health research

co-designed research

models

frameworks

health services research

scoping review

protocol

Citizen engagement in research is defined as the effective and systematic involvement of members of the general public or persons from affected community groups such as patients, caregivers, advocates, and representatives in research processes (1). Organizations worldwide have suggested that citizen engagement in research may confer benefits to the conduct and uptake of scientific research through improving the relevance of study findings, encouraging the representation of diverse groups in research studies, minimizing waste by facilitating stewardship over resources, promoting mutual learning and understanding, and allowing broad dissemination of research findings beyond traditional academic audiences (2–6). Most importantly, citizen engagement has demonstrated utility in building public trust in science and research (7). This has led to citizen engagement becoming a priority for research funding organizations (e.g., the Canadian Institutes of Health Research [CIHR], National Institutes of Health [NIH] in the United States, and the National Institute for Health Research [NIHR] in the United Kingdom), policymakers, academic journals, and researchers themselves (1, 8–12). As a result, citizen engagement is now changing scientific research, particularly health research, to incorporate real world lived-experiences and perspectives of members of the public.

Engaging citizens in research necessitates researchers to modify their research approach and critically evaluate their work to improve its generalizability and impact. However, while citizen engagement in research has been conceptually substantiated, it is unclear when, for whom, and how it can be most impactful (13). Despite the availability of frameworks and guidelines for citizen engagement in health research (1, 2), there remains a lack of evidence-based methods to incorporate effective and systematic, or meaningful, engagement and valid measures to assess its impact. To effectively change health research so that it meaningfully includes the citizen perspective, there is a pressing need for a systematic synthesis of the broad field of citizen engagement in health research. The findings may be used to develop a new framework of citizen engagement or enhance and refine existing frameworks with a strong evidentiary basis, ultimately leading to effective and unified citizen engagement strategies. To address the gaps in knowledge around citizen engagement in health research, we will conduct a scoping review to synthesize the state of knowledge on methods to incorporate and evaluate citizen engagement in this field. The outcomes of this scoping review will provide a descriptive summary of the literature on citizen engagement including frameworks, models, and interventions aimed at improving engagement in health research. This will yield foundational results including strengths and weaknesses in existing frameworks, models and interventions, and help pinpoint areas that can be targeted to improve the relevance and effectiveness of future frameworks and recommendations.

This scoping review will be conducted according to recommendations specified by Arksey and O’Malley (14), and enhanced by Levac and colleagues (15). The conduct and reporting of this scoping review with be further guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guide review reporting (16).

Stage 1: Identification of the research question

We designed the research question to be broad in order to capture all potentially relevant information sources, as recommended by Arksey and O’Malley (14). This scoping review aims to answer the question: “what is the state of knowledge on methods to incorporate and evaluate citizen engagement in health research?”. The proposed research question provides direction to the subsequent steps, and defines the scope of inquiry with respect to population, concept, and outcomes of interest (14).

For the purpose of this scoping review, our target population “citizens” will be defined as consumers of health services, informal caregivers, advocates and representatives from community organizations, and members of the general public. Our target concept is engagement or participation in health research. For the purposes of this scoping review, we will adopt the CIHR definition of health research, which includes research in the biomedical, clinical, health systems and services, and social, cultural, environmental and population health fields (17). We recognize that engagement may take many forms, including but not limited to: priority-setting, planning, decision-making, research conduct, implementation, evaluation, or dissemination. Additionally, citizen engagement may occur when citizens are joint grant holders or co-applicants on research grants, help to identify research priorities, hold membership in a project advisory or steering group, comment and develop research materials, interact with research participants, and/or help carry out research activities.

Stage 2: Identification of relevant studies

As recommended by Arksey & O’Malley and Levac and colleagues (14, 15), we will identify relevant literature using a pre-determined plan for data sources and search strategy, including search terms, languages, and dates of search. The search strategies will be designed to maximize the comprehensiveness and breadth of the search, while considering time and personnel workload as limiting factors (14, 15).

Search strategy

A search strategy will be developed by the study team, including a medical librarian (DLL), who have led multiple initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research (18–22). The following databases will be searched from inception: MEDLINE (1879-present), EMBASE (1947-present), Cochrane Library (1996-present), CINAHL (1961-present), PsycINFO (1927-present), Scopus (1970-present), and Web of Science (1964-present). Our search will be broad to encompass relevant terminology in this area, including subject headings and keywords and relevant synonyms related to three concepts: (1) citizens (e.g., community member, lay person, public, stakeholder), (2) engagement (e.g., collaboration, engagement, participation), (3) health research (e.g., biomedical research, health research, public health research). No exclusion criteria will be placed on language, though we will exclude studies published prior to the year 2000. Prior to implementation, the search strategy will be independently reviewed using the Peer Review of Electronic Search Strategies (PRESS) checklist (23). A draft search strategy (MEDLINE) is shown in Additional File 1.

A targeted search of the grey literature will also be conducted, searching relevant local, provincial, national, and international organizations’ websites and related scientific or national funding organizations (i.e., OpenGrey, Trip, Involve, CIHR, Patient-Centered Outcomes Research Institute [PCORI]) for additional studies, frameworks, and reports on citizen engagement. Finally, the reference lists of included studies and related systematic reviews will also be screened to identify potentially relevant literature in health research.

Stage 3: Study selection

Following developing a strategy to identify studies, we will screen and select relevant studies for inclusion in the scoping review. As recommended by Levac and colleagues, we will develop inclusion and exclusion criteria a priori, with additional meetings with the study team to refine the study selection process at the beginning, midpoint, and endpoint of the citation screening process in case there are any unforeseen factors to be considered (15).

Eligibility criteria

Studies will be eligible for inclusion if they: (1) are primary (e.g., observational or interventional studies) or secondary (e.g., systematic or scoping reviews) research, frameworks, reviews, or reports, (2) report citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health, as defined by CIHR) and (3) report method(s) to conduct, measure, or evaluate citizen engagement. No restrictions will be placed on language or study design. Studies published prior to the year 2000 will be excluded.

Selection process

Retrieved articles will be imported to Covidence (Veritas Health Innovation, Melbourne, Australia) for title & abstract screening, completed independently and in duplicate by two reviewers. Reviewers will pilot screen the titles and abstracts of 50 articles to ensure consistency of inclusion and exclusion criteria and once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved, proceed to screen the remaining articles. If one reviewer indicates an article as potentially relevant at the title & abstract screening phase, the article will proceed to full-text review to ensure inclusivity. Following title & abstract screening, the full- text of selected studies will be screened independently and in duplicate by two reviewers. Reviewers will pilot screen the full text of 20 articles, and proceed to screen the remaining articles once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved. Both reviewers must agree on inclusion status and reason for exclusion, if relevant. Any disagreements between the reviewers will be resolved by discussion or the involvement of a third reviewer, if required.

Stage 4: Charting the data

We will chart the data according to Arksey & O’Malley and Levac’s recommendations (14, 15), beginning with defining the information to be extracted from the studies by developing an initial data charting form and then refining this form iteratively and through regular discussion with the study team.

The study team will develop a standardized data extraction form in Microsoft Excel (version 16.29.1). The data extraction form will be piloted by two reviewers with ten included studies, and revised as needed. Once the final data abstraction form is developed, all relevant articles will then be abstracted independently and in duplicate by two reviewers.

Extracted variables in the initial data extraction form will include study characteristics, participants, type and goals of citizen engagement, engagement frameworks and other strategies, engagement uptake, and findings and outcomes of citizen engagement in the context of health research. A draft list of variables to be extracted is shown in Table 1.

Table 1

Draft list of variables to be extracted
Document identification	List of authors (surnames) Year of publication Publisher (e.g., journal, institution) Research area (e.g., health sciences, social sciences, and natural sciences and engineering)
Document type	Framework OR Model OR Report Primary study OR Secondary research
Characteristics of intended participants	Population of focus (patient, family, healthcare provider, researcher, knowledge-user, decision-maker) Age Sex Gender Geographical location
Citizen engagement	Operationalization	Definition of citizen engagement Type and goals of engagement (e.g., priority setting, decision-making, research conduct, implementation, evaluation, dissemination) Engagement frameworks used Engagement uptake
	Methods (if applicable)	Study design Sample size Outcome measurement
	Results and Discussion (if applicable)	Findings and outcomes of citizen engagement Barriers to citizen engagement Facilitators of citizen engagement Benefits of citizen engagement Challenges and risks of citizen engagement Strategies to mitigate challenges and risks of citizen engagement Author-stated conclusions (if applicable)

Stage 5: Collating summarizing and reporting the results

We will collate, summarize, and report the results according to the Arksey & O’Malley and Levac guidelines, which recommend undertaking synthesis of results by i) analyzing the data, ii) reporting results, ii) and evaluating their meaning (14, 15). More specifically, these guidelines suggest that researchers execute a detailed thematic analysis of their data guided by qualitative analytical techniques before reporting results in a suitable form such as a narrative synthesis, a table of barriers/facilitators or knowledge gaps, or a framework depending on the goal of the review (14, 15). Arksey & O’Malley and Levac further recommend that researchers aim to contextualize their findings within existing literature and research, practice, and policy (14, 15).

We will use a PRISMA-ScR checklist to guide reporting and the flow diagram to report the number of unique articles identified, excluded articles, reasons for exclusion, and included articles in the final scoping review (16). We will present a descriptive summary of characteristics of the included documents, and the characteristics of the intended participants or audiences for these documents. If applicable, we will conduct a narrative synthesis of extracted data variables, including, but not limited to: definition of citizen engagement, goal of citizen engagement, impact of citizen engagement, outcome measurement, author stated conclusions, barriers, facilitators and benefits, risks, and challenges of citizen engagement.

Stage 6: Consultation

As recommended by Arksey and O’Malley (14), we will involve citizens (NF, BS) in study conception and design as well as interpretation and contextualization of the data. This stakeholder consultation (i.e., citizen engagement) will allow for the incorporation of perspectives and insights beyond the literature and is necessary to improve the rigour and uptake of findings (14, 15).

MAIN DELIVERABLE

Results from this scoping review will be synthesized to inform the future development or refinement of a framework for incorporating meaningful citizen engagement in health research. The barriers, facilitators, risks and challenges to citizen engagement identified through this scoping review will be used by our team in future framework development or refinement. Specific focus will be placed on evaluation of citizen engagement in order to improve upon existing methods of incorporating citizen engagement in health research. Once developed, a draft of this framework will be circulated through surveys to stakeholders including health researchers, decision and policy-makers, and citizens who indicate interest to solicit feedback. Focus groups will be conducted to discuss specific items in the framework which will lead the reiterative refinement of this work. While the limitations to citizen engagement have been outlined before (24–26), this applied framework will chart barriers and facilitators to citizen engagement systematically, allowing our team and stakeholders to effectively utilize this knowledge to initiate, perform and carry out an evidence-based process of framework building.

This scoping review aims to synthesize published research, guidelines, and frameworks relevant to citizen engagement in health research. The findings of this scoping review will be applicable to many research disciplines beyond health research such as social sciences research and natural sciences research. The outcome of this scoping review will be a summary of the definitions, goals, methods, outcomes, significance, and evaluation of citizen engagement alongside potential barriers, facilitators, benefits, challenges, or risks.

The potential findings of this scoping review will have several applications. First, we will provide an evidence-based foundation to create novel guidance for citizen engagement in research or enhance existing guidelines, such as those developed by CIHR (1) and NIHR (1, 3). Second, this scoping review may highlight potential gaps in the literature, directing future areas of study in the science of citizen engagement in health research. Third, our scoping review will act as a comprehensive information source of citizen engagement in research that research teams may learn from and apply to their own work. Fourth, this scoping review will identify where sufficient literature exists to warrant systematic reviews that could inform citizens of their potential to be impactful co-developers of research across many disciplines. Overall, we hope that our study will be a preliminary step to meaningful and effective engagement of citizens in research to improve the impact and uptake of research findings, and strengthen the relationship between the scientific community and the public through transparency and collaboration.

Team preparedness and dissemination of research findings

We are well-positioned to complete this project due to our knowledge and experience in engaging and evaluating the impact of citizen engagement in health research. We have led initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research and routine clinical care through our research programs (18–22). We have experience with engaging and training patient and family partners in critical care medicine research (e.g., generating ideas, writing grants, priority setting, conducting research, writing manuscripts), and involving patient advisors in communication of research findings to members of the public at open engagement sessions (18–22). Additionally, we are well-equipped to undertake the methodologies outlined in this proposed research. We have a track-record of high-quality and pertinent systematic and scoping review publications that inform our research (27–30).

The findings of this scoping review will be disseminated widely by the study team members through our research team and institutional websites and presentations at local, national, and international conferences. Our citizen research partners will aid in disseminating the study findings (both to lay and scientific audiences), a step we consider critical for this work to have an impact on citizen engagement in research. We intend to submit the findings for peer-reviewed publication.

CIHR: Canadian Institutes of Health Research

NIH: National Institutes of Health

NIHR: National Institute for Health Research

PRESS: Peer Review of Electronic Search Strategies

PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews

Ethics approval and consent to participate

Not applicable.

Consent to publish

Not applicable.

Availability of data and materials

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Funding

This study is currently unfunded but is seeking local and national sources of funding.

Authors' contributions

All authors (AS, BKR, KDK, DLL, JPL, NF, BS, HTS, and KF) have contributed to the conception and design of this protocol. AS and BKR drafted the manuscript. All authors critically revised the manuscript and approved of the final submitted version.

Acknowledgements

Not applicable.

Canadian Institutes of Health Research. CIHR’s Framework for Citizen Engagement. Ottawa, ON; 2012.
INVOLVE. Exploring Impact: Public involvement in NHS, public health and social care research. 2009.
National Institute for Health Research. Going the extra mile: Improving the nation’s health and wellbeing through public involvement in research [Available from: https://www.nihr.ac.uk/documents/about-us/our-contribution-to-research/how-we-involve-patients-carers-and-the-public/Going-the-Extra-Mile.pdf.
Core Values for the Practice of Public Participation Sooke, BC: IAP2 Canada; 2018 [Available from: https://www.iap2canada.ca/foundations.
Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387-95.
Shimmin C, Wittmeier KDM, Lavoie JG, Wicklund ED, Sibley KM. Moving towards a more inclusive patient and public involvement in health research paradigm: the incorporation of a trauma-informed intersectional analysis. BMC Health Serv Res. 2017;17(1):539.
Haywood BK. Beyond Data Points and Research Contributions: The Personal Meaning and Value Associated with Public Participation in Scientific Research. International Journal of Science Education, Part B. 2016;6(3):239-62.
Walshe K, McKee M, Groenewegen P, Hansen J, Figueras J, Boccia S, et al. Reshaping the agenda of the European Commission for the health systems and policy research in Europe within Horizon 2020. Epidemiology Biostatistics and Public Health. 2013;10.
Horobin A. Going the extra mile - creating a co-operative model for supporting patient and public involvement in research. Res Involv Engagem. 2016;2:9-.
Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8(3):e020452.
Social Sciences and Humanities Research Council. Community Engagement 2019 [Available from: https://www.sshrc-crsh.gc.ca/society-societe/community-communite/index-eng.aspx.
Natural Sciences and Engineering Research Council of Canada. NSERC 2020 A Strategic Plan 2020 [June 23 2020]. Available from: https://www.nserc-crsng.gc.ca/nserc-crsng/nserc2020-crsng2020/index_eng.asp.
Marris C, Rose N. Open engagement: exploring public participation in the biosciences. PLoS Biol. 2010;8(11):e1000549.
Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19-32.
Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Science. 2010;5(1):69.
Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467-73.
Research CIoH. CIHR background: Government of Canada; 2019 [Available from: https://letstalk-cihr.ca/about.
Fiest KM, Krewulak KD, Ely EW, Davidson JE, Ismail Z, Sept BG, et al. Partnering With Family Members to Detect Delirium in Critically Ill Patients. Crit Care Med. 2020;48(7):954-61.
Fiest KM, Krewulak KD, Sept BG, Spence KL, Davidson JE, Ely EW, et al. A study protocol for a randomized controlled trial of family-partnered delirium prevention, detection, and management in critically ill adults: the ACTIVATE study. BMC Health Serv Res. 2020;20(1):453.
Krewulak KD, Sept BG, Stelfox HT, Ely EW, Davidson JE, Ismail Z, et al. Feasibility and acceptability of family administration of delirium detection tools in the intensive care unit: a patient-oriented pilot study. CMAJ Open. 2019;7(2):E294-e9.
Boyd JM, Burton R, Butler BL, Dyer D, Evans DC, Felteau M, et al. Development and Validation of Quality Criteria for Providing Patient- and Family-centered Injury Care. Ann Surg. 2017;266(2):287-96.
Potestio ML, Boyd JM, Bagshaw SM, Heyland D, Oxland P, Doig CJ, et al. Engaging the Public to Identify Opportunities to Improve Critical Care: A Qualitative Analysis of an Open Community Forum. PLoS One. 2015;10(11):e0143088.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40-6.
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.
Grand A, Davies G, Holliman R, Adams A. Mapping public engagement with research in a UK University. PLoS One. 2015;10(4):e0121874.
Powell MC, Colin M. Participatory Paradoxes:Facilitating Citizen Engagement in Science and Technology From the Top-Down? Bulletin of Science, Technology & Society. 2009;29(4):325-42.
Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452-7.
Cherak SJ, Fiest KM, VanderSluis L, Basualdo-Hammond C, Lorenzetti DL, Buhler S, et al. Nutrition interventions in populations with mental health conditions: a scoping review. Appl Physiol Nutr Metab. 2020;45(7):687-97.
Rosgen B, Krewulak K, Demiantschuk D, Ely EW, Davidson JE, Stelfox HT, et al. Validation of Caregiver-Centered Delirium Detection Tools: A Systematic Review. Journal of the American Geriatrics Society. 2018;66(6):1218-25.
Cherak SJ, Rosgen BK, Amarbayan M, Plotnikoff K, Wollny K, Stelfox HT, et al. Impact of social media interventions and tools among informal caregivers of critically ill patients after patient admission to the intensive care unit: A scoping review. PLoS One. 2020;15(9):e0238803.

AdditionalFile1.pdf

Download PDF

Editorial decision: Minor revision
19 Jul, 2021
Review #1 received at journal
18 Jul, 2021
Editor assigned by journal
01 Jul, 2021
Reviewers invited by journal
01 Jul, 2021
Reviewer #1 agreed at journal
01 Jul, 2021
Reviews received at journal
01 Jul, 2021
Editor invited by journal
01 Jul, 2021
Submission checks completed at journal
06 Feb, 2021
First submitted to journal
29 Jan, 2021

You are reading this latest preprint version

Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol

Status:

Version 1

Abstract

Background

Methods

Stage 1: Identification of the research question

Stage 2: Identification of relevant studies

Search strategy

Stage 3: Study selection

Eligibility criteria

Selection process

Stage 4: Charting the data

Stage 5: Collating summarizing and reporting the results

Stage 6: Consultation

MAIN DELIVERABLE

Discussion

Team preparedness and dissemination of research findings

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1