Working with stakeholders to rapidly identify and prioritise COVID-19 health and social care evidence needs for evidence-informed policy and practice decision making: processes, results, and lessons from the Wales COVID-19 Evidence Centre

doi:10.21203/rs.3.rs-3286253/v1

Download PDF

Research Article

Working with stakeholders to rapidly identify and prioritise COVID-19 health and social care evidence needs for evidence-informed policy and practice decision making: processes, results, and lessons from the Wales COVID-19 Evidence Centre

https://doi.org/10.21203/rs.3.rs-3286253/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

The COVID-19 pandemic reinforced the role of health and social care research in policy and practice decision-making. Global research priorities were rapidly identified. However, the diverse health, social and economic impacts in different countries and communities, intersecting with local pre-existing inequalities, warranted a rapid, localised approach so that evidence was maximally useful to policy and practice decision makers. The aim of this paper is to describe how the Wales COVID-19 Evidence Centre engaged multiple stakeholders in a rapid COVID-19 evidence needs prioritisation process, to present the results of this exercise and evolving evidence needs at three time points during the pandemic, and to describe key learning from working with multiple stakeholders.

Methods

Stakeholder groups across policy, practice, and public domains were invited to submit up to ten COVID-19-related health and social care evidence needs via our Stakeholder evidence needs Prioritisation Exercise (ScoPE) across three prioritisation rounds (Spring/Summer 2021, Autumn/Winter 2021, and Spring/Summer 2022). Urgent, high-priority needs were also submitted via our fast-tracked process. Returned evidence needs were assessed against agreed prioritisation criteria.

Results

44 stakeholder groups proposed 223 questions (212 ScoPE and 11 fast-tracked routes); a total of 78 questions were accepted onto our work programme and answered across 58 different studies (50 evidence syntheses, eight primary research). Evidence needs answered by our work programme mapped closely to the broader and evolving COVID-19 context.

Conclusion

We successfully engaged multiple stakeholders (including the public) in one process, demonstrated the feasibility of virtual methods, were able to track a rapidly evolving and complex public health situation, and identified evidence needs that directly informed policy and practice guidance. Further investment and resource in the Science-Policy-Practice Interface during early phases of prioritisation exercises could help to enhance research question specificity and impact planning, thus expediting evidence prioritisation, delivery, and impact.

COVID-19

Science-Policy-Practice Interface

Health policy

Pandemic

Prioritisation

Research priorities

Knowledge transfer

Evidence-informed policy

The COVID-19 pandemic reinforced the role of health and social care research in policy and practice decision-making. Global priorities were rapidly set by the World Health Organization soon after the outbreak,(1) which provided an effective roadmap for the most urgent and widely transferable research priorities, including the natural history of the virus, its transmission, diagnosis and treatment. Whilst there were many common COVID-19 pandemic questions that warranted a shared response, in different countries and communities the epidemic had diverse health, social, and economic impacts,(2) intersecting with local pre-existing inequalities.(3, 4) For COVID-19 evidence to be maximally useful to policy and practice decision-makers, it must be rapidly available, applicable to the local context and reflect the highest priority evidence needs of key stakeholders - a process we describe in this paper.

The science of COVID-19 progressed at extraordinary speed and new evidence became available at a pace not seen previously. The Wales COVID-19 Evidence Centre (WCEC)(5) was established in March 2021 with the remit of rapidly synthesising COVID-19 evidence to answer the most important policy and practice questions for Wales. The COVID-19 pandemic was characterised by complexity (multiple factors interacting and influencing outcomes), urgency (decisions needed in days or weeks, not months) and high levels of threat (the consequences of doing nothing could be catastrophic).(6) A swift response to identifying locally relevant research priorities was essential. In line with Greenhalgh et al’s argument for a shift to ‘Evidence Based Medicine +’ during pandemic periods, we also needed to think differently about how we rapidly collated evidence (moving away from gold standard systematic reviews to rapid reviews), and about the type of evidence we collated - considering a wider range of evidence beyond traditional randomised controlled trials, when rapid decisions are required to save lives and protect health.(6)

The number of potential COVID-19 research questions that the WCEC could explore was limitless, but the time-urgent and high-threat nature of COVID-19 decision-making meant that we had to prioritise the research most likely to impact beneficially on local policy and practice decision-making in Wales and benefit the population. Further, the COVID-19 pandemic evolved rapidly, and we observed differential health, economic and societal impacts as we moved in and out of the different pandemic ‘waves’.(3, 4) Therefore, the window of opportunity to influence decision-making was unknown and limited. Several established research priority-setting methods exist (7–10) and are recommended by the World Health Organization.(11) However, these processes are time intensive, a luxury not afforded in the pandemic context, and we needed to design a robust and transparent yet pragmatic and rapid approach to identifying COVID-19 research priorities for the WCEC.

The aim of this paper is to describe how the WCEC engaged multiple stakeholders in novel rapid evidence needs prioritisation process, to present the results of this prioritisation exercise and the evolving COVID-19 priorities at three time points during the pandemic, and to describe key learning from working with multiple stakeholders, including the public.

2.1 Identifying key stakeholders

At WCEC inception (March 2021), we conducted a stakeholder mapping exercise to identify key individuals or groups that could inform COVID-19 health and social care evidence needs in Wales. We adopted an inclusive approach, ensuring coverage of stakeholders across policy-makers, those involved in COVID-19 scientific advice for Welsh Government, members of the public, health and social care service providers, health and social care professionals, third sector organisations (e.g. charity / voluntary groups, social enterprises), and health and social care researchers. Four public groups were identified as being disproportionately affected by the pandemic, (3, 12) including children and young people, disabled people, ethnic minority groups, and people living in areas of higher deprivation; targeted efforts were made to involve these groups in this process.

The stakeholder list was iterative and adapted as new stakeholders were identified. This was important as we moved from the reactive phase and through the subsequent recovery pandemic phases, where the response shifted from issues regarding, for example, infection prevention and control measures in different settings to addressing backlogs across health and social care services. We worked closely with Welsh Government’s Technical Advisory Cell (TAC) and Technical Advisory Group (TAG), who coordinated scientific and technical advice to support Welsh Government decision-makers during the pandemic. Regular TAC / TAG meeting attendance provided insight into the emerging COVID-19 landscape and urgent priorities and provided linkages (via RJL) with key COVID-19 stakeholders in Wales.

2.2 Stakeholder evidence needs Prioritisation Exercise (ScoPE)

Identified stakeholder groups were invited to take part in the Stakeholder evidence needs Prioritisation Exercise (ScoPE), conducted over three rounds between March 2021 and November 2022: Round 1 (Spring / Summer 2021), Round 2 (Autumn / Winter 2021), and Round 3 (Autumn / Winter 2022).

With our remit of ‘good questions, answered quickly’ we needed to support stakeholders to return priority questions that were specific and answerable, so that we could expedite the subsequent evidence synthesis / research and knowledge mobilisation processes. We drew upon research principles of developing ‘good questions’ to develop the ScoPE proforma and accompanying guidance document. In addition to the evidence needs, we asked stakeholders to provide further information so that we could assess each question against the following 5 criteria for inclusion in our work programme: 1) the relevance to the current or predicted future COVID-19 context; 2) the importance of the evidence gap; 3) the potential benefits of the question being answered (potential impact); 4) how the evidence would be used (knowledge mobilisation plans); and 5) the timeframe required for the evidence to be useful (see section 2.5 for further information). Stakeholders were also encouraged to ‘Horizon Scan’ and consider potential future priorities as we progressed through the pandemic, enabling us to be proactive to longer-term evidence needs.

Stakeholders were invited by email to participate including up to 10 evidence needs or research questions in the proforma, ranked in their order of priority. The email (usually following an introductory email) introduced the WCEC, the purpose and process of the ScoPE exercise, and explained that questions assessed as high priority with potential for impact could be included in our work programme. Two follow-up emails were sent in the result of no response. Completed ScoPE proformas were returned to the WCEC via email.

The ScoPE process was amended for our online WCEC public-facing symposium event (March 2022) and for our public consultation workshops with groups disproportionately affected by the pandemic (described fully elsewhere). Participants were not required to return the ScoPE proforma; instead, individual participants were supported during the workshops (online or face-to-face) by a facilitator to describe their evidence needs (including importance) and to participate in a follow-up ranking exercise (online or face-to-face) to identify their ‘top ten’ research priorities as a group.

2.3 Fast-tracked urgent priorities

Stakeholders with urgent policy or practice evidence needs, out of the rapidly evolving pandemic situation, could submit urgent questions to the WCEC at any time (not restricted to ScoPE consultation timing or completion of the proforma), facilitated through our links with the TAC; these were fast-tracked to the assessment stage (see section 2.5).

2.4. Categorising the questions

All questions received (via ScoPE or fast-tracked) were mapped against Welsh Government’s ‘Five Harms Arising from COVID-19’ (see Table 1).(13) These five harms consider both direct and indirect harms; the latter being less understood and measured during the height of the pandemic. An ‘other’ category was created when a question could not be mapped to a harm (e.g. benefits associated with health innovation). Questions were also categorised by theme (e.g. infection prevention and control) and sub-theme (e.g. hospital settings). Where a question mapped to more than one harm / theme, the main harm / theme category was chosen.

Table 1

Welsh Government’s Five Harms of COVID-19 (13)
Harm 1.	Harms directly arising from SARS-CoV2 infections
Harm 2.	Indirect COVID-19 harms due to surge pressures on the health and social care system and changes to healthcare activity, such as cancellation or postponement of elective surgeries and other non-urgent treatments (e.g. harm from cessation of screening services) and delayed management of long-term conditions.
Harm 3.	Harms arising from population-based health protection measures (e.g. lockdown) such as, educational harm, psychological harm, and isolation from shielding and other measures
Harm 4.	Economic harms such as employment and reduced business income arising both from COVID-19 directly and population control measures, like lockdown
Harm 5.	Harms arising from the way COVID-19 has exacerbated existing, or introduced new, inequalities in our society.

2.5 Assessing questions for importance, impact, and urgency

All questions (via ScoPE or fast-tracked) were collated in a master spreadsheet. Each question underwent research priority assessment by at least two members of the WCEC core team (AE, NJW, AC, RJL, RL) and by at least one member of our Public Partnership Group (NP, CP). Questions were assessed against the five criteria included in Box 1 and allocated a grade of high, medium, low priority. An audit trail of reasoning and decision-making was kept. Questions were checked against our ongoing work programme and with other groups that could be conducting work in this area to avoid duplication.

Where agreement was not reached in the first instance, the assessors discussed these, and agreement was sought between them. Using these assessments as guidance, the core team allocated an overall priority recommendation for each question: Category A) recommended onto the work programme (no significant changes; with refinements agreed with stakeholders, sometimes merged with a duplicate question); Category B) not currently recommended onto the work programme (monitored and considered against changing pandemic priorities); Category C) not recommended onto the work programme (already addressed by WCEC/externally; beyond remit; issues with relevance, importance, benefits, or potential for impact). The team also agreed whether the question was suitable for evidence synthesis, rapid primary research, or both.

2.6 Agreeing the work programme

Category A recommended questions (including rationale) were shared with Welsh Government’s TAG, who oversaw and signed-off our work programme. For transparency, Category B and C questions were also shared. Once agreed, stakeholders were notified about the outcome and an up-to-date version of our rolling work programme was published on the Health and Care Research Wales website.(14) Where a question was not adopted because it had already been addressed (by WCEC or externally), stakeholders were signposted to that information. Questions adopted onto our work programme were then allocated to an appropriate Collaborating Partner (research team linked with the WCEC), and the stakeholder was asked to engage in the planning, research, and knowledge mobilisation processes (described in full elsewhere). If necessary, research questions were also refined at this stage to enable rapid evidence synthesis.

2.7 Patient and public involvement statement

Public partners are key on all our work, from WCEC strategies / processes, evidence needs prioritisation, research work, and knowledge mobilisation. Two members of our WCEC Public Partnership Group were involved in the prioritisation assessment process (see section 2.5), making decisions about our work programme, and are co-authors on this manuscript. Members of the public were also consulted via the ScoPE for their evidence needs, including underserved groups (see section 2.2).

Figure 1 provides a summary of results for the evidence needs prioritisation process for the WCEC, conducted at three time points during the pandemic. It includes number of stakeholders consulted, and a summary of the number of questions proposed for our work programme, decision-making for inclusion, and questions accepted onto our work programme.

3.1 Stakeholders

We invited a total of 52 stakeholders to participate in the evidence needs prioritisation process; 44 (85%) stakeholders engaged in the process and submitted priority COVID-19 questions to the WCEC. A broad range of stakeholder groups engaged across the three rounds, including: Welsh Government (n = 16); local health boards / trusts (n = 6); public (n = 6); academia / research (n = 5); professional Royal Colleges (n = 5); other NHS Wales (n = 4); social care national body (n = 1); UK Government (n = 1). Five stakeholder groups submitted questions in at least two rounds; the social care stakeholder group submitted questions for all three rounds.

3.2 Stakeholder evidence needs Prioritisation Exercise (ScoPE) – questions proposed

A total of 212 questions was proposed by a total of 44 different stakeholders across the three prioritisation rounds via the ScoPE: 81 during Round 1 (Spring / Summer 2021), 53 during Round 2 (Autumn / Winter 2021), and 78 during Round 3 (Spring / Summer 2022). The number of ScoPE proformas completed and returned during each Round was 10, 15, and 19 respectively.

3.3 Fast-tracked urgent priorities included in our work programme

A total of 11 fast-tracked questions were returned and accepted across the three prioritisation rounds from various stakeholders, including Welsh Government (n = 6), public health (n = 1), academia (n = 1), local health board trust (n = 1), and a national research body (n = 1). These questions covered wide-ranging topics, including: COVID-19 transmission in vaccinated populations; innovations to tackle cancer screening backlogs for upper and lower gastrointestinal cancers; the risk of catching COVID-19 in semi-outdoor versus indoor or outdoor settings; evidence of direct harm from COVID-19 infection; COVID-19 vaccination in pregnant women; the efficacy, effectiveness, and safety of ozone disinfection machines for schools; and health-related impact of the cost-of-living crisis (see Supplementary Table 1 for full list of fast-tracked questions and links to reports). Of these 11 questions, nine have been completed to date (December 2022); 20 of the completed questions have been used to directly inform Welsh Government pandemic guidance or health care practice to date (see Supplementary Table 1 for all outputs).

3.4 Work Programme Summary – questions accepted

Of the 223 questions proposed via the ScoPE and fast-tracked routes (as of December 2022), 78 questions were accepted onto the WCEC work programme (67 ScoPE; 11 fast-tracked), across the three rounds: Round 1 = 27, Round 2 = 18, and Round 3 = 33. After merging duplicate questions, these 78 questions were answered across a total of 58 studies (50 evidence syntheses and eight primary research studies). The breakdown of number of questions proposed and accepted for each prioritisation round (via ScoPE and fast-tracked routes) can be found in Fig. 1. Supplementary Table 1 provides a list of all questions answered by the WCEC, including date, proposing stakeholder, research partner, and links to the final report (when available).

3.5 Question categories

Questions proposed via the ScoPE and fast-tracked routes mapped across all ‘Five Harms of COVID-19’, set out by Welsh Government (see Table 1).(13) Questions accepted onto our work programme also mapped across all ‘five harms’. Frequency of the five harm categories, for all questions submitted and for questions accepted on our work programme, across the three prioritisation rounds, is included in Fig. 2. Submitted questions mapped to Harm 3 were the most common across all three rounds (‘harms arising from population-based health protection measures…’). The most common harm category of accepted questions changed over the three rounds. The most common harm category for questions accepted onto the work programme was Harm 3 (‘harms arising from population-based health protection measures…’) for Round 1, and Harm 2 (‘indirect COVID-19 harms due to surge pressures on the health and social care system and changes to healthcare activity…’) for Rounds 2 and 3.

<<INSERT > > Figure 2. Frequency of the ‘Five Harms of COVID-19’ for all questions submitted and questions accepted onto the WCEC work programme

Questions submitted to the WCEC (via ScoPE and fast-track) mapped to a broad range of 16 themes and a further 83 sub-themes (see Table 2). The top five question themes and sub-themes accepted onto our work programme across all three rounds were (theme/sub theme): health services delivery (n = 21) / addressing NHS backlogs (n = 12); infection prevention and control (n = 27) / behavioural / community settings / education settings / health and social care settings (all n = 2); children and young people (n = 7) / education (4); long covid (n = 2) / support (n = 2); and inequalities (n = 7) / vulnerable groups (n = 4). We observed a reduction in accepted infection prevention control and children and young people questions across the three rounds, and an increase in accepted questions exploring how NHS services can address the backlogs caused by the pandemic.

Table 2. Frequency of themes for all questions submitted and questions accepted onto the WCEC work programme across the three prioritisation rounds (ScoPE and fast-tracked) between March 2021 – September 2022 (in order of overall theme frequency of questions submitted)

Theme (sub-themes)

Questions submitted

(n=223)

Questions accepted

(n=78)

Round 1

Round 2

Round 3

Submitted

(n=87)

Accepted

(n=27)

Submitted

(n=55)

Accepted

(n=18)

Submitted

(n=81)

Accepted

(n=33)

Health service delivery

Addressing NHS backlogs / changed service use

Cancer services

Elective services

Chronic conditions

Mental health

Paediatric

Complaints and investigation

Innovation

Paediatric

Remote care

Staff recruitment and retention

Training and Education

Children and young people

Education

Employment

Health outcomes

Other
Other outcomes

Vulnerable groups

Infection prevention and control

Behavioural

Community setting

Education settings

Health and social care setting

Leisure

Multiple settings

Non-pharmaceutical interventions

Transport

Surveillance

Long COVID

Aetiology

Children and young people

Contributing factors

Definitions

International practices

Mental health and wellbeing

NHS services (use)

Other conditions

Outcomes

Prevalence

Studies

Social care

Support (other)

Treatment / management

Inequalities

Children and young people

COVID-19 outcomes

Education and employment

Older people

Social care

Vulnerable / underserved groups

Mental health and wellbeing

Children and young people

Health and social care workforce

Inequalities

Key workers

Social Care

Addressing backlogs / displacement of services

Discharge to community settings

Community interventions

Financial impact on staff

Inequalities

Innovation

Training and education

Workforce recruitment and retention

Planning and learning

Communication and messaging

Future pandemics (general)

Health and social care services

Informing funding calls

Mental health and wellbeing

Vaccination

Vulnerable groups

Vaccination

Behavioural

Delivery

Immunity

Outcomes

Pregnant / post-partum women

Transmission rate

Uptake

Vulnerable / underserved populations

COVID-19 treatment / management

Community setting

Hospital setting

Outcomes

Financial and economic

Cost of living

Financial support

Impact

Social impacts

Communities

Digital exclusion

Domestic violence

Physical activity

Refugees

Returning to normal

Bereavement

Experiences

Long term impact

Shielding (health and social care workforce)

Environmental

Out of Scope (not within remit of the WCEC Centre)

3.6. Evolving priorities mapped to the broader COVID-19 context

The questions accepted onto our work programme are consistent with the broader and evolving COVID-19 context. The most common harm category of accepted questions during Round 1 (Spring / Summer 2021) was ‘harms arising from population-based health protection measures’. This fits with the reactive COVID-19 context at that time in Wales and across the UK, where wide-scale restrictions were still in place or only recently relaxed (15) and the focus was on controlling rates of infection (16). Measures included school closures, stay-at-home requirements, mask wearing, non-essential business closure, international and local travel restrictions, large-scale testing and isolation practices, and early stages of vaccination roll-out. An area of specific interested was the significant and disproportionate disruption to children and young people caused by wide-scale school closures and stay-at-home mandates (15).

As we moved to the second (Autumn / Winter 2021) and third (Spring / Summer 2022) prioritisation rounds, the most common harm category for accepted questions was ‘indirect harms due to surge pressures on health and social care systems’; with the number of accepted questions mapping to this harm doubling between round two and three. We also see an increase in accepted questions mapped to the sub-theme ‘addressing NHS backlogs’. Again, our work programme mirrored the wider COVID-19 recovery context in Wales and the UK at the time (18). As population level restrictions eased and the COVID-19 position became more stable, policy and practice focus shifted to investment and interventions that could help to recover and rebuild health and social care services that had been significantly impacted by postponed, delayed, reduced, or altered access (19) and supporting those with Long-COVID. The number of accepted questions mapping to economic harms increased between round one and three, and questions addressing harms of inequalities increased between round one and two. Prioritisation round two took place as we moved out of the third wave. Here the focus had turned to inequalities caused and exacerbated by COVID-19, the impact of severe economic injury caused by the pandemic recession, and threats to an equitable recovery (15, 17, 18, 19); again, our work programme aligns with the wider COVID-19 policy and practice decision-making context.

<<INSERT Table 2HERE>>

The WCEC successfully engaged 44 stakeholder groups from eight domains to participate in our rapid COVID-19 evidence needs prioritisation process. Stakeholders submitted 223 COVID-19-related health and social care priority questions covering 16 different themes (see Table 2) and five categories of COVID-19 harm(13) over an 18-month period (March 2021 - September 2022), spanning different circumstances of the COVID-19 pandemic. Questions on the WCEC work programme are consistent with the priorities stakeholders deemed most important and reflected the broader and evolving COVID-19 context and key policy decisions being made in Wales, and the UK (15, 17).

Evolving COVID-19 research priorities – from reactive response to equitable recovery

A total of 78 stakeholder questions was addressed by 58 studies on our work programme (50 evidence syntheses and eight primary research studies), mapping across all five COVID-19 harms set-out by Welsh Government (see Table 1).(13) Balancing of each of these harms guided policy decision-making through the evolving pandemic phases. Coverage of each harm by our work programme demonstrates that the priorities proposed by our stakeholder groups were consistent with the key pandemic harms identified by those involved in providing scientific advice and policymakers in Wales (July 2021). Further, our work programme closely aligned with the broader evolving COVID-19 context. As the policy and practice focus moved from reactive measures that focused on infection prevention and control to recovery measures that focused on recovering and rebuilding our health and other services, so did the questions that we addressed.

The 58 studies included in our work programme covered 13 of the 16 themes of all submitted questions. Some important themes were not covered by our work programme. Questions were submitted about shielding and COVID-19 bereavement, but through our networks we identified parallel Wales-based studies that were addressing these issues, (20, 21) and so these were excluded to avoid duplication. Further, we received 11 questions relating to pandemic planning and learning, which were not covered by our work programme. Whilst these questions were important with the potential for longer-term impact, we could not rapidly address these questions, and it was agreed that some may be better answered by the ongoing UK COVID-19 inquiry.(22) However, our priority setting exercise indicates that learning from COVID-19 pandemic experiences is critical and a key task for UK Governments.

What worked well

We established a novel method for rapidly engaging a broad range of multi-sectoral stakeholders in COVID-19 health and social care during a pandemic period. There are several learning points about what worked well and key challenges (see Table 3 for a summary).

Table 3

Summary of key lessons and recommendations for rapid evidence needs prioritisation processes
What worked well
Broad and inclusive involvement – involving multiple stakeholders in one process o Our methods were suitable for a wide range of stakeholders, including policy makers / advisors, health and social care professionals, members of the public, and researchers o Multiple stakeholder perspectives synthesised in one prioritisation process
Online and virtual methods – easier access for stakeholders o Connected with a broad range of stakeholders that were geographically dispersed o Online methods saved time o Adapted online methods (e.g. facilitated online workshop, British Sign Language support, palantypists) supported harder-to-reach groups to engage in the process
Rapid and recurrent process – tracking evolving priorities o Able to track the evolving COVID-19 context and stay up-to-date with important priorities, adapting our work programme when necessary o Additional fast-tracked route allowed for urgent questions to be addressed quickly
Strong links with Government teams – enabling knowledge translation and impact o Brokering relationships between the WCEC research teams and key advisors and decision makers o Expedited research to impact cycle o Helped to ensure we were conducting research that could directly inform important health and care policies
Key challenges and recommendations
Obtaining specific research questions with clear trajectory to impact o Different stakeholders have different levels of experience developing specific and answerable research questions; some questions proposed lacked specificity required o Pathway to impact not always clear – ‘interesting to know’, but would not necessarily change practice o Further training on research question development and identifying pathways to impact up-front could expedite translation of priority questions into deliverable evidence reviews or primary research, and subsequent impact.
Awareness and understanding of the Science-Policy-Practice interface o Effective research prioritisation processes must be underpinned by a strong co-productive working at the science-policy-practice interface, defined by good relationships and a mutual understanding of the role of all three domains o Training / education programmes targeted at researchers, policy makers/advisors and practice leads should cover ♣ how research questions are formulated and how different study designs can answer those questions (including the limits of the research) ♣ the range of factors that influence policy decision-making ♣ how policy is translated into health and social care settings (always considering issues of routine implementation in everyday health and care).

Broad stakeholder involvement using virtual processes

Our engagement with a wide range of stakeholders was supported by our strong links with Government teams, who were able to help us identify key stakeholders, and brokered relationships between the WCEC and key advisors and decision-makers. Whilst there are many UK and international examples of COVID-19 priority setting exercises, these typically focus on the views of practitioners and researchers and tend to be domain or topic specific; for example, nurses (23), psychological / mental health (24) primary care (25), maternal and child health, (26). To our knowledge, no other UK-based COVID-19 priority setting exercise explored and synthesised the priorities of multiple groups, which included policy, practice, and public domains.

The pandemic restricted face-to-face contact, but virtual communication methods with stakeholders proved successful; we were able to use email and virtual meeting software to invite, engage, and successfully receive priorities from the various stakeholder groups across Wales and where required internationally. The adapted virtual process (virtual facilitated priority workshops with email follow-up and various accessibility options; see methods) also proved effective for flexibly engaging members of the public and harder-to-reach groups in this process.

Tracking the evolving pandemic context with multiple rapid consultation rounds

Typically, research prioritisation exercises take longer to conduct (e.g. 12–18 months for a James Lind Alliance Priority-Setting Partnership process),(10) and take place less frequently (e.g. annually). Due to the rapidly evolving and unpredictable nature of the pandemic, we opted for multiple and regular prioritisation rounds across the WCEC lifespan. Once a prioritisation round had taken place, the average time for inclusion in the work programme was 1–3 months and as little as one week for some fast-tracked questions. This rapid and recurrent process, complemented by fast-tracked routes, proved effective in closely tracking the evolving pandemic context, from epidemiological, practice and political perspectives. This was evidenced by the way in which our evolving work programme mapped to the evolving reactive and recovery pandemic context (see previous section), and the key policy decisions being made. Our work programme also has a strong focus on COVID-19 inequalities and equity in pandemic recovery, another significant policy and practice priority.(13)

Identifying relevant questions that informed policy and practice guidance

The questions submitted by multiple stakeholders using our brief ScoPE proforma (supported by guidance notes) were highly relevant to the WCEC brief of health and social care related questions that are important in the Welsh COVID-19 context; only four of the total 223 questions submitted by stakeholders over the three rounds were deemed as ‘out of scope’ for the WCEC. Submitted and accepted questions covered the key harms of the pandemic outlined by Welsh Government, (13) and 20 of the 39 evidence outputs published to date have gone on to directly inform Welsh Government advisors on policy decision making (e.g. use of Ozone machines in educational settings (27)) and Government plans (e.g. Renew and Reform Post-16 and Transitions Plan: addressing the impacts of COVID-19 on education and training (28)). The research to impact cycle in non-pandemic periods is typically protracted, but our rapid research prioritisation process, combined with our rapid evidence and knowledge mobilisation processes (described elsewhere) and strong Government links, expedited this cycle. One example was the question about vaccine safety in pregnant and post-partum women; upon receiving the fast-tracked request, the evidence synthesis was undertaken within two weeks, and included in practice guidance for midwives within a further one week.(29) This information was synthesised into a lay summary and infographic to enable quick and effective dissemination as part of our knowledge mobilisation strategy.

Balancing comprehensiveness with rapidity and translation to practice

A rapid and pragmatic priority setting exercise was essential for our remit of ‘good questions, answered quickly’ and to ensure we met the rapidly evolving and unpredictable pandemic context. Whilst we took every effort to understand the range of relevant health and social care stakeholders in Wales and used links with policy and practice leads, we acknowledge that some key professional, third sector, and public groups might have been missed from our prioritisation process. Unlike priority setting exercises for specific health conditions, the challenge of a COVID-19 based exercise was the vast number of the population affected, and the innumerable ways in which people had been affected (e.g. health, finance, education, employment, wellbeing, environment). However, the consistency observed in topics submitted across the different stakeholder groups, including Government, health and care leads, and the public, indicates that priority topics of greatest important to the Welsh population and their services were identified.

Key challenges and recommended solutions (see Table 3)

Obtaining specific evidence needs that result in impact

We recognise that different stakeholder groups have differing levels of experience with research and developing specific and answerable research questions. At times, we received broad evidence needs not in the form of a research question, and lacked the specificity required by evidence review teams to rapidly initiate a review (e.g. specific population and outcomes of interest) and / or the trajectory to impact was unclear (i.e. questions were ‘interesting to know’, but would not necessarily change practice).

Ongoing discussions with stakeholders at virtual review team meetings helped to refine both question and identify pathway to impact, but were resource intensive. Brief (4-page) guidance was provided to stakeholders up-front, drawing upon principles of developing ‘good research questions’, but we recommend further time-investment in this early phase for future exercises. For example, this could comprise a 30–60 minute online training session for each stakeholder team before they complete the initial priority exercise, covering the principles of developing a research question, defining outcomes of interest, different types of evidence and the pathway to impact. Whilst this might be challenging to accomplish rapidly in urgent health protection situations, we believe the additional time investment up-front could be efficient overall and expedite translation of priority questions into deliverable evidence reviews or primary research, and subsequent impact.

Awareness and understanding of the Science-Policy-Practice interface

The vast number of COVID-19-relevant topics and the extensive number of relevant stakeholders spanning scientific advice, policy, health and social care, and research (as well as public perspectives) highlighted the complexity of the Science-Policy-Practice interface. The literature covers best-practices for the Science-Policy interface, but discussion about the third essential element of ‘practice’ is more limited.(30, 31) Effective research prioritisation processes must be underpinned by a strong co-productive working at the science-policy-practice interface, defined by good relationships and a mutual understanding of the role of all three domains: how research questions are formulated and how different study designs can answer those questions (including the limits of the research), the range of factors that influence policy decision-making, and how policy is translated into health and social care settings (always considering issues of routine implementation in everyday health and care). Better inter-professional education and training (30, 31) may help to foster these relationships and create the foundation necessary for effective co-produced research prioritisation, particularly when rapid processes are required. This should not be a short-term reactive approach when a public health emergency arises but part of a longer-term and ongoing plan, jointly led by government, research and health and social care bodies.

The WCEC successfully engaged a wide range of stakeholders to rapidly identify various COVID-19 research priorities that mapped to the broader COVID-19 epidemiological and political context and directly informed policy and practice decision-making. The transferable processes could be used to inform other rapid and pragmatic research prioritisation exercises, within high threat and urgency contexts. Our work highlights the importance of the critical Science-Policy-Practice interface, complemented by public involvement, in identifying evidence priorities that can lead to direct policy and practice impact.

COVID-19

SARS-CoV2 infection

ScoPE

Stakeholder evidence needs Prioritisation Exercise

TAC

Technical Advisory Cell

TAG

Technical Advisory Group

WCEC

Wales COVID-19 Evidence Centre

Ethics approval and consent to participate

In line with guidance, ethical approval was not required for this study.

Consent for publication

Our manuscript does not contain personal data and does not require consent for publication.

Availability of data and materials

All data relevant to the study are included in the article or uploaded as supplementary materials.

Competing interests

The authors have no conflicts of interest to declare.

Funding

The Wales COVID-19 Evidence Centre is funded by Health and Care Research Wales through Welsh Government.

Authors’ contributions

Natalie Joseph-Williams: Conceptualisation, data curation, formal analysis, investigation, methodology, project administration, writing – original draft, writing – review and editing, visualisation, validation

Alison Cooper: Conceptualisation, data curation, formal analysis, investigation, methodology, project administration, writing – original draft; writing – review and editing, validation

Ruth Lewis: Formal analysis, investigation, writing – review and editing, validation

Jane Greenwell: project administration, data curation, writing – review and editing, validation

Elizabeth Doe: project administration, data curation, writing – review and editing, validation

Micaela Gal: writing – review and editing, validation

Rebecca-Jane Law: investigation, writing – review and editing

Adrian Edwards: Conceptualisation, data curation, formal analysis, methodology, project administration, writing – review and editing, validation

Acknowledgements

We are grateful for the engagement and support of the various stakeholder groups in our evidence needs prioritisation process.

World Health Organization (WHO). A coordinated global research roadmap. WHO., 2020. Available from: https://www.who.int/publications/m/item/a-coordinated-global-research-roadmap.
Van Damme W, et al. The COVID-19 pandemic: diverse contexts; different epidemics – how and why? BMJ Global Health. 2020;5:e003098.
Chief Medical Officer for Wales. Restoring our health: Chief Medical Officer for Wales annual report – June 2022. Welsh Government. June 2022. Available from: https://gov.wales/sites/default/files/publications/2022-06/chief-medical-officer-annual-report-2021-to-2022_0.pdf.
Welsh Government Science Evidence Advice. Coronavirus (COVID-19) and health inequalities. October 2022. Available from: https://gov.wales/sites/default/files/publications/2022-11/science-evidence-advice-coronavirus-and-health-inequalities.pdf.
The Wales COVID-19 Evidence Centre. Available from: https://healthandcareresearchwales.org/about-research-community/wales-covid-19-evidence-centre.
Greenhalgh T, Fisman D, Cane DJ, Oliver M, Raina Macintyre C. Adapt or die: how the pandemic made the shift from EBM to EBM + more important. BMJ Evidence-Based Medicine. 2022;27:253–60.
Okello D, Chongtrakul P, COHRED Working Group on Priority Setting. A manual for research priority setting using the ENHR Strategy. Geneva: Council on Health Research for Development; 2000. Available from: http://www.cohred.org/publications/library-and-archive/a_manual_for_researc_1_0/.
Ghaffar A, Collins T, Matlin SA, Olifson S. The 3D combined approach matrix: an improved tool for setting priorities in research for health. Geneva: Global Forum for Health Research; 2009. Available from: http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf.
Rudan I, Gibson JL, Ameratunga S, et al. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croatian Med J. 2008;49(6):720–33.
James Lind Alliance. Priority Setting Partnerships. Online. Available from: https://www.jla.nihr.ac.uk/.
World Health Organisation. A systematic approach from undertaking a research priority-setting exercise: guidance for staff. Geneva: World Health Organisation. 2020. Available from: https://apps.who.int/iris/bitstream/handle/10665/334408/9789240009622-eng.pdf?sequence=1&isAllowed=y
The Health Foundation. The COVID-19 impact inquiry report. July 2021. Available from: https://www.health.org.uk/publications/reports/unequal-pandemic-fairer-recovery#:~:text=the%20pandemic%20has%20revealed%20stark,19%20than%20those%20in%20wealthiest.
Welsh Government Technical Advisory Group. Five harms arising from COVID-19: consideration of potential baseline measures. July 2021. Available from: https://gov.wales/sites/default/files/publications/2021-07/technical-advisory-group-5-harms-arising-from-covid-19_0.pdf.
Wales COVID-19 Evidence Centre. Work Programme. Accessed. December 2022. Available from: https://healthandcareresearchwales.org/sites/default/files/2022-11/Work_programme_October_2022.pdf.
Welsh Parliament. Coronavirus timeline: Welsh and UK governments’ response (period December 2019 – May 2021). Available from: https://research.senedd.wales/research-articles/coronavirus-timeline-welsh-and-uk-governments-response/.
Fisayo T, Tsukagoshi S. Three waves of the COVID-19 pandemic. Postgrad Med J. 2021;97:332.
Welsh Parliament. Coronavirus timeline: Welsh and UK governments’ response (period June 2021 – September 2022). Available from: https://research.senedd.wales/research-articles/coronavirus-timeline-the-response-in-wales/.
British Medical Association. BMA Covid Review 3: Delivery of healthcare during the pandemic. June 2023. Available from: https://www.bma.org.uk/media/5816/bma-covid-review-report-3-june-2022.pdf.
World Bank. 2022. Chapter 1. The economic impact of the COVID-19 crisis. In, World Development Report 2022: Finance for an Equitable Recovery. Washington, DC: World Bank. 10.1596/978-1-4648-1730-4. Available from: https://www.worldbank.org/en/publication/wdr2022/brief/chapter-1-introduction-the-economic-impacts-of-the-covid-19-crisis.
Evans AB, Akbari A, Bailey R, Bethell L, Bufton S, Carson-Stevens A, et al. Evaluation of the shielding initiative in Wales (EVITE Immunity): protocol for a quasiexperimental study. BMJ Open. 2022;12:e059813.
Harrop E, Goss S, Farnell D, Longo M, Byrne A, et al. Support needs and barriers to accessing support: Baseline results of a mixed-methods national survey of people bereaved during the COVID-19 pandemic. Palliat Med. 2021;35(10):1985–97.
UK COVID-19 Inquiry. Available from: https://covid19.public-inquiry.uk/.
Manning JC, Bramley L, Coad J, Evans C, Evans K et al. Priorities for research during the Coronavirus SARS-CoV-2 (COVID-19) pandemic and beyond: a survey of nurses, midwives and health visitors in the United Kingdom. J Res Nurs Online First Volume 26, Issue 5. https://doi.org/10.1177/17449871211018737.
O’Connor DB, Aggleton JP, Chakrabarti B, Cooper CL, Creswell, et al. Research priorities for the COVID-19 pandemic and beyond: A call to action for psychological science. Br J Psychol. 2020;11(4):603–29. https://doi.org/10.1111/bjop.12468.
Hubbard G, Grist F, Pope LM, Cunningham S, Maxwell M, Bennie M. Survey to identify research priorities for primary care in Scotland during and following the COVID-19 pandemic. BMJ Open. 2022;12(5):e056871. 10.1136/bmjopen-2021-056817.
Etti M, Alger J, Salas SP, Saggers R, Ramdin T, et al. Global research priorities for COVID-19 in maternal, reproductive and child health: Results of an international survey. PLoS ONE. 2021;16(9):e0257516. https://doi.org/10.1371/journal.pone.0257516.
Wales COVID-, 19 Evidence Centre. The efficacy, effectiveness and safety of SARS-CoV-2 disinfection methods (including ozone machines) in educational settings for children and young people. 2021. Available from: https://healthandcareresearchwales.org/efficacy-effectiveness-and-safety-sars-cov-2-disinfection-methods-including-ozone-machines.
Welsh Government. Renew and Reform Post-16 and transitions plan: addressing the impact of COVID-19 on education and training. 2022. Available from: https://www.gov.wales/sites/default/files/publications/2022-03/renew-and-reform-post-16-and-transitions-plan.pdf.
Wales COVID-, 19 Evidence Centre. What is the evidence of direct harm from COVID-19 infection and COVID-19 vaccine in pregnant / post-partum women and the unborn child? 2021. Available from: https://healthandcareresearchwales.org/what-evidence-direct-harm-covid-19-infection-and-covid-19-vaccine-pregnantpost-partum-women-and.
Sokolovska N, Fecher B, Wagner GG. Communication on the science-policy interface: An overview of conceptual models. Publications. 2019;7(4).
Maas TY, Pauwelussen A, Turnhout E. Co-producing the science–policy interface: towards common but differentiated responsibilities. Humanit Social Sci Commun. 2022;9(93).

SupTable1.docx

Download PDF

Reviewers agreed at journal
09 Nov, 2023
Reviewers invited by journal
24 Oct, 2023
Editor assigned by journal
23 Aug, 2023
First submitted to journal
22 Aug, 2023

You are reading this latest preprint version

Working with stakeholders to rapidly identify and prioritise COVID-19 health and social care evidence needs for evidence-informed policy and practice decision making: processes, results, and lessons from the Wales COVID-19 Evidence Centre

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

1. Background

2. Methods

2.1 Identifying key stakeholders

2.2 Stakeholder evidence needs Prioritisation Exercise (ScoPE)

2.3 Fast-tracked urgent priorities

2.4. Categorising the questions

2.5 Assessing questions for importance, impact, and urgency

2.6 Agreeing the work programme

2.7 Patient and public involvement statement

3. Results

3.1 Stakeholders

3.2 Stakeholder evidence needs Prioritisation Exercise (ScoPE) – questions proposed

3.3 Fast-tracked urgent priorities included in our work programme

3.4 Work Programme Summary – questions accepted

3.5 Question categories

3.6. Evolving priorities mapped to the broader COVID-19 context

Discussion

Conclusion

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1