Evidence to practice – lessons learnt in developing an implementation strategy for an online digital health intervention (Eczema Care Online)

doi:10.21203/rs.3.rs-3415611/v1

Download PDF

Research Article

Evidence to practice – lessons learnt in developing an implementation strategy for an online digital health intervention (Eczema Care Online)

https://doi.org/10.21203/rs.3.rs-3415611/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background – Research is not often readily taken up in healthcare, which results in research waste. Eczema Care Online (www.EczemaCareOnline.org.uk/) has been shown in randomised controlled trials to be an effective online self-management intervention to improve eczema outcomes. Implementation requires adoption at patient/family, clinician and organisational levels. This paper aims to describe the lessons learnt from developing and delivering an implementation strategy for an online digital health intervention for eczema self-management.

Methods – Data from systematic reviews, stakeholder consultation meetings, interviews with trial participants, intervention usage data during the trial, and existing eczema information websites informed our implementation plan. Using Normalisation Process Theory, an implementation plan combined these findings with practical, context-specific actions to encourage wider adoption of the intervention.

Results – Barriers and facilitators to implementation were identified. Key lessons learnt include 1) start implementation work early 2) maintain flexibility to explore multiple routes to implementation 3) use secondary data sources 4) balance theory with practicalities 5) consider longer-term maintenance beyond the life of the research project.

Conclusion – Implementation planning is a key stage of the research process that is often not adequately resourced. Implementation planning ensures effective interventions developed and evaluated in research studies are utilised in everyday practice.

digital health intervention

implementation

atopic eczema

normalisation process theory

The implementation challenge.

Research has the potential to improve healthcare, but the reality is that new ideas are often not taken up in healthcare settings [1, 2]. This lack of implementation is often referred to as the ‘implementation gap’. There have been numerous calls to reduce avoidable research waste across the lifecycle of research [3, 4]. To improve the chances of an innovation being adopted, research communities are increasingly engaging in implementation science [5]. A related body of literature uses the term ‘knowledge mobilisation’, which has been described as “the process of moving knowledge to where it can be most useful” [5].

Normalisation Process Theory (NPT) was developed to provide a theory for considering implementation challenges and it suggests “complex interventions become routinely embedded (implemented and integrated) in their organisational and professional contexts as the result of people working, individually and collectively, to implement them” [6]. The theory outlines four key processes, which are included in Box 1.

Box 1. Normalisation Process Theory (NPT) four key processes [6].

Despite the growing recognition that intervention developers need to invest resources into implementation, efforts are not regularly documented or reflected upon. Where we found papers outlining strategies, this provided a valuable resource for creating our own plan [7-9]. We want to offer lessons learnt from our own experiences with implementing Eczema Care Online that can be applied to other digital health implementation efforts.

Eczema Care Online

People often receive little advice on how to manage their or their child’s eczema and obtain variable advice from the internet [10, 11]. Eczema Care Online is the result of an evidence, theory and person-based approach to developing two interventions for eczema self-management [12, 13]. One intervention was aimed at parents/carers of children with eczema (0-12 years), and another aimed at young people (13-25 years) who are starting to manage their eczema themselves [12, 13]. Randomised controlled trials recruiting from primary care practices across the UK demonstrated that the Eczema Care Online interventions had a small but sustained improvement in eczema symptoms compared to usual care [14]. Nested qualitative interviews suggest that users found the interventions easy to use, relatable, and trustworthy, and perceived that the intervention helped them to better manage their eczema [15]. A quantitative process evaluation concluded that engagement was high, and the intervention required minimal time commitment [16]. Health economic analysis suggests cost-effectiveness [17]. Together, this evidence suggests that adoption of this intervention in healthcare or community settings could have benefits for people with eczema. Both interventions were combined into one website for implementation: www.EczemaCareOnline.org.uk.

Self-management digital interventions such as Eczema Care Online bring with them distinct implementation challenges [7]. Successful implementation of Eczema Care Online involves both 1) individuals with eczema, or parents/carers of children with eczema, being able and willing to use and engage with the intervention as well as 2) healthcare professionals being able and willing to advocate for the use of the intervention. Uptake of Eczema Care Online requires adoption at patient/family, clinical, and organisational levels.

Aim: To describe the lessons learnt from developing and delivering an implementation strategy for an online digital health intervention for eczema self-management.

Objectives:

To illustrate the value of using existing data to support implementation.
To illustrate the importance of using a theoretical framework to support implementation.
To describe the lessons learnt for other research teams wanting to invest in implementation within their programme of research.

Guided by the work of Ross et al. [7], we developed our implementation strategy by both drawing on a range of information sources and using NPT as a theoretical framework for implementation. We then combined these findings with specific contextual knowledge to produce an actionable plan.

Data used to inform implementation.

Data source 1: Incorporating new evidence.

Parallel to developing and testing the interventions, a Cochrane systematic literature review was conducted on strategies for using topical corticosteroids to ensure the intervention would be implemented with content based on the most up to date evidence. This evidence was compared to existing UK eczema guidelines to help assess what evidence from the review provided new insight [18-20]. Key messages about eczema treatments were created based on the most up to date and high-quality literature, and these were taken to stakeholders to review.

Data source 2: Stakeholder consultation

Stakeholder consultation took place throughout the 5-year research programme. Key meetings where we gained key insights for implementation include:

A 1-day face-to-face meeting with 34 attendees including researchers (n=11), healthcare professionals (n=13), patient partners/organisations (n=10) (Sept 2019). This meeting involved reviewing preliminary Cochrane systematic review findings.
Five 90-minute online meetings where particular stakeholder views were invited from primary care (n=4), secondary care (n=7), pharmacy (n=5), people with eczema/patient organisations (n=7), and parents of children with eczema/patient organisations (n=7) (March 2021). These were to refine key messages about eczema treatments for the intervention and decide how to engage stakeholders with the intervention.
An additional 2-hour online meeting with 18 individuals or patient organisation representatives gathered views from people concerned about a safety concern of eczema treatments termed ‘topical corticosteroid withdrawal’ (May 2021).
A 1-day results reveal meeting (16 in person and 16 online attendees) including researchers, healthcare professionals, patients, and patient organisations (April 2022) where implementation actions were a key discussion point.

Data source 3: Interview data

Process evaluation interviews with trial participants randomised to both the intervention and control group were nested within the randomised controlled trial design, and the methods and findings from a primary analysis of these qualitative data has been reported in detail elsewhere [15]. To inform implementation, we considered these individuals as potential users of the Eczema Care Online website. The topic guide explored participants’ experiences of the intervention (for the intervention group only) and their views about different ways of accessing the online information for eczema. Secondary analysis of the qualitative process evaluation dataset (20 young people and 20 parents) was deductively coded using Framework Analysis (by LH, discussed regularly with PL) to the four core NPT constructs [21]. These findings were presented and discussed with the wider team.

Data source 4: Usage statistics

Via the online software used to deliver the interventions in the randomised controlled trials, individuals’ interactions with the intervention were recorded automatically. This provided usage data from 171 parents and carers using the intervention and 168 young people using the intervention. This allowed us to see which aspects were being used most frequently and guided our decision about layout and content of the intervention for implementation.

Data source 5: Existing eczema websites

To understand how Eczema Care Online fits within the landscape of currently available online information for eczema, we identified what other online eczema resources were available. We looked for a) websites or webapps providing information about the cause, management, treatment or living with eczema b) both public facing and targeted to professionals c) freely accessible to members of the public. We did not consider any resources that required registration or payment to access. Resources were identified as websites of key stakeholders we were aware of previously, or through searching www.google.co.uk with country filter UK between 3^rd-9^th Feb 22 using a variety of simple queries relating to eczema management (to replicate how an average google user may find resources). This search was not intended to be comprehensive or replicable. Two researchers (LH, PL or NR) independently reviewed website content and provided a subjective rating of whether the website presented itself as having the same ‘unique selling points’ we believed Eczema Care Online had (Figure 1). The three researchers met to discuss the independent ratings and agreed a final rating, with the third researcher who had not rated the website acting as arbitrator.

Figure 1 Value Propositions

Theoretical Framework

We created a theoretical framework that articulates how NPT principles relate to our needs in this specific context, which is outlined in Figure 2.

Figure 2 NPT informed framework

Development of an action plan

To develop an action plan for implementation, it required a negotiation between insight from the data interpreted via the NPT lens and practical and context specific requirements. For example, the data could identify that the National Health Service (NHS) was a ‘trusted source’ of information (support for NPT construct collective action), but it required practical considerations about which NHS sources were feasible and relevant to embed an eczema online intervention.

This results section synthesises how our approach developed learning about the barriers and facilitators for implementation relating to the four NPT constructs. We have numbered throughout which data sources were the main source for informing this learning. We outline for each construct how this learning informed our planned actions for implementation.

Coherence

Learning:

Eczema Care Online is offering something distinct and is valued by users and healthcare professionals (Data sources 2, 3, 5). For example, one user said:

“The ECO website is great, like I said, because it looks at the family as a whole and your life as a whole and the whole – management of it in your life […] This is very much part of your life, this is how we can help you, like the whole nursery stuff and the school stuff, like – that’s not on the NHS website” – Parent, aged 34, female, White British

However, the distinctions from other sources are subtle, and marketing needs to emphasise them (Data sources 2, 3, 5). Marketing literature proposes a ‘value proposition’ plays a critical role in communicating how a company aims to provide value to customers [22].

Actions:

We defined value propositions using features that NPT analysis suggested were distinctive and valuable about Eczema Care Online (Fig. 1). The ‘unique selling point’ of Eczema Care Online, compared to other eczema websites reviewed (data source 5), was that it is ‘proven to help people with eczema’. Marketing materials emphasised the value propositions. For example, we created an animation (https://www.bmj.com/content/379/bmj-2022-072007#media-1) and social media advertisements that highlighted value propositions (Fig. 3).

Cognitive Participation

Learning:

Users were generally willing to engage with the platform and found it easier than other routes to access information about eczema (e.g., challenges accessing GP, overwhelmed by information via search engines) (Data sources: 3, 4).

The Cochrane review demonstrated some evidence that is not in existing guidelines and should be incorporated in Eczema Care Online to ensure up to date evidence is used (Data source: 1). There was a concern from some stakeholders that Eczema Care online could ‘add to noise’ in an online space already saturated with ‘mixed messages’ and we need to be careful not to undermine other trusted sources accessed by healthcare professionals and patients (e.g., guidelines, manufacturer packaging, healthcare professional advice to patients) (Data source: 2). This highlights how we need to update messaging within Eczema Care Online, while also encouraging up to date evidence to be included in other information sources. Healthcare professionals also needed easy tools to incorporate the messages into their practice, with electronic systems in primary care being increasingly important (Data source: 2).

Actions:

We used a general principle to not contradict messaging from other key sources, unless it was considered important to ensure evidence-based information was being provided. To influence messaging in other sources, we lobbied key places (e.g., UK guidelines) and disseminated Cochrane review findings widely to encourage change in practice across the board. To ensure messaging from healthcare professionals was synergistic with Eczema Care Online, we developed and circulated within primary care (practices included in the trial), as well as some pharmacies and secondary care settings ‘two treatments used well’ booklets (Fig. 3) so healthcare professionals could share Eczema Care Online messages directly in consultations. We also contacted providers of electronic system providers so that links to Eczema Care Online could be accessed by primary care staff as well as sent in SMS to patients. We contacted training providers for healthcare professionals so Eczema Care Online could be embedded into training materials. We also identified secondary care ‘champions’ who are trusted sources of information for both secondary care and primary care colleagues. They could spread the word about Eczema Care Online in presentations and communications. We developed template wording for healthcare professionals working in secondary care to add to ‘advice and guidance’ letters to primary care clinicians and made this available on our website.

Collective Action

Learning:

Ease of use is important to people, and there were some aspects of the original intervention that needed updating to create an easier experience (Data sources: 2, 3). For example, some people found the layout of the website confusing and reported struggling to find the content they were interested in. There was also a desire for it to be immediately accessible to remove barriers to access (e.g., no log in, which was required during the trials).

Both branding and where people hear about the website are important to portray legitimacy. Stakeholder and user feedback suggested having a tool that can help healthcare professionals signpost patients and parents/carers to further information would improve accessibility and trust. Sometimes, people were happy to access information in less trusted but more convenient places (e.g., social media) as long as the branding portrayed legitimacy. Here one user talks about the importance of GP recommendation regarding advice:

“I think for, you know, people seek about treatments and stuff, mainly through from their GP, because they trust their GP and they trust their doctors to give them advice that’s good for them. So having it through the NHS would be useful because people will trust the website, firstly, […] and will feel like, okay, well this is legit; they’re not giving them anything that could be wrong for them or could, you know, make their condition worse. […] So I think – yes – the one thing is that people trust doctors and having it maybe, you know, people talking about it maybe on social media, so it makes it a bit more – a bit more – reachable.” (Young person aged 24, female, Indian)

ACTIONS:

In re-designing the website for implementation, we addressed users’ reported barriers to use. We ensured branding and logos of involved organisations were clear on the homepage and in advertising materials. We contacted numerous legitimate resources to embed the Eczema Care Online link into (e.g., NICE guidelines, professional body websites, eczema charity websites). To support engagement from trusted sources, we created a concise summary we could easily share based on the NICE evidence summary framework to emphasise the legitimacy of the intervention (Fig. 3) [23].

Reflexive Monitoring

Learning:

In terms of delivery at organisational level, trial data showing evidence of effectiveness of Eczema Care Online helps clinicians and organisations feel confident that it works and supports ongoing usage and maintenance of delivery (Data source: 2). Users within the trial reported finding filling out the trial questionnaires that monitored their symptoms helpful to see if the changes they are making as a result of the website are working for their eczema (Data source: 3) and would value some way of monitoring their eczema when the intervention is implemented:

I think – like – just the psychology of being part of the study has just been – in some ways helpful, even just doing the questionnaires, I found quite helpful because it makes me stop and assess how she’s doing and I don’t know that I would do that in quite the same way in – you know – in a different context. So I think just kind of being prompted to think about it on a regular basis, is quite helpful.

ACTIONS:

Trial findings that Eczema Care Online improves eczema were included as a value proposition and emphasised to support ongoing usage. Further evidence is needed to understand whether organisations and clinicians feel it works well when used alongside their current practice and how to support this monitoring process. To allow users to engage in ongoing monitoring outside of a trial setting, we re-designed the website to include a link to My Eczema Tracker (https://www.nottingham.ac.uk/research/groups/cebd/resources/my-eczema-tracker-app.aspx), which is an app where users can track their eczema.

Box 2 summarises the key lessons learnt from this process.

Compiling data and reviewing it through an NPT lens allowed for learning that informed our key actions. Our considerable efforts to plan for implementation throughout the 5-year research programme has resulted in some successful implementation outputs. Approximately one year from launching the website it had been accessed by over 20,000 people from 153 countries around the world (as of Aug 2023). Links to our website have been embedded in multiple eczema charity websites, professional bodies websites, healthcare professional training resources, and resources used in the NHS (e.g., primary care electronic templates by Ardens [24]. The website is referenced on the NICE ‘information for the public’ webpage that supports the ‘Atopic eczema in under 12s: diagnosis and management’ clinical guideline [25].

Strengths and weaknesses

Implementation has been central to research team discussions throughout the project lifespan and there was a commitment to implementation of our final website within the team. However, this is not the traditional role of a research team delivering a clinical trial and so there were times of uncertainty about the best approach to take. For example, we chose to use NPT, but there are other implementation frameworks that we could have used. NPT was flexible, but there is also a fluidity of the concepts which can make it difficult to disaggregate the key implementation steps. We also needed to think of NPT on multiple levels of adoption (user level, healthcare practitioner level and organisational level). Some of the team also found the language of NPT difficult to understand, and so we created our own adapted version of the framework to help it work more specifically for our context and team (Fig. 2).

The project involved wide stakeholder consultation, for example we held a stakeholder meeting to address those with topical corticosteroid concerns. However, the healthcare professionals, organisations and potential users who were involved in the data informing organisation were somewhat engaged in the process, and therefore we may be missing views of important groups of people who are less likely to engage in research.

Another example is that the team lacked marketing and commercialisation experience. There was also a desire within the team to ensure that the intervention (that was developed and tested using public funding) remained free to access to the public, which can be a challenge in a financially driven health market. Whilst we have had some implementation success to date, ongoing success is contingent on resources and having funding to maintain a free at point of use intervention [26].

Research projects that aim to develop an online health intervention should consider implementation early to help mitigate against the intervention falling into the ‘implementation gap’ which often happens due to lack of resources and strategy to take the idea from research to practice. Implementation requires a flexible approach, and whilst it is helpful to draw on available data and theoretical frameworks to inform strategy, understanding of the context specific requirements and practical constraints is necessary to make a plan that will be feasible and achievable.

NHS – National Health Service

NICE – National Institute of Health and Care Excellence

NIHR – National Institute of Health Research

NPT- Normalisation Process Theory

Ethics approval and consent to participate

Ethical approval was granted for participant data collection included in this article by South Central–Oxford A Research Ethics Committee (19/SC/0351). The University of Nottingham Faculty of Medicine & Health Sciences Research Ethics Committee confirmed that stakeholder consultation activity did not constitute research participation and did not require ethics committee review (FMHS 134-1220).

Consent for publication

Not applicable. No identifiable information was presented.

Availability of data and materials

Authors will consider reasonable request to make relevant anonymised participant level data available.

Competing interests

LH has received consultancy fees from the University of Oxford on an educational grant funded by Pfizer, unrelated to the submitted work. SLan is funded by a Wellcome Senior Clinical fellowship. SLan is an investigator on the European Union Horizon 2020-funded BIOMAP Consortium (http://www.biomap-imi.eu/), but not in receipt of industry funding. The other authors have no competing interests to declare.

Funding

This study presents independent research funded by the National Institute for Health and Care Research (NIHR) under its Programme Grants for Applied Research programme (grant ref No RP-PG-0216-20007). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The entire programme of work will be co-published in the NIHR journal as a final funding report.

Authors' contributions

LH, KG, KS, MSt conducted the participant interviews. SW and JHo contributed to trial and programme delivery. LH, JHa, SJL conducted the systematic review. LH, PL, KT, MSa, IM, AR, AA facilitated or supported the stakeholder workshops. AR and AA provided patient & public involvement. HCW, Slaw, SLan provided clinical involvement. NR provided data visualisation and design work. LH and PL conducted the NPT analysis. MSa, KT, IM managed the research team and provided strategic direction. All authors were involved in the design of the work. All authors read and approved the final manuscript.

Acknowledgments

We thank all the stakeholders who engaged in consultation to support the development of our implementation strategy. Thank you to the research participants who took part in the qualitative interviews. Thank you to the wider Eczema Care Online study team. Thank you to Kerry Oliver (PRIMIS Managing Director) and Dr Sarah Rodgers (PRIMIS Head of Research Services) for their advice. Thank you to the East Midlands Academic Health Sciences Network (AHSN) for their guidance. Thank you to the wider Eczema Care Online team for their contributions to the programme of research, including a thank you to Dr Jo Chalmers for her input to design and conduct of the implementation project.

Ashrafzadeh S, et al. Using Implementation Science to Optimize the Uptake of Evidence-Based Medicine into Dermatology Practice. Journal of Investigative Dermatology; 2019.
May CR, et al. Development of a theory of implementation and integration: Normalization Process Theory. Implement Sci. 2009;4(1):1–9.
Williams HC. Avoidable research waste in dermatology: what is the problem? Br J Dermatol. 2022;186(3):383–5.
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. The Lancet. 2009;374(9683):86–9.
Ward V. Why, whose, what and how? A framework for knowledge mobilisers. Evid Policy. 2017;13(3):477–97.
Murray E, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8(1):63.
Ross J, et al. Developing an implementation strategy for a digital health intervention: an example in routine healthcare. BMC Health Serv Res. 2018;18(1):1–13.
Lloyd A, et al. Patchy ‘coherence’: using normalization process theory to evaluate a multi-faceted shared decision making implementation program (MAGIC). Implement Sci. 2013;8:1–9.
Band R, et al. Intervention planning for a digital intervention for self-management of hypertension: a theory-, evidence- and person-based approach. Implement Sci. 2017;12(1):25.
Santer M, et al. You don't know which bits to believe’: qualitative study exploring carers’ experiences of seeking information on the internet about childhood eczema. BMJ open. 2015;5(4):e006339.
Teasdale E, Muller I, Santer M. Carers' views of topical corticosteroid use in childhood eczema: a qualitative study of online discussion forums. Br J Dermatol. 2017;176(6):1500–7.
Greenwell K, et al. Eczema Care Online: development and qualitative optimisation of an online behavioural intervention to support self-management in young people with eczema. BMJ open. 2022;12(4):e056867.
Sivyer K, et al. Supporting families managing childhood eczema: developing and optimising eczema care online using qualitative research. Br J Gen Pract. 2022;72(719):e378–89.
Santer M et al. Eczema Care Online behavioural interventions to support self-care for children and young people: two independent, pragmatic, randomised controlled trials. BMJ, 2022. 379.
Greenwell K et al. Eczema shouldn’t control you; you should control eczema: qualitative process evaluation of online behavioural interventions to support young people and parents/carers of children with eczema. Br J Dermatol, 2022.
Greenwell K et al. Online behavioural interventions for children and young people with eczema: Quantitative evaluation. manuscript in review.
Sach T. and, Cost effectiveness of two online interventions supporting self-care for eczema for parents/carers and young people. manuscript submitted.
National Institute for Health and Care Excellence. Atopic eczema in under 12s: diagnosis and management. 2007 [cited 2022 11th November]; Available from: https://www.nice.org.uk/guidance/cg57.
SIGN. Management of atopic eczema in primary care. 2011 [cited 2022 11th November]; Available from: https://www.sign.ac.uk/our-guidelines/management-of-atopic-eczema-in-primary-care/.
British Association of Dermatologists. Atopic Eczema. 2020 [cited 2021; Available from: https://www.bad.org.uk/pils/atopic-eczema/.
Ritchie J, Spencer L. Qualitative data analysis for applied policy research, in Analyzing qualitative data. Routledge; 2002. pp. 187–208.
Payne A, Frow P, Eggert A. The customer value proposition: evolution, development, and application in marketing. J Acad Mark Sci. 2017;45(4):467–89.
National Institute for Health and Care Excellence. Evidence standards framework (ESF) for digital health technologies. 2022 [cited 2023 15th September]; Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/evidence-standards-framework-for-digital-health-technologies#how-use.
Ardens Health Informatics Ltd. [cited 2023 25th August]; Available from. Ardens Health Informatics Ltd.; 2023.
National Institute for Health and Care Excellence. Atopic eczema in under 12s: diagnosis and management. 2023 [cited 2023 25th August]; Available from: https://www.nice.org.uk/guidance/cg57/ifp/chapter/More-information.
Santer M, et al. How to make Eczema Care Online freely available. BMJ. 2022;379:o2973.

Competing interest reported. LH has received consultancy fees from the University of Oxford on an educational grant funded by Pfizer, unrelated to the submitted work. SLan is funded by a Wellcome Senior Clinical fellowship. SLan is an investigator on the European Union Horizon 2020-funded BIOMAP Consortium (http://www.biomap-imi.eu/), but not in receipt of industry funding. The other authors have no competing interests to declare.

Download PDF

Editorial decision: Major revision
13 Oct, 2023
Editor assigned by journal
13 Oct, 2023
Submission checks completed at journal
13 Oct, 2023
First submitted to journal
06 Oct, 2023

You are reading this latest preprint version

Evidence to practice – lessons learnt in developing an implementation strategy for an online digital health intervention (Eczema Care Online)

Status:

Version 1

Abstract

Figures

Introduction

The implementation challenge.

Eczema Care Online

Methods

Data used to inform implementation.

Theoretical Framework

Development of an action plan

Results

Coherence

Cognitive Participation

Collective Action

ACTIONS:

Reflexive Monitoring

ACTIONS:

Discussion

Strengths and weaknesses

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1