Leaders'&nbsp;Perspectives on Learning Health Systems: a qualitative&nbsp;study

doi:10.21203/rs.3.rs-35988/v4

Download PDF

Research article

Leaders' Perspectives on Learning Health Systems: a qualitative study

https://doi.org/10.21203/rs.3.rs-35988/v4

This work is licensed under a CC BY 4.0 License

Journal Publication

published 26 Nov, 2020

Read the published version in BMC Health Services Research →

You are reading this latest preprint version

Background: Integrated utilisation of digital health data has the power to transform healthcare to deliver more efficient and effective services, and the learning health system (LHS) is emerging as a model to achieve this. The LHS uses routine data from service delivery and patient care to generate knowledge to continuously improve healthcare. The aim of this project was to explore key features of a successful and sustainable LHS to inform implementation in an Academic Health Science Centre context.

Methods: We purposively identified and conducted semi-structured qualitative interviews with leaders, experienced in supporting or developing data driven innovations in healthcare. A thematic analysis using NVivo was undertaken.

Results: Analysis of 26 interviews revealed five themes thought to be integral in an effective, sustainable LHS: (1) Systematic approaches and iterative, continuous learning with implementation into healthcare contributing to new best-practice care; (2) Broad stakeholder, clinician and academic engagement, with collective vision, leadership, governance and a culture of trust, transparency and co-design; (3) Skilled workforce, capability and capacity building; (4) Resources with sustained investment over time and; (5) Data access, systems and processes being integral to a sustainable LHS.

Conclusions: This qualitative study provides insights into the elements of a sustainable LHS across a range of leaders in data-driven healthcare improvement. Fundamentally, an LHS requires continuous learning with implementation of new evidence back into frontline care to improve outcomes. Structure, governance, trust, culture, vision and leadership were all seen as important along with a skilled workforce and sustained investment. Processes and systems to optimise access to quality data were also seen as vital in an effective, sustainable LHS. These findings will inform a co-designed framework for implementing a sustainable LHS within the Australian healthcare and Academic Health Science Centre context. It is anticipated that application of these findings will assist to embed and accelerate the use of routine health data to continuously generate new knowledge and ongoing improvement in healthcare delivery and health outcomes.

Health Economics & Outcomes Research

Health Policy

Learning health system

informatics

data-driven healthcare

evidence

healthcare improvement

There is a growing interest in how best to use evidence and health data to inform decision making in healthcare delivery [1]. Systems are required to ensure that the most relevant and current information is used to guide decisions related to healthcare delivery [2, 3]. Improved health outcomes will require informed decision making at all levels of healthcare, including decisions made by policy makers, hospital executives, clinicians and by patients themselves [4, 5]. Improving the ability to access, and understand, quickly visualise and compare high quality data related to patient care is known to enable improvements in health outcomes [4, 6]. However, only a minority (less than fifteen percent) of healthcare organisations worldwide are adept in data-driven processes to support informed decisions at point of care, as reported in a large study involving over 700 health care leaders [7]. The Learning Health System (LHS) has been proposed as a framework for incorporating best practice and taking a systems approach [3, 8-10]. The LHS involves processes that generate and apply the best available evidence and use data to support collaborative healthcare decision making with patients and clinicians. It drives the natural integration of research into high quality patient care and practice and promotes innovation, quality, safety and value in healthcare [1].

In a recent systematic review, at least 23 LHS environments were identified globally that are translating data-driven evidence into clinical practice [11]. These LHS were co-developed with many stakeholders (including multidisciplinary teams of frontline clinicians, researchers and community members), provide timely data access and analysis and require a diverse skilled workforce to make sense of the data arising from complex healthcare environments [11]. These LHS reported having direct health impact, and are therefore suggested as the necessary environments to shape the healthcare systems of the future.

In implementing complex systems level interventions, such as in a LHS, there are a myriad of frameworks and models, as well as barriers and enablers known to impact uptake in health care [1]. Key factors include leadership commitment and behaviours [12-14], technology acceptance, timing of new innovations and technology [15, 16], innovation cultures [17], cross sector integration and readiness to change [18], and occupational stress [19]. Relevant disciplines and expertise include consumer and stakeholder engagement, data technology and adoption, implementation and complexity science and health care improvement. The literature in these areas is broad, applies inconsistent terminology, encompasses overlapping theories and concepts, and often lacks focused and pragmatic insights to implementation within complex health care settings. To address the gap, in systems level approaches to the LHS, available evidence on the common elements in ‘theories/frameworks/models’ required for complex intervention implementation in health care has been summarised by Melder et al [1]. Here the common elements required were: (1) clinician and stakeholder engagement and clinician leadership; (2) access to the best available evidence and measurement; (3) rigorous yet pragmatic processes; and (4) application of iterative models that learn from successes and failures [1].

The Australian Health Research Alliance (AHRA) is a collaboration of ten Research Translation Centres, including three Regional Centres in Australia [20] accredited by the National Health and Medical Research Council. These Centres are modelled on international Centres including Academic Health Science Centres and Applied Research Centres in the UK [21]. AHRA, across these Centres covers over 90% of government-funded health and medical researchers in Australia, and over 80% of hospitals with significant primary care engagement. AHRA has developed three priority areas for Data-Driven Healthcare Improvement activity [20]. One is to create health data hubs to stimulate partnerships across academic, clinician and other stakeholders. Monash Partners, an Australian Academic Health Science Centre in Melbourne Australia, led the development of the Learning Health System (LHS) framework through a systematic review [11] and other key literature [1], qualitative research and co-design workshop with the goal of developing a network of sustainable LHS in Australia.

Although definitions vary, including those that identify an LHS as having operational precision medicine capabilities at point of care [8], here we took a broader definition, informed by broad expert stakeholder needs across community, healthcare professionals and managers, data and information technology experts, academics and policy makers. We defined the LHS as a system in which routine data, from service delivery and patient care, can lead to iterative cycles of knowledge generation and improvement in healthcare, as a result of daily practice [22, 23]. Despite the availability of big data from healthcare, little is known about how to create an effective, sustainable LHS that stimulate partnerships across different stakeholders, to utilise data iteratively for better healthcare and health outcomes. The aim of this project was to explore core elements of a sustainable LHS to inform development of a framework for LHS in Australia.

Design:

This was a qualitative study using semi-structured interviews. The study was methodological informed by Bhaskar's (2008) critical realism [24]. Qualitative research is compatible with a critical realist inquiry [25] as it seeks to explain and understand complexity, and views the experiences and perspectives of people as ‘real’ [26]. The design was a purposive, qualitative study informed by an expert Academic Health Science Centre committee and a National AHRA committee, who had identified that a wealth of relevant information could be sourced from leaders who have experience of supporting or developing data innovations in healthcare nationally and internationally. The study design was also informed by the previously mentioned evidence synthesis on elements required for complex health care interventions [1] and evidence from our recent systematic literature review [11].

This study was approved by the Monash University Human Research Ethics Committee (Project ID: 19969).

Setting:

The Commonwealth of Australia has a population of 26 million and is highly urbanised with almost 90% of people residing in cities of 100,000 people or more. Australians are covered by a tax-financed universal public health insurance scheme, which provides rebates against the cost of medical fees. Private healthcare insurance is also available and the multi-payer health system is high quality but complex [27]. It is managed by different levels of Australian government (federal, state and territory, and local) and private providers to deliver (public and private) services through a range of funding and regulatory mechanisms [28]. Although complex, the Australian health system has relatively good outcomes and has been ranked as the second best health system (after the United Kingdom) by the Commonwealth Fund’s International Health Policy survey [27, 29]. However, there is room for improvement, especially as the ageing population increases, and demand on the system is predicted to exceed capacity in the coming decades [28]. Waiting times for some elective procedures in public hospitals are considered too long, with up to 15% of patients waiting over twelve months in 2015-16 [28, 30]. Inefficiency, waste and provision of low value care are recognised in the Australian health system, and are estimated to account for 30-40% of national health costs, through ineffective health interventions, administrative inefficiencies and inefficient pricing [31, 32]. Quality and safety considerations are also recognised with up to 17% of total hospital activity and expenditure related to adverse events; and one in ten patients will experience some form of harm while in the health system (such as infections, medication complications, delirium and cardiac complications) estimated to cost 9% of total hospital expenditure [32-34].

The Australian government now requires research to be embedded in health care and have charged the Academic Health Science Centres to drive this as well as establishing a National Clinical Trials Governance Framework and including research in hospital accreditation metrics [35]. AHRA Research Translation Centres as described earlier, have collectively prioritised data-driven, system-level and embedded initiatives such as the LHS, to address gaps in evidence translation, to meet government requirements and to assist health care settings to build quality research [20].

Procedures and data collection:

A purposive sample of leaders from a range of clinical and health informatics areas, who had experience of creating or developing data hubs, were invited to take part in a semi-structured interview. The interview guide was developed for this study (Figure 1). Leadership roles in managing data, as well as overseeing other leaders responsible for collection of clinical and health data were targeted. Purposive sampling also included those in a leadership role in healthcare delivery and/or health services research nationally or internationally. To maintain confidentiality of those interviewed, demographic and role details have been withheld.

The inclusion criteria for interviewees aligned with the above and included: 1) A nominated representative from each AHRA centre, 2) A nominated representative from each health service organisation of Monash Partners Academic Health Science Centre, and 3) international experts with published papers in LHS, or from leading organisations with known expertise in LHS or management of health related data. Nominated experts suggested by other interviewees were also considered for interview. Consumer representatives who were members of Centre data committees were also interviewed. Snowball sampling was used when a referral was suggested in a study interview. Contact details for potential participants were obtained from organisation’s websites or through referral. The only exclusion criteria was non-English speakers.

Potential participants were invited to take part in the study by an introductory email or phone call and then followed-up by one of the authors ([AJ]1) to provide study information, answer questions and organise mutually agreeable interview times. Those who agreed to participate were emailed a participant information form before the interview occurred. At the start of the interview, they were asked if they had received the explanatory statement. If they hadn't received it, then they were then immediately emailed it directly by the interviewer. They were asked to read the explanatory statement and ask any questions. In the explanatory statement, all participants were notified that any information they provide would be confidential and any details that might identify them would not be published or made public; these points were also verbally explained by the interviewer. Participants were also advised to keep the explanatory statement. Participants were then asked if they consented to participate in the interview study and if they consented to the interview being recorded and later transcribed. Only after verbal consent was given, did the interview begin. This consent was recorded with the remainder of the interview as a record of the participant’s consent.

Twenty-eight people were invited to take part in an interview. One potential candidate declined interview and another candidate cancelled and did not reschedule. Each interview commenced with the interviewer stating the purpose of the interview and a brief description of the LHS, ascertained from a recent systematic review [11]. The interview guide covered questions about what barriers and enablers were encountered when creating or sustaining LHS environments (Figure 1). Other questions explored what is important to ensure the sustainability of these environments, and sought insights into if they could create a LHS again, what might they do differently.

All interviews were conducted by a single interviewer who is a health services researcher trained in semi-structured interviewing techniques. Twenty-four interviews were undertaken using teleconferencing software and two face to face between 2nd July and 11^th October 2019. Interviews were audio recorded and professionally transcribed. The mean interview time was 55 minutes. After 26 interviews, no new themes were evident and recruitment was ceased.

Sample size:

The sample size was based on the criteria outlined by Malterud (2016) [36]. This included the (a) exploratory aim of the study, (b) decision to interview targeted key informants, (c) ongoing analysis as data was collected, (d) detailed information obtained in the interviews and evidence of repetition in the dataset, and (e) employing a thematic analysis. Thematic saturation was apparent with repetition emerging during the ongoing analysis.

Analysis:

All interview transcripts were loaded into NVivo 12 (QSR International, Doncaster) for data management and coding. A thematic analysis was conducted using iterative and inductive processes following the phase of analysis outlined by Braun and Clarke [37]. An inductive approach was undertaken, as the research aim was exploratory, with the purpose of generating new knowledge.

To ensure familiarity with the dataset, transcripts were read multiple times. Coding involved generating initial codes based on content and meaning, relevant to the research question. Next, themes were searched for among the codes. Codes were then organised by separating, combining and refining to form overarching themes [37]. The entire dataset was subsequently reviewed to ensure the themes and subthemes represented a coherent pattern and to check if any additional data needed coding. Themes were then named and checked for the content captured and for the overall ‘story’ they conveyed. Data analysis was conducted by the first and third authors and discussed at regular meetings with the research team. Varying perspectives were considered and incorporated as the analysis was undertaken.

Rigor:

Records of key analytic and methodological decisions were made to ensure rigor throughout the analysis [38]. Regular consultation between the first author and third author, and meetings with the larger project team, fostered discussions about data interpretations enhancing trustworthiness. Presentations of the evolving analysis to peers and stakeholders enabled peer review to enhanced data validity [39]. A descriptor after each excerpt in the results refers to a unique participant identifier.

The 26 expert participants interviewed were nominated lead representatives from: ARHA Research Translation Centres (n=10), Monash Partners Academic Health Science Centre member organisation nominees (n=8), the Australian Digital Health Cooperative Research Centre (n=1), State Government (n=1), National Australian Digital Health Agency (n=1), Public Health Research Network (n=1), National Stroke Registry (n=1), a consumer advocate (n=1) and international experts from the UK (n=1) and Canada (n=1).

Whilst clear themes were identified and little disagreement on the fundamentals emerged, there were nuances related to the role or experiences of participants. A conceptual model of the findings is shown in Figure 2. The key themes and subthemes are summarised Table 1. Excerpts from the transcripts are provided to exemplify and illustrate the subthemes. Extra quotes from participants are found in Table 2.

1) Systematic frameworks, approaches and iterative, continuous learning with implementation of outcomes into healthcare

Continuous learning

Establishing an iterative innovation cycle early in the implementation of a LHS was identified by interviewees as important. A cycle of innovation was required to continuously drive better patient outcomes and improve services. Some participants reported that it was easy to make changes when initiating some type of quality improvement exercise, but the challenge was to make evidence based changes that were sustainable over time.

ID 01: ‘You need that loop, you need that feedback so that the people who are entering the data could see the value of it and they start acting on it and they start making sure the data is of a high quality when it goes in.’

ID 02: ‘This stuff takes time, it’s not an instantaneous improvement you’ll get, and you do need to invest in time. And it’s a bit like having to do a plan cycle; it is a continual improvement cycle. You are not going to get it right straight up.’

These key informants emphasised how a sustainable LHS requires continuous learning, needing persistence and investment of time. Further, the informants identified that a cyclic improvement process, whereby helping clinicians to see the usefulness of data outcomes for patient care, was key to ensuring high quality data was entered. As such, systems should be interconnected, iterative and cyclical.

Creating a vibrant learning culture that promoted continuous iterative learning through use of data and evidence within a LHS was noted by participants to be unsustainable without top down (e.g., CEO, leader, manager) and bottom up ( e.g., clinician and consumer or patient) support. Support from all stakeholders was required to ensure continuous learning and improvements, underpinned by data and measurement. Further, a positive cultural environment was reported to foster ownership and a sense of responsibility for the quality and relevance of care and services provided.

ID 04: ‘You attract more people and more organisations and so then you create a vibrant ecosystem where this kind of activity is going on all the time’.

ID 05: ‘I think the other thing that’s key to sustainability is if it [data] is being used. So if it’s being well utilised, not just by researchers, but by the health service as well, then people will see value in it. And that will help with the ongoing funding as well.’

These key informants highlighted the necessity for organisations to adopt a ‘business as usual’ attitude to using data as part of a continuous learning cycle.

Interviewees identified that a LHS must have a framework, systems and processes to survive when inevitable changes such as staff turnover and organisational restructures occurred. It was also noted that access to routinely stored data is often tied to knowledge held by individuals, in healthcare environments, rather than at a systems level. This limits sustainability of these systems with knowledge lost with staff movement. In a LHS, underlying frameworks, systems and processes should be documented and embedded.

One informant described how a LHS could be adaptable to be robust and flexible in the face of workforce changes and to manage sustained growth into the future:

ID 06: ‘I think so far a lot of these [LHS environments] have been done on goodwill and have worked best where people who have connected well get together and do some stuff. And we’re now moving into a place, they’re getting more substantial so it needs to be in more formal structures. I think where I’ve seen previously is you need to transition from the interested parties who can work well together to then actually the formalised structure that allows it sustainability of that, regardless of those particular individuals leaving that provide the sustainability.’

2) Broad stakeholder, clinician and academic engagement, with collective vision and leadership

Engaging diverse stakeholders from all levels and disciplines and governance

Diverse stakeholders influenced how structures were set up in the LHS. Participants recommended that governance committees should include stakeholders from clinical, community, research, human resources, finance, information technology, electronic medical record and data analytics teams. Involvement from analytics teams was considered important, as people from these teams could create meaningful knowledge from data, and use effective ways to improve data visibility. Further, participants considered the involvement of consumers as important in a LHS as they could make valued contribution to prioritising data, commenting on measures to be collected and analysed, and deciding time frames for data collection.

One informant emphasised that engaging diverse stakeholders in LHS environments was similarly important as effective governance and processes:

ID 07: ‘So you need governance, you need process, you need engagement. You need to stare down the barrel of the self-interested groups and hold them to account through consumer, health service and health professional engagement. Then, you need to convince everyone that a system like this is the only way to go.’

Clinician engagement and ensured inclusion

Participants stated that clinician involvement was an important resource in a sustainable LHS. However competing clinical workloads often made this challenging, as highlighted by one informant:

ID 02: ‘Our priority is patient care, and I know data does drive improvement in care and service delivery, but you are continually battling with other priorities and it’s the ability to show worth of the data.’

Clinician input into all stages of a LHS from engaging in the vision, to setting clinical priorities, to bringing in expertise and evidence, to analysing data and implementing this back into practice improvement, was seen as important. A LHS was necessary to evaluate and optimise the quality and accuracy of the data, use of data and to implement and drive improvements. Interviewees expressed that clinician buy-in would not be possible without protected time to contribute to the LHS as most clinicians were already operating at capacity. For example, without additional resources and data entry being an integrated and useful component of workflow, the quality of data entered was likely to be poor.

ID 05: ‘Strong multidisciplinary approaches are really important. And then I think obviously a really important aspect of the team is having clinicians involved.’

ID 08: ‘We showed you could get the clinicians in the hospital together, they look at the gaps in care, ‘Ok, we missed out on people getting thrombolysis’ or ‘We’re not meeting the indicator for patients with a care plan. What can we do about it?’ And they get together and they look at their gaps in their quality of care. And then an external facilitator comes in and they do a workshop and they develop an action plan of how they can improve those processes. And by getting them all in together and getting buy-in that does actually improve care. And so we’ve found with the [data-driven project], it improves care by 14% over the time period.’

These key informants highlighted the requirement for clinician involvement in all stages of a LHS to improve health care.

Vision and leadership

Participants stated that stakeholder engagement, collective vision and leadership were all important factors in a sustainable LHS. Developing a common vision informed by all stakeholders was essential for the coordination of multiple resources and for targeted approaches to achieve that vision. Some participants recommended that agreed priorities should be set and worked towards.

ID 07: ‘You need a vision that everybody has. That needs to be co-designed, which means we need all the key stakeholders in the room.’

ID 05: ‘If you can start with projects that are important to the health service and showcase what you can do with their data, that perhaps they’re not currently able to do, that could be a good example to use to support the LHS.’

These above informants described that to showcase the LHS approach, it is recommended to have a clear and shared vision, and start with agreed vision statements is likely to optimise prompt and positive change.

While consensus on a collective vision was crucial, leadership actively committed to the vision was also identified as important. Leadership and strategic planning to embed appropriate processes and systems, was also reported as essential for creating a sustainable LHS.

ID 08: ‘Leadership is really important. If you’ve got really good leaders and they can get really good buy in….’

ID 10: ‘The key enabler is leadership in my view – leadership that sets the direction, frames what you are trying to do and then brings people along. You will need some smart people, you will need some money to cover salaries and you will need some good technology.’

The above informants describe leadership as necessary to ensure resources were available and directed towards supporting the vision, to foster the buy-in from multiple stakeholders.

Leadership to cultivate trusting relationships between clinicians, other healthcare staff and managers was also reported as important aspect of a LHS. A sustainable LHS requires leaders to facilitate the agreement of measured outcomes, and to foster trust that these measures will be used for the agreed purposes and not for penalising individuals or the withdrawal of funding. This informant highlighted trust as important:

ID 03: ‘Trust is the barrier, if you don’t trust what data is being used appropriately. People get pretty tired, particularly in hospital settings, of data being used as a club, ‘You’re not hitting your four hour targets, bastards’

3) Skilled workforce, capability and capacity building in a sustainable LHS

A data literate, skilled clinical and technical workforce is vital

An important resource in a sustainable LHS identified by participants was data literate healthcare and technical staff. Data skills and knowledge identified as important for healthcare staff included familiarity with data collected, and understanding and interpreting data.

ID 02: ‘I think we have a real skill shortage. How we address that, I am not sure. I think that the skill shortage is in the technical fields, and also in the clinicians that work on the ground. If we are moving towards more data-driven improvements and using data to help with workflows or patient flows, clinicians need to understand how to use the data. How do you drive improvements through the use of data? ‘

This informant highlighted the necessity for data-literate staff and also the need to upskill many current healthcare staff with data acquisition, analysis, knowledge generation using data and how it can drive better outcomes.

Further, participants suggested that a sustainable LHS has people with highly developed data management and analysis skills embedded at various levels in a LHS.

ID 11: ‘There needs to be a core group of people who are well versed in several disciplines and that might mean handling of large data, informatics, computing, information technology people, machine learning and other similar related initiatives, data visualisation, the whole lot of infrastructure that comes with it; people infrastructure. And the trick will be to develop at least a core group who are constant and that’s where the sustainability financial need will be critical.’

This informant identified that people with skills related to data management, linkage, analysis and the different ‘languages’ from related disciplines must be available in a LHS for growth and sustainability.

Collaboration between data specialists, clinical staff and academics was seen as beneficial in a LHS. University academics specialising in IT and statistical analyses were important resources that could provide external expertise when required.

ID 02: ‘There needs to be really good relationships and partnership between the clinical and the data people. Because I think with that a symbiotic relationship, they’re both dependent on each other. The clinical people are dependent on the data people to get it out and make sure it’s the right thing and the data people were reliant on the business to actually run with it, know their own business, and understand how it can drive improvement.’

ID 11: ‘Not one hub can do everything by itself; the one hub will have buy-in from related organisations industry, university organisations.’

These key informants underscored the importance of developing connections and relationships to facilitate a sustainable LHS.

4) Resources and sustainable funding

A key resource for creating a sustainable LHS was guaranteed funding and long-term investment. Costs were incurred as implementing a new LHS into a health system already running at capacity requires a coordinated and resourced approach. Additionally, costs were associated with obtaining, storing, analysing and using data.

ID 02: ‘You need funding. It needs to be funded. And not a one-off funding but recognition that data changes and these types of hubs need some degree of maintenance as well, so there’s the technical and obviously the workforce element to be able to pull the data out, and protect it.’

ID 14: ‘I want to create a structure that allows us to be a unit that has longevity and sustainability. So I’d really like to move towards three to five year plans where we can put data scientists in that aren’t pre-committed to part of individual projects but can help build a resource to allow you to have a genuine hub.’

These key informants identified protected longer-term funding, was key to the viability of a LHS, as the resources/investment to establish a LHS were required over a number of years before benefits were seen, involving significant cost

5) Data systems, processes, access and use

Data sharing, transparent processes and consent

Transparent governance, systems, structures, agreements and clarity of roles, responsibilities and activity within the LHS were all seen as vital for its continued success. This informant described the importance for clear data access and consent processes.

ID 05: ‘We need to be more and more transparent around how to use people’s personal data and having community education and transparency. That is, I think, very important.’

Participants perceived a key part of sustainable a LHS was transparent legislation that informed how patient data would be used within the system to create better healthcare. Timely access to data, information and reports was essential to a LHS. If layers of patient consent, agreements, permissions and sign offs resulted in major delays, then the LHS would not be sustainable. This informant called attention to system efficiency as a key LHS component.

ID 12: ‘Make a learning health system function efficiently, within the workflow of an organisation, so all of the necessary approvals are done with the least amount of pain.’

Infrastructure, data storage, timely access and streamlined data linkage and processes

Interviewees identified that there are available affordable, secure, accessible and integrated systems for data storage. It was stated that there are many options available in the current market and that emerging LHS environments would use existing infrastructure and technologies. Timely access to relevant data was reported as fundamental to sustainable LHSs. One informant described access to data for informed decision making in real-time was pivotal in a LHS:

ID 04: ‘We want to move more towards identified, real time and live data that can be used to help patients, clinicians and researchers.’

Hospitals already generate mandated reports for the Government such as length of stay, waiting time in ED, and number of re-admissions. Participants identified that data linkage was important for producing information about quality, models of care, and health outcomes. Several participants mentioned a desire to link health data to other relevant data. Health and healthcare does not exist in an isolated bubble. The social determinants of health are particularly relevant when evaluating value based care or health outcomes when taking into consideration a wider social picture rather than a siloed health perspective. It was expressed that there is much that could be learnt through the linking of health data with that for housing, pensions, education, and crime and corrections data. However, current data linkage processes are often drawn out and convoluted and therefore unfavourable for current LHS environments in Australia. One participant suggested that streamlining federal and state based legislation would facilitate data linkage and more timely access to various data sources:

ID 01: ‘So many things have been duplicated across Australia, every state is doing its own thing; it’s so inefficient.’

A sustainable LHS efficiently produces practical, informative and up to date data. This participant described how data from a LHS could be strategically used and the subsequent benefits:

ID 01: ‘Now the beauty of it was that suddenly not only could you follow what was happening in real time in your department, but you could also interrogate and get all sorts of stats and there are all sorts of standard tables that they had, there were reports that would spit out. Using this reporting system, we could also refine the reports even more. You could also set up triggers, so - whether it was at triage or with the nurse inside with the doctor - when they put in certain words you could set it up so up come half a dozen questions as a result. So suddenly it just transformed the way that we were doing business.’

The importance of a culture and processes that make the integration of data, information and research, routine in healthcare delivery was important in a LHS. For this approach to be sustainable in a LHS, organisational level support was seen as essential.

ID 02: ‘this is about improving clinical care by getting organisations to understand the use of their data and the benefits of using their data, in many ways much more than it is about the academic use of the data.’

ID 08: ‘We want to be more internally-driven so that internal facilitation, the quality managers in the hospital look at these live reports and say, ‘How can we improve that in our hospital rather than externally driven?’

These informants conveyed the importance of embedding systems in the culture of an organisation to automate data retrieval and reports which included benchmarking.

There is a requirement for transformation in healthcare to deliver better health outcomes. Digital health data has the potential to underpin this transformation towards more efficient and effective services. This project leveraged multidisciplinary leaders nationally and internationally to explore core elements of a sustainable LHS and inform development of a LHS framework for the Academic Health Science Centre context in Australia. Key themes were identified including the need for systematic approaches and iterative, continuous learning with implementation of outcomes into healthcare; broad stakeholder, clinician, academic and community engagement with collective vision, leadership, governance and trust; skilled workforce, capability and capacity building; resources in human capital and investment, and; data access, systems, standards and processes.

A major theme identified was that a sustainable LHS requires systematic frameworks, approaches and iterative, continuous learning with implementation of outcomes into healthcare. Planning to embed processes into usual practice to capture and use data and learnings was key to success, consistent with literature on registry based LHS [4, 6]. Ad hoc LHS-like environments were acknowledged as common in healthcare, but participants agreed that these were unsustainable and waned, when key personnel moved on. These offered little for those at the frontline in terms of tangible healthcare improvement, yet frontline engagement was vital for success and sustainability. Transparent and iterative, cyclical improvement processes in the LHS to enable timely access, sharing and streamlined linkage of data, with mechanisms to feedback into frontline care, were recognised as important in ensuring the LHS approach was impactful and sustainable. These findings are commensurate with the emerging literature that describes systematic approaches and processes that are common in all LHS environments; see further below for more discussion about this [1, 2, 9, 11, 40-42].

Broad stakeholders including health professionals, managers, academics and community engagement was deemed important. Indeed partnering with all people who contribute to a healthy LHS including patients and the wider community, alongside all staff working in healthcare, was prioritised. The continuous learning within a LHS was dependent on diverse groups engaging and working together to innovate and solve complex healthcare problems around a shared vision. In this context, strong credible leadership, identification and commitment to a collective LHS vision was important. Transparent governance and clear structure was identified as a sub-theme. Collecting and storing data, without clear purpose or vision, was felt to contribute to lack of sustainability, because the worth of the data and interactive cyclical improvement benefits were not seen. This is worthy of consideration when a LHS is focused on data use only for research or health service management, rather than extension into improved clinical care. These views are in line with the vast literature on health care improvement and implementation science research [1, 43, 44], and also highlight the importance for improvement in complex health systems to assist clinicians and others to engage and lead change [1].

Skilled workforce, capability and capacity building in a sustainable LHS was another major theme. A data literate, skilled clinical and technical workforce, with access to education and training was reported as key to a LHS. Highly developed data management and analysis skills in staff embedded at various levels was also recognised. Collaboration between data specialists, clinical staff and academics was seen as beneficial and a strength of an LHS. University academics specialising in IT and statistical analyses were seen as important resources and can provide external expertise when required. These skills and resources in human capital underpin a sustainable LHS and also should be iteratively enhanced and expanded over time. This is supported by a recent systematic review that identified health improvements arising within 23 LHS environments, and all had combined people with relevant workforce capacities and people with data analytic capabilities to make sense of the complex data arising from complex improvement cycles to address areas of unmet need, public interest and priorities [11].

Other resources and investment were also noted as essential in building a sustainable LHS. In addition to investing in workforce, there are costs associated with obtaining, storing, analysing and using data. It was reported that it takes significant resources and protected funding, sustained for a number of years to establish a LHS and often this investment is required before benefits emerge. An essential component of a LHS is a collaborative platform that provides connectivity across silos, organizations, and professions. In the literature there is evidence that automated reports using the data from the entire LHS has led to the efficient identification of patients for standardised care, specialised care, follow-up or clinical trials [45-47]. Collection of information directly from patients before the clinical encounter has also been reported as beneficial due to time efficiencies [48], as well as creating PROMs (patient reported outcome measures) that are saved within the electronic medical record and can enable longitudinal tracking of individual patient outcomes and are used in aggregated research [4, 5, 48]. There are also examples of existing LHSs that invested infrastructure to enable patients to self-track their condition [4, 5], for example the electronic patient-reported outcome measurement system to identify distress and despair in cancer patients, which then lead to more referrals to psychological care [5].

Data access, systems and processes to producing useable, useful and quality data was identified as a theme and is generally reported in literature on the impact of LHS models [3, 8-11]. We found that appropriate infrastructure and transparent processes are required to operationalise data access, linkage, analysis and applications. Participants clearly expressed that from the patient perspective, there is a responsibility to ensure their data can be compiled, accessed and analysed to benefit their own health and that of the broader community. This requires a culture of trust, transparency, partnership and co-design. For example, a Swedish system enables a patient to record symptoms, health status, and quality of life directly into their electronic medical record. Patients access their record at a clinic or at home. The LHS combines their data with other data (clinical examinations and laboratory results) in graphical displays of health status over time. The patient and clinician can view this together, or separately, and this helps patient and clinician partnerships to optimize health. Data is also exported into the national registry, enabling research and ongoing learnings, contributing to improving population health. Evaluations have found that patients greatly value this system [4, 6].

Connection to extant literature:

These findings are commensurate with the broad literature that describes systematic approaches and processes required for system-level complex interventions in health care, with our research suggesting this is also required for a functioning LHS [1, 2, 9, 11, 40-42]. As recently summarised by Melder et al. [1], such complex interventions require rigorous yet pragmatic processes and iterative applications that learns from success and failures and drives health care improvements [1]. This continuous learning culture was identified as our first key theme. Skilled workforce, capability and capacity building was identified as another of our key themes and these are essential to power the rigorous (yet pragmatic) processes to drive the LHS continuous learning culture.

Clinician and stakeholder engagement and clinician leadership is another recognised element in the literature on complex interventions in health care [1]. This aligns with our second theme: broad stakeholder, clinician and academic engagement, with collective vision and leadership. Trust was identified as a sub-theme, which resonates with trust appearing in a number of leadership models [13, 14] and trust in technology has been found to be influential in organisation wide adoption studies [49].

Access to the best available evidence and measurement is also purported to be required in the literature about complex interventions in health care [1]. This was captured in our key themes of ‘Data access, systems and processes’ and ‘Resources and investment over time’.

The semi-structured interviews were informed by a national steering committee in data-driven healthcare improvement and international systematic literature review. The nature of the questions may have influenced the breadth or responses provided. We aimed to inform development of LHS in the Australian context and hence interviews from Australian experts were preferenced and results may not be generalisable. Overall, diverse stakeholders were interviewed and thematic saturation was reached, however additional stakeholders and stronger consumer and international participation may have influenced the outcomes.

The themes identified in this qualitative study in data-driven healthcare improvement, provide insights into the core elements in a sustainable LHS. Fundamentally, an LHS requires continuous learning, generating evidence for implementation back into frontline care to improve outcomes. Structure, governance, trust, culture, vision and leadership were all seen as important, along with a skilled workforce and sustained investment. Processes and systems to optimise access to quality data were also seen as vital in an effective, sustainable LHS. These findings will inform a co-designed framework for implementation within the Australian Academic Health Science Centre context. Application of these findings, offers the potential to accelerate the use of health data to iteratively produce new knowledge to in-turn improve healthcare delivery and health outcomes.

LHS Learning Health System

AHRA Australian Health Research Alliance

AHRTC Advanced Health and Research Translation Centres

CIRH Centre for Innovation in Regional Health

Ethics approval and consent to participate: Those who were contacted and invited for an interview were emailed a participant information form. All interviews were recorded, and verbal consent provided at beginning of the interview. This study was approved by the Monash University Human Research Ethics Committee (Project ID: 19969) who approved the method for verbal consent which is explained in the methods section and also here:

Potential participants were invited to take part in the study by an introductory email or phone call and then followed-up by a project researcher to provide study information, answer questions and organise mutually agreeable interview times. Those who agreed to participate were emailed a participant information form before the interview occurred. At the start of the interview, they were asked if they had received the explanatory statement. If they hadn't received it, then they were then immediately emailed it directly by the interviewer. They were asked to read the explanatory statement and ask any questions. In the explanatory statement, all participants were notified that any information they provide would be confidential and any details that might identify them would not be published or made public; these points were also verbally explained by the interviewer. Participants were also advised to keep the explanatory statement. Participants were then asked if they consented to participate in the interview study and if they consented to the interview being recorded and later transcribed. Only after verbal consent was given, did the interview begin. This consent was recorded with the interview as a record of the participant’s consent.

• Consent for publication: Not applicable.

• Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

• Competing interests: The authors declare that they have no competing interests.

• Funding: This work was supported by a fellowship program for one of the authors (JCE) from Monash Partners and funded by the Medical Research Future Fund.

• Authors' contributions: HT, [AJ]1, [AJ]2 conceived the project. JE, SB and [AJ]1 created the interview schedule. JE conducted all interviews. SB and [AJ]1 analysed the data and analysis was finalised by ongoing discussions until consensus was reached between JE and [AJ]1. Manuscript was prepared by JE with significant contributions by SB, [AJ]1 and HT. All authors reviewed, provided inputs and approved the manuscript.

• Acknowledgements: We would like to thank the interview participants, and the wider Monash Partners team.

Melder, A., et al., An Overview of Healthcare Improvement: Unpacking the Complexity for Clinicians and Managers in a Learning Health System. IMJ, 2020.
Flynn, A., Informatics and technology enable us to learn from every patient: Pharmacists' many roles in learning health systems. Am J Health Syst Pharm, 2019. 76(15): p. 1095-1096.
Budrionis, A. and J.G. Bellika, The Learning Healthcare System: Where are we now? A systematic review. J Biomed Inform, 2016. 64: p. 87-92.
Ovretveit, J., E. Nelson, and B. James, Building a learning health system using clinical registers: a non-technical introduction. J Health Organ Manag, 2016. 30(7): p. 1105-1118.
Smith, S.K., K. Rowe, and A.P. Abernethy, Use of an electronic patient-reported outcome measurement system to improve distress management in oncology. Palliat Support Care, 2014. 12(1): p. 69-73.
Hvitfeldt, H., et al., Feed forward systems for patient participation and provider support: adoption results from the original US context to Sweden and beyond. Qual Manag Health Care, 2009. 18(4): p. 247-56.
Harvard Business Review Analytic Services., Leading a new era in health care. 2019.
McLachlan, S., et al., The Heimdall Framework for Supporting Characterisation of Learning Health Systems. J Innov Health Inform, 2018. 25(2): p. 77-87.
Menear, M., et al., A framework for value-creating learning health systems. Health Res Policy Syst, 2019. 17(1): p. 79.
Medicine., I.o., Roundtable on Value and Science-Driven Health Care: The Learning Health System and its Innovation Collaboratives: Update Report. 2011: Washington, DC: IOM.
Enticott, J.C., Johnson A., and T. H, Learning Health Systems translating data-driven research into healthcare: A systematic review. Submitted, 2020.
Laukka, E., et al., Identifying the Roles of Healthcare Leaders in HIT Implementation: A Scoping Review of the Quantitative and Qualitative Evidence. International Journal of Environmental Research and Public Health, 2020. 17(8): p. 2865.
Gillespie and Mann, Transformational leadership and shared values: the building blocks of trust. Journal of Managerial Psychology, 2004. 19(6): p. 588 - 607.
Graen and Uhl-Bien, Relationship-Based Approach to Leadership: Development of Leader-Member Exchange (LMX) Theory of Leadership over 25 Years: Applying a Multi-Level Multi-Domain Perspective. Leadership Quarterly 1995. 6(2): p. pp. 219-247. .
Gaddam, S., Timing and Assimilation of New Technology Adoption in Healthcare. Journal of Health & Medical Informatics, 2019. 10:3.
Ammenwerth, E., Technology Acceptance Models in Health Informatics: TAM and UTAUT. Stud Health Technol Inform, 2019. 263: p. 64-71.
Anderson and West, Measuring climate for work group innovation: development and validation of the team climate inventory. JOURNAL OF ORGANIZATIONAL BEHAVIOR, 1998(19): p. 235 - 258.
Harvey, G. and A. Kitson, PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci, 2016. 11: p. 33.
Martin, P., M. Salanova, and J.M. Peiro, Job demands, job resources and individual innovation at work: going beyond Karasek's model? Psicothema, 2007. 19(4): p. 621-6.
Teede H, et al., Australian Health Research Alliance: National priorities in data driven healthcare improvement. [Accepted 2019]. MJA, 2019.
Robinson, T., et al., Flipping the Paradigm: A Qualitative Exploration of Research Translation Centres in the UK and Australia. 2020, IN PRESS at Europe PMC.
Psek, W.A., et al., Operationalizing the learning health care system in an integrated delivery system. EGEMS (Wash DC), 2015. 3(1): p. 1122.
Flum, D.R., et al., Implementation of a "real-world" learning health care system: Washington State's Comparative Effectiveness Research Translation Network (CERTAIN). Surgery, 2014. 155(5): p. 860-6.
Bhaskar, R., A Realist Theory of Science. 2008, Verso, London.
Danermark, B., et al., Explaining society: An introduction to critical realism in the social sciences. . 2002, London:. Routledge.
Clark, A.M., Critical Realism, in The SAGE Encyclopedia of Qualitative Research Methods., L. Given, Editor. 2008: Thousand Oaks, California.
Schneider, E., et al., The Commonwealth Fund report: Mirror, Mirror 2017: International Comparison Reflects Flaws and Opportunities for Better U.S. Health Care. 2017: New York.
Duckett, S., Australia’s health system is enviable, but there’s room for improvement. The Conversation, 2017.
Schneider, E.C. and D. Squires, From Last to First - Could the U.S. Health Care System Become the Best in the World? N Engl J Med, 2017. 377(10): p. 901-904.
Australia', C.o., The Senate Select Committee on Health Final report. 2016: Parliament House, Canberra.
Carpenter, A., et al., Health policy, administration and expenditure Submission to the Senate Select Committee on Health. 2015, The George Institute for Global Health and The Menzies Centre for Health Policy, University of Sydney.
Braithwaite, J., P. Glasziou, and J. Westbrook, The three numbers you need to know about healthcare: the 60-30-10 Challenge. BMC Med, 2020. 18(1): p. 102.
Care", A.C.o.S.a.Q.i.H., The state of patient safety and quality in Australian hospitals. 2019: Sydney NSW.
Braithwaite, J., et al., Bending the quality curve. Int J Qual Health Care, 2020. 32(Supplement_1): p. 1-7.
Care;, N.C.o.H.a.S.i.H. National Clinical Trials Governance Framework and User Guide. 2019 25 August 2020]; Available from: https://www.safetyandquality.gov.au/publications-and-resources/resource-library/national-clinical-trials-governance-framework-and-user-guide.
Malterud, K., V.D. Siersma, and A.D. Guassora, Sample Size in Qualitative Interview Studies. . Qualitative Health Research, 2015. 26(13): p. 1753-1760.
Braun, V., et al., Thematic Analysis, in Handbook of Research Methods in Health Social Sciences, P. Liamputtong, Editor. 2019, Springer Singapore: Singapore. p. 843-860.
J., C., Qualitative inquiry & research design: choosing among five approaches. In: Creswell J, editor. Qualitative inquiry and research design. . 2007: Thousand Oaks: Sage Publications.
Morse, J.M., Critical Analysis of Strategies for Determining Rigor in Qualitative Inquiry. Qualitative Health Research, 2015. 25(9): p. 1212-1222.
Friedman, C. and J. Macy. Toward Complete & Sustainable Learning Systems. [Presentation slides] 2014; Available from: https://medicine.umich.edu/sites/default/files/2014_12_08-Friedman-IOM%20LHS.pdf.
Friedman, C., et al., Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inform Assoc, 2015. 22(1): p. 43-50.
Forrest, C.B., et al., PEDSnet: a National Pediatric Learning Health System. J Am Med Inform Assoc, 2014. 21(4): p. 602-6.
Best, A., et al., Large-system transformation in health care: a realist review. Milbank Q, 2012. 90(3): p. 421-56.
Braithwaite, J., D. Marks, and N. Taylor, Harnessing implementation science to improve care quality and patient safety: a systematic review of targeted literature. Int J Qual Health Care, 2014. 26(3): p. 321-9.
Marsolo, K., et al., A Digital Architecture for a Network-Based Learning Health System: Integrating Chronic Care Management, Quality Improvement, and Research. EGEMS (Wash DC), 2015. 3(1): p. 1168.
Fung-Kee-Fung, M., et al., Piloting a regional collaborative in cancer surgery using a "community of practice" model. Current Oncology, 2014. 21(1): p. 27-34.
Fung-Kee-Fung, M., et al., Regional process redesign of lung cancer care: a learning health system pilot project. Curr Oncol, 2018. 25(1): p. 59-66.
Eriksson, J.K., J. Askling, and E.V. Arkema, The Swedish Rheumatology Quality Register: optimisation of rheumatic disease assessments using register-enriched data. Clin Exp Rheumatol, 2014. 32(5 Suppl 85): p. S-147-9.
Li, X., T.J. Hess, and J.S. Valacich, Why do we trust new technology? A study of initial trust formation with organizational information systems. The Journal of Strategic Information Systems, 2008. 17(1): p. 39-71.

Table 1: Themes and subthemes for a sustainable Learning Health System.

Themes and subthemes:
1	Systematic frameworks, approaches and iterative, continuous learning with implementation of outcomes into healthcare
1	Continuous learning
2	Broad stakeholder, clinician and academic engagement, with collective vision and leadership
	Engaging diverse stakeholders from all levels and disciplines, including active health professional engagement Governance, structure, culture
	Vision and leadership

3	Skilled workforce, capability and capacity building
3	A data literate, skilled clinical and technical workforce is vital
4	Resources and investment over time
5	Data access, systems and processes
	Data sharing, transparent processes and consent
	Infrastructure, data storage, timely access and streamlined data linkage and processes

Table 2: Table of extra quotes from participants

ID 03: ‘You try and then you try again and then you try again and again and then you do it again and then you take a breath and then you do it again and again and again and again and again, and again and again and again and again. I’ve been doing this for 17 years now.’

ID 01: ‘If you set something up and it just goes into a black hole and no one ever sees it you have problems. Not only are clinicians not interested in it, but the data you get will be rubbish. You need that loop, you need that feedback, so that the people who are entering the data can see the value of it and they start acting on it, and they start making sure the data is of a high quality when it goes in.’

ID 05: ‘Strong multidisciplinary approaches are really important. And then I think obviously a really important aspect of the team is having clinicians involved.’

ID 06: ‘There’s got to be initially a really clear vision about what is it you want to achieve from it, and not just in the next two years but the next five to eight years.’

ID 09: ‘It’s all about expert leadership.’

ID 05: ‘Sometimes I will know what I want and I will be trying to say, ‘Can we do this? Can we do this this way?’

ID 06: ‘And data doesn’t come for free. It’s expensive in terms of the setup of it and also the ongoing stories of that data in whatever form it is.’

ID 12: ‘But right now, the situation is the funding for these things comes from multiple sources and sustainability is a major issue because people are usually unwilling to commit funding for more than a year, two years, two years and after that, they need to revisit things’

ID 13: ‘It takes a brave CEO, who tends to have an appointment term of about five years, to invest in something that may not bear fruit for a few years.’

ID 04: ‘There’s nothing more powerful than demonstrating a positive outcome.’

ID 06: ‘The informatics profession is an emerging profession and is going through its own process of specialisation. Where you previously would’ve had people putting data in and then you’ve had people getting data out, there’s now a growing community. This includes data scientists, perhaps a visualisation expert, structural infrastructure expert, as well as those who design the architecture of the warehouse. There’s growing specialisation. Historically, things were pretty simple. Whereas now that’s probably about 10 different professions along that line.’

ID 15: ‘Sustainability requires that this produces value which can be quantified that can then turn into profits or savings or increased values, which can go back into this spot. But it can’t be left for the people in the periphery to find that money because the people in the periphery will always be short of money and will use this money to soak up their deficits wherever they happen to have them. Has to be allocated centrally, for that purpose.’

ID 02: ‘It needs to be embedded in the business that information is valuable. ’

Download PDF

Journal Publication

published 26 Nov, 2020

Read the published version in BMC Health Services Research →

Submission checks completed at journal
20 Nov, 2020
Editorial decision: Accept
12 Nov, 2020

You are reading this latest preprint version

Leaders' Perspectives on Learning Health Systems: a qualitative study

Status:

Journal Publication

Version 4

Abstract

Figures

Background

Methods

Results

1) Systematic frameworks, approaches and iterative, continuous learning with implementation of outcomes into healthcare

Continuous learning

2) Broad stakeholder, clinician and academic engagement, with collective vision and leadership

Engaging diverse stakeholders from all levels and disciplines and governance

Clinician engagement and ensured inclusion

Vision and leadership

3) Skilled workforce, capability and capacity building in a sustainable LHS

A data literate, skilled clinical and technical workforce is vital

4) Resources and sustainable funding

5) Data systems, processes, access and use

Data sharing, transparent processes and consent

Infrastructure, data storage, timely access and streamlined data linkage and processes

Discussion

Limitations

Conclusion:

Abbreviations

Declarations

References

Tables

Status:

Journal Publication

Version 4